Mid-IR hyperspectral imaging for label-free histopathology and cytology

Since the first demonstration of a working QCL in 1994 by Capasso et al [30] with peak power of 8 mW at 4.2 μm, the laser has been very successfully further developed. By now, the peak power has risen to over 200 W [31] and even in continuous-wave (cw) operation the average output power exceeds several watts [32, 33]. The emission spectral range has improved as well, and lasing has been demonstrated on spectral bands spanning from 2.63 μm [34–36] to 24.4 μm.

Similar to diode lasers (DL) and interband cascade lasers (ICL) [37], pumping of the laser is realised by applying a voltage across the QCL chip. Carefully designed periodic heterostructures as visualised in figure 1 enable electrons to move into an upper laser level in the conduction band, where these electrons undergo a radiative transition to a lower energy state, also within the conduction band [38]. Finally, the electrons exit the lower laser state by non-radiative transition and are re-injected into the upper laser state of the next cascade (period). After moving through all cascades, typically 30–40 within one laser structure, the electrons are collected at the positive contact. The electrons' wavefunctions are composed of spatially-modulated Bloch states of the conduction band during the entire process, so no holes are involved, which is not the case for DL and ICL.

Zoom In Zoom Out Reset image size

Increasing the number of cascades can technically archive total quantum efficiencies greater than 100% because each cascade can result in up to one photon per electron [39] and the electron can be reused by the next cascade. The power efficiency is largely independent on the number of cascades, but increases with the cascade number due to better overlap of the optical mode with the gain. For high power, it appears that having 10–15 cascades [40] is optimum because of a need to balance thermal conductance with power efficiency. One can also enhance the emission range of a single chip by heterogeneous stacking of cascades with gain centred at different wavelengths [41–44].

To achieve individual energy levels, in an otherwise continuous band of energies, quantum confinement has to be established. This is realised by surrounding a thin (∼nm) layer of low bandgap material with a different lattice matched material of higher conduction band offset. This creates a quantum hole with individual energy levels along one spatial dimension. Additional energy levels need to be introduced into the sequence for proper functioning of the laser (e.g. for efficient depopulation of the lower laser state), resulting in a set of quantum holes and barriers. Such a set is called a period or a cascade, and as is apparent from its name they can be stacked together quite easily as the major carrier are electrons. By heterogeneous stacking of periods, a gain spectrum at different wavelengths can be designed. Therefore, they can be periodically stacked. Several pairs of quantum wells and barriers on top of each other form a cascading structure enhancing the quantum efficiency of the process [39] and giving the lasers their name (QCL).

QCL chips emit a Fabry–Perot (FP) spectrum centred at a particular wavelength (design dependent) that can be broader than 10% of the central wavelength. One way to achieve single longitudinal mode lasing is to etch a diffraction grating on top of the laser stripe, which acts as a distributed Bragg reflector and provides feedback for a narrow band of wavelengths; such a laser is called distributed feedback laser (DFB). Tuning is achieved by either directly changing the temperature of the chip or by modifying its pumping current. This technique however only allows tuning of less than 1% of the central wavelength [45]. To overcome this limitation, several DFB lasers with different grating periods can be manufactured on one chip. Lasing from 5.9 to 10.9 μm with over 50 mW peak power throughout a wide range of wavelengths has been demonstrated [43].

A different approach is to use just one stripe, but use an asymmetric sampled grating DFB with an additional amplifier section [46]. With this configuration, over 5 W power in pulsed operation at 4.8 μm and tunability of 270 nm (300 nm) has been achieved, as well as 1.25 W in cw operation. By combining these two approaches [44], thereby creating a set of eight stripes with different sampled grating DFB with additional on-chip sections responsible for combining the beams, tuning from 6.2 to 9.1 μm has been demonstrated [44]. All the above mentioned designs emphasise probably the biggest benefits of semiconductor versus other laser types, i.e. small size, completely electronic tuning and enormous mass production potential.

Single mode lasing and tuning can also be achieved by mounting a FP laser into an external cavity (EC) [47]. Applying an antireflection coating on at least one facet of the QCL chip suppresses the FP modes and allows wavelength selection by tuning an optical element in the cavity. In Littrow and Littman–Metcalf configuration, a diffraction grating is utilised for this purpose, achieving laser line widths of below 1 cm⁻¹ [48, 49]. An alternative to this is the use of interference filters and retroreflectors. This has been successfully demonstrated [50] over a wavelength range of 4–4.3 μm and shows comparable results to grating based designs. If broader emission wavelength can be tolerated, acousto-optic modulators (AOM) and simple mirrors can also be used for tuning, thus achieving switching time between wavelengths under 1 μs [51] instead of several milliseconds, typical for micro-electro-mechanical systems. Dual wavelength operation and tuning is also possible with AOM tuning [52].

EC-QCLs with tuneability in pulsed mode from 7.6 to 11.4 μm, with peak power of up to 1 W and average power of 15 mW have been realised [41]; higher powers have been achieved in the narrowed band of 7–8.9 μm with a maximum power of 200 mW [42]. Even by placing the QCL chip in these complex external cavities, the size is still reasonably small, of the order of a few tens of cm. However, the spectral range coverage of most commercially available laser chips is not sufficient to cover the full wavelength range of interest. To overcome this limitation, several individual EC-QCL chips are combined into a system with a common output port.

Supercontinuum sources are well known to be powerful tools for microscopy and spectroscopy applications due to their high brightness and very broad spectrum. Supercontinuum generation was first demonstrated in bulk borosilicate glass by Alfano and Shapiro [53]. In this process, narrow band optical pulses undergo significant spectral broadening through the interplay of a variety of nonlinear optical effects in the propagation medium, yielding a spectrally continuous output. Optical fibres have proven to be particularly suitable for this purpose, as the light is strongly confined over long interaction lengths, which enhances the efficiency of nonlinear effects. With the advent of fibre technologies, such as high power pump diodes and gain fibres, as well as the development of new optical fibre materials, geometries and dispersion profiles, it is now possible to design and build broadband supercontinuum sources at wavelengths from ultraviolet to the infrared [54, 55].

In recent years, a great deal of effort has been put into extending emission capability into the MIR. This pursuit requires the use of novel fluoride based fibres with transmission windows in the 2–5 μm region and chalcogenide materials that allow supercontinuum generation extending to 13.3 μm [56, 57]. In the 2–4.5 μm region, power spectral density values of several hundred μW nm⁻¹ have been reported [56]. Beyond 5 μm, experimental demonstrations have shown power spectral density levels of 1–2 μW nm⁻¹ [58]. Simulations of alternative conversion schemes indicate that there is potential to achieve power densities of over 10 μW nm⁻¹ at wavelengths as long as 10 μm [59]. However, it is worth noting that current commercially available MIR supercontinuum sources do not extend beyond a wavelength of 4.5 μm (around 2200 cm⁻¹).

Fibre based supercontinuum sources have a high inherent beam quality, enabling diffraction limited performance and high spatial coherence. Due to their large spectral bandwidth, supercontinuum sources exhibit extremely low temporal coherence effectively eliminating speckle in broadband illumination situations. However, applications that require narrow filtering of the supercontinuum may exhibit speckle [60].

Furthermore, fibre based MIR supercontinuum sources can be made compact, robust, and affordable ($50K–$100 K) thus making them very attractive for practical applications. While only just appearing on the market, commercially available MIR supercontinuum sources are relatively inexpensive and can be purchased for comparable prices to those of current commercial turn-key QCL systems in the same wavelength range.

As supercontinuum sources are inherently broadband, their direct use for hyperspectral imaging requires a spectrometer or wavelength selection element. Utilising acousto-optic tuneable filters (AOTF), they can also be operated in a narrowband imaging scheme which has been demonstrated for wavelengths up to 4.5 μm [60, 61]. Yet, at any longer wavelengths, there are currently no commercially available AOTFs. Thus, the use of longer wavelength supercontinuum source requires an interferometer, monochromator or a novel upconversion scheme [62].

In summary, globars are still the most universal low cost option available to cover the entirety of the spectral range of interest, but lacking brightness, especially for high resolution imaging. Synchrotrons and FELs are bright tuneable light sources, but very expensive and relatively inaccessible. Supercontinuum sources are close to performing as well as synchrotrons, but do not cover the full MIR spectrum with sufficient power. QCLs are becoming commercially mature (in the short wavenumbers 800–1800 cm⁻¹) and offer spatially and temporally coherent radiation in the longer wavelength MIR range, but may require a multi-chip design to cover arbitrarily wide spectral ranges.

A thermal detector is based on the temperature increase produced by absorbed incident photons, which can be detected as a change in a temperature sensitive physical property of the material. These detectors generally have slow temporal response, often in the millisecond range, set by the thermal relaxation time of the sensor material. The spectral response of thermal detectors depends on the absorption characteristics of the material used. Thermal detectors are efficient for detection of high brightness signals, and can be operated without cryogenic cooling. One of the major sources of noise in thermal detectors relates to ambient thermal fluctuations transformed into a variation in the conductance of the sensor material. In general, thermal detectors need to have a low heat capacity for fast response time and good thermal isolation from the surroundings [63, 64]. Applications of thermal detectors are generally limited by their slow response time and lack of sensitivity, due to unwanted black-body radiation from surroundings and temperature fluctuation in the detector material particularly in the non-cooled versions. Thermal detectors are divided into different categories based on the physical property responsible for generating the output signal in response to the temperature variation. Two commonly used thermal detectors are presented below.

3.1.1. Bolometers

The bolometer is a widely used thermal detector whereby the incident radiation is absorbed in the sensor material, increasing the temperature of the material relative to the surroundings. The sensor material has a temperature-dependent electrical resistance, which can be read out as a voltage when passing a current through the bolometer. The voltage can be expressed as:

$$\begin{eqnarray}&&{\rm{\Delta }}V=IR\alpha {\rm{\Delta }}T;\,\alpha =\displaystyle \frac{1}{R}\,\displaystyle \frac{{\rm{d}}R}{{\rm{d}}T},\end{eqnarray} \tag{ 1 }$$

where $\alpha$ is the temperature coefficient of resistance of the sensor material that determines the sensitivity of the bolometer. Preferred materials for bolometers should have low heat capacity, low room temperature resistance and a large thermal coefficient of resistance.

IR imaging in the long wavelength regime, e.g. 8–14 μm, can be achieved using microbolometer arrays. These arrays consist typically of 320 × 240 pixels, however larger arrays with 640 × 512 pixels are emerging. The individual pixel dimension is of the order of 20 μm × 20 μm; however, as the pixel size decreases and thus the number of pixels per unit area increases, a larger noise equivalent temperature difference results due to smaller pixels being less sensitive to IR radiation [65].

3.1.2. Thermopiles

The thermopile is another type of thermal detector, where the temperature change is measured through a series of thermocoupled devices. An infrared absorption film is deposited on one side of the junction of a thermopile consisting of a series of thermocouple pairs. Each pair has a junction of two conductors on either side of a thermal resistance layer, resulting in a voltage in the range 10–100 mV proportional to the temperature difference,

$$\begin{eqnarray}&&{\rm{\Delta }}V=N({\alpha }_{a}-{\alpha }_{b}){\rm{\Delta }}T,\end{eqnarray} \tag{ 2 }$$

where ${\alpha }_{a}\,{\rm{and}}\,{\alpha }_{b}\,$ are the Seebeck coefficients (the magnitude of an induced thermoelectric voltage in response to a temperature difference across that material) of the two conductors forming the junction of the thermocouple pair, N is the number of thermocouple pairs and ${\rm{\Delta }}T$ is the temperature difference caused by the absorbed IR radiation.

The sensitivity and the frequency response of thermopiles are inferior to bolometers, nevertheless thermopiles are cost efficient and have found several applications in the medical, farming and automotive industry [63, 64].

The working principle of photodetectors is based on the photon absorption in a semiconductor material resulting in an electronic transition generating a free charge carrier. The bandgap of the semiconductor dictates the working wavelength range. Silicon (Si) is the preferred material for detectors in the visible and near-infrared (NIR) range, between 350 and 1100 nm. Moving towards longer wavelengths, indium gallium arsenide (InGaAs) is the detector material of choice covering the range between 0.85 and 1.7 μm [66]. Indium antimonide (InSb) can be used in the range 3–5 μm, whereas mercury cadmium telluride (MCT) is commonly used in the wavelength ranges 3–5 and 8–12 μm [63, 67, 68].

Generally, Si and InGaAs detectors can have very high quantum efficiency reaching more than 90%, thus approaching single photon counting sensitivity, and have a rise time in the order of 10–100 ps. In contrast, InSb and MCT based detectors typically have rise times in the order of 10–100 ns or more. The low bandgap energy of the IR detector has a great impact on the noise performance, both in terms of internal noise as well as sensitivity to black-body radiation originating from the detector surroundings. In practical applications where high sensitivity is needed, the InSb and MCT detectors are actively cooled in order to obtain low noise levels.

3.2.1. InGaAs and InSb detector

Si and InGaAs detectors are divided into PIN diodes and avalanche photodiodes (APD). In both cases the detector operates in the reverse-biased region, which is known as photoconductive mode. The APDs work in the high voltage reverse-biased region, where the signal-to-noise ratio can be improved by the internal current gain due to the avalanche effect. APDs can be applied for single photon detection when operating in the Geiger mode.

A great variety of InGaAs detectors has been developed for different applications; table 1 lists the properties of two typical APDs, working in the linear regime and Geiger mode, respectively. Generally for all detectors, there is a trade-off between high quantum efficiency, low noise and high bandwidth.

Table 1.  Properties of IR detectors.

Detector type	Wavelength range (μm)	Working temp. (K)	Quantum efficiency	Band width/maximum count rate	NEP $({\rm{pW}}{{\rm{Hz}}}^{-1/2})$/dark count rate s−1
InGaAs APD [69]a	0.85–1.65	260	64%@1550 nm	0.9 GHz	0.24
MCT [70]	2–12	195	≥31%@6 µm	∼20 kHz	0.5
Bolometer [71]	15–2000	4.2	N/A	300 Hz	0.12
InSb photovoltaic detector [72] (Hamamatsu)	1–5.5	77	—	—	1.1
InGaAs APD [73]b	0.9–1.7	183	25%@1550 nm	∼200 KHz	<100
SNSPD [74]	0.8–1.8	<1	>75%	>20 MHz	∼300
Upconversion detector [75]	1–20c	∼300	93%@1550 nm	>1 GHz	100

^aThe detector works in the non-Geiger mode. ^bThe detector works in the Geiger mode. ^cThe upconversion detector in [26] is based on PPLN crystal, and it has only a few sub-nanometres detection range centred at 1550 nm. In order to achieve detection wavelength range from 1 to 20 μm, different kinds of nonlinear crystal should be applied, as shown in table 3.

InGaAs can also be applied for NIR imaging. Table 2 lists the parameters of an InGaAs camera provided by Hamamatsu [76]. Compared to silicon-based cameras used for visible light imaging, the InGaAs cameras suffer from higher dark current and larger readout noise as a result of the lower bandgap. InSb photovoltaic detectors reach further into the MIR then InGaAs systems and cover the wavelength range from 1 to 5.5 μm [77]. They however require to be cooled with liquid nitrogen like MCT detectors. Their noise performance is better than an MCT but as their spectral, coverage is smaller they are not used for biomedical imaging applications often.

Table 2.  Properties of IR cameras.

Detector type	Spectral band (μm)	Frame rate (FPS)	Working temp. (K)	Field of view	Dark Current (electron/pixel s−1)	No. of pixels
InGaAs [76] (Hamamatsu)	0.95–1.7	7.2	203	—	130	640 × 512
Bolometer $({\rm{V}}{{\rm{O}}}_{x})$ [78]	8–14	100	273–323	30° × 23°	—	384 × 288
MCT FPA [79]	2–12	1500	77–190	—	100	320 × 255
Upconversion detector [80]	1–20	25	300	20° × 20°	—	265 × 265a

^aThe numbers mentioned are for a specific system which vary accordingly based on several parameters, such as wavelength range and nonlinear crystal used in the module.

3.2.2. MCT based detector

Upconversion based detection is a fundamentally different approach for IR sensing. Using upconversion the low energy IR radiation is not detected directly, rather it is mixed with a high brightness laser in a transparent nonlinear medium, shifting both the spectral and spatial information from the low energy infrared range to the NIR or visible range, for which sensitive and fast detectors exist. Using a transparent medium for the nonlinear process means that no black-body radiation from the medium is added to the IR signal, which is in strong contrast to direct detection where the medium has to be highly absorbing. Secondly the nonlinear process imposes a strong spectral and spatial filter to the upconversion process, limiting the spectral bandwidth and field of view of the detector, hence reducing the black-body noise picked up from the surroundings.

Upconversion is a parametric process and follows the principle of energy and momentum conservation (equation (3) and figure 2)

$$\begin{eqnarray}&&\displaystyle \frac{hc}{{\lambda }_{{\rm{I}}{\rm{R}}}}+\displaystyle \frac{hc}{{\lambda }_{{\rm{p}}{\rm{u}}{\rm{m}}{\rm{p}}}}=\displaystyle \frac{hc}{{\lambda }_{{\rm{u}}{\rm{p}}}},\,\,\,{\vec{k}}_{{\rm{I}}{\rm{R}}}+{\vec{k}}_{{\rm{p}}{\rm{u}}{\rm{m}}{\rm{p}}}={\vec{k}}_{{\rm{u}}{\rm{p}}},\end{eqnarray} \tag{ 3 }$$

where h is the Planck constant, c is the speed of light, $\lambda$_i and k_i are the wavelength and the wave vector for each specific photon. It should be emphasised that the phase matching condition acts in all spatial dimensions; therefore the upconversion photon preserves the spatial information of the IR photon, which makes upconversion imaging possible.

Zoom In Zoom Out Reset image size

Upconversion technology has been exploited for both point detection [81] and for imaging configurations [80, 82, 83]. Recently, a conversion efficiency of 20% has been demonstrated which presents the upconversion scheme as an emerging technique for MIR detection [74]. Figure 3 shows an example of upconversion-based measurement of MIR hyperspectral imaging.

Zoom In Zoom Out Reset image size

Here, a globar is used as broadband illumination source, whilst an upconversion unit converts the transmitted radiation from the sample from MIR to NIR and a silicon-based camera captures the images. The acquired broadband images are then post processed to obtain a hyperspectral datacube made of a series of monochromatic images.

Tables 1 and 2 list the performance properties of different IR detectors including the detector based on upconversion technology. Upconversion can be used over a substantial part of the IR spectral range depending on the nonlinear crystal used. Table 3 shows the properties of some commonly used nonlinear materials for the upconversion.

The choice of the nonlinear material is very important for a given application, not only should the material be as transparent as possible to the IR wavelengths in order not to introduce thermal noise, but it should also be able to phase match the nonlinear process while preferably having a high nonlinear coefficient for efficient frequency conversion. So far lithium niobate (LN) has been the preferred material for upconversion detection. It has been used in an intracavity configuration both for spectroscopic and imaging applications, however limited by the transparency range (to below 4.5 μm). At wavelengths below 12 μm, silver gallium sulphide (AGS) has been used both for imaging [82] and spectroscopy [84]. However, other features may also be important depending on the application, e.g. AGS has very poor thermal conductivity, which means that it cannot be used in intracavity configurations for imaging applications as thermal lensing within the crystal will distort the images. Silver gallium selenide (AGSe) may be an interesting candidate even at longer wavelengths; however, AGSe can only be phase matched with relatively longer mixing wavelengths, e.g. 1.9 μm, and consequently needs to be combined with InGaAs detectors rather than Si detectors, thereby increasing the overall cost and complexity of the system.

Upconversion detection has also been demonstrated using periodically poled LN waveguides (PPLN-W). Comparing PPLN-W with upconversion in the bulk nonlinear crystals, high internal conversion efficiency in PPLN-W can readily be achieved even with modest mixing powers due to the spatial confinement of the interacting fields inside the waveguide [81, 85]. If a broadband illumination source is used, the spectral resolution of the system can be set by the phase matching properties on the nonlinear process.

Furthermore, upconversion can be used for IR imaging. In this case, the spatial resolution of an upconversion detection system is primarily determined by the beam diameter of the mixing field, which serves as a soft aperture in the imaging system. However, both the spatial and spectral resolution of the imaging system depends on the IR input wavelength [83, 86].

Upconversion detection has a very small acceptance angle, which can strongly inhibit the ambient noise. In contrast, conventional detectors need a cold shield to block the ambient radiation from almost 2$\pi$ solid angle. A second source of noise is false IR photon generation through a spontaneous parametric down conversion process (SPDC). These IR photons will be indistinguishable from the signal IR photons. High pump power is usually needed to achieve high conversion efficiency, however the upconverted SPDC noise is roughly quadratically proportional to the pump power and can be considered a trade-off between high efficiency and low-noise. For some cases, the noise power can be as high as few pW. The widely used periodic poled nonlinear crystal, PPLN, is very attractive, below 4.5 μm however random duty cycle errors induced in the fabrication of the poling structure enhances USPDC noise [87] in comparison to the bulk crystal. Further techniques utilising quantum phenomena with potential to improve MIR imaging technology are induced quantum-coherence imaging [88] and field-resolved spectroscopy [89, 90], although they have not been applied to real world samples yet.

A number of preprocessing steps are often required prior to undertaking detailed multivariate analysis of hyperspectral images in order to obtain meaningful results and interpretation. This section focuses on methods to analyse the spectral domain i.e. the unfolded data cube. These techniques have been developed in the context of FTIR data and some of them harness the continuous nature of the data, therefore application to DF-IR has to be considered with caution.

A key assumption when using the Beer–Lambert law is that the optical path length b (and molar absorption coefficient) is the same for each pixel. For histopathology applications, where the sample is a tissue section cut by microtome, variations in the sample thickness across the section result in a different path length. A normalisation step is therefore essential for comparing variations in absorption between pixels in an image. One simple approach is vector normalisation, which maps the spectra onto a unit sphere in multivariate space [114]. Other simple approaches include normalisation to constant total [114] and min–max normalisation, which scales each spectrum to between 0 and 1. Without normalisation, those pixels with lower signal because the tissue section is thinner will be given less weight in subsequent multivariate analysis, which may mask important variations due to compositional changes that may relate to pathology.

Another common approach to normalisation in IR spectroscopy is multiplicative scatter correction (MSC) [115, 116]. It attempts to account for scaling and offset effects by calculating a least squares fit of the measured spectrum to a reference spectrum. The measured spectrum is the corrected using the using the slope and offset of the fit. An extended multiplicative scatter correction (EMSC) has been described that allows for normalisation, baseline subtraction and filtering of unwanted signals in a single step [117, 118]. Examples of application of EMSC in histopathology include resonant Mie scatter EMSC (RMieS-EMSC) [119] and paraffin removal [120, 121].

Standard normal variate (SNV) has also been suggested as an alternative method for normalisation [122]. SNV centres a spectrum and scales it by its own variance. SNV gives more weight to values in the spectrum that deviate most from the mean value, which for FTIR corresponds to peaks in the absorbance spectrum. It is usually most successful when the spectra are quite similar e.g. between pathology groups. It has been shown that SNV and MSC are mathematically very similar [123, 124].

5.2.1. Baseline subtraction

5.2.2. Derivative spectra

5.3.1. Atmospheric water vapour removal

5.3.2. Resonant Mie scatter extended multiplicative scatter correction (RMieS-EMSC)

5.3.3. Paraffin removal

Although all of the multivariate methods described in this article ignore spatial information, it is still advantageous to interpret the information in the image domain in order to give some biological context. For example, it would be very difficult to identify a spectrum as being from a cell nucleus if it is not possible to see it in the context of the surrounding cell. The following methods are the most commonly applied for microspectral histopathology.

5.4.1. Intensity based images

5.4.2. Principal components analysis (PCA)

PCA is a data reduction technique that decomposes a dataset into a set of linearly uncorrelated variables called principal components (PCs) [145, 146]. PCA is useful in hyperspectral image analysis [147] because it provides information about the pixels in the form of principal component scores T, and the corresponding biochemistry in the loadings P.

$$\begin{eqnarray}&&X=TP^{\prime} ,\end{eqnarray} \tag{ 7 }$$

where X is the unfolded array of pixel spectra. Because the scores for each component are per-pixel, they can be refolded into intensity images (see figure 5). In addition, examining the loadings for that component alongside the scores-based intensity image allows interpretation in both spectral and image domains. For histopathology applications, this enables spectral differences between pathology groups, for example, to be discussed in terms of biological structures in the tissue section. This is vital for comparison with standard histology that relies almost entirely on interpretation of the biological structures and only uses the biochemistry as a means to improve image contrast through the application of chemical stains, such as haematoxylin and eosin.

PCA outputs are the principal components listed according to a decreasing order of explained variance. Alternative approaches include minimum noise fraction [148, 149] and Min/Max autocorrelation factors [150] which order the components by signal-to-noise ratio.

PCA is applied as a processing method or, more often, as a preprocessing step before a classification method such linear discriminant analysis (LDA) or support vector machines (SVM) is applied. Including a PCA step reduces the influence of noise (by excluding components) and prevents over fitting by reducing the number of variables in the model, by combining those collinear variables found in the dataset together in one principal component, leaving only orthogonal components.

5.4.3. k-means clustering

k-means clustering is an exploratory algorithm that aims to divide the spectra into k groups, or clusters [151]. Each spectrum belongs to the cluster with the nearest mean, or centroid. Its primary use in hyperspectral imaging is as a form of visualisation. Clustering creates a new vector that contains a value indicating the cluster membership of each pixel. An image can then be generated by shading each pixel according to its cluster membership. In image processing terms, the image has been segmented and is now easier to interpret. For histopathology applications, not all of the clusters will be relevant and all spectra associated with those clusters can now easily be removed from further analysis. For example, in figure 6, an image of a human colon polyp section has been segmented by k-means clustering. It can be seen that cluster x corresponds to the substrate i.e. where there is no sample. These pixels can now be excluded from further processing e.g. by PCA, and the analysis can focus on describing the sample and not the differences between the sample and the substrate.

Zoom In Zoom Out Reset image size

k-means clustering does have some disadvantages. For example, it can be difficult to choose an appropriate value for k i.e. the number of clusters. Too few and the segmented image is not informative. Too many and the clusters represent noise. Methods such as the silhouette width [152] exist to try and solve this problem but are often computationally intensive and can over, or under, estimate the number of informative clusters. In histopathology applications, in particular, the value of k that generates meaningful clusters biologically is often much larger than the number determined by the silhouette width. Other disadvantages include initialisation, which may result in different clustering each time the algorithm is applied to the same image, and the inability to use any distance measure other than the Euclidean distance. Modifications such as k-mediods [152] have been proposed to allow for the use of alternative distance measures, and other extensions are available such as fuzzy C-means (FCM) [153, 154], which allows for soft-cluster membership i.e. each pixel has some degree of membership to all clusters, and spatially-guided FCM [155] that takes into account both the spectral and spatial information during clustering. Furthermore, clustering performed on separate hyperspectral images often leads to regions with similar spectral signals being allocated to clusters with spectrally different centroids and colours in the different images. Common k-means on the whole dataset can overcome this problem at the expenses of significant computing resources. Self-organising maps, an unsupervised neural network [156], have also been used as alternative form of clustering for microspectral histopathology [157].

A long-standing goal of microspectral histopathology has been to make predictions for the diagnosis/detection of disease. In a data analysis context, this is usually framed as a classification problem. Applied approaches include LDA [114], partial least squares-discriminant analysis (PLS-DA) [158, 159], SVM [160, 161], random forests [162] and neural networks [163]. The approach taken by a number of studies is to generate healthy/normal/benign versus disease type models using pixels of interest extracted from hyperspectral images. Selection of these regions of interest is either guided by a pathologist or extracted using a technique like k-means, or by a preliminary classification model trained to identify different tissue types (e.g. stroma), or a combination thereof. The predicted pathology labels can be generated on a per-pixel basis and refolded for visualisation in a similar approach described previously for k-means. Figure 6 shows an example human colon section where the red and green pixels are the outputs of a predictive algorithm to detect cancer. Black pixels have been rejected as 'not-epithelium' and white pixels indicate the substrate, or absence of sample.

Using FT-IR imaging is popular because use of the Fourier transform allows rapid collection of the entire spectrum. For biospectroscopy applications though there are regions of the spectrum that are not informative, either because they only contain noise or because they do not provide relevant information e.g. indicating the presence of disease. DF-IR has been proposed as an alternative that only collects IR images at specific ('discrete') wavelengths that are relevant to the problem being studied. In general, the number of relevant wavelengths is expected to be far lower than those in a whole FTIR spectrum, and thus using DF-IR may actually speed-up image acquisition, which is critical for translation of IR imaging into histology applications, for example. Because of the complexity of the samples in biospectroscopy, relevant targets for DF-IR imaging typically fall into bands, or blocks, covering regions of overlapping peaks that correspond to biologically important molecules, such as proteins. Thus, DF-IR imaging falls somewhere between multi- and hyper-spectral imaging. As such, many of the approaches described before, which work well for hyperspectral images, may no longer be suitable for DF-IR images. Alternative normalisation methods such as autoscaling (standardisation), or block scaling, will be more suitable due to the discrete nature of the wavelength regions being collected. Research is ongoing in this area as new instrumentation for collecting DF-IR images becomes available.

For DF-IR imaging to be successful, the right wavelengths have to be measured. The approach generally taken in the literature so far has relied on FT-IR imaging combined with variable selection methods to identify the relevant wavelengths for a particular problem. A number of variable selection methods are available for this purpose, and some recent examples for wavenumber selection in DF-IR imaging include Gini coefficients [164] and variable importance in projection scores [165]. Whichever method is used, a robust validation scheme is required to prevent overfitting and accidental selection of non-informative wavelengths, both of which would be detrimental to translation of DF-IR into clinical applications. Ultimately, the selected wavenumber targets have to be tested using a large external validation set to verify their suitability for a given problem. The problem-specific nature of wavelength selection required the selection to be applied afresh for every research question.

As already discussed in the data processing section, IR hyperspectral images are directly correlated with the underlying biochemistry of the sample. Plotting metrics computed from IR spectra can be described as 'digital staining' i.e. an alternative to the chemical stains currently used in histopathology. Digital staining has the potential to provide more information than current chemical staining technologies. Increasingly complex information can be generated by treating three different intensity based images as a single RGB image. An example is shown in figure 6 which is the composite image of three selected principal components. One criticism of this approach is that the pseudo colour maps can be too colourful, which some pathologists may find off putting. A suggested alternative is to use approaches such as neural networks to generate images that look more like stained images and provide the equivalent of a chemically stained image without any chemical staining; so called stainless staining (124). An example using a PLS regression model to digitally stain a colon section is shown in figure 6. Advantages include visualising multiple stains for the same section and otherwise impossible stains e.g. for biomarkers or pathology indicators that do not currently have a chemical stain. The primary disadvantage is the decreased spatial resolution in IR images versus the visible, because of the longer wavelengths involved and the corresponding diffraction limit.

Digital staining, however, still relies upon a pathologist to interpret the resulting spatial distribution of biochemical signals as they would for conventional chemical staining, although each unstained section could be digitally stained for various different biomolecules of interest. The true benefit is gained by predicting pathology directly from the combined chemical and spatial information. Old et al demonstrated classification accuracies of 100% while denying decision for 30% of the samples for the diagnosis of oesophageal adenocarcinoma [171]. Tiwari et al demonstrated the use of a QCL microspectroscopy setup for the identification of infiltration and fibrosis in endomyocardial biopsy samples to assess transplant rejection, achieving 82.87% overall accuracy without neglecting samples [111]. There are numerous more examples of proof of concept experiments achieving classification accuracies between 70% and 95% [164, 172–182].

The most common problem in this research however is that sample numbers are usually relatively small which makes classification models susceptible to the curse of dimensionality [170, 183]. If a single spectrum consists of more data points than there are samples, classifiers have significant chance of picking up random variations as features for classification. This can be overcome by applying dimensionality reduction methods such as PCA or increasing the sample size. Higher sample numbers however are difficult to collect, measure and analyse. A common shortcut, here, is to utilise tissue microarrays (TMA) which are paraffin blocks consisting of selected biopsy sub-sections from different samples arranged in an array specifically used for medical research. Individual sections can then be taken that include cores of many tens to hundreds of tissues samples. This provides a good distribution of tissues, pathologies and patients, and it is simpler for the pathologist to classify each sample. The disadvantage of this is that it does not represent the large samples and variance found within individual samples that a pathologist would usually make their diagnosis from. A very good example of utilising TMAs for this purpose is given by Bhargava et al [184]. Another big challenge in the field is to find a uniform approach for data acquisition to make use of the spectral information, suppressing perturbation from individual instrument response functions [185].

FTIR imaging of single cells is of interest for a number of reasons. Firstly, cultured cell lines form the basis of many enquiries in the biological sciences research field. They have proven to be robust model systems for assessing metabolic, morphological and general biochemical changes occurring both in healthy and disease states. Cultured cell lines, being clonal cells, provide high reproducibility. Furthermore, the cellular environment can be tightly controlled through media composition and CO₂ incubators. This consistency is important for a valid association of spectral changes with specific biochemical processes. FTIR imaging of cultured cell lines has been used to investigate a range of cellular events, such as changes in chemical composition during the cell cycle [186–189], and changes in protein acetylation levels [190, 191]. FTIR imaging further enables label-free identification of subcellular structures as shown by Clède et al [192]. Ratios of specific IR bands were used to identify organelles such as the nucleus and the Golgi apparatus. The results were corroborated by fluorescence microscopy.

Secondly, some lines of enquiry deal with cells that naturally exist in a unicellular form. These include unicellular organisms, such as bacteria [193, 194] and yeast [195, 196], as well as highly specialised cell types from multicellular organisms, including blood cells [197, 198] and sperm cells [199]. A recent review explores the application of vibrational spectroscopy, including FTIR imaging, on microbial cells [200]. As for imaging of tissue sections, FTIR imaging of single cells also has clinical potential. For example, a label-free non-invasive method for assessing the health of single sperm cells has great potential for assisted reproduction.

Details of sample preparation vary depending on the cell type and the aim of the study. Adhesive cells can be grown directly on a FTIR suitable substrate, such as CaF₂. Cells grown in suspension need to be deposited onto the substrate in a different manner. However, in broader terms, sample preparations can be divided into two main approaches: fixation of cells and live cell preparation.

6.2.1. FTIR imaging of fixed single cells

6.2.2. FTIR imaging of live single cells