Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions

Abstract

Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.

Graphical Abstract

1 |. INTRODUCTION

Optical imaging techniques capable of accurately assessing and visualizing tissue molecular and biochemical characteristics have significant applications in both early disease screening as well as intraoperative surgical guidance. Technical and methodological advancements over the last decades have facilitated the transition of many optical imaging modalities from bench-top instrumentation used in fundamental research to clinical devices suitable for bedside investigations [1,2]. These include fluorescence lifetime imaging (FLIm) optimized to image tissue at clinically relevant mesoscopic scales.

While the widespread clinical adoption of optical techniques is still far from the status achieved by radiology and nuclear-based imaging methods (ie, X-rays, magnetic resonance imaging [MRI], computed tomography [CT] or positron emission tomography [PET]), intraoperative tissue assessment based on optical methods have gained significant ground. Optical imaging is well suited for providing clinicians with real-time, in situ feedback of tissue pathology. The competitive advantage of optical imaging resides in portable, cost-effective instrumentation that allows for seamless integration with existing clinical workflows. Furthermore, optical imaging relies on non-ionizing radiation (unlike X-rays or PET) and can provide morphological, molecular and biochemical contrast to distinguish tissue lesions that may not be apparent to the unaided eye.

White light reflectance imaging is the most basic optical imaging technique routinely used in clinical practice to visualize tissues, but lacks sensitivity and specificity to diagnose pathological conditions. Beyond white light imaging, several imaging technologies that exploit the properties of light propagation in tissue (ie, absorption, scattering) are valuable tools for clinical diagnosis and visualization of tissue content and morphology. Optical coherence tomography (OCT) is currently the standard-of-care in ophthalmology [3]. OCT has also been explored intraoperatively [4] for detecting abnormalities in the oral mucosa [5,6], brain gliomas [7,8] and positive tumor margins in breast conserving surgery [9,10] among other oncological applications [11]. Intraluminal OCT modalities are further used for endoscopic imaging of the gastrointestinal tract [12,13] and for intravascular imaging of atherosclerosis [14,15]. Narrow-band imaging, which highlights mucosal structure and microvasculature with selective wavelength illumination (blue, green and red), is used on endoscopic approaches to detect Barrett’s esophagus [16,17], ulcerative colitis [18,19] and colorectal tumors [20]. Diffuse reflectance spectroscopy and diffuse optical imaging exploit the heterogeneous absorption and scattering properties of light in tissue for breast cancer management [21–23] and has also been applied to assess brain tissue [24,25]. Raman spectroscopic imaging has remarkable molecular specificity and it has been used in preliminary studies for guiding surgeries and identifying tumors and cancerous tissue in head and neck [26–32], brain [33–35], skin [36,37], breast [38–41] and the gastrointestinal track [42–45], among others [46].

Fluorescence contrast has been used for medical diagnosis for over three decades. The first studies demonstrating the use of fluorescence for tissue characterization and pathology diagnosis were reported in the 1980s and early 1990s. Fluorescence from photosensitizing agents and tissue molecules were used to identify neoplastic tissue [47–49], to characterize atherosclerotic plaque [50] and to guide biopsy collection [51]. Today, fluorescence contrast is routinely used in surgical guidance [52–56]. Most of the established fluorescence imaging approaches employ contrast agents that consist of non-targeted fluorescent dyes typically used to highlight the vasculature and, more recently, “smart probes,” obtained by combining a targeting ligand and a fluorescent molecule. There are currently four fluorescence-guided surgery contrast agents approved by the FDA, and dozens more are on trial [1,57]. Most of these contrast agents fluoresce in the near-infrared (NIR) range, where tissue scattering and absorption properties are optimal.

Despite the superior molecular specificity of targeted fluorescent contrast agents, the administration of exogenous drugs and dyes complicates clinical protocols and requires a thorough analysis of potential toxic pharmacologic effects [58]. Endogenous fluorescence (ie, intrinsic tissue fluorescence or tissue autofluorescence) is an alternative approach capable of discriminating between tissue types and malignancies [49,59,60]. Clinical studies that investigate the use of intrinsic tissue fluorescence for the evaluation of tissues are simplified, as they do not require the thorough evaluation of pharmacologic effects required for novel exogenous contrast agents. The main disadvantage of instruments based on autofluorescence intensity analysis is their low biochemical specificity. In part, this is due to the overlapping emission spectra of multiple endogenous fluorophores [61]. Measuring the temporal dynamics of tissue autofluorescence (eg, fluorescence lifetime), together with the spectral features, provides a means to overcome such limitation and have found application in clinical settings [62–64].

Ultimately, three general criteria drive the clinical adoption of a new technology [65]: (a) the existence of a true clinical need, (b) the ability of the new technology to solve the problems posed by specific clinical needs and (c) the seamless integration of the new technology with standard clinical workflows. The focus of this article is on label-free mesoscopic FLIm as an optical imaging technology for tissue assessment in real-time within clinically relevant settings. Here, we review the fundamental principles of FLIm including mesoscopic FLIm (Section 2), the technical characteristics of the most advanced clinical FLIm instrumentation with a particular emphasis on the pulse-sampling technique (Section 3), and the most commonly used analytical methods to interpret FLIm data (Section 4). We conclude with a summary of recent applications of mesoscopic FLIm in surgical oncology, particularly in intraoperative assessment of oral and oropharyngeal cancer, FLIm-assisted brain tumor surgery and identification of positive margins in breast cancer, and we highlight recent results on the assessment of atherosclerotic cardiovascular disease (Section 5). This review aims to underscore the FLIm’s capability for real-time tissue characterization and to provide a general outlook of the field.

2 |. FUNDAMENTAL PRINCIPLES OF MESOSCOPIC FLIM

Autofluorescence has been extensively used for tissue diagnostics as it is inherently linked to specific molecular species present in tissue, thus providing biochemical information. Biological tissues emit fluorescence upon appropriate irradiation (Figure 1A). Fluorescence is the radiative process that occurs after a molecule absorbs a photon that promotes an electron from the ground state (S₀) to an excited state (S₁) and this electron decays back to the ground state, resulting in the emission of another photon (Figure 1B). Fluorescence is typically characterized by the absorption cross-section of the molecule, the coupled absorption and emission spectrum, the quantum yield (number of emitted photons over the total number of absorbed photons) and the lifetime (τ) of the fluorescence signal. Due to non-radiative vibrational relaxation, fluorescence emission is of lower energy than the absorbed photons (Stokes shift; Figure 1C). Fluorescence lifetime describes the average amount of time a molecule resides in the excited state after absorbing a photon [61,66]. For a single fluorophore, fluorescence decays as a monoexponential function and the fluorescence lifetime corresponds to the time of the decay where the intensity has decreased to 1/e of the amplitude (I₀; Figure 1D).

FIGURE 1.

Tissue autofluorescence. A, Schematic of light-tissue interaction, where incident excitation light results in the generation of fluorescence emission. The tissue penetration depth depends on the tissue scattering (μ_s) and absorption (μ_a) properties, which are wavelength (λ_ex) dependent. The longer the wavelength, the deeper light penetrates. B, Simplified Jablonski diagram for fluorescence, where a photon (hν_ex) excites the electrons from the ground state (S₀) to an excited state (S₁). The radiative relaxation back to S₀ emits fluorescence photons (hν_em). C, Absorption (abs.) and emission (em.) spectra featuring the Stokes shift. D, Temporal intensity decay of the fluorescence emission characterized by the fluorescence lifetime (τ) following an excitation pulse

Fluorescence lifetime varies between molecular species and it is sensitive to environmental factors such as the conformation and binding state of the absorbing molecule, the temperature, the pH, or the solvent viscosity [67]. Such dependencies can be used for probing tissue properties including composition, structure and metabolic state, leading to an overall assessment of the tissue’s health without exogenous contrast administration [62,64].

Intrinsic tissue fluorophores often present as aromatic molecules. Amino acids like tryptophan and tyrosine absorb and emit in the ultraviolet (UV) range. Enzymatic cofactors Nicotinamide Adenine (Phosphate) Dinucleotide (NAD(P)H), Flavin Adenine Dinucleotide (FAD) and pyridoxal phosphate fluoresce in the visible range. Other molecules fluorescing in the visible range include collagen, elastin, structural protein crosslinks, keratin, lipopigments and porphyrins. The fluorescence properties of these molecules and other tissue fluorophores are detailed elsewhere [59,61,66,68].

Approaches to measure fluorescence lifetime in biological tissue divide in to time- and the frequency-domain methods (see [68] for a recent review). Widespread methods in the time-domain include Time-Correlated Single-Photon Counting (TCSPC), which generates a histogram of the photon time of arrival and time-gating and pulse sampling directly quantify the fluorescence decay curve. The frequency-domain methods measure the amplitude and phase shift of the fluorescence signal in response to a modulated excitation light. Fluorescence lifetime imaging microscopy (FLIM), through various implementations using time- and frequency-domain methods, has been extensively used to study biological systems at microscopic resolution including cells [69–73], organoids [74–76] ex vivo animal tissue sections [77–80], human tissue sections [80,81] and tissue biopsies [82,83]. FLIM has also been adapted to image diseased tissue in vivo on animal models [84,85] and in human patients [82,86–88]. For further clinical applications that require access to constrained spaces, endoscopic FLIM was developed [89–92]. In spite of the successful implementation of endoscopic FLIM, the rather complex optical systems involved, and the typically expensive setups have limited its applications in clinical settings. Flexible and inexpensive optical fiber probes, cost-effective short-pulsed UV illumination sources and a multispectral detection scheme combined with the pulse sampling method led to the development of the current, and most advanced, clinical FLIm instrument. This generation of FLIm instruments demonstrate versatility for rapid image acquisition of tissue areas with variable shape and scale.

When compared to multi-photon FLIM imaging based on NIR fluorescence excitation, the use of single-photon UV excitation for FLIm may be counter-intuitive as it excites a wider focal spot, suffers from out-of-focus background, has a shorter penetration depth in tissue, and is more prone to cause cellular damage. However, a careful choice of design parameters makes it well suited for tissue evaluation in clinically-relevant settings. Single-photon UV excites multiple tissue fluorophores simultaneously, overcoming the need to switch between different excitation wavelengths and simplifying illumination schemes. Tissue fluorescence is then detected at multiple spectral bands tailored to the emission spectra of cells (eg, NAD(P)H, FAD) and extracellular matrix components (eg, elastin and collagen and its crosslinks such as pyridinoline) [66,93]. The shallow penetration depth of UV light, which is determined by tissue scattering (σ_S ∝ 1λ⁴) and absorption (eg, by water and hemoglobin) properties, confines the measurements to the tissue surface (≲250μm, varies with tissue type). It is therefore well suited for assessing pathological conditions occurring on tissue surfaces such as epidermal diseases, cardiovascular disease (ie, atherosclerosis) and alterations of the mucosal epithelium of the gastrointestinal tract. Similarly, it is also suitable during surgical procedures such as intraoperative tumor resection surgery. Finally, the excitation wavelength currently employed for FLIm (355 nm) is in the UVA range (>315 nm), which does not pose a risk of direct DNA damage [94,95]. However, UVA is known to produce reactive oxygen species that may in turn induce DNA and protein damage. Indirect tissue damage by UVA on internal organs is poorly understood, although reported studies indicate that doses below 10 mJ/cm² have negligible effects on tissue [95,96]. As shown in the following, FLIm studies can be performed with irradiation levels that are well below this value (eg, 1–3 mJ/cm²), which comply with the American National Standards Institute (ANSI) guidelines (more on safety in Section 3.2).

3 |. CLINICAL FLIM INSTRUMENTATION

3.1 |. Evolution of fluorescence lifetime spectroscopy and imaging instrumentation for tissues

The acquisition of time-resolved fluorescence from tissue was initially documented using pulsed nitrogen lasers as light sources (337 nm excitation wavelength and <20 Hz repetition rate) and TCSPC as detection method [50,97]. The availability of more energetic light sources and fast data acquisition electronics resulted in the ability to directly sample the entire fluorescence decay signal [98–100]. In initial studies, the ability to record fluorescence emission spectra with high spectral resolution (using monochromators) was achieved at the cost of large optical losses and required individual measurements for each wavelength. Nevertheless, this approach, combined with fiber optic probes for delivery of the excitation light and collection of the fluorescence signal was used in clinical settings to study brain tumors ex vivo [101] and in vivo [102]. The potential for clinical translation of this approach was limited by the long data acquisition time of individual measurements. The data acquisition speed was markedly improved by a novel design of the fluorescence lifetime measurement system, where the optical signal, decomposed in a limited number of wavelength or spectral bands, was acquired simultaneously using a single photomultiplier tube [103]. This approach relied on a temporal multiplexing scheme where signals from each spectral band were delayed in time by the different lengths of delay fiber optics. The introduction of new laser sources with higher repetition rate (in the kHz range) [104] and frequency tripled excitation wavelength at 355 nm (eg, fiber lasers [105] and solid state lasers [106,107]) allowed the generation of higher fluorescence signals and led to higher data acquisition speeds, which enabled fast helical intravascular imaging [108]. At the same time, these new sources were smaller and cost-effective, allowing the decrease of the overall size and cost of the apparatus. The more compact designs are easier to integrate in clinical settings [109,110]. This configuration has enabled a real-time point-scanning approach [111]. In combination with the incorporation of an aiming beam to clearly identify the measured location, it has become possible to create FLIm parameter maps from free-hand scans [107]. Currently, the clinical FLIm instruments primarily employed in clinical research studies are built on this concept.

3.2 |. Design requirements for clinical FLIm

In the following, we summarize the design considerations for a practical clinical FLIm instrument.

3.2.1 |. Integrability with existing surgical instrumentation and workflow

This is an important feature for both accessibility to sites of interest during procedures and to minimize interruption of the conventional clinical workflow.

3.2.2 |. Adjustable field-of-view and resolution

The imaging system has to be adapted to the different dimensions and geometries of the areas of interest (tissue sample size, surgical field), and should achieve an imaging resolution suitable for each application, from a few hundred microns to provide accurate guidance for tumor resection, to sub-micron when identification of single cells is required. For example, wide-field imaging allows the display of centimeter-scale field-of-view of relatively flat surfaces where tumors can be visualized within their surrounding healthy tissue for rapid margin delineation [112]. For uneven surfaces with flaps and creases, as well as for intraluminal imaging of small diameter vessels or airways, a point-scanning approach may be better suited. Optical fiber-based imaging provides the flexibility to address free-hand scanning on small and irregular surfaces for tumor detection [109,110] as well as helical scanning for cardiovascular imaging [113,114]. The image resolution may be adapted using microscope and microendoscope objectives or distal-end optics on fiber optic probes according to the level of detail necessary for each application [115]. Screening for disease or bulk tumor resection can be addressed through mesoscopic imaging, while identification of individual infiltrating cells on tumor margins benefits from microscopic resolution.

3.2.3 |. Rapid data visualization and feedback

Clinicians need access to the acquired qualitative or quantitative information in real-time in order to make informed decisions. Therefore, the instrument must perform fast data analysis. Several methods to rapidly compute fluorescence decay parameters from FLIm data have been developed [116,117] (see Section 4.2). A recently reported study details real-time display of FLIm maps intraoperatively during transoral robotic surgery [118].

3.2.4 |. Robustness to environmental factors

To be adopted, a new imaging instrument must not interfere with current standard of care protocols. This involves facile interfacing with sterile fields and operating with room lights or other illumination sources (eg, endoscopic lights or head-lamps) on. These considerations affect the size and form of the instrument (portable and compact are preferred), the materials used (ie, the optical fiber probes can be sterilized with processes easily accessible in a hospital setting), and the chosen optoelectronic detection scheme (ie, amplifiers and filters). The current clinical FLIm instrumentation described below, able to detect weak endogenous fluorescence, employs a pulse sampling approach that allows operation in presence of other illumination sources (see Section 3.3).

3.2.5 |. Safety

Above all, the instrumentation must be safe to use, for both operating room personnel and patients. The emission of the clinical FLIm instrument reported here is categorized under class 2 (considered safe for normal operation) under the requirements of the Code of Federal Regulations (21 CFR 1040.10) and the International Electrotechnical Commission (IEC 60825–1). Therefore, no specific eye protection is required from the system operator or any personnel in the instrument’s vicinity. Tissue exposure of the patient, determined by the energy of the light and exposure duration used (considering both the UV pulsed laser and the visible aiming beam laser), is below the limits stipulated by the ANSI guidelines for eye/skin exposure (ANSI Z136.1 [119]). In particular, compliance with limits of exposure for thermal damage is ensured by defining illumination parameters such that the maximum permissible exposure (MPE) is not exceeded for a stationary exposure of up to 5 seconds. Additionally, the system graphical interface allows us to highlight the measurement location where the exposure is reaching half of the MPE, to indicate to the operator that these locations should not be further exposed.

3.2.6 |. Ergonomics

Another factor that can facilitate clinical adoption is the ease of use of the proposed technology. Clinicians must feel comfortable operating the instrument to acquire and visualize data. For this reason, when possible, it is useful to integrate the imaging technology with conventional instruments used in clinical practice. For example, with point-based fiber-based imaging, the user can comfortably position the flexible fiber probe wherever desired within complex and tortuous anatomies (eg, the oropharynx) and still orient it normal to tissue to enable better signal collection during scanning. For endoscopic, or robotic surgeries, for example, the acquired measurements are then displayed on the same screen used to navigate the surgical field-of-view [118].

3.3 |. Example of clinical FLIm implementation

Below we describe a FLIm instrument fully compatible with the clinical environment that has been extensively evaluated in various operating room scenarios [109,110,120]. Specifically, we describe a fiber optic-based multispectral pulse sampling FLIm device. This instrument employs a 355 nm pulsed laser (0.6 ns FWHM pulse width, 0.1–4 kHz repetition rate; STV-02E-1×0, Teem Photonics, France) delivered through a single fiber imaging probe (eg, 365 μm core diameter multi-mode fiber [MMF] for surgical oncology applications, and 100–200 μm core UV grade MMF for cardiovascular imaging). Tissue autofluorescence is collected by the same fiber and guided to a wavelength selection module (WSM) that splits the signal into four spectral bands (SB). A commonly used configuration is SB1: 390 ± 20 nm, SB2: 470 ± 14 nm, SB3: 542 ± 25 nm and SB4: 629 ± 26.5 nm, which broadly cover the emission spectra of structural proteins, metabolic cellular cofactors and porphyrins. The spectral bandwidth can be adapted according to the application changing the corresponding filters and dichroic mirrors on the WSM. The detection scheme consists of a single microchannel plate photomultiplier tube (MCP-PMT; R3809U-50, Hamamatsu, Japan), an amplifier (AM-1607–3000, Miteq Inc., Hauppauge, NY), and a fast digitizer (12.5 GS/seconds, 3GHz, 8-bit; PXIe-5185, National Instruments, Austin, Texas) [103,105].

In the pulse sampling approach, short (sub-ns) and intense (~0.1–10 μJ) excitation pulses generate a large amount of fluorescence photons that are detected by a high-bandwidth photodetector [121,122]. A fast digitizer measures the resulting electrical transient signal with a resolution of tens of picoseconds, and full fluorescence intensity decays are recorded within a few microseconds. With this implementation, room illumination has a negligible effect on the fluorescence signal (a large number of fluorescence photons generated within a nanosecond) that can be even further minimized by low-frequency filtering [105,107,123].

Other groups have adapted the basic concept of this instrument [117,124,125] to make it compatible with galvanometer scanners for in vitro and in vivo tissue diagnosis with hand-held endoscopes [126] or through scanning microscopy [124].

For intraoperative applications, where the ultimate goal is to provide visual biochemical feedback to the surgeons, FLIm imaging is achieved by free-hand point-scanning measurements. An aiming beam (445 nm continuous-wave diode laser; TECBL-50G-440-USB, World Star Tech, Canada) was integrated into the WSM of the FLIm system to highlight the location where FLIm point measurements have been acquired to facilitate image reconstruction (see Section 4.4) [107]. This aiming beam is delivered through the same optical path used to excite tissue autofluorescence. The amplifier of the instrument is AC coupled with a cut off frequency of 10 kHz to filter out any signal contribution from the aiming beam and other sources of light, such as lights in the operating room.

The core of a recently reported FLIm instrument is compatible with multiple imaging optical fiber probes that adapt to different scanning modalities and that can couple with existing clinical tools. Examples include catheterized helical scanning mechanisms for intravascular imaging [108,113] and highly flexible free-hand scanning during intraoperative interventions. The latter include brain tumor resection surgery, where the FLIm instrument interfaces with a neurosurgical microscope (eg, OPMI Pentero 900 (Carl Zeiss Meditec, Jena, Germany) [109]; and oral and oropharyngeal surgery for tumor removal with surgical robotic platforms (eg, da Vinci Surgical System [Intuitive Surgical, Sunnyvale, California] introducer sheaths [Si model] and graspers [SP model]) [110,123,127], which display the acquired FLIm data in an image format onto the surgeon’s field-of-view. Finally, FLIm is also compatible with multimodal imaging and it has been combined with ultrasound [108,128], OCT [84,117,129] and Raman imaging [130–133]. The majority of these integrated approaches were implemented in bench-top systems and with ex vivo samples, establishing proof of principle.

3.4 |. FLIm-compatible optical fiber probes

Optical fiber probes are readily sterilizable; their length allows the FLIm instrumentation to be located outside of the sterile surgical field. Fiber probes used in earlier studies were based on bifurcated configuration, where one fiber was used to deliver the excitation beam and another fiber or a set of fibers were used to collect the fluorescence emission [134] (Figure 2A). This configuration has been typically used in steady-state (intensity or spectral-based) tissue autofluorescence studies because they prevent detection of fiber autofluorescence signal generated along the excitation path [134]. When performing time-resolved measurements, the fiber probe autofluorescence, however, can be easily separated from the useful fluorescence signal due to its earlier arrival at the detector. This allows for much simpler configurations of fiber optic probes where a single fiber is used for excitation delivery and signal collection (Figure 2B). In this approach, the excited and collection areas are inherently overlapping without the need for distal-end optics, with benefits in terms of complexity, robustness and flexibility. Single-photon FLIm is typically performed using multi-mode fibers (MMF), which size results from a trade-off between fiber flexibility, improved using thinner fibers, and signal collection, improved by larger cross-sections of the fiber’s core. Such approach is currently in use for applications in surgical oncology, with MMF of 365 μm core diameter (0.22 NA, pure silica core, high-OH and fluorine-doped silica cladding; FG365UEC, Thorlabs Inc., New Jersey) that enables short term bending radii of less than 10 mm, and is well suited to both free-hand and robotic applications [135].

FIGURE 2.

Fiber probe configurations compatible with clinical applications of FLIm. A, Bifurcated probes, where excitation and collection light is guided in independent optical fibers bundled together within a common jacket. B, Enface multi-mode fiber (MMF), where excitation and collection light is guided through the same optical fiber. C, Side-viewing MMF directs the excitation light at an arbitrary angle (typically 90°) through distal-end polishing or the addition of distal-end optics (eg, prism). This configuration allows for intraluminal imaging when combined with a rotation mechanism. D, Distal-end optics include GRIN lenses (terminated with a prism for side-viewing modalities), ball lenses and freeform reflective optics. E, Multimodal probes include pairing with ultrasound transducers for FLIm-IVUS, double-clad fibers for FLIm-OCT and fiber bundles for FLIm-Raman

The single-fiber approach also opens the possibility of more advanced probes and scanning schemes. The combination with an optical rotary junction and side-viewing optics enables the acquisition of FLIm data from luminal surfaces [136] (Figure 2C). Intravascular applications such as coronary artery imaging require flexible probes and thus rely on smaller MMF (100 μm core). Here, adequate signal collection requires the use of distal-end optics (Figure 2D), but optics developed for other intravascular optical imaging techniques are poorly suited for use in the UV, or in combination with large-core MMFs. Only low-NA, lithium-doped GRIN lenses are suitable for use below 370 nm, requiring the use of a long GRIN element (>5 mm), detrimental for accessing tortuous anatomies. On the other hand, polished ball lens designs are not optimal in combination with large core fibers, as they typically do not allow for proper beam expansion before focusing [114]. These issues were addressed by the development of a novel freeform side-viewing optic where beam reflection and focusing were performed by a curved reflective surface [137]. This design also addressed well-known astigmatism and chromatic dispersion issues reported with refractive elements [138].

Morphological imaging techniques are a great complement to spectroscopic imaging, and thus FLIm has been integrated with other imaging modalities such as ultrasound or OCT. Integration with ultrasound can be performed as a forward-viewing system [128] and for intravascular imaging, also known as intravascular ultrasound (IVUS; Figure 2E). The last generation of a FLIm-IVUS device combines an optical channel for FLIm and a 40 MHz ultrasound transducer into an 800 μm diameter rotating imaging core (overall cross-section including imaging sheath: 3.7 Fr), suitable for the interrogation of coronary arteries in vivo [108,113].

Another example of integration consists in substituting IVUS for OCT to create a fully optical multimodal system using a single double-clad fiber [129] (Figure 2E). In this configuration, OCT light is transmitted through the single-mode core of the fiber, whereas the inner cladding is used to transmit FLIm excitation and fluorescence signals. With this approach, it is now possible to integrate FLIm capability in a device similar to current clinically available intravascular OCT systems [114].

The biochemical specificity of Raman spectroscopy has demonstrated clinical value for cancer detection [34] as well as for detection of the main components in atherosclerotic plaque [139]. Bimodal FLIm and Raman imaging may yield a label-free detection system with overall increased sensitivity. A combined probe that facilitates the simultaneous acquisition of both modalities has been previously tested on atherosclerotic samples [130–132]. A more recent configuration consists of a fiber bundle with a central MMF for FLIm excitation and collection, and eight peripheral fibers, one for Raman excitation (785 nm) and seven for Raman collection (Figure 2E, see [132,133] for more details).

4 |. FLIM DATA ANALYSIS AND VISUALIZATION

Time-domain FLIm instrumentation records fluorescence emission pulses at distinct wavelengths or wavelength bands. Absolute fluorescence intensity and intensity decay parameters can be extracted from the measured fluorescence pulses in multiple ways. Since the physical representation of the fluorescence emission decays follows a mono-exponential (single fluorophore) or multi-exponential (multiple fluorophores as found in tissue) function, fitting the intensity decay to a multi-exponential curve is an intuitive solution. However, this method is computationally expensive, slow and not well suited for typical biological samples where the number of fluorescent species is large and undetermined. Fast computational methods that do not require a prior assumption on the number of fluorescent species are preferred to extract FLIm parameters. A large number of algorithms have been proposed for FLIm data analysis and are summarized elsewhere [68]. Here, we focus on methods used to process clinical FLIm data.

4.1 |. Pre-processing requirements

A schematic of the data pre-processing steps for the multispectral FLIm system is illustrated in Figure 3. Fluorescence from four spectral bands is temporally multiplexed into a single MCP-PMT detector. The raw waveforms are therefore a concatenation of the fluorescence signal from each spectral band (Figure 3A). Averaging the waveforms over multiple pulses improves the signal-to-noise ratio (SNR). The fiber probe introduces a non-negligible background from fluorescence generated at the proximal end, inside and the distal end of the fiber (most noticeable in the first spectral band, Figure 3B). Only the distal-end background temporally overlaps with the sample fluorescence signal, reducing the dynamic range of the system. The fiber background is subtracted from the raw waveform (Figure 3C). After background subtraction, the sample fluorescence is truncated to isolate the decay waveform corresponding to each spectral band (Figure 3D).

FIGURE 3.

Data processing pipeline for time domain FLIm using pulse sampling technique. A, Raw waveform, signal. B, Fiber background. C, Background subtracted from the signal. Each box outlines the signal in one spectral band (j). D, Truncated signal y_j(k). E, Instrument impulse response function (iIRF) h_j(k). F, Deconvolved fluorescence impulse response function (fIRF) I_j(k) from which to extract intensity and lifetime parameters

4.2 |. Extraction of FLIm parameters

4.2.1 |. Deconvolution with Laguerre expansion

The sample fluorescence is obtained through a non-parametric model based on a Laguerre expansion of the fluorescence impulse response function (fIRF) followed by a constrained least-square deconvolution with the instrument impulse response function (iIRF), a method previously described in detail [105,116]. This method has demonstrated extensive use for the analysis of biological systems [140] and for disease detection [141].

Briefly, for each spectral band, the measured fluorescence signal y(k) is a convolution of the iIRF h(k) and the fIRF I(k), or sample fluorescence, with additive noise ε_k:

$$y(k)=h(k)*I(k)+{\epsilon }_k$$

where k indicates the discrete index of sampling.

The iIRF h(k) is measured at each spectral band with a known fluorescent standard (eg, 2-DASPI, 2-[4-(dimethylamino)styryl]-1-methylpyridinium iodide; τ~ 10–60 picoseconds [142]; Figure 3E).

The sample fluorescence, I(k), is expanded on a linear combination of an ordered set of Laguerre basis functions b_l(k; α) (LBFs),

$$I(k)=\sum _0^{L-1}{c}_l{b}_l(k;\alpha )$$

where c_l is the coefficient corresponding to each LBF. This expansion does not make assumptions on the number of fluorophores present in the sample. Instead, the LBFs set is specified by two parameters: scale α and dimension L. The selection of these two parameters will determine the lifetime range that can be accurately retrieved. For clinical applications, L = 12 and α = .916 are well suited to a lifetime range from 2 to 8 nanoseconds (sampling rate 12.5 GS/seconds). Increasing L will increase the lifetime range that can be accurately estimated at the cost of increased processing complexity [116, 143].

The measured signal can then be represented by a linear combination of this set of LBFs convoluted with iIRF:

$$y(k)=\sum _0^{L-1}{c}_l\left[h(k)*b_l(k;\alpha )\right]+{\epsilon }_k$$

The Laguerre-expanded I(k) is deconvolved from the measured signal y(k) using Least Square Deconvolution (LSD; Figure 3F) and the Laguerre coefficients c_l for each LBF are determined. Noise in the signal that results in negative intensity values or non-monotonous behavior (oscillations) can lead to large errors in the estimation of the average lifetime. To solve this, a Constrained Least Squares Deconvolution (CLSD) approach was developed by adding constraint criteria that force the candidate I(k) to be convex, positive and monotonically decreasing [116].

Finally, the fluorescence intensity detected in each spectral band j is determined by the area under the curve of each resulting decay and is used to compute intensity ratios that convey the spectral information of the sample as:

$$I{R}_j=\frac{\sum _k{I}_j(k)}{\sum _j\sum _k{I}_j(k)}$$

and the average fluorescence lifetime as:

$${\tau }_j=\frac{\sum _k{I}_j(k)t_k}{\sum _k{I}_j(k)}$$

The processing time for all four spectral bands (each one has 680 data points) is less than 1 millisecond.

This method also generates a series of additional parameters (ie, Laguerre coefficients) that can be useful for further data analysis and interpretation (ie, classification methods, see Section 4.3).

4.2.2 |. The analog mean delay (AMD) method

The AMD method [144] was proposed to estimate average fluorescence lifetimes at high speed. Briefly, the arrival time of a fluorescence photon is linearly determined by the delays corresponding to physical processes like excitation pulse duration, vibrational relaxation, internal transition, fluorescence emission and detector response. The vibrational relaxation and the internal transition are much faster (picoseconds) than the rest of the processes (nanoseconds), and thus can be ignored. The average fluorescence lifetime, defined as the expected value of the delay corresponding to the fluorescence emission (fIRF), is calculated by subtracting the expected value of the other delays (iIRF) from that of the measured fluorescence photon arrival time (measured signal). The corresponding expected values are calculated from the probability distribution functions associated with each signal (ie, measured fluorescence, iIRF). The algorithm is based on simple algebra, so a high processing speed can be achieved (~10⁵ measurements per second), making it an attractive alternative for real-time data processing in clinical applications.

4.2.3 |. The phasor approach to fluorescence lifetime imaging

In FLIm applications, the phasor representation of the fluorescence lifetime was initially applied to frequency-domain FLIM [145,146] and has been more recently adopted for processing time-domain data [147,148]. To obtain the phasor plot, the temporal decay is Fourier transformed and represented in a polar plot as two-dimensional histograms. This method makes no assumptions on the number of fluorescence species present in the sample, permits fast processing speed and allows to quantify the concentration and interactions of the involved fluorophores [149,150] without necessarily associating a lifetime value to each phasor. If the phasor is located on the universal circle of the polar plot, it consists of a radiative decay with a unique lifetime. If the measurement contains contributions from several fluorophores (complex decay), it will appear inside of the universal circle and its location may provide information about the pure constituents [145]. The phasor space is governed by the law of linear addition, which facilitates data analysis and interpretation [151]. Furthermore, a common implementation of this method permits to map each phasor point back to a specific pixel on the image. Such a convenient approach enables rapid spatial exploratory analysis and estimation of ratios and concentrations of the imaged species [150]. A comparative study between the performance of the Laguerre deconvolution and the phasor methods for online characterization of tissue properties concluded that the phasor’s method provides more accurate lifetime estimations, whereas the Laguerre approach has a higher precision [152].

4.2.4 |. Convolutional neural networks

Machine learning and deep learning techniques have been recently proposed to perform accurate deconvolution of acquired fluorescence signals, with efficient processing and enhanced robustness to noise [153,154]. Parameter extraction through CNNs largely improves the processing speed for compressive FLIm. However, the parallel computation using GPU makes this method more suitable for wide-field imaging instead of point scanning that is currently widely used in clinical applications of FLIm.

4.3 |. Analysis of FLIm parameters

Data modeling and machine learning techniques have been employed to analyze and interpret FLIm parameters (including fluorescence lifetime and intensity ratios for each spectral band) beyond conventional univariate statistics. When using the Laguerre expansion based deconvolution approach [116], a set of Laguerre expansion coefficients can also be incorporated into the analysis for additional discrimination value [127,155]. Several multivariate analysis methods using these parameters have been explored, adapting to the goal of each study and the complexity of the captured data.

Multiple linear regression [156] was performed to identify correlations among FLIm parameters and between FLIm parameters and measured signals from other imaging modalities. For example, a multi-regression approach was applied to isolate the sources of FLIm-based contrast when categorizing atherosclerotic lesions through combination with Raman spectroscopy [132].

Linear discriminant analysis (LDA) [157] can be used to identify the optimal linear combination of FLIm parameters to distinguish labeled tissue conditions in a multivariate approach and has been used to quantify intrapatient FLIm contrast for oral and oropharyngeal cancer [110].

Nonlinear classification models such as random forests (RF) [158], k-nearest neighbors (KNN) [159] and support vector machines (SVM) [160] have been used to recognize acute tissue conditions that vary with experimental context. SVM and RF have been investigated for interpatient discrimination of breast cancer specimens imaged ex vivo [120,155] and oral and oropharyngeal cancer [127] (both in vivo and ex vivo). SVM has also been used to identify non-melanoma skin lesions using FLIM microscopy [161]. A KNN classifier was used to detect actinic cheilitis in the lips and distinguish between normal tissue, mild dysplasia and moderate dysplasia [162]. While not all methods have been evaluated for every application discussed, the ensemble classification approach of random forests has been shown for multiple anatomical locations [127,155] to result in improved generalization between patients when identifying cancer using fiber-based FLIm, suggesting this method produces robust classifiers for FLIm data.

Phasor plots can also be used in a FLIm analysis context to provide a 2-D graphical representation of fluorescence lifetime in which distinct distributions within the captured FLIm data can be clearly observed and projected back to the imaged region [79,145,147,150]. This graphical approach allows for interactive data exploration to be carried out providing a greater understanding of the underlying FLIm spatial distribution, particularly when exhaustive tissue condition labels are not available.

4.4 |. Real-time visualization of FLIm parameters

Feedback to clinicians is best provided by real-time mapping of optical parameters, or the result of classification and predictive algorithms, to the exact position within the tissue area from which they were acquired, typically displayed in a white light image or video stream. Real-time feedback through optical imaging provides a clear advantage over more conventional and slower analysis methods such as frozen section histology. We demonstrated that real-time pre-processing and parameter extraction for optical data is possible. A remaining challenge consists in the real-time registration of FLIm point-measurements with the tissue surface and the visualization of optical parameters when only sparse and non-uniform sampling is performed. With the point-scanning approach used in free-hand fiber-based FLIm, registration with a white light video stream must also account for camera and tissue motion for an accurate augmentation of the FLIm data on the surgical field-of-view. Examples of the three key processing steps needed for real-time visualization of FLIm data are illustrated in Figure 4.

FIGURE 4.

Overview of the key processing steps required for real-time visualization of FLIm data acquired in a clinical setting

4.4.1 |. Aiming beam segmentation

The aiming beam allows for the localization of FLIm point-measurements during acquisition [107]. This location is then used to overlay FLIm parameters or analysis output onto the white light image acquired by an external or device-integrated camera. Initially, the position of the aiming beam was identified through color space thresholding [107]. More recently, we developed a more robust CNN-based segmentation method [118] (Figure 4A).

4.4.2 |. Improved registration

Camera and tissue motion correction must be performed for intraoperative fiber-based FLIm scans in which camera and tissue motion can occur. This is currently performed through a combination of block-matching based motion estimation and motion vector summation to ensure correct point-measurement registration is maintained throughout a FLIm scan [118] (Figure 4B).

4.4.3 |. Augmentation

Real-time visualization of point-scanning FLIm is achieved by combining corrected locations and FLIm data (eg, average lifetime, classifier output) for a set of point-measurements to generate a transparent overlay used to augments a white light image of the tissue region. Initially, the segmented aiming beam for each measurement was fitted with an ellipse. Maps combining FLIm data from multiple point measurements were then obtained by overlapping and averaging these ellipses and applying a colormap (eg, jet) to generate the final image overlay [107]. Sparse sampling of the tissue surface can occur in certain cases with a point-measurement approach that leads to the generation of noisy or incomplete visualizations using the overlapping ellipse method. To overcome this issue, an inverse distance weighted interpolation method was developed for FLIm visualization [118,155](Figure 4C).

5 |. APPLICATIONS

5.1 |. Surgical oncology

Tumor resection is a common treatment for cancer patients. The extent of resection is closely associated with survival rates, with more complete resections leading to better outcome [163,164]. To optimize the extent of tumor resection it is necessary to identify malignant from viable and functional tissue during surgery. Pre-operative imaging (eg, MRI, CT…) is used for surgical planning. However, issues with the initial registration and further tissue movement during the intervention do not allow for accurate delineation of the tissue to be removed during surgery solely based on pre-operative imaging. There is, therefore, a need for real-time and non-invasive technologies that improve diagnostic quantification and can be integrated with the current surgical workflow. FLIm has the potential to provide visual feedback based on biochemical contrast for real-time in situ decision making.

Below, we summarize the applications of fluorescence lifetime spectroscopy and imaging in surgical oncology. In particular, for guiding surgical resection of oral and oropharyngeal cancers during robotic and non-robotic surgery, detection of brain tumors at the resection margins during craniotomy procedures, and identification of positive margins in breast tumor specimens following breast-conserving surgery. These examples, illustrated in Figure 5, underscore the mesoscopic FLIm utility as intraoperative diagnostic tool.

FIGURE 5.

Illustration of, A, various FLIm integration schemes, B, data visualization strategies and, C, validation against histopathology evaluation for applications in oral and oropharyngeal cancer (adapted from Reference [110, 127]), brain cancer (adapted from Reference [109]) and breast cancer (adapted from Reference [155])

5.1.1 |. Intraoperative assessment of oral cavity and oropharyngeal cancers

Oral cavity and oropharyngeal cancers occur with similar incidence [165] and currently represent 2.9% of all new cancer cases in the United States [166]. As with other cancers, adequate tumor resection is the key factor for long-term survival [163]. The surgical resection of oral cavity cancer is typically performed by hand, while oropharyngeal cancers are generally resected via transoral robotic surgical (TORS) platforms. The advantages of TORS over endoscopic procedures include deeper access to anatomical sites, which enables precise operation in tight spaces without large open incisions, improved patient functional outcomes, and enhanced dissection ability of lesions and neoplastic growths [167]. However, TORS eliminates the surgeon’s ability to sense tissue and bone resistances [167–169], resulting in a loss of haptic feedback and making the procedures more challenging [168,169].

Endogenous fluorophores (eg, collagen, NADH, FAD and porphyrins) are abundant in the oral and oropharyngeal epithelium. FLIm is, therefore, well suited for intraoperative delineation of tumors based on rapid evaluation of the extent of molecular changes (neoplastic area). Time-resolved autofluorescence research in head and neck anatomy has evolved in the last two decades from early feasibility studies (both in animal models [90] and human cancer subjects [170]) to the recent integration of advanced instrumentation with TORS surgical robotic platforms used in the operating room [110,135]. The main contributions from these studies are listed below with key milestones using fluorescence lifetime in oral and oropharyngeal cancer summarized in Figure 6.

FIGURE 6.

Key milestones and studies of label-free time-resolved fluorescence spectroscopy and imaging for tissue assessment in oral, brain and breast cancer. FLIM, fluorescence lifetime imaging microscopy; FLIm, fluorescence lifetime imaging; GBM, glioblastoma multiforme; HGG, high-grade glioma; IR, intensity ratio; LT, lifetime; LGG, low-grade glioma; PAI, photoacoustic imaging; TRFS, time-resolved fluorescence spectroscopy; TCSPC, time-correlated single-photon counting; UBM, ultrasound backscatter microscopy

Initial findings demonstrating the feasibility of time-resolved contrast to distinguish between healthy tissue and cancer in patients date back to 2004. The first time-resolved autofluorescence spectroscopic studies (using TCSPC) on human patients (n = 33–55) leveraged the temporal profile of Protoporphyrin IX (PpIX) fluorescence as an indication of carcinogenesis in vivo [170,171]. In 2009, a multispectral endoscopic FLIM instrument with a gated intensified CCD camera was used to differentiate normal tissue from dysplasia, carcinoma in situ, and squamous cell carcinoma in vivo on hamster buccal pouch models using collagen (390/70 nm) and NADH (450/65 nm) fluorescence [90]. Other reported studies served to further establish the potential of time-resolved fluorescence spectroscopy and imaging for intraoperative surgical guidance of oral cavity and oropharynx cancers [92,172–174].

Multimodal approaches for oral cancer detection

Multimodal optical imaging has been tested in animal models of oral cancer. FLIm was first combined with OCT in 2010 to characterize oral cavity cancer [84]. Collective findings using a FLIm-OCT instrument on hamster cancer models in vivo demonstrated that multimodal evaluation of cancer and healthy tissue improved discrimination capacity by improving both sensitivity and specificity [84,85,124,175]. A subsequent study in 2013 coupled FLIm with photoacoustic imaging and ultrasound backscatter microscopy to distinguish normal tissue from precancerous and carcinoma tissue, also in hamster buccal pouch models [128]. Optimal performance for differentiating between healthy and cancer tissue, with the highest sensitivity and specificity, was reported when FLIm was used in combination with additional imaging modalities that provide structural and morphological information besides the biochemical contrast provided by FLIm.

Integration into transoral robotic surgical platforms (TORS)

Recently, a multispectral FLIm system was integrated with the da Vinci Si system as a means for detecting surgical margins during TORS platform [135]. The initial validation of this approach was reported for both swine (n = 3) and human patients (n = 4) [135]. The effect of electrocautery on multispectral FLIm data was also studied on live Yorkshire pigs during TORS procedures [123]. The integration of FLIm with TORS procedures has provided the opportunity to study the FLIm signatures of various oropharyngeal cancers that are deeper in the neck anatomy and harder to access (eg, palatine tonsil, lingual tonsil, base of tongue). For example, recent results demonstrate that FLIm could consistently resolve cancer from healthy palatine tonsil tissue [110].

Automated data analysis in oral and oropharyngeal cancer research

The first use of automated quantitative image analysis through data classification methods was performed on a 2016 3D FLIM and OCT study for the diagnosis of oral cavity cancer in animal models and suggested that the output of the classification algorithm could be mapped on the OCT volume for diagnostic interpretation [85]. More recently, in 2020, advanced machine learning techniques were used on a human patient cohort (n = 53) in vivo and ex vivo for tongue (oral cavity) and tonsil (oropharynx) tumors [127]. Incidentally, the results demonstrated that in vivo autofluorescence lifetime and intensity parameters provided superior contrast to ex vivo scans between healthy and cancer tissue, likely due to the impact of surgical tissue resection on metabolic contrast, where metabolic cofactors NADH and FAD were reported as the key sources of contrast [127].

Co-registration with histopathology

Accurate registration of imaging data with histopathology is of paramount importance for clinical validation of any new modality. Optical data from initial studies based on static point-measurements (where a probe was held in position on tissue) or using a hand-held endoscope capable of scanning a small field-of-view (mm-scale) were directly coregistered to biopsy specimens collected from the analyzed area [85,126,170,172,173,176,177]. In the later studies using the current FLIm device, imaging was performed over a larger tissue area (multiple centimeters) that may contain different anatomies and pathological conditions. A more accurate and detailed co-registration strategy was implemented, where the imaged tissue was fully resected, sectioned multiple times (Figure 5C), and analyzed by pathologists to associate imaged regions to biochemical status [110,127,174].

It is important to note that comparison methods between healthy and cancer tissue varies among different research groups. Some groups report healthy vs. cancer comparisons based on contralateral sites (eg, cancer of the left tonsil compared to healthy right tonsil tissue) [172,177],, whereas others report comparisons on the same anatomical side of the body (eg, compare a region of cancer of the left tonsil to the healthy region of the left tonsil) [110,127]. This may be an important consideration in the comparison and interpretation of FLIm results across studies, since factors such as field cancerization may affect a primary tumor site as well as the healthy-looking peripheral tissue [178].

5.1.2 |. FLIm-assisted brain tumor surgery

Complete and accurate resection of brain tumors is also responsible for longer survival rates [164,179–181]. Therefore, the main goal of brain tumor resection surgery is to maximize the removal of tumor-involved tissue while preserving normal and functional brain tissue. Achieving this goal is currently limited by the inability to unambiguously distinguish tumor from normal brain tissue in real time during surgery, a task particularly difficult for glioblastoma multiforme (GBM), the most malignant type of brain tumor, characterized by poorly defined edges that infiltrate into the brain parenchyma.

Neurosurgeons rely on pre-operative imaging (eg, MRI, CT) for surgical planning, stereotactic neuronavigation systems to navigate the brain during surgery, and ancillary tools that provide snapshots of the brain (eg, intraoperative MRI and ultrasound) or that enhance tumor contrast via fluorescent molecular probes. The metabolic biomarker 5-aminolevulinic acid (5-ALA) [182–185] has been used in Europe for over a decade, and was recently approved for use in the United States [185] for intraoperative visualization of GBM. Epidermal growth factor receptor (EGFR) targeted molecular probe cetuximab-IRDye800 has also been successfully used in humans to detect GBM [186]. In spite of these encouraging advances, their best performance is limited to identifying the confined core of GBM, underperforming on the tumor infiltrative edge, necrotic tissue and low-grade gliomas (LGGs) [182,187–189]. Tactile and white-light visual feedback remain the main source of information during surgery [190,191].

The fluorescence lifetime technique has shown promise to discriminate between normal brain tissue and different lesions including low- and high-grade gliomas and areas affected by radiation necrosis. If successfully validated, FLIm could be used as an intraoperative adjunct to enhance tumor visualization and provide a robust means for optical in situ biopsy to help guide tumor resection, ultimately improving the clinical outcome of patients.

The use of fluorescence lifetime on the brain has evolved significantly from the first examinations of human brain tissue on ex vivo freshly excised samples [102,192,193] to in vivo spectroscopy and later imaging during neurosurgery [91,109,194,195]. The main findings from these studies are highlighted below and key milestones on the use of FLIm on brain tissue are summarized in Figure 6.

Each type of brain tissue presents unique time-resolved fluorescence features

Gray matter, white matter, different tumors types (low- and high-grade gliomas) and necrotic tissue exhibit different contrast attributed to differences in their composition and metabolic activity [91,102,109,192–196]. Alternations in the metabolic activity affect the optical properties of metabolic cofactors NAD(P)H and FAD, the main sources of tissue autofluorescence in the brain, that the mesoscopic clinical FLIm instrument detects at the spectral bands centered at 460 nm and 540 nm, respectively. Further understanding of the involved fluorophores and increased specificity in detecting their emission will greatly enhance the tissue discrimination ability of FLIm. Given that the main contributors to tissue fluorescence are metabolic cofactors, in vivo and ex vivo measurements differ significantly. The usefulness of FLIm for intraoperative tumor detection will be determined by its performance in vivo. Yet, the ability of FLIm to discriminate between tissue types ex vivo could have clinical value in rapid intraoperative histology of biopsied tissue, as recently demonstrated with stimulated Raman imaging [35].

Observed differences between brain tumor types

LGG has been resolved with higher accuracy than high-grade glioma (HGG) [195], likely due to the increased biochemical heterogeneity present in HGG [197]. Necrotic tissue due to radiation treatment or advance stages of GBM exhibited significantly longer fluorescence lifetime than viable tissue across the spectrum. Consistent trends were found in live animal studies of radiation necrosis [196] and in necrotic cores present in human patients [109].

Combination of spectral and lifetime information for optimal discrimination

Similar to results obtained in the oral and oropharyngeal area, both spectral- and time-dependent fluorescence emission provide discriminating contrast and combining the output of both provides the best results [195]. The optimal instrument, therefore, must combine both spectral and time-resolved fluorescence information.

Integration with neurosurgical instrumentation

The current FLIm device is integrated with the OPMI Pentero 900 surgical microscope from Zeiss to obtain the white-light video feed required to provide surgeons with visual feedback (FLIm data overlay) [109].

Co-registration with histopathology

Validating FLIm data from brain tissue requires collecting biopsies for histopathological evaluation, which is not always an option with brain tissue specially to obtain viable control samples. Fluorescence lifetime measurements are typically obtained from areas that will be biopsied or resected according to the standard of care, and histopathological analysis of the excised tissue is used for a direct evaluation of the imaged area. Imaged areas excluded from the biopsy-collection plan (eg, healthy cortex areas) are assessed by the surgeons during the procedure.

5.1.3 |. Identification of positive margins in breast cancer

Breast-conserving surgery (lumpectomy) is the preferred strategy for breast tumor removal, especially at early stages [198,199]. Typically, surgeons will identify the tumor area with pre-operative imaging (eg, X-ray mammography, ultrasound, or MRI) and define a resection margin that aims to contain the entire tumor. A highly conservative approach that removes large amounts of viable tissue leads to undesirable cosmetic results. Alternatively, failing to remove all the tumor will leave behind positive margins with cancerous cells that are susceptible to recur [200]. Therefore, finding precise margins is yet again a well identified clinical need. Because pre-operative imaging cannot provide the necessary real-time feedback during surgery, novel alternatives are required. Several intraoperative optical imaging methods are under investigation to provide real-time guidance and positive margin assessment for breast-conserving surgery [201], including diffuse reflectance spectroscopy [202], Raman spectroscopy [203] and time-resolved fluorescence imaging. Key milestones on the use of fluorescence lifetime for evaluating breast cancer features and the evolution of the field are briefly summarized in Figure 6.

Fluorescence lifetime provides contrast for breast tumors

The first study that used FLIm on human breast cancer excised tissue demonstrated that benign and malignant tissue have different autofluorescence characteristics [81]. This early study compared the endogenous signals detected with a wide-field time-gated imaging approach (excitation at 415 nm) to the H&E histopathological results on fixed thin (10 μm) tissue sections. Later studies using multi-photon FLIM further demonstrated the ability of the technique to discriminate between tissue types relevant to breast cancer on animal models [204,205]. The first intraoperative approaches to assess breast tissue types utilized optical fibers to interface with fresh lumpectomy specimens and consisted of point-spectroscopic time-resolved measurements through TCSPC (ex. 447 nm) [206] or a gated multichannel plate photomultiplier tube and a digital oscilloscope (ex. 337 nm) [207]. Both of these studies were capable of differentiating between fibrous tissue, adipose tissue and invasive ductal carcinoma. An imaging approach was later introduced with FLIm (ex. 355 nm) [120]. The use of the aiming beam to precisely localize the measurement locations enabled image formation from free-hand scanning and an SVM classifier was used to discriminate normal fibrous and adipose tissues from cancer tissue with high accuracy (97%).

Co-registration with histopathology

Accurate labeling of imaging data based on histopathology, a key factor to develop accurate classification protocols, was addressed by a co-registration method consisting of a hybrid model that combines fiducial markers (burn marks on the tissue) and specimen shape information [208].

Real-time classification and display

An augmented classification approach of the FLIm data onto the specimen was recently developed [155]. This approach demonstrated real-time tissue diagnosis of excised tissue, provided an intuitive visualization of tissue types, and it reported an iterative imaging refinement method for the surgeon to switch between rapid scan with low resolution and slow scan with higher resolution to adapt to the area of interest. This approach could be used for rapid intraoperative histology to assess positive margins on excised tissue inside of the OR.

5.2 |. Cardiovascular disease

Identification and characterization of atherosclerotic plaque using laser-induced autofluorescence have been reported extensively. Recent work has demonstrated the practical implementation of FLIm for imaging of coronary arteries in vivo.

5.2.1 |. Studies in ex vivo human specimens

Initial spectroscopic measurements [50, 209], followed by time-resolved fluorescence spectroscopy and more recent FLIm technique for analysis of atherosclerosis lesions [210–214] have demonstrated how tissue autofluorescence properties enable the characterization of atherosclerotic lesions. Figure 7 depicts results from a recent study [113] conducted in coronary segments from 32 human hearts (obtained from donors/transplants). This study has provided additional insights into key biochemical and morphological features of human plaques correlated with FLIm parameters. Using serial histological sections (n = 204) of the arterial segments and a systematic tabulation (12 sectors) in each section, we were able to associate FLIm parameters from three spectral channels with the histology at each measurement location (n = 33 980). The key results from this study are summarized below.

FIGURE 7.

Intravascular FLIm enables label-free identification of biological species associated with plaque progression (representative results); adapted from Reference [113] Left panel: lesions consistent with new plaque formation are associated with an increase in 390 nm lifetime and intensity (AIT, adaptive intimal thickening; PIT, pathological intimal thickening). Central panel: the amount of superficial mFC assessed using a semi-quantitative scale using CD68 immunohistostaining (0, absent; 1, <10%; 2, 11%−25%; 3, 25%−50%; and 4, >50% of superficial 200 μm) is associated with a corresponding increase in 540 nm band lifetime. Right panel: this finding allowed for the creation of a predictor (piecewise linear interpolation of 540 nm lifetime) that can map the location and degree of mFC infiltration over the vessel lumen surface with high accuracy

New matrix formation

Plaques with new matrix formation (eg, pathological intimal thickening and healed thrombus regions) present increased lifetime values in the spectral band centered at 390 nm. This finding points to FLIm sensitivity to a structural protein associated with the early stages of cap/-plaque remodeling, such as a type of proteoglycan [215]. Plaques rich in proteoglycans and smooth muscle cells were associated with increased risk of plaque erosion [216, 217].

Calcifications

Plaques with superficial calcifications present a lifetime decrease in the spectral band centered at 450 nm. Superficial calcification and micro-calcifications can give rise to foci at high stress and were associated with increased risk cap destabilization [215, 218].

Macrophage foam cells

Plaques characterized by the superficial presence of macrophage foam cells (mFC) and by extracellular lipid content present lifetime increase in the spectral band centered at 540 nm. We found that lifetime values in this spectral band can be used to predict the presence of mFC quantitatively. Interestingly, this wavelength range has been linked with the fluorescence emission properties of ceroids [219, 220], which are byproducts resulting from the uptake of oxidized low-density lipoprotein (LDL) by macrophages [220–222]. Due to oxidized LDL’s central role in transforming macrophages into mFCs, a key initiating factor in the progression of atherosclerosis [223], FLIm can provide valuable insight into the inflammatory activity of atherosclerotic lesions, a key hallmark of plaques with increased risk of rupture. Also, we found that locations with iron-rich macrophages showed a similar red-shifted emission but a shorter lifetime, suggesting that these macrophages could potentially be differentiated from lipid-rich mFCs.

In brief, our findings indicate that autofluorescence lifetime provides valuable information for characterizing atherosclerotic lesions in coronary arteries. Specifically, FLIm can be used to identify key phenomena linked with plaque progression (eg, recent plaque formation, superficial calcification, mFC accumulation). The findings are consistent with a previous study performed on a smaller sample size using a bench-top FLIm system [224] that also reports on the ability of FLIm to detect mFCs.

5.2.2 |. Demonstration in animal models

Acquisition of FLIm data in an intravascular setting presents challenges due to the strong absorption of UV light by blood and large variations of signal intensity due to varying probe to tissue distance. Initial tests were performed by imaging through a transparent balloon [225]. Subsequently, blood clearing via bolus injection of Dextran solution was retained, due to its transparency in the UV-visible-NIR wavelength range and high viscosity. Dextran solution is already routinely used as a low toxicity alternative to iodinated contrast as an OCT flushing agent [226]. FLIm data acquired in swine demonstrated that a consistent lifetime from the vessel wall autofluorescence could be obtained in such challenging conditions [108] (Figure 8A). More recent work in animal models of atherosclerosis (eg, rabbit Figure 8B,C) confirmed that locations of plaque show increased lifetime in locations where mFC were present [114, 227]. It is now possible to acquire FLIm data in conditions representative of future use in patients, with devices similar to currently available intravascular imaging modalities (ie, IVUS, OCT). This paves the way for the application of intravascular FLIm in combination with existing morphological imaging modalities to improve the characterization of atherosclerotic lesions in patients.

FIGURE 8.

Demonstration of applications of FLIm in vivo. A, Optimization of flushing procedures enable the acquisition of FLIm-IVUS data from coronary arteries in swine. Fluorescence obtained from the vessel wall demonstrates a consistent lifetime despite large variations in fluorescence intensity and the presence of residual blood; adapted from Reference [108]. In vivo imaging in rabbit aorta using a FLIm-OCT catheter demonstrates, B, increased lifetime in the location of a lipid-rich plaque created by balloon injury and, C, increased lifetime in locations with macrophage activity; adapted from Reference [114]

6 |. CONCLUSION

Mesoscopic FLIm for clinical applications continues to show great potential for real-time identification of disease. The technical advances implemented over the past decade have enabled the translation of time-resolved fluorescence devices from the bench-top to the operation rooms and subsequently, a more systematic evaluation of their ability to characterize and diagnose tissue during medical interventions.

In situ in patient evaluation of tissue pathological conditions using optical techniques requires direct access to the areas of interest, fast data acquisition and real-time feedback or display of clinically valuable data. Current intraoperative FLIm instrumentation is mainly implemented with a point-scanning approach with multimode fibers for optimal accessibility to tissue [109, 110]. Most practical FLIm devices are based on pulse sampling detection methods using sub-nanosecond pulsed lasers (0.1–1 ns) with fast repetition rate (0.1–4 kHz) and fast electronics along with software for rapid data acquisition, analysis and real-time visualization of fluorescence parameters encoding diagnostic information [127]. Recently, a TCSPC fiber-based imaging setup was reported [228]. This real-time alternative approach integrates stroboscopic illumination of the field-of-view to make TCSPC, which uses ultrasensitive detectors, compatible with surgical applications that require operation under bright illumination. The practical implementation of TCSPC-based imaging with stroboscopic illumination would require most light sources in the operating room to be synchronized with the acquisition of the fluorescence signal, which could be a significant practical hurdle for immediate clinical implementation. Thus, as discussed by Lagarto et al., this approach is envisioned to work mainly for endoscopic-based applications where only the endoscope illumination will need to be synchronized with the acquisition system.

Point-scanning FLIm with MMF, as described in this review, provides a practical approach for a broad range of clinical applications. Tissues of interest are accessible regardless of their location, shape and surface geometry, from brain cortex exposed during craniotomy procedures [109] to restricted internal organs such as vasculature [108, 113, 114] or the oropharynx [127], which can be reached through catheterization and robotic surgery, respectively. Furthermore, imaging through point-scanning is suitable for both free-hand [109, 118, 208] and mechanical scanning [108] approaches. For example, point-scanning through optical fibers is compatible with helical imaging of luminal surfaces (eg, vasculature) through rotation and pull-back motion, as well as with raster scanning (the probe or the sample) with 3D-stages that profile the sample’s surface without necessary contact between the probe and the tissue. An alternative approach to imaging large exposed surfaces is wide-field imaging [91, 112, 229], which can cover from millimeters to centimeter square areas in a few seconds, up to a minute. Wide-field snapshots of larger areas allow surgeons to rapidly visualize tumors in context, surrounded by viable tissue, which may facilitate margin detection. For example, a wide-field fiber-bundle endoscope was used to acquire 4 mm-diameter FLIM images of brain tumors in patients during surgery in less than 120 s [91]. A recent study demonstrated frequency-domain FLIM integrated with a neurosurgical microscope to obtain wide-field images (6.5 × 6.5 mm² in 65 s) of ex vivo brain samples with LGG labeled with 5-ALA for improved tumor delineation [229]. Wide-field FLIm will improve with further development of time-of-flight sensors and rapid cameras.

Besides imaging over large areas, the ability to inspect tissue over multiple scales (from micro- to macroscopic) will increase the breath of applications of FLIm. For intraoperative tumor resection, for example, it is required to first locate relatively large tumor masses (millimeters to centimeters). To achieve a higher extent of tumor resection, which has a direct impact on patient survival rates [163, 164, 179], it is then necessary to directly identify individual microscopic infiltrating cells, often responsible for cancer recurrence. Ideally, clinicians would benefit from seamlessly switching between sub-cellular resolution and tissue-scale imaging during surgery. A fluorescence lifetime micro-endoscope may fulfill this need. Although fluorescence lifetime microscopy is in widespread use, the challenges associated with the development of a FLIm micro-endoscope suitable for use in surgical settings are still to be overcome, despite ongoing efforts in this direction [115, 230, 231]. A practical demonstration of scale-switching was recently proposed with fast, large area, multi-photon exoscope (FLAME) for label-free in vivo imaging of human skin [88]. FLAME enables time-resolved millimeter-scale imaging through tile-mosaic acquisition with microscopic resolution for zoom-in detail. This concept was specifically designed for skin imaging where the microscope objective can be placed in close proximity to the patient’s skin, with the ultimate goal to help guide localized pigmentary skin disorders. Unfortunately, the large size of the device’s scanning head precludes endoscopic use.

Tissue light exposure is of paramount importance for the clinical adoption of optical devices. The data acquisition speed of the pulse sampling approach used in mesoscopic FLIm instrument is ultimately determined by the laser power and repetition rate. These parameters are in turn determined according to safety considerations regarding tissue exposure to avoid thermal and photochemical damage. The current standard used with the mesoscopic clinical FLIm instrument is set at 5 seconds of exposure at the limit when the fiber is in direct contact with the tissue, without exceeding the MPE as defined with ANSI Z136.1 [119]. This corresponds to a repetition rate of 120 Hz (considering four-waveform average). New FLIm instrumentation with increased detection sensitivity will allow to decrease laser power and increase laser repetition rate without increasing tissue exposure for even faster imaging.

The broad clinical adoption of FLIm in the operating room also requires rapid FLIm data analysis and visualization of key optical parameters associated with distinct tissue characteristics. Currently available software based on the Laguerre deconvolution method allows for real-time analysis of the autofluorescence signal. FLIm parameters are retrieved to assess the spectral characteristics (via the intensity ratio values at each spectral band) and the average fluorescence lifetime of the inspected tissue. The incorporation of an aiming beam to track the measurement location permits real-time display of individual or combined FLIm parameters superimposed to an image of the surgical field-of-view for an augmented reality visualization experience [110, 118]. For applications where the current standard of care does not require an imaging system (ie, cameras), mixed-reality visualization approaches with goggles or ancillary screens could enable enhanced surgical guidance [232–237].

Characterization and diagnosis of tissue based on its autofluorescence lifetime properties has inherent advantages but also poses challenges. The absolute fluorescence lifetime value is not always a robust comparison parameter as it can be affected by the experimental situations as well as biological variability from patient-to-patient. Factors that can affect fluorescence lifetime include in tissue blood perfusion or metabolism (eg, in vivo vs ex vivo measurements [110]), processing (eg, fresh vs frozen/thawed resected tissue [238]) and specific biochemical, biophysical and biomechanical tissue parameters (ie, pH, temperature, viscosity) [61]. Additionally, inter-patient variability due to inherent patient characteristics (ie, inflammatory response, co-morbidities and clinical history, demographics) will lead to errors when working with small sample numbers. Besides studies with large patient numbers, a combination of FLIm parameters (intensity and lifetime) over multiple spectral bands, rather than single metrics, also result in more accurate and improved global discrimination power [110, 127].

The diagnostic ability of fluorescence lifetime spectroscopy and imaging methods has been evaluated in both ex vivo specimens from surgical resected tissue or biopsies as well as, to a lesser extent, in vivo in patients. As demonstrated by an increasing body of literature, characteristic fluorescence lifetime from both ex vivo and in vivo measurements can be associated with distinct tissue types and pathological conditions [109, 110, 113, 120, 123, 194]. However, measurements acquired in vivo may not always match those acquired from ex vivo samples. For applications where the bulk of the signal raises from tissue composition and structural features (eg, proteins of the extracellular matrix, lipids aggregates, calcium deposits), ex vivo results may be similar to those acquired in vivo [113]. In those cases, ex vivo FLIm imaging of freshly resected specimens may be used as a means of rapid-biopsy, for example, to identify positive tumor margins from adipose and fibrous tissue during breast-conserving surgery [120, 208]. In contrast, a recent study showed how fluorescence lifetime values of in vivo tissue in the oral cavity change within minutes once this tissue is resected and imaged ex vivo [110]. Such a change in lifetime is expected due to the lifetime dependence on microenvironmental factors, severely affected upon tissue manipulation. For example, metabolic activity rapidly deteriorates after tissue removal from its natural environment. Additionally, a recent report comparing tissue discrimination ability from in vivo and ex vivo measurements demonstrated that results from in vivo data, both in terms of univariate statistics and classifiers, outperform those obtained ex vivo [127]. Therefore, clinical applications that rely on metabolic contrast to discriminate between healthy and diseased tissue (eg, tumor detection in the brain or head and neck tissue) are more appropriately conducted in vivo intraoperatively.

The main challenge for in vivo optical imaging techniques is establishing the relationship between optical parameters and corresponding tissue histopathological features. Often, this relies on multi-step registration methods where the location of optical measurements in vivo is mapped on the location of the tissue resected specimen with corresponding detailed histopathological/molecular evaluation [110, 127]. Alternative methods may involve the collection of micro-biopsies on small imaged areas for one-to-one co-registration. Creative and customized approaches have to be developed in close collaboration with pathologists in order to validate tissue identification methods based on FLIm data acquired in vivo. The implementation of robust and systematic annotation/labeling methods is an important step to drive the widespread clinical adoption of FLIm as a clinical imaging technique. Another step involves the collection of large data sets and big-data analysis methods to account for inter-patient variability of the autofluorescence signal as well as uncontrollable sources of errors such as tissue heterogeneity and variations in excitation-collection efficiency during free-hand scanning (due to small changes in distance and angle between sample and fiber probe). Machine learning data analysis methods have already uncovered the classification ability of FLIm data [120, 127, 195]. Deep learning approaches are currently employed to retrieve fluorescence lifetime parameters [153] and to improve the real-time visualization experience of FLIm data [118]. Further development of machine learning and deep learning algorithms will help elucidate trends and patterns from FLIm parameters and their correlations with complementary imaging modalities (eg, MRI, CT, OCT, Raman), histopathology and other patient specific characteristics that will lead to enhancing the diagnostic ability of FLIm. Eventually, novel artificial intelligence analysis methods may be trained to self-identify contrast and enable personalized diagnosis.

The future of mesoscopic FLIm appears promising for continued investigation and development for clinical applications in surgical oncology, cardiovascular diagnostics and potentially tissue engineering and regenerative medicine [133, 239–241]. The functionality and clinical integrability of FLIm devices will continue to advance. Improvements in terms of higher performance including speed and sensitivity, lower cost and ergonomics are expected to lead to a new generation of FLIm instrumentation that can be seamlessly integrated in intraprocedural workflows.

Abbreviations:

FAD

Flavin Adenine Dinucleotide

FLIm

fluorescence lifetime imaging

FLIM

fluorescence lifetime imaging microscopy

IRF

instrument response function

NAD(P)H

Nicotinamide Adenine (Phosphate) Dinucleotide

NIR

near-infrared

TCSPC

time-correlated single-photon counting

TRFS

time-resolved fluorescence spectroscopy

UV

ultraviolet

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.