Enhancing Fluorescence Lifetime Imaging With Differential Transformer

Abstract

Conventional analytical methods for fluorescence lifetime imaging (FLI) require the instrument response function (IRF) alongside temporal fluorescence decay to accurately estimate lifetime parameters. While the IRF remains spatially and temporally invariant in microscopy, it exhibits significant pixel-wise spatial variation and temporal offsets at mesoscopic and macroscopic scales due to sample size and surface topography. Existing deep learning approaches, trained on limited synthetic or simplified experimental datasets, often fail to generalize to complex biological or in vivo imaging. Here, we introduce MFliNet, a deep learning framework working in accordance with the physics of photon time-of-flight deconvolution for accurate multiexponential decay parameter estimation. Utilizing a Differential Transformer encoder–decoder architecture, MFliNet jointly processes temporal fluorescence decay and corresponding IRF inputs to correct topographical distortions in photon arrival distributions. Validation with tissue-mimicking phantoms and preclinical tumor models demonstrates exceptional robustness and precision, enabling reliable, real-time macroscopic FLI for complex biological and intraoperative applications.

1 |. Introduction

Fluorescence lifetime imaging (FLI) is a superior molecular imaging technique with high sensitivity and the ability to provide unique signatures with high specificity [1, 2]. Fluorescence lifetime and its associated parameters enable multiplexing studies [3–5] and can report on numerous unique biological signatures, including micro-environmental parameters, protein conformations, metabolic states, protein-protein interactions, and/or ligand-target engagement [6–8]. FLI has experienced constant growth over the last three decades, with a significant acceleration in its dissemination thanks to the availability of user-friendly FLI microscopes [9, 10]. Simultaneously, FLI has found increased utility in translational applications over the last two decades, ranging from mesoscopic (mFLI) [11] to macroscopic regimes (MFLI) [12–16]. Compared to microscopic implementations, mFLI and MFLI are significantly more challenging due to the requirement of using near infrared (NIR) fluorophores for deeper tissue penetration. As fluorophores are red shifted, it is typical that their lifetimes are shorter (nanosecond [ns] or sub-nanosecond compared to a few nanoseconds in the visible) [13] whereas large-format detectors exhibit low quantum efficiency (<10%) [17, 18]. Hence, quantifying lifetime and its associated parameters can be challenging due to very short fluorescence decays and/or low photon counts [15, 19, 20]. Unlike microscopic imaging, where sample preparation allows for precise control over the imaging plane, mFLI and MFLI samples can exhibit a large depth of field (DOF), leading to significant variations in the photon time-of-arrival in acquired data that must be accounted for accurate lifetime quantification. This is especially important in clinical systems, such as endoscopic or image-guided surgical instruments, where tissue profiles can lead to DOF variations of several centimeters.

The temporal histogram of fluorescence photon time-of-arrival following each pulsed excitation, also referred to as the temporal point spread function (TPSF), represents the periodic convolution of the instrument response function (IRF) and the sample fluorescence decay (SFD). The IRF characterizes the temporal response of the imaging system to pulsed illumination, encompassing both the excitation source’s temporal profile and the detector’s electronic response time; its shape varies depending on the detector type [21]. As the SFD is an intrinsic, molecule-specific property, any offset observed in the TPSF primarily originates from the IRF, which reflects variations in photon time-of-flight caused by surface profile differences. Consequently, although the IRF’s temporal profile remains relatively constant across the imaging field, its temporal position can shift on a pixel-by-pixel basis according to local surface geometry, leading to heterogeneous photon arrival times that must be explicitly corrected to ensure accurate lifetime estimation in samples with complex morphology [22].

Given the complexity involved in estimating FLI parameters across diverse imaging conditions with pixel-wise IRF variations, fast and advanced data processing techniques are necessary to enhance both the precision and efficiency of these analyses. Recently, the field has seen a shift toward rapid, fit-free deep learning (DL) methodologies [23] to alleviate the computational burden and reliance on user expertise typically associated with methods such as nonlinear least squares fitting (NLSF) for FLI parameter estimation [21, 24]. This advancement makes real-time FLI a possibility [25–27], driven by the development of novel DL methods that eliminate traditional time-consuming computational approaches. FLI-Net, a DL model developed for FLI parameter estimation [28], analyzes FLI data quickly, producing 2D quantitative images of lifetime parameters directly without requiring manual parameter adjustments. FLI-Net is versatile across imaging domains, including visible and NIR imaging, making it adaptable to a large range of biomedical applications. However, FLI-Net and similar existing models share a fundamental limitation: they process only the TPSF as input, with the IRF incorporated implicitly during training data generation rather than as an explicit pixel-wise input during inference [19]. Consequently, accurate lifetime recovery requires pixel-wise compensation for IRF variability. Conventional deep learning models such as FLI-Net implicitly embed the gate-jitter in IRF during training but cannot adapt to large pixel-level differences. In contrast, the proposed dual-input MFliNet architecture explicitly incorporates both the TPSF and corresponding pixel-wise IRF, enabling the network to disentangle instrument-induced temporal distortions from the underlying fluorescence decay dynamics.

To contextualize the advancements introduced by MFliNet, it is instructive to compare its design principles and functional capabilities with prior deep learning approaches for FLI. Table 1 summarizes the key methodological differences between FLI-Net and MFliNet. Prior deep learning approaches such as FLI-Net incorporate IRF variability during training through experimentally derived IRF libraries acquired from planar reference surfaces (e.g., flat white paper) and augmentation strategies, which capture pixel-level jitter and detector nonuniformities but not the large systematic temporal offsets induced by topographic height variations. Furthermore, these models operate on TPSF-only inputs at inference and therefore cannot explicitly adapt to the measured pixel-wise IRF when strong topography-driven temporal offsets are present. In contrast, MFliNet explicitly integrates pixel-wise IRFs through a dual-stream encoder–decoder design, enabling accurate lifetime estimation even in the presence of topography-induced temporal offsets. Moreover, the incorporation of a differential attention mechanism enhances signal fidelity under low-photon conditions by actively suppressing noise-dominated temporal components.

TABLE 1 |.

Comparison between FLI-Net and MFliNet architectures. MFliNet introduces explicit IRF awareness and a differential-transformer framework that enhances robustness under spatial and photon-count variability.

Feature	FLI-Net (Smith et al. [28])	MFliNet (This work)
Input type	Temporal point spread function (TPSF) only	Dual input: TPSF + pixel-wise instrument response function (IRF)
IRF treatment	IRF library acquired from planar reference surface (flat white paper) with stochastic temporal jitter augmentation (random shifts 0–3 time points) incorporated during training; inference operates on TPSF-only input	Explicit pixel-wise IRF provided as secondary input stream at inference (dual-input architecture)
Architecture	3D convolutional neural network	Differential transformer encoder–decoder
Noise handling	Implicit through training data	Differential attention mechanism for noise suppression
IRF offset correction	Implicitly learned through IRF variation during training	Explicitly modeled via cross-attention between TPSF and IRF
Domain adaptability	Effective for IRF variations within training distribution; limited generalization to large topographic variations outside training range	Robust to topographic and photon-count variations through explicit IRF conditioning
Computation speed	High (fit-free)	Comparable, with improved robustness
Output	Lifetime and amplitude maps	Lifetime, amplitude, and fractional component maps with IRF-aware correction

Transformer model architectures, primarily through the use of self-attention, have revolutionized time-series data processing by enabling the effective capture of long-range dependencies and providing superior performance in forecasting, classification, and anomaly detection [29]. MFliNet leverages the transformer architecture to handle sequential data, a choice motivated by several critical advantages. Unlike recurrent neural networks that process temporal sequences element-by-element [21], transformers can analyze entire photon time-of-arrival histograms simultaneously through parallel processing, enabling efficient training and inference on the temporal data structures inherent to FLI [30–32]. More importantly, transformers excel at capturing long-range dependencies in sequential data, which is essential for FLI applications where accurate lifetime extraction requires analyzing relationships across the entire temporal decay profile from the initial photon arrival through the exponential tail. The self-attention mechanism in transformers allows the model to dynamically focus on the most informative temporal regions of both TPSF and IRF inputs, while the cross-attention capability enables learning the complex relationship between these two input streams, a requirement that conventional convolutional or recurrent neural network architectures cannot adequately address.

Specifically, we employ a differential transformer (DIFF transformer) architecture, which has been proposed as a new approach to amplify attention to relevant task features while canceling noise [33]. The differential attention mechanism computes the subtraction between two attention maps rather than relying on a single attention output. This operation inherently produces sparse attention distributions, concentrating model capacity on temporal regions exhibiting strong fluorescence signal while attenuating contributions from detector noise and stochastic photon fluctuations. The learnable scaling parameter governing this subtraction adapts during training to the signal-to-noise characteristics of time-domain FLI data.

The contributions of this work are twofold. First, we demonstrate that explicit pixel-wise IRF integration as a secondary input stream overcomes the fixed-IRF assumption limiting prior deep learning models. Second, we show that differential attention within both encoder self-attention and novel decoder cross-attention blocks enables parameter estimation for challenging conditions.

2 |. Methods

2.1 |. Imaging Setup

All experimental data used in this work were captured on our MFLI system, detailed information about which can be found in [19, 34]. Briefly, the system uses a large-format Intensified Charge-Coupled Device (ICCD) camera (Picostar HR, LaVision GmbH, Germany), for wide-field detection over a 8 × 6cm² in combination with a Digital micro-mirrors device (DMD) (DLi 4110, Texas Instruments, Texas, USA), for wide-field illumination. As an excitation source, we used a tunable Ti-Sapphire laser (Mai Tai HP, Spectra-Physics, California, USA), which delivers 100 femtosecond pulses at 80 MHz. A gate width of 300 picoseconds (ps) and gate delay of 40 ps were used for capturing time-resolved fluorescence decays (for in vivo and in vitro experiments) with a total of 176 time points, which is referred to as a number of gates $(G=176)$. An emission filter at 740 ± 10 nm (FF01–740/13–25, Semrock, IL, Rochester, New York, USA) is used to capture the TPSFs, with the laser set at a 700 nm wavelength.

2.2 |. Classical Fluorescence Lifetime Processing

Fluorescence lifetime decay follows exponential decay kinetics. Depending on the number of fluorescent components present in the sample, the decay can be described by a combination of multiexponential functions. Most FLI imaging experiments involve up to two components, hence a bi-exponential model is typically used. The two-component or bi-exponential model also includes mono-exponential cases where fractional amplitudes $A_R$ equal to one or zero. The intrinsic fluorescence decay function $F_0(t)$ can be expressed as

$$F_0(t)=A_R{\text{e}}^{-\frac{t}{{\tau }_1}}+\left(1-A_R\right){\text{e}}^{-\frac{t}{{\tau }_2}},$$ (1)

where lifetime decays are denoted as ${\tau }_1$ and ${\tau }_2$, and $A_R$ is the amplitude fraction.

The emitted signals are recorded using multiple instruments (detectors, electronics, etc.) which have a characteristic response function $E(t)$ to an instantaneous signal $\delta (t)$ (e.g., a single photon). For an excitation with period $T$, the emitted signal from the $k$-th period can be expressed as

$${\epsilon }_k(t)=\underset{-\infty }{\overset{+\infty }{\int }}{x}_k(u)F_0(t-u)\text{d}u={\epsilon }_0(t-kT),$$ (2)

where $x_k(t)$ represents the excitation pulse and ${\epsilon }_0(t)$ is the emitted signal. The recorded signal $S_0(t)$ can be written as a convolution of the emitted signal and the instrument response:

$$\begin{gathered} S_0(t)=E(t)*{\epsilon }_0(t) \\ =E(t)*\left(x_0(t)*F_0(t)\right) \\ =\left(E(t)*x_0(t)\right)*F_0(t) \\ =I_0(t)*F_0(t). \end{gathered}$$ (3)

Equation (3) introduces the IRF $I_0(t)=E(t)*x_0(t)$, which characterizes the system’s temporal response to the excitation pulse. Substituting the fluorescence decay from Equation (1), the recorded signal becomes

$$S_0\left(t\right)=I_0\left(t\right)*\left(A_R{\text{e}}^{-\frac{t}{{\tau }_1}}+\left(1-A_R\right){\text{e}}^{-\frac{t}{{\tau }_2}}\right).$$ (4)

The relationship between $I_0(t)$ shifts and fluorescence lifetime estimation can be understood through the mathematical framework described in Equation (4). When topographic variations introduce a temporal shift in the IRF, denoted as $\Delta t$, the shifted IRF can be expressed as $I_0(t-\Delta t)$. Substituting this into Equation (4) yields a modified recorded signal of the form:

$$S_0(t)=I_0(t-\Delta t)*\left(A_R{\text{e}}^{-t/{\tau }_1}+\left(1-A_R\right){\text{e}}^{-t/{\tau }_2}\right).$$ (5)

This temporal offset effectively shifts the entire measured photon time-of-arrival histogram forward in time without altering the underlying fluorescence decay kinetics.

However, when conventional fitting algorithms or deep learning models assume a fixed IRF and attempt to deconvolve this shifted TPSF, the deconvolution process interprets the temporal offset as a change in the decay profile itself. Specifically, a right-ward shift in the IRF causes the fitted model to compensate by either increasing the estimated lifetime values to account for the delayed photon arrivals or by adjusting the fractional amplitudes to reweight the exponential components. The magnitude of this artifact scales with $\Delta t$, which in macroscopic imaging scenarios can be approximated as $\Delta t=2d/c$, where $d$ represents the change in sample-detector distance and $c$ is the speed of light in the medium. For macroscopic FLI applications where depth-of-field variations can reach several centimeters, these temporal shifts become significant relative to the nanosecond-scale lifetimes of NIR fluorophores.

2.3 |. Generation of Training Data

The in silico data used for training and validating the proposed model was generated using Equation (4). Initially, time-resolved fluorescence lifetime images with dimensions of 28 × 28 pixels were generated by using the MNIST dataset. Fluorescence decays were generated for a range of lifetime values commonly used in NIR applications [21, 28]: 0.2 to 0.8 ns for ${\tau }_1$ (short-lifetime component) and 0.8 to 1.5 ns for ${\tau }_2$ (long-lifetime component). The range of the $A_R$ (fraction amplitude) was set from 0% to 100%, respectively (both bounds corresponding to mono-exponential cases).

To ensure that our simulated data accurately represents experimental applications, pixel-wise IRFs were used. To capture experimental IRFs, a white diffused paper was placed on the imaging table and illuminated using the DMD with an excitation wavelength of 700 nm. The reflected light was captured using a neutral density (ND) filter. Following the publicly available data generation pipeline described by Smith et al. [28] for gated-ICCD data, we expanded the white-paper IRF library to include a broader range of height-induced temporal offsets, thereby capturing the systematic IRF variations caused by topographic changes in addition to pixel-level jitter and detector non-uniformities. Subsequently, each TPSF was generated by convolving a randomly selected IRF from the dataset with simulated fluorescence decay profiles. To emulate laser jitter and temporal uncertainty in gate timing, each TPSF was subjected to stochastic temporal shifts (0–3 time points) before final processing, with shift direction and magnitude randomly assigned to capture variations in the point of TPSF ascent [28]. To approximate the noise characteristics of real-world measurements, system-derived noise, including read-out noise, dark noise, etc., as explained in [35], was incorporated into the simulated TPSFs. This approach ensures that the simulated data closely matches the noise dynamics observed in the actual system. The generated dataset was used to train all baseline models (FLI-Net, standard Transformer) and MFliNet, ensuring fair comparison where performance differences arise solely from architectural distinctions.

To evaluate and compare the model’s performance in the absence of experimental ground truth, we used the NLSF method, which is commonly used to estimate the FLI parameters described in Equation (4). Traditionally, FLI parameters are estimated from experimental data through iterative fitting optimization methods such as the NLSF (see [22, 36] for detailed explanation), which incorporates the Levenberg–Marquardt algorithm [37], or center of mass (CMM) analysis [38]. For our NLSF analysis, we utilized software named AlliGator [36], allowing adjustments and constraints on fitting parameters, including short and long lifetimes, fraction amplitudes, and offsets, depending on experimental conditions. We selected between single and double exponential decay models according to the complexity of our datasets. AlliGator also provides an option for offset correction when there is a mismatch between the TPSF and IRF. We evaluated the importance of offset correction in our NLSF analysis by comparing data with and without this feature in our phantom experiments. Additionally, we benchmarked our results against those obtained using FLI-Net to provide a comprehensive evaluation of our approach in the context of established methodologies. Furthermore, to assess the impact of the DIFF Transformer model, we compared our results with those from a transformer model of the same architecture and parameters, trained on the same dataset. This comparison effectively serves as a focused ablation analysis, isolating the contribution of the differential attention mechanism. Because both models share identical architectures and training conditions, the observed differences in performance can be attributed specifically to the inclusion of differential attention, demonstrating its role in enhancing noise suppression and parameter estimation accuracy.

2.4 |. Deep Learning Network Architecture for Reproducibility

MFliNet is a novel architecture designed for FLI parameter estimation, as shown in Figure 1, particularly effective under varying IRFs. The model leverages a DIFF Transformer framework, incorporating a unique differential attention mechanism to enhance feature extraction while mitigating the effects of noise. The theoretical background on the differential attention mechanism is detailed in [33].

FIGURE 1 |.

Proposed transformer-based deep learning network architecture.

At the core of MFliNet is the differential attention mechanism, which computes attention scores by contrasting two separate multi-head attention outputs. Specifically, the mechanism calculates the difference between two attention maps, scaled by a parameter $\lambda$, to focus on relevant patterns and suppress noise. The differential attention is defined as:

$$\text{DiffAttn}(X)=\left(\text{softmax}\left(\frac{Q_1{K}_1^{\top }}{\sqrt{d_k}}\right)V_1\left.-\lambda \cdot \text{softmax}\left(\frac{Q_2{K}_2^{\top }}{\sqrt{d_k}}\right)V_2\right)\right.,$$ (6)

where:

• $Q_1,K_1,V_1$ are the query, key, and value matrices for the first attention head, derived from the input $X$ using learned projections.

• $Q_2,K_2,V_2$ are the query, key, and value matrices for the second attention head, also derived from $X$.

• $d_k$ is the dimensionality of the key vectors.

• $\lambda$ is a learnable scalar parameter that balances the contributions of the two attention maps.

The queries, keys, and values are computed as:

$$Q_i=X{W}_i^Q,K_i=X{W}_i^K,V_i=X{W}_i^V,\text{for}i=\mathrm{1,2},$$ (7)

where $W_i^Q,W_i^K$, and $W_i^V$ are the learned weight matrices for the $i$-th attention head.

The parameter $\lambda$ in Equation (6) is a learnable scalar that controls the balance between the two attention maps during training (see [33] for details). Unlike traditional hyperparameters that require manual tuning, $\lambda$ is initialized and optimized through backpropagation alongside other network parameters using the Adam optimizer. During training, the gradient descent process adjusts $\lambda$ to determine the optimal weighting between the two attention heads, effectively learning how much the second attention map should be subtracted from the first to maximize signal extraction while suppressing noise. The value of $\lambda$ directly impacts the model’s noise cancelation capability, with higher values amplifying the differential attention effect and lower values reducing it. In the context of FLI parameter estimation, this adaptive mechanism allows MFliNet to automatically calibrate its sensitivity to relevant temporal features in the TPSF while attenuating contributions from instrument noise, detector dark counts, and stochastic photon arrival variations. The learned value of $\lambda$ reflects the signal-to-noise characteristics inherent in the training dataset, enabling the model to generalize to experimental data with similar noise properties without requiring manual adjustment or prior knowledge of noise distributions.

Building upon this differential attention mechanism, the MFliNet architecture processes input sequences through encoder blocks that integrate this noise cancelation capability at each layer. Each input sequence passes through two stacked encoder blocks, followed by layer normalization and a feed-forward network (FFN) with SwiGLU activation. The encoder block operates as follows:

$$\begin{gathered} \text{Attention}\text{Output}=\text{DiffAttn}\left(X\right), \\ {\text{Add\&Norm}}_1=\text{LayerNorm}(X+\text{Attention}\text{Output}), \\ \text{FFN}\text{Output}=\text{FFN}\left({\text{Add\&Norm}}_1\right), \\ \text{Encoder}\text{Output}=\text{Layer}\text{Norm}\left({\text{Add\&Norm}}_1\right.+\text{FFN}\text{Output}). \end{gathered}$$ (8)

where the feed-forward network is defined as:

$$\text{FFN}(x)=\left(\text{Swish}\left(x{W}_1+b_1\right)\odot \left(x{W}_2+b_2\right)\right)W_3+b_3,$$ (9)

with $W_1,W_2,W_3$ being learned weight matrices, $b_1,b_2,b_3$ bias vectors, $\odot$ representing element-wise multiplication, and Swish being the activation function defined as $\text{Swish}(x)=x\cdot \sigma (x)$, where $\sigma (x)$ is the sigmoid function.

The decoder blocks incorporate both self-attention and cross-attention mechanisms. The self-attention layer within the decoder uses the differential attention mechanism to capture intra-sequence relationships. The cross-attention layer aligns the decoder’s inputs with the encoder outputs, integrating information from both inputs. The decoder block operates as:

$$\begin{gathered} \text{Self}-\text{Attn}\text{Output}=\text{DiffAttn}\left(X\right), \\ {\text{Add\&Norm}}_1=\text{LayerNorm}(X+\text{Self}-\text{Attn}\text{Output}), \\ \text{Cross}-\text{Attn}\text{Output}=\text{Attention}\left({\text{Add\&Norm}}_1,E,E\right), \\ {\text{Add\&Norm}}_2=\text{LayerNorm}({\text{Add\&Norm}}_1+\text{Cross}-\text{Attn}\text{Output}), \\ \text{FFN}\text{Output}=\text{FFN}\left({\text{Add\&Norm}}_2\right), \\ \text{Decoder}\text{Output}=\text{LayerNorm}({\text{Add\&Norm}}_2+\text{FFN}\text{Output}). \end{gathered}$$ (10)

where $E$ represents the encoder outputs from the corresponding input sequence, and the standard attention mechanism is defined as:

$$\text{Attention}(Q,K,V)=\text{softmax}\left(\frac{Q{K}^{\top }}{\sqrt{d_k}}\right)V.$$ (11)

MFliNet’s architecture includes three parallel output pathways, each dedicated to predicting one of the FLI parameters: short lifetime, long lifetime, and fractional amplitude. Each pathway processes the decoder outputs through additional layers to refine the predictions.

The final outputs are obtained by reshaping the decoder outputs and applying a convolutional layer with kernel size 1 × 1, using the Exponential Linear Unit (ELU) activation function and L2 regularization to prevent overfitting:

$$\begin{gathered} {\text{Output}}_i=\text{Conv}2{\text{D}}_{\text{ELU}}(\text{Reshape}(\text{Decoder}{\text{Output}}_i\text{)),} \\ \text{for}i=1,2,3. \end{gathered}$$ (12)

where $\text{Conv}2{\text{D}}_{\text{ELU}}$ denotes a 2D convolutional layer with ELU activation and L2 regularization applied to the reshaped decoder output corresponding to each parameter.

Training was conducted using the Adam optimizer with an adaptive learning rate starting from 0.001. The loss function for each output branch was mean squared error (MSE). The dataset comprised 1568000 generated time-resolved photon signals and corresponding IRFs, with 10% reserved for validation. The model’s design allows it to capture both local and global patterns in the data, effectively modeling variations in time-domain signals and encoding the relationships between inputs and FLI parameters. For reproducibility, model architecture code is publicly available [39].

2.5 |. Phantom Preparation

For experimental validation, we designed a step ladder phantom to introduce variations in sample-detector distance, as depicted in Figure 2a. A 3D printable case was designed to accommodate five discrete containers arranged at various heights: ground level, 5, 10, 15, and 20 mm. Each container was crafted with dimensions of 20 × 10 × 3.5 mm³ to accommodate the fluorescent dye solutions. Each container was filled with Alexa Fluor 700 dye dissolved in phosphate-buffered saline to achieve a concentration of $20\mu \text{M}$.

FIGURE 2 |.

Illustration of designed 3D step ladder phantom and the IRF shifts as a result of variations in height: (a) display of the 3D phantom (up) and plots of average of IRFs from each height (down); (b) a side view of a mouse, highlighting height differences between anatomical regions (up) and IRF plots of randomly selected pixels on liver, urinary bladder (UB), tumors (down); (c) a distal ventral view of the mouse highlighting the tumors and the liver (up) and the IRF plots of the randomly selected pixels on the left tumor (down).

To examine the offset variation across different heights on the phantom and in live intact animals, we plotted the pixelwise IRFs for comparison as shown in Figure 2. For each specified height in the step ladder phantom, we plotted the average IRFs in Figure 2a. Moreover, we also illustrated the IRFs of various anatomical regions in live intact animals, including the liver, urinary bladder (UB), and tumors, in Figure 2b. Lastly, in Figure 2c, we examined the variability of the IRF within a single tumor. This plot contains IRFs at three points within a tumor: the top, middle, and bottom. The top refers to an IRF from the upper region of the tumor, the middle corresponds to an IRF from the central area in terms of height, and the bottom shows an IRF from the lowest part of the tumor.

2.6 |. In Vivo Experiment

For in vivo MFLI imaging experiments, we imaged HER2+ breast tumor xenografts HCC1954 in athymic nude mice. The cell line was sourced from ATCC (Manassas, Virginia, USA) and maintained in RPMI 1640 media enriched with 10% fetal bovine serum (ATCC) and $50\text{units}/\text{mL}/50\mu \text{g}/\text{mL}$ penicillin/streptomycin from ThermoFisher Scientific (Waltham, Massachusetts, USA). We initiated tumor xenografts by subcutaneously injecting 5 ×10⁶ HCC1954 cells suspended in PBS and mixed in a 1:1 ratio with Cultrex BME (R&D Systems Inc., Minneapolis, Minnesota, USA) into the inguinal mammary fat pads of female athymic nude mice aged 4 weeks (CrTac: NCR-Foxn1nu, Taconic Biosciences, Rensselaer, New York, USA). Tumors were monitored daily for 4 weeks. The mouse was administered with a retro-orbital injection of AF700 conjugated with Meditope Trastuzumab (MDT-TZM) (MDT-TZM-AF700) at $20\mu \text{g}$ and AF750 conjugated with MDT-TZM (MDT-TZM-AF750) at $40\mu \text{g}$ in a 2:1 acceptor to donor ratio through staggered injection [40]. MDT-TZM is an engineered antibody enabling site-specific NIR dye conjugation via a 12-amino acid cyclic peptide binding site in the Fab region, which preserves HER2 binding affinity while enabling consistent, reproducible imaging of HER2-positive tumors [41–43]. Donor injection was performed 2 h ahead of acceptor injection through the retro-orbital route. MFLI Imaging was conducted 24 h post-injection. Throughout the imaging process, the mouse was anesthetized with isoflurane, and the body temperature was maintained with a Rodent Warmer X2 (Stoelting, Illinois, USA). All animal procedures were conducted with the approval of the Institutional Animal Care and Use Committee (IACUC) at both Rensselaer Polytechnic Institute and Albany Medical College. The animal facilities of both institutions have been accredited by the American Association for Accreditation for Laboratory Animals Care International.

2.7 |. Statistical Analysis

Statistical comparison between NLSF and MFliNet lifetime parameter estimates was performed using pixelwise paired $t$-testing across two tumor regions identified via connected component analysis of the binary tumor mask. For each tumor and lifetime component, a paired $t$-test was applied with test statistic $t=\frac{\bar{d}}{s_d/\sqrt{n}}$, where $\bar{d}$ represents the mean difference, $s_d$ the standard deviation of differences, and $n$ the number of paired pixels. To control family-wise error rate across four independent comparisons (two tumors, two lifetime components), Bonferroni correction was applied with adjusted significance threshold ${\alpha }_{\text{adj}}=0.05/4=0.0125$. Complementary to difference testing, two one-sided tests (TOST) [44] for equivalence were conducted to establish practical interchangeability within prespecified bounds $\pm \epsilon$, where $\epsilon =0.05\text{ns}$ reflects typical FLI measurement precision. The TOST procedure evaluates two composite null hypotheses $H_{0,\text{lower}}:{\mu }_d\le -\epsilon$ and $H_{0,\text{upper}}:{\mu }_d\ge \epsilon$ via corresponding $t$-statistics $t_{\text{lower}}=(\bar{d}+\epsilon )/\left(s_d/\sqrt{n}\right)$ and $t_{\text{upper}}=(\bar{d}-\epsilon )/\left(s_d/\sqrt{n}\right)$, declaring equivalence when both null hypotheses are rejected at ${\alpha }_{\text{adj}}$, which is equivalent to requiring the entire ($1-2{\alpha }_{\text{adj}}$) confidence interval to fall within $[-\epsilon ,\epsilon ]$. Method agreement was further quantified through Bland–Altman analysis [45] computing mean bias $\bar{d}$ and 95% limits of agreement $\bar{d}\pm 1.96s_d$, Pearson correlation coefficients assessing spatial pattern concordance, and error metrics including mean absolute error $(\text{MAE})=\frac{1}{n}\sum _{i=1}^n\left|x_i-y_i\right|$, root mean square error $(\text{RMSE})=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(x_i-y_i\right)}^2}$, and mean absolute percentage error $(\text{MAPE})=\frac{100\%}{n}\sum _{i=1}^n\frac{\left|x_i-y_i\right|}{x_i}$, where $x_i$ and $y_i$ denote NLSF and MFliNet estimates for pixel $i$, respectively.

3 |. Results

We conducted the ladder phantom experiment designed to validate the MFliNet model under controlled conditions that mimic biological tissues. Figure 3 shows the phantom experiment results where the analysis was done using four methods: NLSF, FLI-Net, transformer model, and MFliNet. To compare the precision and stability of each method under varying conditions reflective of real-world applications, results were evaluated across five different heights: ground, 5, 10, 15, and 20 mm (see Figure 2). A 160 ps shift in the IRF was observed from ground level to a height of 20 mm, with a shift of approximately 40 ps for each 5 mm increment in height. For comparative analysis, the amplitude-weighted average lifetime was calculated using Equation (13) for all outputs.

FIGURE 3 |.

Phantom experiment results. (a) Image overlay of the lifetime estimation results, (b) violin plots of NLSF analysis, FLI-Net, transformer model and MFliNet.

$${\tau }_M=\left(A_R{\tau }_1+\left(1-A_R\right){\tau }_2\right)$$ (13)

NLSF analysis using pixel-wise IRF showed consistency in lifetime estimation across all tested heights (see Figure 3). The mean fluorescence lifetime values obtained by NLSF were clustered around 1.01 ± 0.02 ns. NLSF without offset correction results deteriorated with each increase in height. At the ground level, NLSF without offset correction began with a mean value of 1.04 ± 0.01 ns. However, as the height increased, a steady decline in lifetime estimation was observed, reaching a mean value of 0.93 ± 0.01 ns at 20 mm. FLI-Net, in contrast, demonstrated a wider variation in estimated fluorescence lifetime values. At the ground level, it reported a mean value of 0.96 ± 0.01 ns, which was lower than the NLSF values. As the distance increased, FLI-Net’s estimations deviated further, peaking at 1.14 ± 0.01 ns at 10 mm and estimating 20 mm with a mean value of 1.10 ± 0.02 ns. The transformer model showed variable performance across different heights; at 10 mm, a decrease was observed with a mean value of 0.91 ± 0.04 ns, while estimations at other heights were closer to the NLSF values. In contrast, MFliNet, showed closer results with NLSF, where the mean values were within the same range as NLSF. Moreover, in terms of processing speed, NLSF took approximately more than 1 h to analyze 598 pixels (covering only the tumor area), whereas MFliNet analyzed the entire dataset of 90 480 pixels in just 63 s.

Following the phantom studies, in vivo experiments were conducted using the HER2+ breast tumor xenograft model in mice to evaluate the model’s performance in a more complex, biologically variable environment. The experimental results, illustrated in Figure 4, demonstrate the comparative analysis of NLSF and MFliNet in HER2+ HCC1954 tumor xenograft mice model. For the smaller tumor (HCC1954 (A)), the NLSF method reported a short fluorescence lifetime of 0.56 ± 0.06 ns and a long lifetime of 1.17 ± 0.03 ns, whereas MFliNet yielded 0.52 ± 0.05 and 1.19 ± 0.02 ns for ${\tau }_1$ and ${\tau }_2$, respectively. In the case of the larger tumor (HCC1954 (B)), the NLSF method reported a short fluorescence lifetime of 0.55 ± 0.07 ns and a long lifetime of 1.16 ± 0.04 ns, while MFliNet reported 0.58 ± 0.04 and 1.18 ± 0.03 ns for ${\tau }_1$ and ${\tau }_2$, respectively. Pixelwise paired statistical testing revealed significant differences between methods for all lifetime parameters after Bonferroni correction ($p<0.0125$ for all comparisons), with mean biases ranging from − 0.028 to 0.039 ns (see Figure S1a for bias and 95% limits of agreement and Figure S1b for equivalence decisions relative to ± 0.05 ns). However, TOST for equivalence within prespecified bounds of ± 0.05 ns confirmed practical equivalence for tumor A ${\tau }_2$ (TOST $p<0.0001$), tumor B ${\tau }_1$ (TOST $p<0.0001$), and tumor B ${\tau }_2$ (TOST $p<0.0001$), while tumor A ${\tau }_1$ approached but did not achieve formal equivalence (TOST $p=0.050$) (see Figure S1b for the 90% confidence intervals positioned within, or marginally outside, the equivalence bounds). Pearson correlation analysis demonstrated strong linear relationships between NLSF and MFliNet estimates, with correlation coefficients of $r=0.747,r=0.598,r=0.762$, and $r=0.773$ for tumor A ${\tau }_1$, tumor A ${\tau }_2$, tumor B ${\tau }_1$, and tumor B ${\tau }_2$, respectively (all $p<0.001$). Bland–Altman analysis yielded mean biases of 0.039 ns (tumor A ${\tau }_1$), – 0.022 ns (tumor A ${\tau }_2$), – 0.028 ns (tumor B ${\tau }_1$), and – 0.025 ns (tumor B ${\tau }_2$), with 95% limits of agreement spanning [– 0.041,0.119], [− 0.077,0.033], [− 0.125,0.069], and [− 0.070,0.020]ns, respectively (see Figure S1a for Bland–Altman agreement between NLSF and MFliNet across short and long lifetimes in tumors A and B). Mean absolute errors ranged from 0.028 to 0.049 ns, with root mean square errors between 0.034 and 0.057 ns across all comparisons.

FIGURE 4 |.

Comparison of in vivo results for both NLSF and MFliNet (a) image overlays of the short and long-lifetime results for both NLSF and MFliNet (b) plot of means and standard deviations of the predicted lifetime values of both methods. Asterisks denote statistical significance from pixelwise paired testing (Bonferroni-corrected $\alpha =0.0125$): **** p < 0.0001. Mean differences (NLSF—MFliNet): Tumor A ${\tau }_1=+0.039\text{ns}$, tumor A ${\tau }_2=-0.022\text{ns}$, tumor B ${\tau }_1=-0.028\text{ns}$, tumor B ${\tau }_2=-0.025\text{ns}$. TOST equivalence testing (±0.05 ns bounds) confirmed practical equivalence for tumor A ${\tau }_2$, tumor B ${\tau }_1$, and tumor B ${\tau }_2$.

4 |. Discussion and Conclusion

The experimental validation of MFliNet across phantom and in vivo studies demonstrates its capability to address a fundamental challenge in macroscopic FLI: accurate lifetime parameter estimation in the presence of topography-induced temporal variations in the IRF. The data (Figure 2) show a spatially varying offset in IRFs, originating from differences in sample height and organ-specific surface geometries. For instance, an intermediate height variation of ~10 mm yields an offset of approximately 80 ps, compounded by detector noise that introduces differential variations in the temporal fluorescence decay. This spatially varying IRF offset leads to pixelwise differences in photon time-of-flight, demonstrating as temporal shifts in fluorescence decay signals and consequently results in FLI parameter estimation bias. The integration of pixelwise IRFs within the MFliNet framework directly addresses this challenge, providing a crucial correction often overlooked in other deep learning (DL)-based FLI models.

MFliNet demonstrates robust fluorescence lifetime parameter estimation even under substantial variations in biological topography and sample height. The challenging “ladder phantom” experiment (Figure 3), which introduced large distance changes at each step, produced corresponding temporal shifts in fluorescence decay curves. Notably, even the state-of-the-art FLI-Net model exhibited systematic, distance-dependent lifetime bias, reflected in an artificial increase in lifetime estimates consistent with unmodeled time-of-flight variations. The observed distance-dependent bias in FLI-Net reflects the fact that the model processes only TPSF inputs at inference; even though training includes IRF libraries and temporal jitter augmentation, systematic IRF offsets introduced by large height/topography changes can exceed what is represented in the training distribution, leading to biased lifetime recovery. MFliNet effectively overcomes this limitation through its explicit dual-input architecture that directly conditions on the measured pixel-wise IRF through cross-attention, which enables accurate lifetime estimation across diverse imaging geometries without retraining.

The issue of fluorescence decay offset is not unique to this field. Pixelwise variations in photon time-of-flight are commonly corrected using analytical preprocessing algorithms such as thresholding or autocorrelation-based alignment of the fluorescence decay and IRF rising edges. However, these methods often perform poorly in low-photon-count regimes dominated by Poisson noise and require iterative computations that substantially increase latency. In practice, conventional NLSF analyses frequently rely on experimentally measured pixelwise temporal IRFs derived directly from the biological sample (e.g., tumor or organ surface) rather than from planar diffusive references (e.g., white paper), to circumvent preprocessing-based corrections. MFliNet’s dual-input architecture further mitigates these limitations by inherently compensating for IRF offsets without explicit preprocessing, achieving both robustness and computational efficiency. Comparative analysis demonstrates that MFliNet matches the accuracy of offset-corrected NLSF analysis while dramatically reducing processing time, from several hours to approximately 1 min. Specifically, a dataset requiring more than 1 h for NLSF fitting was processed in 63 s by MFliNet, corresponding to an improvement of approximately ~57× faster.

The in vivo tumor imaging results highlight an important distinction between statistical detectability and practical equivalence. While pixelwise paired testing identified significant differences between NLSF and MFliNet for all lifetime parameters, these findings reflect the inherent statistical power of large sample sizes rather than clinically problematic discrepancies. The observed mean biases ranged from − 0.028 to +0.039 ns, representing systematic offsets in the tens-of-picoseconds range. To contextualize these biases within measurement precision constraints, we applied two one-sided equivalence tests with ± 0.05 ns bounds, which confirmed that MFliNet operates within acceptable tolerance for three of four comparisons. The correlation coefficients exceeding $r=0.7$ indicate that MFliNet preserves the spatial heterogeneity patterns across tumor regions, which is critical for distinguishing biologically distinct microenvironments. The Bland–Altman analysis further supports this interpretation, with limits of agreement and mean absolute errors remaining well within the noise characteristics of macroscopic fluorescence lifetime imaging systems (see Figure S1a,b for agreement limits, bias magnitudes, and equivalence decisions). These findings demonstrate that MFliNet achieves quantitative fidelity comparable to NLSF while maintaining the computational efficiency necessary for real-time applications.

Under photon-starved conditions, the spatial and temporal offset on photon time-of-flight becomes a particularly difficult obstacle to overcome using preprocessing algorithm. Traditional computational methods, such as NLSF, often underperform in these low-photon, high-offset situations. While deep learning approaches, like FliNet, have demonstrated high reproducibility and accuracy in the low-photon regime where temporal offset is absent. In our proposed method, MFliNet, showed remarkable performance in both offset correction and lifetime parameter estimation even in low-photon biological imaging. Unlike conventional transformer architectures, MFliNet employs a differential attention mechanism via its DIFF Transformer module. This approach selectively focuses on the most informative temporal features, specifically, pixelwise fluorescence decay offsets, while suppressing noise contributions, thereby improving the fidelity of lifetime estimation. As shown in Figure 3, the differential attention mechanism enhances performance by computing the difference between two attention maps, effectively canceling stochastic fluctuations and isolating signal-bearing temporal components. This enables reliable FLI parameter estimation even in photon-starved or geometrically complex imaging scenarios. This results in systematic underestimation of lifetime parameters (Figure 3), with mean lifetime values of 0.91 ± 0.04 ns at 10 mm compared to the NLSF reference of 1.01 ± 0.02 ns.

While the present study includes a limited number of biological samples and tissue types, this scope reflects the current technical and logistical constraints of macroscopic fluorescence lifetime imaging at the preclinical stage. MFliNet represents the first implementation of a differential-transformer architecture for pixel-wise, IRF-aware fluorescence lifetime estimation, and the collection of large-scale, multi-animal datasets remains experimentally prohibitive due to the complexity of time-resolved imaging, acquisition duration, and the need for precise IRF calibration for each animal. Importantly, the phantom and in vivo results were designed to span a wide range of photon-count conditions, depths, and surface geometries, thereby capturing the dominant variability encountered in practical imaging scenarios. Future work will extend this framework to multi-site and multi-species datasets as the instrumentation and data-sharing pipelines mature. We thus view the present validation as a proof-of-principle demonstration establishing methodological robustness rather than a statistical population study.

Despite the limited number of in vivo datasets, the experiments presented here encompass a diverse range of photon-count regimes, topographic variations, and fluorescence lifetimes that effectively capture the dominant sources of variability encountered in practical macroscopic FLI. This breadth of experimental conditions provides a robust proof-of-concept validation of MFliNet’s capacity to generalize across geometrical and optical contexts. Moreover, the modular dual-input design of MFliNet, explicitly incorporating both the TPSF and pixel-wise IRF, renders the framework inherently extensible. This architecture can be retrained or fine-tuned on new datasets acquired from different tissues, fluorophores, or imaging geometries without requiring structural modifications, thereby ensuring adaptability to future large-scale studies as data availability increases.

The practical significance of this advancement extends to complex imaging environments such as intraoperative ex vivo imaging of freshly resected tumors, where rapid and accurate delineation of tumor margins is critical for guiding surgical decisions and improving outcomes. Beyond clinical applications, MFliNet offers broad utility in preclinical imaging for quantifying drug–target interactions, receptor–ligand binding dynamics, and probe pharmacokinetics in whole-animal studies. Furthermore, the enhanced accuracy in measuring molecular interactions enabled by MFliNet can deepen our understanding of cellular functions and disease mechanisms, potentially accelerating the discovery and validation of targeted therapies.

Supplementary Material

Additional supporting information can be found online in the Supporting Information section. Figure S1: Agreement and equivalence between NLSF and MFliNet lifetime es timates in vivo. (a) Bland–Altman plots for short (${\tau }_1$, “Short Lifetime”) and long (${\tau }_2$, “Long Lifetime”) components in tumors A and B show the pixelwise difference (NLSF–MFliNet) versus the mean of the two methods; solid red lines denote mean bias, dashed red lines denote the 95% limits of agreement, and the dotted gray line marks zero difference. (b) Two one-sided tests (TOST) for equivalence using prespecified ± 0.05 ns bounds; the shaded region indicates the equivalence interval, vertical dashed lines mark the bounds, the filled circle shows the mean difference, and the horizontal bar indicates the 90% confidence interval.

Abbreviations:

2D

two-dimensional

AF700

Alexa Fluor 700

AF750

Alexa Fluor 750

ATCC

American Type Culture Collection

BME

basement membrane extract

CMM

center of mass

DIFF Transformer

differential transformer

DL

deep learning

DMD

digital micro-mirrors device

DOF

depth of field

ELU

exponential linear unit

FFN

feed-forward network

FGS

fluorescence-guided surgery

FLI

fluorescence lifetime imaging

FLI-Net

fluorescence lifetime imaging network

G

number of gates

HCC1954

human breast carcinoma cell line HCC1954

HER2+

human epidermal growth factor receptor 2 positive

IACUC

Institutional Animal Care and Use Committee

ICCD

intensified charge-coupled device

IRF

instrument response function

L2

L2 regularization

MDT-TZM

Meditope trastuzumab

MDT-TZM-AF700

Meditope trastuzumab–Alexa Fluor 700 conjugate

MDT-TZM-AF750

Meditope trastuzumab–Alexa Fluor 750 conjugate

MFLI

macroscopic fluorescence lifetime imaging

mFLI

mesoscopic fluorescence lifetime imaging

MFliNet

macroscopic FLI network

MNIST

Modified National Institute of Standards and Technology database

ND

neutral density

NIR

near-infrared

NLSF

nonlinear least squares fitting

PBS

phosphate-buffered saline

RPMI

Roswell Park Memorial Institute medium

SwiGLU

Swish-Gated Linear Unit

Ti-Sapphire

titanium–sapphire

TPSF

temporal point spread function

UB

urinary bladder

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.