Improving the spatial resolution and signal-to-noise ratio of ultrasound switchable fluorescence imaging

Abstract

Ultrasound switchable fluorescence (USF) imaging, a hybrid imaging technology that combines the advantages of both fluorescence sensitivity and acoustic resolution in centimeter-deep tissue, has great potential for biomedical different applications. A camera-based USF imaging system reveals its capability of capturing both spatial and temporal dynamics of the USF signal in tissue. In this study, various algorithms were explored to enhance the spatial resolution and signal-to-noise ratio (SNR) of USF images, utilizing temporal and spatial information from a camera-based time-domain USF imaging system. The correlation method proved effective in boosting SNR, while the ascending-slope-weighted method enhanced spatial resolution. Additionally, the spatially back-projection method significantly improved spatial resolution in silicone phantoms. The results underscore the advantages of incorporating temporal and spatial information from USF signals.

Graphical Abstract

A camera-based ultrasound-switchable-fluorescence imaging captures both temporal and spatial information of the acoustic-optical signals from tissue, which has the potential to improve the image quality. Compared to a traditional raster scan method, this work explored several algorithms using the information that successfully enhanced the image’s spatial resolution and signal-to-noise ratio.

Introduction

High-resolution fluorescence imaging in centimeters-deep tissue is highly desirable for different biomedical applications and has been actively investigated during past years.^1-8 Ultrasound switchable fluorescence (USF) is one of the techniques for achieving this purpose.^9-11 In USF imaging, an ultrasound beam is tightly focused into centimeters-deep tissue.¹² Within its focal volume, USF contrast agents (such as fluorophore-encapsulated nanoparticles) are thermally (or ultrasonically) switched on and emit fluorescence after absorbing excitation light from a laser. Thus, the generation of these fluorescence emission is confined within the ultrasound focal volume in deep tissue. Once generated, these fluorescence photons will propagate along all the directions via scattering. If a 2D planar camera is positioned above the tissue surface as a detector, the received fluorescence photons will form a temporal and spatial distribution function, although they are generated from the ultrasound focus well below the surface. The detected signal is called the USF signal at that specific location of the ultrasound focus. Currently, the USF signal is simply averaged spatially and temporally. Thus, a single number can be generated representing the USF signal strength at each location of the ultrasound focus. To acquire a 2D or 3D USF image, the ultrasound focus will be scanned 2- or 3-dimensionally in the tissue. Such a mechanism leads to the spatial resolution of USF imaging mainly depends on the ultrasound focal size, which is usually much higher than the spatial resolution of diffused photons based optical imaging methods (such as diffuse optical tomography).

USF imaging as a hybrid imaging technique has been fast developed over the past decade^10-20, which can reveal sub-millimeter structural and functional information in centimeters-deep tissue through its unique fluorescent probes^21-26 as well as imaging system.^16-20 It breaks through optics’ high-scattering low-resolution limit in biological tissue by confining the fluorescence emission in an acoustic focus. Because an acoustic signal scatters much less than an optical signal²⁷, USF achieves a high resolution (hundreds of microns) and a high detection sensitivity (picomole fluorophore concentration). USF has successfully demonstrated in both ex-vivo and in-vivo tissues and has been shown to reach an imaging depth up to 5 cm in chicken breast tissue.²⁰ Because of its good image quality as well as safety and cost-efficiency, this imaging technique shows great potential in many biomedical applications, such as early-stage breast cancer detection and monitoring^10,11,23,24.

In our previous work, USF images have been successfully acquired through camera-based systems.^10,18,19 One work¹⁹ finds the trade-off relationship between USF’s spatial resolution and signal-to-noise ratio (SNR) due to the spread of acoustic-thermal spatial profile as time elapses, captured by the camera’s different exposure and acquisition time. It shows the benefits of adopting a camera instead of a single-element photon detector (such as photomultiplier, PMT), which provides a larger collection aperture thus a higher detection efficiency. The camera also provides the spatial distribution of scattered USF photons escaping out of tissue, which potentially indicates the location of the fluorescent agents relative to the tissue structure. For example, by knowing tissue optical properties and analyzing a 2D Gaussian-distributed scattered photons, it may reveal the depth of the USF emission source in tissue¹⁰. Nevertheless, the previous work simply collected all the USF photons to present the signal intensity and didn’t use the spatial information.

We further speculate that the temporal and spatial information of a USF signal should contain extra information of tissue-optics properties that has not been analyzed in a regular raster-scan method. In this work, we presented several signal and image processing algorithms, which used the temporal and spatial information of USF signals, to improve the image qualities (i.e., the spatial resolution and SNR of USF images). First, we demonstrated that using the temporal information of the USF signals, the SNR of USF images could be improved by a correlation method^11,16. Second, we characterized the shape of the normalized USF signal’s temporal information in three parameters: (1) the ascending slope; (2) the descending slope; and (3) the time-duration-above-half-intensity (TDHI). By adopting an ascending-slope-weighted method, we showed an improvement of the spatial resolution of USF images. Third, using the spatial information of the USF signals, we developed a back-projection algorithm for USF imaging. By adopting this method, the spatial resolution was significantly improved.

Methods

Experiments setup

In this study, we adopted the same system, the ICCD camera time domain USF imaging system developed in our previous work.¹⁹ Multiple objective lens were used for different field of views (FOVs). A 4× objective lens provides a FOV = ~ 3.2 mm; a 2× objective lens provides a FOV = ~ 6.4 mm, and a 1× objective lens provides a FOV = ~ 12.8 mm. The different FOVs were adopted for adapting the size of different silicone phantoms and the size of the USF 2D scattering spots, which aims for an optimized USF photon collection efficiency and spatial resolution. It is worth mentioning that, to apply the algorithms developed in this work for future studies at a larger image-depth such as in-vivo tumor imaging at centimeter-depth, the FOVs should also be enlarged to capture majority of USF photons for signal-collection efficiency and for applying algorithms.

Multiple silicone and tissue phantoms were adopted as imaging targets in this study. The silicone phantom protocol was developed in our previous work.¹⁶ The first one is a silicone phantom embedded with one tube with an inner-diameter I.D. = 0.31 mm and target depth = 6 mm. The second one is a silicone phantom with two adjacent tubes (I.D. = 0.31 mm and target depth = 5 mm), and the distance from the two tube centers is around ~ 800 μm. The third is a silicone phantom with three tubes (I.D. = 0.31 mm) with a shape of a triangle 2D structure, with target depth = 5 mm. It is noted here the silicone phantom adopts the same material as the silicone tube so the tube wall thickness (i.e., the outer diameter) does not impact the USF signal. The last is a piece of porcine muscular tissue with three silicone tubes (I.D. = 0.31 mm) inserted into and arranged with a shape of a triangle 2D structure. The O.D. of the tube is 0.64 mm. It is noted here that despite that the tube wall (i.e., silicone) has different thermal absorption and diffusion coefficients compared to the tissue, the USF signal is directly dependent on the thermal profile of the USF contrast agents, which are confined in the I.D. of the tube. Thus, the I.D. of the tubes are used for USF imaging quality analysis. The thickness of the porcine muscular tissue is 5 mm. The estimated absorption coefficient of the silicone phantom is μ_a = 0.03, and the reduced scattering coefficient μ_s' = 3.5 cm⁻¹. Although the exact values of μ_a and μ_s' of the porcine muscular tissue sample are unknown, they should be larger than those of the silicone phantom and can be used for comparison with the silicone phantoms.

In USF imaging, the focused ultrasound (FU) power was set at a Vpp range 130-160 mV with a power amplifier and an exposure time 400 ms. The CCD camera exposure time was set at 400 ms unless specified and its first signal frame synchronized¹⁹ with the ultrasound exposure time. The USF contrast agent ADP(OH)₂-Bottom encapsulated in 5% pluronic F98 nanoparticle²¹ was adopted and filled in the tubes in the phantom for USF imaging.

Temporal information of a USF signal

The intensity of USF signal changes as a function of time when the ultrasound-induced thermal spatial profile spreads. The temporal variation of USF signal intensity can be used to improve the USF image quality. First, the temporal USF signal has been shown to improve the SNR using a correlation method in our previous work. ¹⁶ This algorithm has been developed for a PMT-based USF imaging system.¹⁶ When a camera-based USF system is used, the SNR should be also improved. When a USF signal is appropriately acquired, the signal intensity will first increase and then decrease in duration from a few to tens of seconds with a unique shape. If noise is acquired, the shape is random and at most time not correlated to the typical shape of a USF signal. Thus, the noise can be effectively suppressed. Here we apply the mask of correlation coefficient (CrC) vector to the raw USF profile through the equations below:

$$CrC=\mid corr(I{(t)}_{signal},I{(t)}_{ref})\mid$$ Equation (1)

$${I(x,y,z)}_{corrected}=I(x,y,z)\ast CrC(x,y,z)$$ Equation (2)

Where ${I(t)}_{signal}$ is any signal/noise acquired at a given $(x,y,z)$, ${I(t)}_{ref}$ is a known USF signal which has a typical temporal shape, $corr()$ is the function of computing the correlation coefficient between two vectors, and $CrC(x,y,z)$ is a spatial matrix of the absolute correlation coefficients in a USF scan.

Second, the shape of a temporal USF signal contains the information regarding the distance between the ultrasound focus and the fluorescence emission source. In the case of a single-tube silicone phantom is adopted, the three circles (in red, orange, and green) in Figure 1 (a) represents the three scan points. The red circle represents on the center of the profile (or tube), the orange one represents some distance away from the center (i.e., the scan point is nearby the half of the maximum of the profile), and the green one represents on the edge of the profile. Figure 1 (b) shows their corresponding temporal USF signals. When the USF signal was acquired at x = 0.00 mm (i.e., the red circle), the USF signal increased fast but decreased slowly, and its duration was the longest. In the opposite, when the USF signal was acquired at x = −0.61 mm (i.e., the green circle), the USF signal increased slowly but decreased fast, and its duration was the shortest. At x = −0.38 mm (i.e., the orange circle), the USF signal had a medium increase, decrease and duration. This indicates that the temporal shape of a USF signal was correlated to the distance between the ultrasound focus (i.e., the scan point x) and the tube’s (i.e., fluorescence emission source) location. Thus, the temporal shape should be utilized to improve the spatial resolution of a USF image.

Figure 1.

(a) Non-correlated USF profile. (b) The normalized USF signals at different x.

We characterized the shape of the normalized USF signal in three parameters: (1) the slope of the ascending signal when t is between 1.8 and 2.4 s (note that 1.8 s is the starting point of the data acquisition); (2) the slope of the descending signal when t is between 3.6 and t = 4.2 s); and (3) the TDHI of the signal, which is used to represent the duration of the USF signal. It is worth mentioning that we selected the very-first slopes for both ascending and descending phases because they should have a highest sensitivity to thermal diffusion in each corresponding phase due to the thermal diffusion property. Thus, selecting the very-first slope can provide the best improvement in spatial resolution through this algorithm. These choices are not specific to the phantom and will be the same principle for a different material. Figure 2 (a)-(c) respectively show the ascending slope, the descending slope, and the TDHI of USF signals as a function of the location of ultrasound focus in the scanning (i.e. x). Correspondingly, Figure 2 (d), (e), and (f) shows the normalized USF profiles after multiplying the USF profile in Fig. 1(a) with data in Fig. 2(a), (b), and (c). Note that during the data processing, a correlation method same as the one used in the reference of ¹⁶ is also applied so that we can eliminate the background noises. The equations are described below:

$${I(x,y,z)}_{corrected}=I(x,y,z)\ast CrC(x,y,z)\ast M(x,y,z)$$ Equation (3)

where $M(x,y,z)$ is spatial mask of the ascending slope, the descending slope, or the TDHI of USF signals, and $CrC(x,y,z)$ is the spatial matrix of the absolute correlation coefficients.

Figure 2.

(a) Ascending slopes of normalized USF signals at different x. (b) Descending slopes of normalized USF signals at different x. (c) TDHIs of normalized USF signals at different x. (d) Ascending-slope-weighted correlated USF profile. (e) Descending-slope-weighted correlated USF profile. (f) TDHI-weighted correlated USF profile.

Compared the width of the USF profile in Figure 1(a) (in which FWHM=0.80 mm), both Figure 2(d) and (f) show a narrower width of the USF profile (FWHM = 0.66 mm and 0.76 mm respectively), while the Figure 2(e) provides a wider USF profile (FWHM = 0.87 mm). Obviously, the ascending-slope-weighted correlated USF profile (i.e., Figure 2(d)) demonstrates the highest improvement in spatial resolution compared with the other two. Accordingly, it will be adopted for a higher spatial resolution imaging in this work and more examples will be demonstrated in the Results section. It is worth mentioning that, the shape-weighted profile (e.g., the ascending slope of USF signals) only weights the shape when the USF signal is presented. When it is a noise, the slope/TDHI is random and thus not meaningful (e.g., Figure 2 (a) where x = [−2.0 mm −1.0 mm] or [1.0 mm 2.0 mm]), in which case, this algorithm is usually combined with the correlation method. In theorem, the ascending-slope-weighted method itself would not improve the SNR of the image.

Spatial information of a USF signal

In a camera-based system, USF signal is usually acquired from a surface (such as the surface of skin or tissue samples or phantoms) and presented as a 2D fluorescence image. The single-tube silicone phantom was first adopted for the USF spatial information analyses. Figure 3 (a) shows the setup diagram of the focus ultrasound (FU) transducer and the silicone phantom. The FU transducer had a scanning step size = 76.2 μm with a total scanning range = 3.962 mm (i.e., the total number of scan points = 53). The red crosses represent the scanning points. The ultrasound focus is initially located at the center of tube (x = 0.00 mm). We analyzed the USF signals acquired at t = 3.0 s at different scan points (note that when t = 3.0 s it provides a close-to-peak USF intensity at an earliest t so the spatial information has a high SNR). Figure 3 (b) shows the USF signals acquired at these scanning points. Note that the camera was positioned at the opposite side of the sample relative to the transducer (see the reference.¹⁹ When the FU transducer was focused on the tube (x = 0.000 mm) or nearby the tube (e.g., x = −0.152 mm), a clear USF signal could be clearly detected. When the FU transducer was far away from the tube (e.g., x = −1.095 mm), no USF signal was detected. When the FU transducer was scanned across the tube for acquiring the USF profile of the tube, the ultrasound transducer was physically moved hundreds of microns, however, the hot spot formed by the USF signal on the 2D fluorescence image was barely moved in the camera’s FOV. This might be due to the following reason. The size of the tube (I.D. 0.31 mm) was smaller than the lateral size of the ultrasound-induced thermal focus (FWHM = ~0.8 mm in this experiment), so that the tube could be viewed as a point source. In other words, when the FU transducer switched on the fluorescence in the tube wherever the transducer was located, the USF signal was always generated from the tube, which did not move along with the transducer. Thus, the USF photons captured in the camera’s FOV did not move along with the transducer. If the phantom has a uniformed distribution of scattering coefficient and an even surface, we can assume the tube is located at the geographical center of the hot spot of the USF signal.

Figure 3.

(a) Set up diagram of the FU transducer and the silicone phantom. The FU transducer had a scanning step size = 76.2 μm with a total scanning range = 3.962 mm (i.e., the total number of scan points = 53). (b) USF signals acquired at these scanning points. 1. Can you move the red crosses from the transducer to the sample, which seems better to understand? 2. Can you have the scale of the images, such as add a grayscale bar. 3. Better to have a diagram figure of the system or to show where the camera is or you can clarify the position of the camera. Otherwise, readers may get confused about how the USF images are acquired.

In fact, the tube’s inner diameter here is not an infinite small point source. Instead, the finite size (I.D. = 0.31 mm) will lead to the back-projected centroids having a finite size along the x axis. Here we adopt the spread of these centroids to represent the tube, so it is called the spatially back-projected USF image. Compared with the traditional USF scan method which adopts the ultrasound focus as the spatial coordinate, this method extracts the spatial coordinates of each scanning based on the hot spot of the USF signal on the 2D fluorescence image. The equations below described how a centroid of USF hot spot was extracted from each camera frame:

$$S(ix,iy)=I(ix,iy)>I_{threshold}$$ Equation (5)

$$cx=mean(ix),cy=mean(iy)$$ Equation (6)

In Eq. (5) $S(ix,iy)$ is the mask of USF hot spot image $I(ix,iy)$ that provides the pixel index $(ix,iy)$ where the signal at each pixel has a recording above a user-defined threshold. Thus, by finding the mean of $(ix,iy)$ we acquire the centroid of the USF spot $(cx,cy)$.

It is worth mentioning that by using a user-defined signal threshold, this is a simple way to find the centroids when scanning through a simple silicone tube target. Intensity normalization of each frame and correlation of a reference hot spot can also help further determine the existence of a USF signal. When a phantom has multiple tubes or structures, some hot spots of USF signals could overlap. Thus, it requires segmentation of these overlapped spots before back-projection. It is difficult to determine the threshold and contour of each individual spot during segmentation. A typical way is to adopt a standard USF hot spot from a single tube as the reference, to determine the separation of overlapped contours. The temporal information of a USF signal in the 2D space would also help in identification of USF signals from multiple targets. More details will be presented in the Results.

Results and discussion

USF imaging of a silicone phantom with a single tube embeded with I.D. = 0.31 and 0.76 mm

We first present the improvements of the USF image of the silicone phantom with a single tube (I.D. = 0.31 mm) embedded, as described in the Methods. Figure 4 (a) shows the USF profiles processed using different methods and positioned vertically for comparison. The regular USF means without any algorithm processing and is similar to the one shown in Figure 1 (a). Its SNR = 72.3, and FWHM = 0.80 mm. Note that the signal strength was presented in log scale for better visualization. All profiles were normalized. The correlated USF represents that the USF signal is processed using the correlation method described in our previous work¹⁶. Its SNR and FWHM are 137.4 and 0.80 mm, respectively. Clearly, compared with the regular USF image, this method can significantly improve SNR but not the spatial resolution. The ascending-slope-weighted USF profile (also see Figure 2 (d)) shows an improved spatial resolution (FWHM = 0.66 mm) compared with those of the regular and correlated USF, meanwhile with the same SNR because the slope-weight doesn’t suppress the noise. Note that the correlation method was also adopted when using this ascending-slope-weighted method. The spatially back-projected USF profile shows 0.19 mm of the full-width-at-one-percent-of-the-maximum (FWPM). Obviously, this method significantly improves the spatial resolution compared with the previous three. In the analysis of the spatial back-projected USF profile, it is reasonable to use its bottom width to represent the measured size as this method is more sensitive to the edge-detection due to that intensity is presented by counts of back-projected centroids rather than the absolute fluorescence signal intensity. The real size of the tube (I.D. = 0.31 mm) was also displayed.

Figure 4.

USF profiles of a single silicone tube processed by different algorithms: (a) I.D. = 0.31 mm and (b) I.D. = 0.76 mm.

From Figure 4 (a), we found that the correlation method improved the SNR of the USF image. Both the ascending-slope-weighted and the back-projection algorithms improved the spatial resolution. Interestingly, the profile size acquired from the back-projection algorithm was even smaller than the real size of the tube. We speculated that this might be caused by a threshold effect. When the ultrasound focus was scanned close to the edge of the tube, only a portion of the fluorophores that were nearby the edge of the tube were switched on to generate the USF signal. The signal intensity might be too weak to form a hot spot with a clear shape to allow the algorithm to back project the centroid (i.e. the back-projection algorithm did not provide an outcome when the signal was below a certain level). Thus, those fluorophores nearby the edge of the tube could not be back-projected, which leads to a narrower width smaller than the width of the real tube. To verify this assumption, we analyzed USF imaging of a silicone phantom embedded with a larger tube with an I.D. of 0.76 mm. The sample configuration of the silicone phantom was kept the same. Similarly, figure 4 (b) shows the corresponding USF imaging profiles processed by different algorithms. The regular USF profile has a SNR = 90.9 and a FWHM = 1.24 mm while the correlated USF profile has a SNR = 320.6 and a FWHM = 1.23 mm. The SNR was improved using the correlation method; however, the spatial resolution was not. The ascending-slope-weighted correlated USF profile showed a FWHM of 1.03 mm that was smaller than those of the first two cases, with the same SNR. Similarly, the spatially back-projected USF profile provided a FWPM of 0.55 mm, which was also smaller than the real tube I.D. (0.76 mm). The real tube I.D. (0.76 mm) was also displayed in Figure 4(b) for comparison. When adopting the temporal information of the USF signals, the ascending-slope-weighted correlated USF profile achieved the best SNR and spatial resolution. When adopting the spatial information of the USF signals, the spatially back-projected USF profile provides the highest spatial resolution. This conclusion was verified by both Figure 4(a) and (b). In this work we adopted FWHM to estimate the USF spatial resolution performance. Since the FWHM is the convolution of a tube I.D. size with the USF spatial resolution, having multiple diameters of the tube provides a better insight about its performance. For example, it is realized that in the back-projection algorithm, the movement of centroids is relatively finite when USF scanning through a small tube I.D. (310 um), thus, the same experiment was conducted on a larger tube I.D. (760 um) to verify the feasibility of this method (Fig. 4 (a) & (b) row 4). In theorem, USF image resolution is determined by the acoustic-thermal resolution and not impacted by the size of the tube, however, we would adopt different tube sizes which accommodates the resolution capability in different experiments and also in different post-processing algorithms.

USF imaging of a silicone phantom embedded with two tubes

When applying the spatially back-projected algorithm in USF imaging, it is a challenge to identify individual hot spots that are overlapped and generated from multiple targets (e.g., multiple tubes). Appropriate segmentation of the overlapped spots is the key to solve this challenge. In this study, we adopted a relatively simple method to demonstrate idea, although other methods may exist. The idea is to generate a reference hot spot from a single and small target (such as a tube with a small I.D. embedded in the phantom). Both the spatial and temporal information of this single small hot spot are used as a basic element (or a reference) to segment (or decompose) the overlapped hot spot into individual spots. To demonstrate the idea, we conducted USF imaging of a silicone phantom embedded with two small tubes (I.D. = 0.31 mm) that were positioned closely, whose positions were confirmed by an ultrasound image with a peak-to-peak distance 0.838 mm. configuration of the silicone phantom was shown in Figure 5 (a), which was similar to the previous except that two tubes were embedded. Its target depth was Z1 = 6 mm (on the FU transducer side), and Z2 = 5 mm (on the camera side). Its μ_s' was 3.5 cm⁻¹ and μ_a 0.03 cm⁻¹. The FU transducer scanned across the tube with a scanning step size of 76.2 μm with a total scanning range of 5.563 mm (i.e., the total number of scan points was 74). The USF imaging results were verified with a B-mode ultrasound image of the two tubes (when filled with air) in the phantom.

Figure 5.

(a) Set up diagram of the FU transducer and the silicone phantom. The FU transducer had a scanning step size = 76.2 μm with a total scanning range = 5.563 mm (i.e., the total number of scan points = 74). (b) USF signals acquired at three scanning points: 1) when FU was focused on the first tube; 2) when FU was focused in between two tubes (x = −0.076 mm); 3) when FU was focused on the second tube. (c) USF imaging profile of the two tubes in the silicone phantom by the spatially back-projection algorithm without spots-segmentation. (d) Temporal information of scattered USF spot presented in 2D. (e) USF imaging profile of the two tubes in the silicone phantom by the spatially back-projection algorithm with spots-segmentation.

Figure 5 (b) shows examples of the USF signals acquired. These images represent the USF signals were acquired at a CCD recording time = 3.0 s (i.e., the second frame after FU signal was sent, with background subtracted). The first image was acquired when FU was focused on the first tube (x = −0.457 mm), the second image was acquired when FU was focused between the two tubes (x = −0.076 mm), and the third image was acquired when the FU was focused on the second tube (x = 0.381 mm). The images clearly showed that the 2D scattering spots moved from the left (where the first tube was located) to the right (where the second tube was located) when the FU scanned across. When the FU was focused in between the two tubes, the USF signal showed two overlapped spots which came from both tubes. If we counted this overlapped USF signal as a single large spot and then found the overall centroid of this large spot using the spatially back-project algorithm, we could still get a USF imaging profile of the two tubes. Figure 5 (c) shows the USF imaging profile of the two tubes by the spatially back-projection algorithm without segmentation.

However, many back-projected centers fall in between the two tubes as shown in Figure 5(c). This is reasonable because the USF signals acquired in between them had signals from both tubes and their 2D scattering spots were overlapped. Without segmentation of the overlapped spots, these back-projected centers did not contribute to each tube but instead fell in between the two tubes. To address this issue, we analyzed the contours of the overlapped spots for the purpose of segmentation. We first used the USF signal acquired when FU was focused on the first tube (i.e., the first image acquired at x = −0.457 mm in Figure 5 (b)) as the reference. Second, we determined the size (diameter) of this reference. In addition, we introduced the temporal information of the USF signal in 2D space. Figure 5 (d) shows an example of one overlapped USF signal acquired at x = −0.076 mm (at t = 3.0 s). This image corresponds to the second image in Figure 5 (b) but it represents a mask of the TDHI (unit: s, as indicated in Figure 1(b)). Although the two spots were spatially overlapped, their signals’ temporal information (such as the TDHI) were different because the FU focus (at x = −0.076 mm) was closer to the first tube (at x = −0.457 mm) than to the second tube (at x = 0.381). Thus, using the difference in temporal information, masked with the standard reference signal, the two fluorescence spots from the two tubes were separated. Thus, each spot could be back-projected to its own emission source. Figure 5 (e) shows USF imaging profile of the two tubes by the spatially back-projection algorithm with an overlapped-spots-segmentation method.

Figure 6 shows the corresponding image profiles processed using different algorithms. In the regular USF profile, the FWHM of the image of the left tube is 0.59 mm and the SNR of the overall profile is 35.2. In the correlated USF profile, the FWHM of the image of the left tube is 0.56 mm and the SNR 88.7. In the ascending-slope-weighted correlated USF profile, the left tube shows a FWHM of 0.36 mm, with the same SNR 88.7. In the spatially back-projected USF profile without adopting the segmentation method. In the spatially back-projected USF profile with the segmentation method, the FWPM is 0.24 mm. Although the two FWPMs for both back-projected USF profiles are similar, the background noise between the two tubes are significantly reduced after being processed by the segmentation method. A B-mode ultrasound image was provided as a gold standard to compare with all the USF images. In the ultrasound image, the two tubes were filled with air to increase the acoustic contrast. The FWHM of the left tube is 0.61 mm. Lastly, the real tube I.D. was also displayed in the figure for comparison. These results clearly showed that that the spatial resolution can be significantly improved by using the spatially back-projected algorithm compared with other methods. With the help from segmentation method, the SNR or background noise can be improved compared with that without segmentation method.

Figure 6.

USF profiles of the two adjacent silicone tubes (I.D. = 0.31 mm) in the silicone phantom, processed by different algorithms, and also ultrasound image when the two tubes were filled with air.

USF imaging of a silicone phantom embedded with multiple tubes in 2D

In this section, we conducted 2D USF imaging of a silicone phantom embedded with multiple tubes in a 2D structure. Figure 7 (a) shows the sample configuration of the tubes from top view. Three silicone tubes (shown as white lines, their I.D. = 0.31 mm) were placed in a 2D triangle structure at the same z position and embedded in the silicone phantom. The tubes’ depths are Z1 = 6 mm (FU transducer side) and Z2 = 5 mm (camera side). The FU was focused on the tube on z direction for USF imaging. The FU scanning step size is 0.381 mm on both x and y direction. The scanning area is x = 4.572 mm (i.e., 13 points) and y = 5.334 mm (i.e., 15 points). The red dots represent the USF scanning points in 2D. In this 2D tube-structure imaging, we have provided 3 gold standard images: (1) micro-CT image; (2) fluorescence image; and (c) ultrasound image (C-mode), to best present the performances of different USF image processing algorithms compared to these gold standards. Figure 7 (b) shows the cross-section top view of a micro-CT image of the silicone phantom when the tubes were filled with the CT contrast agent solution ExiTron nano 12000. The dark dots were the hollow caves caused by the needles that were used to anchor the tubes when making the silicone phantom. The needles were already removed when conducting the CT and USF imaging. Figure 7 (c) represents the 2D planar fluorescence image directly acquired from the surface of the phantom using the camera (no FU applied). The USF contrast agent solution was injected in all the three tubes. It shows the fluorescence was diffusive and three tubes could not be clearly separated due to the photon scattering caused by the phantom. Figure 7 (d) represents the ultrasound image of the three tubes when they were filled with air. The same FU transducer was adopted for ultrasound imaging. The ultrasound image was normalized and presented in greyscale bar. The three the hollow caves left by the needles is also slightly visible in the ultrasound image.

Figure 7.

(a) Sample configuration of the silicone phantom. (b) Cross-section top view of micro-CT image of the silicone phantom when the tubes were filled with the CT contrast agent solution (ExiTron nano 12000). The dark dots were the hollow caves caused by the needles that were used to anchor the tubes when making the silicone phantom. The needles were already removed after the phantom was solidified. (c) Diffusive fluorescence image detected from the surface of the phantom by the camera. (d) Ultrasound image of the three tubes when they were filled with air. (e) – (h) USF images of the three silicone tubes (I.D. = 0.31 mm) processed using different algorithms.

In this experiment the camera exposure time was 200 ms. Figure 7 (e) represents the regular USF image of the phantom. If we count the lower left corner area of the image (4 × 4 pixels) as the background, the SNR of this image is 78.7. Figure 7 (f) represents the correlated USF image. Its SNR is 91.7. Comparing Figure 7 (e) and (f), it shows the correlation method improve the SNR of the USF image but its spatial resolution remains similar. Figure 7 (g) represents the ascending-slope-weighted correlated USF image. In Figure 7 (e)-(g), it is difficult to quantify the spatial resolution of a single tube due to the close alignment of the three tubes as well as relatively a coarse scan step size, thus we didn’t provide FWHM for this experiment as well as for the next experiment (Figure 8). Regardless, it shows that in Figure 7 (g) the spatial resolution is visually slightly better than the previous two. Figure 7 (h) represents the spatially back-projected USF image with spots-segmentation algorithm applied. This image provides a best spatial resolution and a good separation of the tubes in visualization. Regarding the three hollows in the phantom structure, although they are seen in both micro-CT image and ultrasound-image, they are not seen at any of the USF images (or the fluorescence image) because there are no USF contrast agent solutions filled in these hollows. Only the tubes were filled with the solutions thus the fluorescent signals acquired can only come from the tubes. This is a special advantage of USF imaging because it carries high detection sensitivity.

Figure 8.

(a) Sample configuration of the porcine muscular tissue phantom. The three silicone tubes embedded in the phantom has a 2D triangle structure and their I.D. = 0.31 mm. (b) Diffusive fluorescence image detected from the surface of the phantom by the camera. (c) – (f) USF images of the three silicone tubes in the tissue phantom, processed by different algorithms.

USF imaging of a tissue phantom embedded with multiple tubes

In this section, we conducted 2D USF imaging of a porcine muscular tissue phantom attached with multiple silicone tubes in a 2D structure. Figure 8 (a) shows the sample configuration of the phantom. Three silicone tubes (shown as white lines, their I.D. = 0.31 mm) were placed in a 2D triangle structure at the same z position and embedded on top surface of a silicone phantom (μ_s' = 3.5 cm⁻¹, μ_a = 0.03 cm⁻¹.). A piece of porcine muscular tissue with a thickness of 5 mm was attached to the silicone phantom on the camera side. Thus, the fluorescence photons from the tube scattered into the tissue until they were detected by the camera on the other surface of the tissue. On the FU transducer side, the silicone has a thickness of 6 mm. The porcine muscular tissue had a higher scattering coefficient and the USF photons should be more significantly scattered. We adopted a 1× objective lens (FOV = ~ 12.8 mm) for fluorescence imaging and USF imaging. In this experiment, we only include diffusive fluorescence image as the reference image, which can be acquired directly from the USF imaging setup. Because the three-tube-structure may deform when the sample was transported to a separate imaging setup, we did not provide micro-CT image or ultrasound image here. Figure 8 (b) shows the diffusive fluorescence image of the phantom (on porcine tissue side) when the three tubes were filled with the USF contrast agents. Compare to Figure 8 (c), it shows the porcine tissue was a more scattering medium (with the same thickness = 5 mm) and the fluorescence from the three tubes could not be separated from each other totally.

In USF imaging, the FU power was Vpp = 130 mV, and the FU exposure time was 400 ms. The CCD camera exposure time was 100 ms in this experiment. All other experimental parameters were kept the same as previous. Figure 8 (c) represents the regular USF image of the phantom. We used the upper right corner area of the image (4 × 1 pixels) as the background, the SNR of this image is 70.4. Figure 8 (d) represents the correlated USF image. Its SNR is 76.9, which is slightly improved comparing to that of Figure 8 (c). Meanwhile, its spatial resolution remains similar. Figure 8 (e) shows the ascending-slope-weighted correlated USF image. It shows the spatial resolution is better than the previous two. Figure 8 (f) is the spatially back-projected USF image and the structure of the three tubes is visible but with relatively lower SNR than the one in Figure 7(h). This is mainly due to that the porcine muscular tissue scatters more than the silicone phantom, and USF photons from different targets were mixed more significantly, which reduces the accuracy to separate them in space and time. In this work we did not characterize the SNR of all the images using the back-projected method, but it is valuable to discuss about its challenge here. The back-projected USF adopts a new image processing method that does not directly use the raster scan points of the ultrasound for image representation, but instead uses the spatial information of USF signals captured in the camera FOV, and it adopts the statistical counts of the back-projected USF signals. Because of this, the more counts collected, a better USF image shall be presented. However, it is a challenge to quantify the SNR in this method due to an unknown noise level. In the experiments, by selecting a user-defined threshold, all USF signals are segmented from the background. The background noise in theorem is zero unless when a background fluorescence mis-segmented and mis-counted, which makes it hard to determine. Due to a limited scan number and limited counts of USF signals in this work, we haven’t developed an appropriate metric for the SNR of this method, although from all the images we see a good representation of the target and conceptually a good SNR. Future work will be conducted for development of this characterization.

Conclusions

In this study we investigated several algorithms to improve USF image’s spatial resolution and SNR, based on the temporal and spatial information of the USF signals acquired in the camera-based time-domain USF imaging system. The results show that the correlation method could improve the SNR of a USF image and the ascending-slope-weighted method could improve the spatial resolution of a USF image. The results also show that the spatially back-projection method provides a significantly improved spatial resolution of a USF image in the silicone phantoms. In these algorithms, it is necessary to acquire a complete USF signal over time for sufficient temporal and spatial information. It is worth mentioning that, unlike the previous work ¹⁹ which considers spatial resolution and SNR trade-offs and adjusts the experimental settings for a better image quality, the algorithms developed in this work are simply post-processing methods, which provides additional improvements of a USF image.

Although characterization of SNR in the back-projection method remains a challenge due to an unknown noise level based on the method’s accuracy, the back-projected USF images show a visually high SNR. The accuracy of using spatial information of scattered USF photons in the back-project algorithm also depends on the heterogeneity of tissue and the roughness of surface. This algorithm may not be sufficient in practice, as implied in Figure 8 (f). Despite this limitation, the results show the benefits of adopting temporal and spatial information of USF signal. To fully take advantage of these temporal and spatial information at each ultrasound scanning position, it may be necessary to develop a reconstruction model to calculate the dynamics of the USF signals, similar to tomographic technology. The models may involve excitation light propagation diffusion, ultrasound heating and thermal diffusion, contrast agents switching, and emission propagation diffusion.