Gaussian Light Model in Brightfield Optical Projection Tomography

Abstract

This study focuses on improving the reconstruction process of the brightfield optical projection tomography (OPT). OPT is often described as the optical equivalent of X-ray computed tomography, but based on visible light. The detection optics used to collect light in OPT focus on a certain distance and induce blurring in those features out of focus. However, the conventionally used inverse Radon transform assumes an absolute focus throughout the propagation axis. In this study, we model the focusing properties of the detection by coupling Gaussian beam model (GBM) with the Radon transform. The GBM enables the construction of a projection operator that includes modeling of the blurring caused by the light beam. We also introduce the concept of a stretched GBM (SGBM) in which the Gaussian beam is scaled in order to avoid the modeling errors related to the determination of the focal plane. Furthermore, a thresholding approach is used to compress memory usage. We tested the GBM and SGBM approaches using simulated and experimental data in mono- and multifocal modes. When compared with the traditionally used filtered backprojection algorithm, the iteratively computed reconstructions, including the Gaussian models GBM and SGBM, provided smoother images with higher contrast.

Introduction

The imaging of samples in the mesoscopic size range of few millimeters to a centimeter is important for understanding the processes of a biological system, such as embroyos, organs or organoids of a small animal. One prominent imaging technique for imaging these mesoscopic samples natively in 3D is optical projection tomography^1,2 (OPT), which has been successfully applied in the examination of zebrafish^3–5, gene expression^6–8, small organs^9–12, tissue structures^13–16, cells^17–19 and hydrogels^20,21. OPT can be used also to complement selective plane illumination microscopy^3,22. To further enhance the information content acquired from OPT, the focus of this study is the inverse problem of reconstructing the sample from the image data.

In brightfield OPT, a wide-spectrum light beam propagates through a semi-transparent sample and projection images (shadowgrams) are recorded from a number of angles around the sample. OPT could be described as the optical equivalent of X-ray computed tomography (CT), and the associated analytic inversion algorithm of X-ray CT, filtered backprojection (FBP), has been used to reconstruct brightfield OPT images with reasonable success. There is, however, one major difference between X-ray CT and OPT concerning the detected ray geometry. The X-rays that penetrate the sample are considered to be straight lines but, in contrast, the focusing objectives in the OPT detection path shape the light beam to a Gaussian bell-shaped intensity profile^23–25. The Gaussian shape has a certain focal distance, and in the projection images, blurring is induced into particles depending on their distance from the focus. This blurring is further seen in a decrease of radial resolution in the reconstructions. Hence, this paper focuses on incorporating the Gaussian shape, i.e., the Gaussian beam model (GBM), into the forward model of brightfield OPT with the aim of increasing the reconstruction accuracy through inversion of this enhanced model. The combined forward model is a nested convolution of type $y=ℛ\circ {ℬ}_K(\cdot )$ where inner functional ${ℬ}_K$ describes the blurring. In standard Radon transform there is no blurring and ${ℬ}_K=\mathrm{id}$.

The blurring properties of the objective can be parametrized using numerical aperture (NA)²⁶. The NA describes a trade-off between lateral resolution and longitudinal resolution and is directly related to the Rayleigh range of the imaging system, which describes the elongation of the Gaussian shape. In our analysis, blurring is considered to be a continuous function, symmetric to both sides of the focal distance, and GBM is represented by equation $K(\tau ,z_L)=\sqrt{2/(\pi w(z_L))}\exp (-2{\tau }^2/w(z_L))$ with w(z_L) = w₀(1 + ((z_L − z₀)/z_r)²) where NA and Rayleigh range are related through NA = w₀/z_r. This approach does not consider the focality as a discrete value as commonly used depth-of-field (defined as twice the Rayleigh range¹). Traditionally, objectives with low NA are used in OPT to cover at least half of the sample into depth-of-field at once. Long depth-of-field attempts to create conditions as close as possible for the assumed detection of straight lines, but also limits the lateral resolution.

Previously, decreasing the effects of the Gaussian beam shape have also been studied using image post-processing^2,27,28, Fourier beam propagation with regularized inversion²⁹, and multifocal acquisition^19,30–32. In multifocal acquisition, several projection images are taken from each imaging angle and then fused into a single image. The fusion algorithm is chosen so that the focal parts should be preserved in the fused image. A practical implementation of multifocal acquisition is helical scanning^3,33. In this study, multifocal acquisition is considered and compared with the results of the GBM.

Regarding fluorescence OPT, which detects the emission of excited fluorescent molecules instead of attenuation shadowgrams, several approaches to counter the blurring of particles in the reconstruction have been published. These include modified FBP and probabilistic inversion with weights based on light properties^34,35. In addition, research on fluorescent OPT reconstruction problem includes the deconvolution of reconstructions using point spread function³⁶ and adaptation of fluorescent X-ray reconstruction method³⁷. Furthermore, the GBM has already been successfully applied to fluorescence OPT³⁸. In contrast to brightfield OPT, which is the transmission of light through a domain, fluorescence is an emission-sources inside a domain problem, and therefore motivating a validation study in the brightfield case.

When coupled with the Radon transform, the GBM enables both the focal and blurred parts of a detected light beam to be included into a single projection operator³⁸. As a result, the projection data can be used for an extended angular aperture. Consequently, the GBM can be expected to improve the reconstruction quality when compared with the inversion of the plain Radon transform. The incorporation of the GBM, however, leads to increased memory consumption and longer computation times during the inversion stage. Furthermore, as a deblurring method, the GBM-based inversion approach can be sensitive to measurement inaccuracies, such as focusing errors.

To decrease the consumption of memory, we used a thresholding approach where low values of beam intensity profile are set to zero based on the on-axis value along the propagation, i.e., the profile is enveloped. Furthermore, we introduce the concept of a stretched Gaussian beam model (SGBM), where the Gaussian beam is scaled in order to avoid the modeling errors related to the error of positioning the sample’s center of rotation in the focal plane. Scaling is achieved by multiplying the Rayleigh range z_r with a constant we call strecthing constant c_S, i.e., $z_{\mathrm{r}}\mapsto c_S{z}_{\mathrm{r}}$, while keeping w₀ in its original value. Analysis on choosing c_S is discussed in the Methods section. Another essential computational improvement in using the SGBM approach is that it allows a significant reduction in the memory consumption when compared with the GBM due to the faster decay of the off-axis values. For computing a reconstruction with the GBM or SGBM, we used a steepest descending minimization method that has already been described in our previous work³⁹. The minimization is regularized with total variation (TV)⁴⁰ to increase the robustness of the computation. The use of the GBM and SGBM in the reconstruction process was tested in simulated and experimental studies using both mono- and multifocal projection modes. The reconstructions are compared with standard FBP reconstructions.

Figure 1 illustrates the different properties of GBM and SGBM imaging in 2D plane. Depicted with a dashed line, the ideal shape of detected GBM light has a Gaussian shape, centered at the focal point. In practice, however, plain hydrogel or cell culturing samples have no known point in the center to focus on. For this reason, it is likely that even an experienced technician would be unable to place the sample center into the focal point, and thus induces a modeling error in the direction of propagation, depicted in the figure with a solid black line. Misplacement orthogonal to propagation can be corrected easier during the imaging⁴¹ or by processing projection data before reconstruction^42–45. However, the light model requires center of rotation to be at focal point in both coordinates. An SGBM beam (marked in the figure with gray solid color) can be interpreted in two ways: firstly, the physical relation is with Rayleigh limit, which is elongated without altering the beam width at focal point; and secondly, in a numerical sense, GBM and SGBM can be thought of as a weighting of the Radon transform beam, where the SGBM is thus a balance of Gaussian details and the plain Radon transform.

Figure 1.

Illustration of GBM light propagation along axis z_L. Dashed line: a GBM beam with focus at z_L = 0. Solid black line: a true Gaussian imaging beam with unknown focal offset. Gray line: SGBM beam, centered at z_L = 0, has Gaussian form and the same beam waist as GBM, but does not underweigh the values at z_L = 0 as much as misplaced GBM.

To analyze the performance of GBM and SGBM, we used both simulated and experimentally acquired data. In the simulations, we computed monofocal data with different focal offsets and also multifocal data. Reconstructions were computed with standard FBP, and iterative solution TV with GBM and SGBM, where different strecthing constant were applied when using SGBM. The experimental data was imaged from two samples: one including sparse population of beads and the other a zebrafish embryo which is a large continous object. In both samples, the base volume was a hydrogel suspension. Reconstructions from experimental data were computed with FBP, GBP and SGBM with stretching constant 5. In both simulated and experimental multifocal cases, the multifocal projections were flattened into single all-in-focus projections before reconstructing. The experiments in this work are summarized in Table 1 and labeled with letteres (A)–(X). In total, we present ten reconstructions from monofocal, simulated data, (A)–(J); four reconstructions from multifocal, simulated data, (K)–(N); four reconstructions from monofocal zebrafish data, (O)–(R); and five reconstructions from multifocal bead data, (S)–(X), one of which, (X), is monofocal by using only one focal plane.

Table 1.

The identifiers and descriptions of the reconstructions.

ID	Focal Offset (s) (μm)	Algorithm	Gaussian model	β 1	β 2	Data acquisition mode
(A)	0	FBP				monofocal, synthetic
(B)	0	TV	GBM: K1	1E-10	1E-10
(C)	75	FBP
(D)	75	TV	GBM: K1	1E-10	1E-10
(E)	225	FBP
(F)	225	TV	GBM: K1	1E-8	1E-10
(G)	225	TV	SGBM: K5	1E-8	1E-10
(H)	225	TV	SGBM: K10	1E-8	1E-10
(I)	225	TV	SGBM: K20	1E-8	1E-10
(J)	225	TV	SGBM: Kδ	1E-8	1E-10
(K)	−300, 0, 300	FBP				multifocal, synthetic
(L)	−300, 0, 300	TV	GBM*: $K_1^{⁎}$	1E-10	1E-10
(M)	−300, 0, 300	TV	SGBM: K5	1E-10	1E-10
(N)	−300, 0, 300	TV	SGBM: Kδ	1E-10	1E-10
(O)	Not specified	FBP				monofocal, zebrafish embryo
(P)	Not specified	TV	GBM: K1	1E-8	1E-10
(Q)	Not specified	TV	SGBM: K5	1E-8	1E-10
(R)	Not specified	TV	SGBM: Kδ	1E-8	1E-10
(S)	Not specified	FBP				multifocal, beads
(T)	Not specified	TV	GBM*: $K_1^{⁎}$	1E-8	1E-10
(U)	Not specified	TV	GBM*: K5	1E-8	1E-10
(V)	Not specified	TV	GBM*: Kδ	1E-8	1E-10
(X)	Not specified	FBP				monofocal, beads

Results

Results with simulated, mono- and multifocal data

Concerning the simulated experiments (A)–(J), we concentrated on the effect of focal offset, i.e., the unknown distance of focal point from the center of the sample. The effect was analyzed by simulating projection data with different offset and comparing the reconstructions of inverse methods that assumed no offset in the projection data. The forward model in simulations was a GBM without the hard thresholding of the beam.

The relative error measure (REM) and total variation error (TVE) values of (A)–(J) are presented in Table 2 and visualized in Fig. 2. The reconstructions (A)–(J) are shown in Fig. 3. With respect to REM, FBP was found to perform generally better than GBM. In both FBP and GBM, the reconstructed particles were blurred tangentially corresponding with the projection directions parallel to the light beam. The inversion errors increased as the offset was increased from 0 μm to 75 μm and further to 225 μm. For these offsets, the REM of GBM was 1.6 and 2.6 times that of the no-offset case, and the REM of FBP was 1.3 and 4.9, respectively. TVE, on the contrary, did not provide a clear difference between the monofocal approaches. As the focal offset was increased, the overall contrast of the reconstruction and tangential blurring decreased, but FBP yielded more prominent streaking artifacts as seen in cases (A) to (C) to (E). Using the SGBM, a trade-off between roundness and smoothing was achieved. The REM decreased, and the contrast of the reconstructions increased along with the stretching, as seen in Table 2. However, the shape of the particles seems to have been slightly better maintained with moderate stretching.

Table 2.

The numerical errors of the reconstructions (A)–(J) obtained with the numerical phantom and simulated Data.

ID	Focal offset	Inversion kernel	Dynamic range		REM (%)	TVE	Data acquisition mode
			min	max
(A)	0 μm	FBP	−0.01	0.82	180	323.0	monofocal simulations
(B)	0 μm	K 1	−0.07	1.13	440	778.1
(C)	75 μm	FBP	−0.01	0.33	229	323.0
(D)	75 μm	K 1	−0.05	0.42	723	774.3
(E)	225 μm	FBP	−0.04	0.25	888	323.0
(F)	225 μm	K 1	−0.00	0.04	1157	387.7
(G)	225 μm	K 5	0.00	0.07	721	403.2
(H)	225 μm	K 10	0.00	0.08	652	405.5
(I)	225 μm	K 20	0.00	0.08	622	406.6
(J)	225 μm	K δ	0.01	0.13	463	432.0
(K)		FBP	0.01	0.29	246	323.0	multifocal simulations
(L)		$K_1^{⁎}$	−0.03	0.46	404	806.8
(M)		K 5	−0.02	0.38	428	754.4
(N)		K δ	−0.02	0.37	350	862.8

Figure 2.

Scatter plot of (a) REM and (b) TVE values of the reconstructions (A)-(N); and (c) a bar chart showing the respective dynamical ranges. The REM values of the SGBM reconstructions are lower than GBM K₁ and TVE values at the same level. Controversially, with multifocal data, REM values are relatively low but TVE values high.

Figure 3.

The reconstructions (A)–(J) obtained with the numerical phantom and simulated monofocal data. The region of interest shown below each reconstruction (rows 2 and 4) is from the bottom left part of the sample. Focal offset is 0 μm in (A) and (B), 75 μm in (C) and (D) and 225 μm in (E)–(J). The reconstructions (K)–(N) obtained with the numerical phantom and simulated multifocal data. The region of interest shown below each reconstruction is from the bottom left part of the sample. The scalebar length is 500 μm in rows 1, 3 and 5 and 50 μm in rows 2, 4 and 6.

In the multifocal simulations, the reconstructions (K)–(N) presented in Fig. 3 had particles with less elongation and with less prominent streaking. In terms of the REM values presented in Table 2, the reconstruction approaches GBM^* (L), SGBM (M) and (N) and FBP (K) were all roughly of the same order, although FBP provided the lowest REM and GBM^* the best contrast. Moreover, compared with the monofocal REM values, multifocal reconstructions were on the lower end of the error value scale. The TVE values, however, were relatively high for reconstructions from multifocal data.

Results with experimental, monofocal data from the zebrafish

The reconstructions (O)–(R) of the experimental monofocal data from the zebrafish embryo are presented in Fig. 4. As shown by the visual analysis of region of interest, the GBM reconstruction (P) provided the best contrast, but was smoother than any of the other methods. In the case of SGMB using the kernel K₅ (Q), a balance between tangential blurring was achieved, but with a subsequent decrease in contrast. FBP (O) and the Radon kernel K_δ SGBM (R) provided very similar reconstructions with the sharpest details of (O)–(R), but with also imminent streaking and halo artifacts.

Figure 4.

The reconstructions (O)–(R) obtained with the experimental zebrafish embryo sample and monofocal data and limited to the regions of interest. The location of the region of interest are marked in the top-left image. Scale bar in upper row is 250 μm and in lower row 25 μm. Region of interest has a width and height of 250 μm.

Results with experimental, multifocal data from the bead sample

In Fig. 5, the multifocal (S)–(V) and monofocal (X) reconstructions are compared using the experimental data of the bead sample. Consistent with the above reported results of multifocal simulations, the multifocal data provided more details in the reconstructions in all cases. When compared with the monofocal FBP reconstruction (X), some particles were reconstructed only from the multifocal data. SGBM reconstruction (U) with the kernel K₅ provided the same effect of trade-off as in monofocal case. Reconstruction (U) is sharper than blurry GBM* (T), and also shows less streaking and halo artifacts around the particles. Additionally, (U) has a smoother background compared with the Radon inversions FBP (S) and K_δ SGBM (V).

Figure 5.

The reconstructions (S)–(V) obtained with the experimental bead phantom and multifocal data. (X) For comparison, the reconstruction obtained with the experimental bead phantom and monofocal data. The regions of interest are marked in image (S) and have a width and height of 500 μm. The scale bar length in row 1 is 500 μm and 50 μm in rows 2 and 3.

Computational times and memory consumption

Computing one reconstruction of the TV-regularized iteration using a single processor thread took approximately 7 minutes CPU time and required 9.7 GB of RAM. With parallel computing using the Lenovo P910 workstation, computing times were in the order of 10 to 60 seconds, mainly depending on the used kernel. Using the present hard-thresholded SGBM approach, the memory consumption during the inversion stage is 5 to 15 times that of the plain inverse Radon transform. However, with GBM, the relative system size is 60 to 100 times that of the plain inverse Radon transform.

Discussion

This study focused on the mathematical modeling of the brightfield version of optical projection tomography (OPT). We introduced the stretched Gaussian beam model (SGBM) and investigated its effect on imaging accuracy and artifacts. SGBM uses an elongated blurring kernel compared with the conventional Gaussian beam model (GBM) in order to provide more robustness with respect to forward errors and measurement noise. Additionally, similar to GBM, it also enables the exploitation of a larger projection aperture than the classical model of one-dimensional light beams. SGBM was evaluated with both numerical and experimental data as well as for both monofocal and multifocal measurements. As reconstruction techniques, we used total variation regularized (TV) iterative solutions for GBM and SGBM. We compared these reconstructions with filtered backprojection (FBP) computed reconstructions.

SGBM was found to be an efficient and robust surrogate for the GBM with significantly lower memory consumption, which enabled computations for larger image resolutions. The SGBM with the kernel K₅ seemed to provide a sought-after trade-off between the image sharpness and the reconstructed object shape, when the forward modeling errors were large. This was observed to be the case for the large 225 μm focal offset in numerical experiments (E)–(J) and, especially, with the experimental data of the bead sample for which the reconstruction (T) based on GBM were overly smooth. Based on the region of interest comparisons, the streaking artifacts and halo effects were reduced in the SGBM reconstructions (Q) and (U) using K₅ when compared with the respective FBP reconstructions (O), (S) and (X) or K_δ reconstructions (R) and (V). Preliminary tests with alternative regularization parameter values suggested that some artifacts could also be diminished by changing the regularization parameter values, but the major structural differences observed between the methods remained the same.

In addition to the gains in computational efficiency, a further motivation to use SGBM instead of GBM is the uncertainty related to the model parameters. In practice, the accurate positioning of the focal plane to the center of the sample is difficult if there is no reference point in the center of the sample. In this study, we tested several values of stretching factor c_S and of the focal offset. The results indicate, that the choice of c_S seems to be stable as the difference between K₅, K₁₀ and K₂₀ was quite small in simulations in terms of relative error measure (REM) and total variation error (TVE).

The monochromatic light beam assumption used in this study included some errors because the Rayleigh range varies according to the wavelength of the light. We used only a single wavelength of 600 nm in our computations, but in reality, the white LED light has a spectrum in the order of 400 to 750 nm, which affects the Rayleigh length by up to ±25%. The GBM might benefit from having a spectral kernel, but in the case of SGBM, the elongation of the kernel already decreases the effect of spectral mismatch.

In multifocal imaging, a higher numerical aperture (NA) objective can be used to decrease the focal spot size, and thereby improve resolution close to the focal spot³⁰. The cost of this is an increased overall level of blurring. However, to tackle the blurring, several methods can be employed to create a composite image with only focused details. In this study, we applied averaging which we consider a valid, although not necessarily the best, approach for multifocal data^32,46,47. Alternatively, a so-called all-in-focus projection image can be formed by combining the focal parts of multiple projection images for a single angle and different focal plane distances. The final reconstruction can then be obtained via a standard inversion approach, such as FBP. Examples of all-in-focus fusion methods include^3,48,49. In this study, we applied multifocal imaging without changing the objective, and hence the same NA throughout. The reconstructions computed from multifocal data provided finer details than reconstructions from monofocal data in all cases, even with the simplifying assumptions regarding creation of the composite image and the choice of objective.

A recent study compared several reconstruction techniques in fluorescence OPT⁵⁰. They concluded that a reconstruction method based on point spread function analysis, which GBM and SGBM are, is good for nonsparse objects, such as the zebrafish embryo. On the other hand, for sparse objects, such as the bead sample, a different approach based on deconvolution produced better results. Our results seem to indicate that same might hold in brightfield OPT as well: visually more benefits are seen using GBM or SGBM with the zebrafish embryo when compared with the bead sample.

For practical applications, further quantitative verification is required and also motivated by the results of this study. The regularization weights in our reconstruction were minimal to make the effects of GBM and SGBM visible. In practice, moderately higher regularization should be applied, especially with the SGBM since visible streaking artifacts and halos were present in the TV reconstructions. The artifacts occurred because the optical and linear beam projections for a given image detail did not match in the directions where optical the beam was out of focus. Therefore, without an enhanced beam model, the OPT has a similarity to a limited-angle or sparse data imaging problem (see, e.g.³⁹). Consequently, in addition to the regularization, the applied beam model was found to be essential to reduce the artifacts. The beam model might also turn out a valuable addition for upgrades of existing instrumentation, such as presented in⁵¹, where non-telecentric performance of low magnification lenses may turn out problematic. Furthermore, implementation of the blurring kernel in three dimensions would be a natural continuation of this work. This might, however, be difficult due to the already relatively large 2D system sizes obtained in this study. For example, inversion of the multifocal model with three different focusing kernels was not possible due to memory constraints. Therefore, it will be necessary to develop methods other than hard thresholding to compress the kernel.

Using different NA objectives, also the balance between actual lateral imaging accuracy and numerical compensation of the longitudinal resolution should be studied. To date, mostly low NA objectives with higher longitudinal resolution have been used. However, appropriate modeling, as shown here, will enable the use of higher NA objectives and take advantage of their higher lateral resolution.

Conclusions

We have reported the use of Gaussian light beam model (GBM) in the reconstruction process of brightfield OPT. Using simulations, we studied the effect of focal offset, i.e., the difference of rotational center to the focal plane along the propagation axis. We found that the model was more robust to the focal offset via stretching the Rayleigh range artificially. This stretched Gaussian beam model (SGBM) was further tested with experimental data where, similarly to the simulations, the standard GBM produced overly smooth reconstructions while SGBM was able to balance between sharp details and a lower number of streaking artifacts.

Materials and Methods

Radon transform and blurring functional

The standard Radon transform⁵² $ℛ:{ℝ}^2\to ℝ$ is a line integral of a target function f along a given line L. Points (x, y) along L can be parametrized as

$$\begin{array}{ccc}x(z_L)& =& z_L\sin \alpha +s\cos \alpha \\ y(z_L)& =& -z_L\cos \alpha +s\sin \alpha \end{array}$$ 1

where s is the distance of L from the origo, α is the angle between normal n of L and x-axis, and z_L is the arclength along L. The geometry of the Radon transform is presented in Fig. 6(a). With respect to the pair (α, s), the Radon transform of the function $g:{ℝ}^2\to ℝ$ is given by

$$ℛg(\alpha ,s)={\int }_{-\infty }^{\infty }f(x(z_L),y(z_L))d{z}_L.$$ 2

Figure 6.

Schematic images for the Radon transform and the related Gaussian beam model (GBM). (a) The Radon transform integrates its target distribution along the line L. (b) The GBM describes a blurring process corresponding to an integral along the line L_K perpendicular to L. The hourglass-shaped contours illustrate the blurring kernel of the Gaussian beam.

The function f typically represents the image to be reconstructed. In this study, we consider f to be a blurred image of the form $f\mapsto {ℬ}_{K}f$, where $ℬ:{ℝ}^2\times ℝ\to {ℝ}^2$ is a linear blurring operator following from the GBM, similarly as already introduced to florescence OPT³⁸. The source space of ${ℬ}_K$ is composed by the Cartesian product ${ℝ}^2\times ℝ$ between the image space and the viewing angle. The operator ${ℬ}_K$ is determined by a parameter we call blurring kernel $K:{ℝ}^2\to ℝ$, i.e., a function from the image space to a light intensity distribution. Illustrated in Fig. 6(b), the kernel is an axi-symmetric function in the coordinates (x_L, z_L) in which x_L is the distance to L. The kernel can also be interpreted as the shape of the individual light beams L propagating through sample. The focal spot of the beam, z_L, is the position of the narrowest part of L. The blurring increases along with the value of |z_L|.

In Fig. 6(b), the blurring kernel and the Radon transform are superimposed. The line L_K is orthogonal to L, and it intersects L at z_L. In the coordinates (1), the blurred image $g={ℬ}_{K}f$ is given by

$$({ℬ}_{K}f)(\alpha ,s,z_L)={\int }_{-\infty }^{\infty }K(\tau ,z_L)f(x_K(s-\tau ),y_K(s-\tau ))d\tau ,$$ 3

where (x_K(z_K), y_K(z_K)) are the coordinates along line L_K and can be expressed similar to (1) as follows:

$$\begin{array}{ccc}{x}_K(z_K)& =& z_K\cos \alpha +z_K\sin \alpha \\ y_K(z_K)& =& -z_K\sin \alpha +z_K\cos \alpha .\end{array}$$ 4

The blurring operator described in Eq. (3) is a convolution between the blurring kernel K and the image f computed at the point (x(z_L), y(z_L)). In other words, for each point in line L, the image function f is convoluted orthogonally along the line L_K with respect to the blurring kernel K.

To have a physical meaning, the blurring kernel $K(\tau ,z)\in {ℝ}^2$. corresponds to the normalized intensity profile of an axi-symmetric Gaussian beam^23,24 and it is given by

$$K(\tau ,z_L)=\sqrt{\frac{2}{\pi w(z_L)}}\exp \left(-\frac{2{\tau }^2}{w(z_L)}\right)$$ 5

with

$$w(z_L)=w_0\left(1+{\left(\frac{z_L-z_0}{z_{\mathrm{r}}}\right)}^2\right).$$ 6

here, z_L is the axis of the propagation and τ is the radial distance from the z_L-axis. The focal spot is placed at z₀, and w₀ is the width of the beam waist, i.e., the beam width at the focal spot. The Rayleigh range is z_r = πw₀²/λ, where λ is the wavelength of the beam. The Rayleigh range and numerical aperture (NA) of the objective are related through NA = w₀/z_r. NA describes the focusing properties of the objective.

Stretched gaussian beam model

To reduce the forward modeling errors related to practical detection of the focal plane, we introduce the stretched Gaussian beam model (SGBM) in which the Rayleigh range z_r is stretched, i.e., substituted with an expanded value z_r → c_Sz_r with c_S ≥ 1.

The SGBM can be interpreted as a surrogate forward approach in which the beam width is narrower when compared to the exact GBM. The motivation for using the SGBM is the ill-conditioned nature of the image deblurring task, because the SGBM allows the use of beam specific parameters in the forward model while ensuring that errors, such as the focal offset, will not be amplified in the inversion process.

The blurring kernel stretched by the factor c_S is denoted with $K_{c_S}$. For c_S = 1, the beam is non-stretched, i.e., K₁ = K. Furthermore, for c_S → ∞ the beam approaches an intensity profile that is constant in the direction of propagation. If w₀ → 0 simultaneously with c_S → ∞, then K_∞ → δ with δ denoting the Dirac’s delta⁵³. We denote the case K = δ with K_δ and it corresponds to no blurring, i.e., the standard Radon transform.

As previously mentioned and illustrated in Fig. 1, the choice of value for c_S depends on knowledge of the imaging system. In this paper, we assume that the shape of the detected beam is several orders more accurate than the focusing distance, i.e., the values of NA, λ and w₀ depend on the components of the system, and uncertainty of their values is not discussed here. Instead, the placement of the sample, and thus the focusing distance z₀ depends on the operator of the image acquisition, and its value is more prone to errors.

In the Gaussian model, the width of the beam is described by the Eq. 6 with respect to propagation coordinate z. Including the stretching of the Rayleigh range z_r, we can use the Eq. 6 to formulate an expression to the stretching factor c_S, which is

$$c_S=\sqrt{\frac{{\left(\frac{z-z_0}{z_r}\right)}^2}{\frac{w(z)}{w_0}-1}}.$$ 7

In Eq. 7, the expression z − z₀ can be used to estimate the discrepancy between of true, unknown focal distance and the range by which the focal distance is assumed to vary. For example, let us consider the case where the desired focal distance is in the center of the sample (z₀ = 0), and we assume that the furthest possible true placement has a maximum 225 μm offset, yielding z − z₀ = 225 μm. For NA = 0.14 and λ = 600 nm, the Rayleigh range has the value z_r = 9.7 μm. To have the maximum weight on the reconstruction on the focal plane at the extreme offset, the beam width of the stretched beam at z = 225 μm should match w₀, in which case c_S → ∞ since w(z) = w₀ (this is natural, since w(z) was supposed to be of Gaussian shape and stretching is trying to force a straight line, i.e., the Radon transform). Thus, the relation $\frac{w(z)}{w_0}$ describes the weighting of the uncertainty. A smaller fraction means more stretching, and that the focal offset is assumed to be in the extreme of its supposed range. A higher fraction, i.e., less stretching supposes that the real focal offset is likely to be close to z₀ = 0. Further, assuming that the focal offset is likely closer to the extreme offset; and weighting the extreme offset to be quite likely and thus allowing the stretched offset to only double the initial beam width at z₀, that is, w(225 μm) = 2w₀; stretching factor should have the value of c_S ≈ 23.2 Similarly, the choice of w(225 μm) = 7w₀ yields c_S ≈ 9.5, or the choice of w(75 μm) = 2w₀ gives c_S ≈ 7.7.

Forward operator and inversion algorithm

Our forward operator describes the image information collected by the lens from the light propagating through the sample. The forward operator combines the Radon transform $ℛ$ with the GBM and SGBM via the blurring functional ${ℬ}_K$. In the following analysis, we additionally assume that the light beams are parallel. The measurement (projection) data y for a given image I, is assumed to obey the following linear model with an additive noise term n:

$$y=ℛ\circ {ℬ}_{K}I+n={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }Kfd\tau d{z}_L+n.$$ 8

We emphasize that the operators $ℛ$ and $ℬ$ cannot be commuted — blurring depends on the depth, but the Radon transform suppresses information in the direction of the light propagation. Hence, physically interpreted, it is a single operator that describes the portion of light propagation collected by the lens. If K(τ, z) = δ(τ) with δ denoting the Dirac’s delta, then the operator of (8) corresponds to the Radon transform, i.e., $ℛf=ℛ\circ {ℬ}_{K}f$.

For reasons of practical memory consumption, the kernel entries below a given (hard) threshold value were set to zero in all computations. We used a threshold value of e⁻²$K_{c_S}$ (0, z_L), i.e., e⁻² times the on-axis value along propagation.

To compute a reconstruction, we use the following iteration:

$$f_{\ell +1}={({\mathrm{L}}^T\mathrm{L}+{\mathrm{D}\mathrm{\Gamma }}_{\ell }\mathrm{D})}^{-1}{\mathrm{L}}^{T}y$$ 9

in which $y\in {ℝ}^m$, $f\in {ℝ}^n$, ${\mathrm{\Gamma }}_{\ell }=\mathrm{diag}({\gamma }_1,{\gamma }_2,\dots ,{\gamma }_n)$ with ${\gamma }_i={\left|\mathrm{D}f\right|}_i^{-1}$, and $\mathrm{D}$ is a regularization matrix given by

$${\mathrm{D}}_{i,j}=\frac{{\beta }_1(2{\delta }_{i,j}-1){\int }_{{\mathrm{P}}_i\cap {\mathrm{P}}_j}\mathrm{d}s}{\underset{i,j}{\max }{\int }_{{\mathrm{P}}_i\cap {\mathrm{P}}_j}\mathrm{d}s}+{\beta }_2{\delta }_{i,j},\mathrm{and}$$ 10

$${\delta }_{i,j}=\{\begin{array}{ll}1,\hfill & \mathrm{if}j=i\hfill \\ 0,\hfill & \mathrm{otherwise}\mathrm{.}\hfill \end{array}$$ 11

Here, P_i denotes the i-th image pixel. The first term penalizes the jumps over the edges and the second one the norm of f and β₁ and β₂ are the regularization parameters, i.e., weights given for the respective terms. If this iteration converges, it minimizes the regularized objective function

$$F(x)={\Vert \mathrm{L}x-y\Vert }_2^2+2{\Vert \mathrm{D}x\Vert }_1$$ 12

in which the latter norm is the total variation of x, if β₂ = 0^39,40. We used a low value for β₂ to ensure the invertibility of $\mathrm{D}$ and thereby the converge of the algorithm. $\mathrm{L}$ is the matrix formulation of the combined transform $ℛ\circ ℬ$, formed by inserting the Gaussian shape along each detection line. A detailed analysis of the algorithm is presented in³⁹. The MATLAB code used in this article is provided at⁵⁴ with example data. Another, complete data set imaged by us is available at⁵⁵.

Numerical experiments

In estimating the blurring kernels, we approximated the brightfield with a single wavelength λ = 600 nm in air, further adjusted for sample immersion liquid, in this case water with a refractive index of 1.33. The Rayleigh range had the value z_r = 9.7 μm following from the experimental setup objective of NA = 0.14. For forward solution, we used standard Gaussian kernel K₁ with the focal offsets 0, 75, and 225 μm. Zero-offset K₁ is shown in Fig. 7(a). In order not to commit an inverse crime, the forward solutions in the simulations were computed with non-thresholded K₁. All of the inverse solutions were computed using thresholded kernels: K₁ without focal offset for GBM, and K₅, K₁₀, K₂₀ and K_δ for SGBM. K₅ and K₂₀ are shown in Fig. 7(b,c), respectively. For comparison, FBP⁵² reconstructions were computed using MATLAB’s built-in function iradon with Hamming filter. Figure 7 also includes the number of non-zero elements of the kernels as they relate directly to the system inverted system sizes.

Figure 7.

(a) The Gaussian beam kernel K corresponding to NA = 0.14 and λ = 600 nm. (b) Extended kernels K₅ and (c) K₂₀. (d) A multifocal kernel K^* = (K⁽¹⁾ + K⁽²⁾ + K⁽³⁾)/3 in which NA = 0.14 and λ = 600 nm and the focal offsets of K⁽¹⁾, K⁽²⁾ and K⁽³⁾ are −300, 0 and 300 μm, respectively. NNZ is the number of non-zero elements in the kernel. Horizontal axis is the propagation axis z_L and vertical axis is the radial distance τ.

In multifocal experiments, the projection data of each angle was averaged into a single projection for the inversion. The multifocal kernel is referred to as GBM^*, and it consists of averaging three K₁ kernels with different offsets, i.e., $K_1^{⁎}=(1/N)\sum _{i=1}^N{K}^{(i)}$ where K⁽¹⁾, K⁽²⁾, K⁽³⁾ of the individual focal offsets i = 1, 2, 3. The multifocal kernel K^* is shown in Fig. 7(d). For reconstructions in simulated cases, we used the GBM with kernel K₁ without focal offset, and the SGBM with kernels K₅, K₁₀, K₂₀ and K_δ; and FBP. The kernels K₁, K₅ and K₂₀ are shown in Fig. 7.

Figure 8(a) shows our numerical phantom that was a square of 512 × 512 pixels with a side length corresponding to 1 mm and particle diameter to 10 μm (approximately 19 pixels). The background had the attenuation coefficient of zero (i.e., that of the void) and the particles had the attenuation coefficient of one. The numerical projection data comprised 400 projections in the range [0.9°, 360°] with 0.9° angular increment. Projection data were simulated using kernels K₁ with no focal offset (shown in Fig. 8(b) and with 75 and 225 μm focal offsets (the 225 μm offset data are shown in Fig. 8(c)). In total, we present 15 simulated studies labeled with the letters (A)–(N) and combinations of these are presented in Table 1 showing the forward and inverse parameters used.

Figure 8.

(a) The numerical 512 × 512 phantom with the side length of 1 mm and particle diameter of 10 μm. (b) The numerical sinogram without focal plane offset and (c) with 225 μm offset in kernel K₁. (d) A projection image from the zebrafish embryo data set. Reconstruction shown in this article are from a height of 1600 (pixel row illustrated with a black line). Image contrast has been adjusted for printability.

As the measurement noise, we used additive normal distributed noise with standard deviation 0.01 relative to the maximal noiseless data entry. For the GBM and SGBM reconstructions, we used three iteration steps. After trying different iteration lengths up to ten steps, three steps were found to be sufficient.

The numerical accuracy was analyzed via the relative error measure (REM) which we defined as the relative 1-norm ${\Vert f\Vert }_1=\int \left|f\right|dxdy$ between the phantom and the reconstruction f_rec with its minimum shifted to zero and maximum scaled to one. We expressed REM in percentages as $\mathrm{REM}=100\frac{{\Vert f_{\mathrm{relative}}-f_{\mathrm{phantom}}\Vert }_1}{{\Vert f_{\mathrm{phantom}}\Vert }_1},$ where $f_{\mathrm{relative}}=\frac{f_{\mathrm{rec}}}{f_{\max }-f_{\min }}-f_{\min }$. The 1-norm was chosen because it is known to be small for sparse distributions with only a few non-zero values^56,57, such as the cell phantom distributions used here. REM value measures the general quality of the image without penalizing the change in dynamic range. To analyze the dynamic accuracy, we evaluated the following relative total variation error (TVE):

$$\mathrm{TVE}=100\frac{{\Vert f_{\mathrm{rec}}-f_{\mathrm{phantom}}\Vert }_{\mathrm{TV}}}{{\Vert f_{\mathrm{rec}}\Vert }_{\mathrm{TV}}}\mathrm{with}{\Vert f\Vert }_{\mathrm{TV}}=\int \left|\nabla f\right|\mathrm{dxdy}\mathrm{.}$$

Experimental data

The experimental data were obtained from two experimental samples, one being a zebrafish embryo and the other composed of hydrogel with small polymer beads as cell phantoms. We used the in-house OPT^20,21 where the projection images were captured using an sCMOS camera (ORCA-Flash 4.0, Hamamatsu, Japan) with an infinity-corrected long working distance objective lens of either 5X for bead sample (Edmund, USA, NA = 0.14) or 10X for zebrafish embryo (Edmund, USA, NA = 0.28). The instrumentation included an iris diaphragm (Thorlabs, USA) and a filter wheel (Thorlabs, USA) between the objective and the camera, but these were not in use for the data used in this article, i.e., the iris was fully open, and there was no filter in the filter wheel. The projection data of both samples were acquired with a 0.9° interval from 0° to 359.1°, 400 images in total. Before imaging, both samples were manually centered in the x-coordinate using horizontally symmetric rotation and along the y-coordinate using the tube walls as a visual aid.

The zebrafish embryo was a two days post fertilization Tg(fli:1a:eGFP) prepared in 1.5% agarose (Sigma Aldrichm, Finland) hydrogel with added Tricaine (Sigma Aldrichm, Finland) for anesthetizing the fish. From the hydrogel, the fish was then sucked into a fluorinated propylene ethylene (FEP) tube with a syringe. The FEP tube had an inside diameter of 2 mm. The sample was imaged after mounting using a 10X objective with an exposure time of 8 ms. Only one focal plane was used for this image set. A single projection from the zebrafish embryo data is shown in Fig. 8(d). The reconstructions are computed from a height (pixel row) of 1600.

The keeping and raising of zebrafish stocks was performed with permission of the State Provincial Office of Western Finland (permission ESAVI/10079/04.10.06/2015). Care and experimental use of zebrafish embryo was carried out in accordance with EU Directive 2010/63/EU on the protection of animals used for scientific purposes; the Finnish Act on the Protection of Animals Used for Scientific or Educational Purposes (497/2013); and the Government Decree on the Protection of Animals Used for Scientific or Educational Purposes (564/2013).

The bead sample was prepared from gellan gum (GG) with physical crosslinking via 0.6% (w/w) spermine crosslinker, both dissolved in a 10% (w/v) sucrose solution (Sigma Aldrichm, Finland)⁵⁸. Spherical, black polystyrene beads (0.5% v/v) with a 10 μm diameter (Polysciences, USA) were mixed in the hydrogel during gelation. For the image acquisition, the sample was placed inside a FEP tube, which had an inside diameter of 2 mm. Two additional acquisitions were performed after translating the sample to 600 nm either closer or further to the objective lens than the original placement. The multifocal data were then processed by averaging through all three projections in each angle. An exposure time of 6.50 ms and 5X objective were used in imaging.

For computations, the data were linearized by scaling the values to interval [10⁻¹⁰, 1] and taking the 10-based logarithm (−log₁₀(·)). The center of rotation was corrected manually by shifting the sinogram with different offsets and choosing the offset with visually the least amount of circular effects caused by center of rotation offset^42,44. For memory reasons, the reconstructions were computed on a 512 × 512 pixel grid, and hence the width of the sinogram was average downscaled to 512 pixels from the original 2048.

From the experimental data, we present nine reconstructions labeled with the letters (O)–(X) the details of which are presented in Table 1. The reconstructions (O)–(R) from the monofocal data of the zebrafish embryo were computed using FBP, GBM with kernel K₁ and SGBM with kernels K₅ and K_δ. Reconstructions (S)–(V) were computed from the multifocal data of the bead sample with FBP, GBM with kernel K^* and SGBM with kernels K₅ and K_δ. For comparison, one reconstruction (X) was computed from the bead sample with monofocal data using FBP. In GBM and SGBM reconstructions, we used three iteration steps. The values of β₁ and β₂ are shown in Table 1. In the SGBM cases, we report only c_S = 5 cases.

Hardware and software used

The computations were performed using a high-end Lenovo P910 Workstation equipped with two Intel Xeon E5-2697 processors and 256 GB RAM. As the computation platform we employed MATLAB version R2016b 64-bit (The MathWorks, Inc.). The MATLAB scripts written for the blurring kernels, Radon transform and the inversion routine (9) have been included in⁵⁴.

Author Contributions

O.K. and S.P. were the main investigators of this study who derived the required mathematics, implemented the computational pieces, designed the experiments and analyzed the results. T.M., B.B., E.F. and J.H. contributed to the evaluation of the model in relation to physics of the instrumentation and to the analysis of the results. In addition, T.M. was responsible of the experimental data acquisition. All authors reviewed the manuscript.

Data Availability

MATLAB codes and data related to this article are shared through Zenodo at 10.5281/zenodo.1469361.

Competing Interests

The authors declare no competing interests.