Challenges and advances in optical 3D mesoscale imaging

Abstract

Optical mesoscale imaging is a rapidly developing field that allows the visualisation of larger samples than is possible with standard light microscopy, and fills a gap between cell and organism resolution. It spans from advanced fluorescence imaging of micrometric cell clusters to centimetre‐size complete organisms. However, with larger volume specimens, new problems arise. Imaging deeper into tissues at high resolution poses challenges ranging from optical distortions to shadowing from opaque structures. This manuscript discusses the latest developments in mesoscale imaging and highlights limitations, namely labelling, clearing, absorption, scattering, and also sample handling. We then focus on approaches that seek to turn mesoscale imaging into a more quantitative technique, analogous to quantitative tomography in medical imaging, highlighting a future role for digital and physical phantoms as well as artificial intelligence.

1. INTRODUCTION – WHAT IS OPTICAL MESOSCALE IMAGING?

Imaging in life sciences is increasingly turning towards 3D cultures, ¹ , ² , ³ whole organs ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ and whole animals ¹¹ as more relevant models to validate findings from 2D cultures and to visualise development at the system level. For example, the development of small animal models (e.g. Caenorhabditis elegans) and the determination of spatial gene expression patterns in tissue have revealed gene interactions that determine the life cycles of parasites. ¹² Importantly, the recent types of (model) organisms used are increasingly diverse, and research horizons are expanding to the study of later developmental stages and thus more complex organ structures. All of these pose new challenges for 3D optical imaging.

This trend towards imaging of ever‐larger samples exceeds the classic microscopy domain and is referred to as ‘mesoscopic imaging’. In optical imaging, this refers to objects between the microscopic and macroscopic scale, while imaging with subcellular resolution; in practice, this implies the imaging of objects from millimetre up to cm size with μm or nm resolution. As such, the mesoscopy field spans the boundary between classic ‘biological’ imaging and preclinical ‘biomedical’ imaging, typically utilising lower magnification objective lenses with a bigger field of view. Figure 1 shows examples of mesoscale imaging and highlights the variety of samples, approaches and the resolution scale achieved in mesoscale imaging.

FIGURE 1.

Examples of optical mesoscale imaging using the Mesolens, Light‐Sheet Microscopy and OPT. (A) Brightfield Mesolens image of a section of mouse embryo at full term, stained with haematoxylin and eosin. Scale bar = 0.5 mm. (B) Digital zoom into the snout of the embryo (shown with a black box in A). The individual cell nuclei are revealed at this level of digital zoom, and the individual chromatin granules can be seen in the nuclei. Scale bar = 150 μm. (C) A confocal fluorescence Mesolens image of a whole mount of fixed mouse ileum that has been prepared with the nuclear marker Propidium Iodide and cleared using Murray's Clear. The image shows the mesoscale architecture of the ileum at a depth of 350 μm into the specimen, and a region of interest is shown with a box. Scale bar = 1 mm. (D) Region of interest boxed in C after a software zoom, with the cell nuclei in the crypts now clearly visible. Scale bar = 50 μm. (E) Tribolium castaneum, treated with RNAse and stained with Propidium Iodide and cleared with benzyl alcohol/benzyl benzoate (BABB), ¹³ imaged with the Mesolens in confocal mode. This image is composed by maximum intensity projection colour‐coded by depth of 160 optical sections taken with an axial separation of 4 μm, forming a z‐stack 640 μm deep. Scale bar = 500 μm. (F) An antibody labelled E10.5 mouse embryo, cleared with BABB, and imaged with Light‐Sheet Microscopy. Cyan: neurofilament. Red: E‐cadherin. The image is a maximum‐value projection through the 3D data set. Scale bar = 500 μm. (G) Autofluorescence of the skin and transmitted light shape of a Xenopus tropicalis frogling imaged by OPT displayed in false colours. Scale bar = 2 mm

Other specimens used in mesoscale imaging include mouse embryos and intact insects, but also frog (Xenopus) tadpoles, ¹⁴ froglings, ¹⁵ zebrafish (Danio rerio), ¹⁶ medaka (Oryzias latipes), ¹⁷ insect larvae (Drosophila melanogaster) ¹⁸ , ¹⁹ and humans (Homo sapiens). ²⁰ , ²¹

The samples used in mesoscale imaging not only differ in size and volume but also in body plan, skeleton structure and composition (e.g. chitin). As such, the sheer size of the samples can pose a challenge for handling and orienting them ideally towards the optical readout and to the staining of specific structures. With increasing sample size, difficulties in imaging originating from the sample have more and more impact on light transmission, for example.

While other techniques such as Computed Tomography (CT), Positron Emission Tomography (PET), Magnetic Resonance Imaging (MRI) and ultrasound‐related technologies can be used to image these kinds of samples, here we focus on optical mesoscale imaging techniques such as the Mesolens, Light‐Sheet Microscopy and Optical Projection Tomography (OPT) strategies (Figures 1 and 2). Additionally, we touch upon the most important challenges for mesoscale imaging and what strategies have already been implemented to overcome such shortcomings. We consider limitations arising from the sample that hamper quantitative imaging, including labelling, absorption, and scattering. We discuss common strategies to tackle absorption and scattering, like intensity correction, clearing, cutting, stripe reduction, multiphoton approaches, adaptive optics strategies and combinations of devices. In addition, we discuss optical mesoscopy in relation to the medical imaging field, specifically imaging preclinical models with microCT, microPET and microSPECT, which have a long tradition of quantitative imaging at the mesoscale range, albeit often at lower spatial resolution than the optical methods highlighted here. We also consider the future use of optical phantoms for calibrating the imaging and analysis strategies for optical mesoscale imaging, including deep learning and other artificial intelligence strategies. ²²

FIGURE 2.

Schematic of optical configurations. (A) Mesolens, a confocal approach using a lens with a unique combination of low magnification and high numerical aperture. (B) Light‐Sheet Microscopy using two lenses with perpendicular orientation, one for illumination and the other for readout. (C, D) Optical Projection Tomography (OPT) approaches. (C) Transmission OPT. (D) Fluorescence OPT. Schematics not to scale, for example, the Mesolens setup is much larger than the other depicted devices

2. LABELLING AND LABEL‐FREE OPTICAL MESOSCALE APPROACHES

For a targeted mesoscale imaging approach, one of the problems is labelling structures of interest. Staining is inherently difficult as the tissue forms a diffusion barrier, and penetration of affinity probes may be hampered (a problem that gets worse with increasing sample volume). When comparing affinity probes, small molecules diffuse easier and stain large volumes more readily compared to antibodies, as can be seen for staining of amyloid plaques in brain tissue of mouse models for Alzheimer's disease. ¹³ , ²³ , ²⁴ Consequently, there is some interest in using ‘classic’ immunohistochemistry protocols for mesoscale imaging (Figure 1A–E). Also, staining is typically more difficult for more mature tissue than embryonic samples (Figure 1F) due to the increased stiffness and limited diffusion within them.

Additionally, coelom cavities can be used, for example, for labelling blood vessels through tail vein injections. There is also the possibility of enhancing label infiltration in tissue electrophoretically. ²⁵ Genetic tagging is another alternative that has become easier through CRISPR strategies. However, despite the progress, sample labelling remains a challenge due to the limited number of established protocols in this emerging field. This is because of the fact that every sample is different and requires adaptations for each species, organ, and developmental stage, thus requiring a divergence from common staining protocols for cell culture.

Consequently, for mesoscale imaging, label‐free imaging presents an opportunity. As such, label‐free imaging can provide complementary information to targeted fluorescent signals. Specifically, by revealing the context in which a fluorescent signal is located, it provides the big picture of the tissue, organ and organism organisation necessary to interpret the targeted fluorescent signals. Examples are the combination of specific fluorescent imaging as in two‐photon imaging with second and third harmonic imaging to access additional structural information. ²⁶ , ²⁷ , ²⁸ Likewise, autofluorescence (Figure 1G) can be used to provide information about the organism in a label‐free manner.

However, label‐free techniques are also increasingly used by themselves to directly access the structure of mesoscale samples. Examples include optical diffraction tomography and quantitative phase imaging approaches used for zebrafish and C. elegans imaging. ²⁹ , ³⁰ In addition, Optical Coherence Tomography (OCT) can be used to image non‐invasively organs like the eye and the brain, but also increasingly in developmental biology, as for the vascularisation of Xenopus. ³¹ , ³²

In contrast to current label‐free techniques, the flexibility of the current fluorescence toolbox allows the study of structures with high specificity. For label‐free imaging of complex samples, like whole‐mount preparations, in general, there is a need for benchmarking to pinpoint the (molecular) nature of the signal. However, label‐free imaging can be an interesting option in the context of artificial intelligence (see below).

3. APPROACHES FOR OPTICAL MESOSCALE IMAGING

Different approaches have been used to tackle imaging on the mesoscale. In this review, we refer to optical mesoscale imaging techniques and focus primarily on the Mesolens, light‐sheet microscopy, and OPT approaches (Figure 2).

3.1. Mesolens – a giant objective lens for 2D and 3D imaging

The Mesolens is a giant, custom‐made objective lens for imaging of tissue volumes up to 100 mm³ with subcellular resolution throughout. As such, the Mesolens can be used for most established microscopy techniques, with the advantage of providing a bigger field of view for imaging large specimens with high spatial resolution. Among the established microscopy techniques for 3D imaging, confocal microscopy is currently the technology most used for biological samples. Confocal microscopy can be said to have founded 3D microscopy, with its potential to perform optical sectioning. ³³ The basis of confocal microscopy is the use of a scanned point‐source for illumination and an aperture in the detection path to reject out‐of‐focus light. Consequently, the optical throughput of the microscope and scattering from the tissue determines the amount of light returning from the sample. Therefore, the maximum imaging depth of a confocal microscope has remained around 150–200 μm for most tissue specimens.

Commercial confocal microscopes also have restrictions for the field of view, as the objective lenses used are built to match the resolving capacity of the human eye. As a result, the size of the eye as a detector is used for reference. This has the consequence that multiple tiles or volumes must be stitched together computationally after the acquisition to image larger specimens with an off‐the‐shelf confocal system. Unfortunately, the process of stitching images is not free from problems and artefacts. ³⁴

Therefore, to image large specimens at high resolution, a complete redesign of the microscope was required (Figure 2A). This led to the development of the Mesolens by McConnell et al. ³⁵ The Mesolens is an optical lens system that allows for 3D imaging of objects up to 6 mm in diameter, which is comparable to the field of view of a 4× lens and can image samples up to 3 mm thick with a depth resolution of a few microns, instead of the tens of microns currently attained with regular off‐the‐shelf objectives (of comparable magnification) used in confocal microscopes. In addition, the Mesolens has an unusually high numerical aperture (NA = 0.47), allowing subcellular detail to be resolved throughout the entire volume of capture. This means that with the Mesolens, subcellular resolution in 3D for specimens in the mm range is easily achieved. In this sense, the Mesolens is geared towards the challenges of the bigger samples of the future. Applications of the Mesolens to date range from the imaging of whole intact adult Drosophila melanogaster ³⁶ to transgenic mouse embryos, ³⁵ imaging of whole mature colony biofilms ³⁷ and mapping Tuberculosis sp. infection in whole lobes of mouse lung. ³⁸

Moreover, the Mesolens is not restricted to confocal imaging and can be used with other imaging modalities as well. It has been used with a sensor‐shifting CCD camera for widefield, brightfield and epifluorescence imaging of large, but thin specimens. Its high collection efficiency (approximately 20× that of a conventional 4× objective lens) allows imaging of weakly fluorescent specimens with no obvious photobleaching. In addition to confocal imaging, a computational approach to 3D reconstruction with the Mesolens has been demonstrated using the HiLo configuration. ³⁹ Given the large field of view, establishing a Mesolens‐based light‐sheet is not trivial. Therefore, the HiLo illumination poses a good compromise for transparent specimens with uniform thickness.

A general drawback of the Mesolens is that due to the size of the lens, all optics need to be customised, and respective detection systems are less readily available in the case of use with a camera. In the case of point detection, scan heads and related parts require dedicated solutions; moreover the scanning of large fields of view is slow.

3.2. Light‐sheet microscopy

In recent years, the reinvention of light‐sheet microscopy (also known as Selective Plane Illumination Microscopy or, abbreviated, SPIM) has proven to be a very successful way to optically slice thicker samples (beyond the range of a standard confocal microscope) without physically sectioning them for 3D imaging. In light‐sheet microscopy, the sample is illuminated perpendicularly to the imaging axis, decoupling the illumination from the acquisition process. ⁴⁰ For this purpose, two perpendicular objective lenses are typically used (Figure 2B). One objective lens is used for illuminating the sample with a sheet of light either generated by a cylindrical lens or by fast lateral scanning of a micrometre‐thin laser beam. ⁴¹ , ⁴² The other objective lens is then used to capture the excited fluorescence. Consequently, the light is efficiently used for excitation, and the optical section directly imaged has only limited out‐of‐focus information.

Light‐sheet microscopy is famous for imaging early embryonic development of Drosophila melanogaster, ¹⁷ , ⁴³ and zebrafish, ⁴¹ but also cleared organs like brain tissue. ¹³ These applications have triggered an explosion of different implementations and extensions. For example, engineering the thickness of the light‐sheet by using an ‘optical lattice’. ⁴⁴ A single‐lens implementation of light‐sheet microscopy called Oblique Plane Microscopy ⁴⁵ (OPM) only requires one objective lens near the sample (instead of one each for illumination and detection), allowing more conventional, planar sample mounting. Examples of OPM are Swept Confocally‐Aligned Planar Excitation (SCAPE) microscopy, which was designed for high‐speed volumetric imaging of freely moving and behaving organisms ⁴⁶ like tardigrades, and diffractive oblique plane microscopy, ⁴⁷ which is an OPM adopted for large samples.

3.2.1. MesoSPIM and related approaches

Different light‐sheet devices have been developed to tackle larger mesoscale samples, including the ultramicroscope from the Dodt group ¹³ and the tiling light‐sheet from the Gao group. ⁴⁸ , ⁴⁹ Likewise, a few commercial setups have been developed to tackle large samples, including commercialisation of the aforementioned approaches from LaVision and 3i, as well as combined approaches using light‐sheet and OPT approaches from Planelight. ⁵⁰ One of the recent light‐sheet‐based open hardware developments for large samples is the mesoSPIM (mesoscale Selective Plane Illumination Microscopy) initiative. It optimises the optical sectioning for large samples by synchronising the camera's rolling shutter with the thinnest part of the light‐sheet (its waist), creating an optimally thin light‐sheet at the imaging plane, which would otherwise be impossible to achieve. ⁵¹ As the mesoSPIM mostly images optically cleared specimens, it uses air lenses to allow the use of all kinds of clearing agents as compared to the more commonly used water‐dipping lenses. The disadvantage is that air lenses limit the achievable resolution because of refractive index mismatches and the lower numerical aperture.

Limitations for mesoSPIM, and light‐sheet microscopy in general, are that with larger samples, these systems have less spatial resolution as the working distance of the applicable objective lenses and the resolution are inversely related. ³¹ , ⁵² A general problem with light‐sheet imaging is the frequent uneven illumination. In addition, the scattering introduced by the sample in a light‐sheet microscope can lead to a progressively defocused light‐sheet with imaging depth, compromising the optical sectioning capability of the setup.

It is an irony that the optical concept of illuminating with a sheet of light perpendicular to the readout axis was originally developed in the early 20th century for imaging nanometre‐size objects by light scattering (using an ‘extreme darkfield’‐style setup), while the reinvented light‐sheet microscope using fluorescence as a readout is hampered by scattering. ⁵³

One of the advantages of the light‐sheet approach is the speed of imaging that can be achieved. ⁵⁴ In general, we believe that many inventions from microscopic light‐sheet imaging ⁵⁵ will be reimplemented on the mesoscale.

3.3. Optical projection tomography

For imaging of small animals and structures, several techniques from medical imaging have been directly adapted. Micro‐Computed Tomography (microCT) scanning is the longest established of these and uses X‐rays to produce a tomographic map of object density, with a range of available contrast agents to better visualise structure both ex vivo and in vivo. ⁵⁶ , ⁵⁷ Inspired by the tomographic approach of CT, in 2002 Sharpe and coworkers developed a new technique called OPT. ⁵⁸ OPT is a 3D imaging technique utilising a series of 2D views to reconstruct a 3D voxel image computationally. For small samples, images are acquired by rotating the sample, and typically, views are collected from 360°. Images are then reconstructed using the same procedures from the CT world, that is, filtered backprojection and helical rotation algorithms. ⁵⁹ The technology can be used in transmitted light (Figure 2C) or fluorescent mode (Figure 2D).

Given that multiple images need to be acquired from different angles, the photostability of the used fluorophores towards bleaching is important. Next to the established use of fluorescent labelling, autofluorescence is used regularly to reveal the structure of the biological specimen. To accommodate out‐of‐focus regions and aberrations, several improvements and plenoptic imaging have been integrated into OPT. ⁵⁰ , ⁶⁰ , ⁶¹ Regardless, this mode of imaging gives the possibility for quantitative 3D analysis of small model organisms or embryos in their original geometry. ⁶² In addition, the reconstructed volumes have a nearly isotropic resolution, which means that the resolution in the z‐direction can exceed that of a confocal microscope with similar optics.

One general drawback of OPT is the inherent need for reconstruction. This process can be computationally demanding and uncouples the acquisition process from the 3D visualisation, standing in the way of quickly assessing the quality of the acquisition process.

Another drawback is that the lateral resolution is limited due to the use of telecentric long working distance lenses. To overcome this shortcoming, light‐sheet and OPT have been combined in one device, ⁶³ the OPTiSPIM. In this way, one can take advantage of the lateral resolution of SPIM and the (isotropic) axial resolution and nonfluorescent contrast imaging of the OPT.

3.4. Other related technologies

Mesoscopic imaging is notably an active research field, and multiple approaches are being developed for depicting samples in the mesoscopic size range. Other interesting approaches are Optical Coherence Microscopy (OCM) and photoacoustics, to name two. OCM is poised to provide additional information with images beyond the technical limitations of fluorescence imaging. Implemented in a confocal scanning microscope and using a high NA objective lens, OCM is an interference method that provides 3D reconstructions based on intrinsic contrasting of backscattered coherent light ³¹ , ⁶⁴ typically in the near‐infrared range. It has been used, for example, to image early embryonic development in a label‐free manner. ⁶⁴

Another possibility is to leverage the deep‐reaching capability of ultrasound in combination with high‐resolution imaging. Combining microscopy with optoacoustics might bring together the best of both worlds, allowing deeper penetration and a way to image small animals as a whole. ⁶⁵ This technology also holds potential for medical applications.

4. OPTICAL MESOSCOPY IMAGING CHALLENGES

Light interacts with matter in multiple ways. We utilise these effects, for example, in the form of fluorescence, when labelling our samples. However, other optical phenomena play a role in mesoscale imaging beyond the desired effects. Here we focus on absorption and scattering as these lead to well‐known artefacts in microscopy, which are even amplified in mesoscale imaging, such as shadowing stemming from bone, chitin and blood vessels and loss of optical performance in the case of scattering.

4.1. Absorption and scattering

Absorption and scattering of light by mesoscale samples are a central problem for all light‐based mesoscale imaging techniques (Figure 3). The two are often combined and regarded as attenuation or extinction.

FIGURE 3.

Schematic for absorption and scattering. (A) Absorption. The intensity of incident light is reduced by absorption. (B) Shadowing. For a number of incident photons shadowing is induced by complete absorption by the particle as indicated by the arrows. (C) Scattering. The direction of incident light is changed by a scattering angle by interaction with a particle indicated by the arrows. (D) Depending on the size, shape and density of the particles the light will be scattered differently. Here a Mie type of scattering is depicted, where more forward scattering is happening, indicated by the thickness of the arrows. (E) Sample complexity can lead to unrecoverable regions marked here by the white space in the middle, where absorption and/or scattering block the region from all possible imaging directions

Attenuation is the gradual loss of flux through a medium that can be caused by absorption, reflectance, and scattering. In the case of the simple question of light going through a uniform medium, Lambert‐Beer's law quantifies the amount of absorbance A as,

$$A=\epsilon lC={\log }_{10}\frac{I_0}{I},$$

with l being the optical path length, ε being the molar attenuation coefficient for a given wavelength and C being the concentration. Further, I ₀ being the incident light intensity and I being the transmitted light (Figure 3A). However, absorption and scattering often occur together and, in practice, they are regarded together, as the energy E of extinction for both phenomena is additive:

$$E_{\mathrm{Exti}\mathrm{ncti}\mathrm{on}}=E_{\mathrm{Scat}\mathrm{teri}\mathrm{ng}}+E_{\mathrm{Abso}\mathrm{rpti}\mathrm{on}}.$$

Consequently, the scattering can be accessed in a similar fashion as the absorption (in the simple transmission case), for example, in turbidimetry. However, the absorbance refers to the amount of light absorbed in the sense of energy of photons taken up as opposed to being reflected or refracted (Figure 3B). In contrast, scattering refers to the light deviating from its path upon interaction with particles depending on the size and shape of the particles.

To emphasise the difference between scattering and absorption, and to describe the difference with Lambert–Beer's law above, the process of scattering can be regarded by two functions, S ₁ (θ,φ) and S ₂ (θ,φ) for amplitude and phase, with θ being the scattering angle and φ being the corresponding azimuthal angle ⁶⁶ (Figure 3C). Consequently, scattering is often measured at an angle, as in nephelometry. Importantly, the amplitude of a scattered wave is inversely proportional to the distance of detection. This means that scattered photons can interfere within a complex optical system and continue to propagate at a different angle (S ₁ & S ₂) while absorbed ones are lost (Lambert–Beer).

In samples imaged by light microscopy, mostly elastic scattering (Rayleigh and Mie‐type) affects the light path, with Mie and Rayleigh referring to approximations of solutions to Maxwell's equations on electromagnetism for differently‐sized particles (Figure 3D). The amount of scattering and the contributions from Rayleigh and Mie scattering for different tissue types have been extensively reported by Jacques. ⁶⁷

Next to the weakly scattering effects described above, depending on the tissue and the scale at which the tissue is observed, biological tissues can progressively be considered strongly diffusive media, in which multiple scattering events occur. For infrared imaging, where absorption is less relevant, scattering becomes dominant. Diffuse optical imaging and tomography (DOI; DOT) are non‐invasive techniques that utilise light in the near‐infrared spectral region, allowing interrogation of translucent tissue like breast or brain. In this strong scattering imaging regime, a major concern is lateral interpixel crosstalk, which degrades the resolution. Consequently, major efforts have been put into the assessment of photon transport in these tissues to improve reconstructions. ⁶⁸ , ⁶⁹ , ⁷⁰ , ⁷¹ , ⁷² Likewise, Monte Carlo simulations are used in PET along with digital phantoms to evaluate reconstruction/quantification protocols, ⁷³ and generally, different approaches have been made to model scattering in tissue. ⁷⁴ It is of note that beyond the optical imaging regime, scattering can be further distinguished as elastic scattering, where there is no change in the energy, or inelastic, where energy is lost (and the intensity is attenuated) in the process of scattering, as in Compton scattering.

However, for the optical mesoscale imaging highlighted here, mostly weak scattering is relevant and the two processes, absorption, and scattering, affect each optical configuration differently (Figure 2). Thus, it is useful to consider them separately (Figure 3A–D). Practically, when scattering affects the excitation light path, it will lead to a less‐defined excitation spot/sheet, and the focal positions will be less bright. If scattering affects the emission path, the imaged position will be more diffuse. In the case of a transmission image, scattering may appear as absorption if the light is scattered out of the detection path, while photons scattered into or within the detection path increase the background, reducing the relative signal strength. With larger samples, scattering‐related problems typically increase.

In fluorescence, if absorption affects the excitation light path, it means that less light will be available to excite the fluorophore. In the case of emission, it means that less light will reach the detector. Depending on the optical configuration, combinations of these effects may occur. For example, in light‐sheet imaging, scattering affects the thickness of the light‐sheet due to broadening the light‐sheet. In addition, resolution is affected due to lateral crosstalk impacting xy information. The latter is a problem for all widefield types of detection schemes. If the sample contains pigments (for example, haemoglobin or melanin) and visible light is used, the signal is attenuated due to the interaction of the light with the pigments. These pigments will lead to photons being absorbed, resulting in a loss of signal for imaging (Figure 3E). Table 1 summarises the effect of absorption and weakly scattering tissue on the different imaging modalities.

TABLE 1.

Simplified effects of absorption and weakly scattering tissue on the illumination/detection light of the methods reviewed here. The effects are different for each technique

	Absorption	Scattering
Confocal Mesolens	Reduced signals	Reduced signals
Light‐sheet microscopy	Shadowing artefacts	Less confocality
Transmission OPT	Depicted as signal	Depicted as signal
Fluorescence OPT	Less signal	Less optical confinement

Diverse measures have been taken to compensate for scattering and absorption in the different optical approaches. However, one of the most universal ways to tackle this is by clearing the sample.

5. DEALING WITH ABSORPTION AND SCATTERING

Clearing is a biochemical process that renders tissue, and even complete animals, significantly transparent by immersing the specimen in a clearing agent. This process was first explored by Spalteholz at the beginning of the last century, ⁷⁵ but it is now experiencing a renaissance thanks to new and emerging mesoscale imaging technologies. ⁷⁶ Clearing mostly reduces scattering by reducing refractive index mismatches in the tissue and between the tissue and the immersion medium, but procedures that include quenching, bleaching, and oxidation of pigments, thus reducing absorption, have also been developed. ⁷⁷ Protocols have been optimised for different cellular materials and for specific tissue types, for example, skin and bone, ⁷⁸ as well as the development of clearing methods that preserve the fluorescence signal from fluorescent proteins ⁷⁹ or that combine clearing with labelling. ⁸⁰ In the context of expansion microscopy, ⁸¹ where the sample is embedded in a hydrogel and swells isotropically in size to permit microscopic imaging of nanoscale structures, the sample is also rendered transparent. ⁸²

One disadvantage for most clearing protocols is that the sample is fixed, with a few notable exceptions of live clearing. ⁸³ In addition, clearing does not always work perfectly, with the problems of scattering and absorption persisting to some extent. Finally, clearing is a series of chemical processes that remove or modify the sample composition, which may have some effects on the structure to be imaged.

Another sample‐based approach to deal with absorption and scattering is the usage of fluorophores that are redshifted. Due to less scattering and absorption of biological tissue in the longer wavelength range dyes and fluorophores that can be used in the near‐infrared and infrared range are increasingly adopted. ⁸⁴ , ⁸⁵

5.1. Compensating for the lost signal

Next to sample‐based approaches, the most commonly used strategy to compensate for a reduction in signal strength due to absorption and scattering is either through post‐processing or during the acquisition itself, by increasing either the excitation power or the detection amplification when possible. The goal of this compensation is typically to improve the signal‐to‐noise ratio across the sample. In this section, we describe how each of the mesoscale imaging techniques apply compensation.

5.1.1. Compensation for the Mesolens

In the confocal Mesolens approach, the optical configuration for excitation and emission makes use of the same giant objective lens. As with confocal imaging, the pinhole is placed in an optical plane that is conjugated to the focal plane, and that pinhole then rejects out‐of‐focus information. At the same time, it also rejects scattered light, which seemingly does not originate from the point of excitation. This explains why confocal systems have a limited penetration depth in tissue, as both the excited and the emitted light gets scattered, resulting in dimmer images. The intensity can be increased by using higher powers for excitation to compensate for the loss by scattering at the price of additional light exposure of the specimen, thus causing bleaching and phototoxicity in living specimens. However, photons lost on the emission side cannot be recovered, leading to a poorer signal‐to‐noise ratio. Similarly, increasing the photodetector gain to compensate is usually at the expense of the signal‐to‐noise ratio. These rather crude compensation mechanisms fall short because they do not take the sample structure per se into account. This is because the exact 3D distribution of attenuating structures in the sample is generally not known, so quantitative estimates of how much to increase the excitation light power and/or detection gain cannot be made.

5.1.2. Compensation in light‐sheet microscopy

In light‐sheet microscopy, both the illumination light path as well as the detection light path can be affected by scattering and absorption, with penetration of the light‐sheet being a concern for the illumination, and the imaging depth relative to the detection lens affecting the emitted light traveling through the sample. Although much effort has been spent in obtaining a greater degree of penetration for light‐sheet microscopy, ⁸⁶ , ⁸⁷ to date, this effort has been focused on obtaining a signal via ‘attenuation compensation’ rather than correcting for inhomogeneity in the sample per se. The presence of differentially attenuating/scattering structures in a mesoscale sample casts ‘shadows’, leading to concealed regions in the sample or distortion of the expected signal strength at the detector. As the typical strategies to address these problems are by multiview imaging, ¹⁸ , ⁸⁸ , ⁸⁹ it is unlikely to be corrected quantitatively. Using different angles for imaging will effectively decrease the thickness of the sample, therefore, limiting the effect of attenuation, but does not address the size and morphology of attenuating biological structures as such. Remarkably, little attention has been given to date to overcome the effect of such attenuation on mesoscale light‐sheet images.

5.2. Stripe reduction

A common problem in light‐sheet imaging is stripes originating from shadows created by absorption, for example, by blood vessels. One way to address this is by using pivot scans that vary the angle of the light‐sheet hitting the sample and therefore limiting the shadow to a cone instead of a stripe. ⁸⁸ Another strategy is using diffuse light‐sheets for illumination. ⁹⁰ While this helps to improve image quality, the problem is just diluted instead of being fully addressed. Multiview imaging of the sample can also reduce stripes to some extent (Figure 3C–E). ⁸⁸ , ⁸⁹

5.3. Multiphoton excitation

One solution to reach deeper into tissue is to use two‐ or three‐photon fluorescence microscopy. ⁹¹ , ⁹² The idea behind these techniques is to use ultrashort pulses of infrared light that are less prone to scatter. Additionally, light in the near‐infrared spectral range is less absorbed than light in the visible spectral range in most biological tissue. The multiphoton effect is then used to excite a spatially confined volume whose point spread function depends on the numerical aperture of the objective lens, the wavelength of illumination, the beam quality of the laser, and the scattering and absorbing properties of the tissue. As the nonlinear absorption effect has a significant probability of happening only in the focal spot of the system and depends on the laser's peak intensity, all fluorescence signals emitted by the sample can, in theory, be detected and used for imaging. Two‐photon strategies have been applied to light‐sheet microscopy ⁹³ and could be applied to the Mesolens and OPT, as well. However, this strategy has its limitations, as creating long, thin sheets of light with two‐photon illumination is not easy. Gaussian optics can create a long sheet of light extending over several millimetres, but it is too thick to give cellular or subcellular resolution over the field of view. It is also necessary to consider the peak intensity that must be spread over the field of view: this is managed easily by the focused scanning spot in a conventional two‐photon microscope, but the peak intensity must be scaled accordingly for widefield or light‐sheet illumination. ⁹⁴ Another challenge in multiphoton microscopy is preserving the ultrashort pulse duration at the specimen plane, but this can be managed through pre‐dispersion compensation using gratings or fibre pulse compressors. ⁹⁵ , ⁹⁶

5.4. Cutting and imaging – HREM

One of the limitations of both two‐photon and confocal microscopy is the axial resolution, which is usually on the order of 0.3–1 μm. In addition, in the 1980s, it became clear that optical sectioning works best for high NA lenses. The problem is that typically high NA lenses have a shorter working distance, while longer working distance lenses have a lower NA, resulting in low‐resolution images. Cutting tissue and imaging the slices is one way around that problem, allowing retrieval of images from inside the tissue with a high NA lens while reducing scattering and absorption. The drawback is the computational challenge to register the images. Alternatively, the blockface can be imaged as in High‐Resolution Episcopic Microscopy (HREM) and related technologies. ⁹⁷ , ⁹⁸ , ⁹⁹ , ¹⁰⁰ These systems can be used to image tissue autofluorescence as well as specific signals from expressed fluorescent proteins targeted to a structure of interest in model organisms and can be combined with confocal or two‐photon imaging. Unfortunately, the technique is dedicated to fixed samples, and artefacts can arise from cutting and embedding samples in a stabilising matrix. For thinner slices, the sample needs to be more rigid which can be achieved, for example, by paraffin embedding or freezing. Interestingly, the combination of two‐photon imaging for brain tissue in an HREM fashion poses a valid alternative for imaging with a cellular resolution. ¹⁰¹

5.5. Adaptive optics

Adaptive optics can be used to correct scattering and distortion stemming from refractive index mismatch in the sample. The concept of adaptive optics originates from astronomy, where an artificial guide star in the form of a laser directed at the night sky is used to measure distortions induced by the atmosphere. ¹⁰² In microscopy, a wavefront sensor is often used to detect the distortions. Consequently, an adaptive optics element corrects the light wavefront to obtain a less distorted image. For the correction, a deformable mirror or a liquid crystal array can be used. Adaptive optics are used extensively in high‐resolution microscopy, for example, in the context of brain imaging ¹⁰³ as well as in light‐sheet imaging ⁸⁴ , ¹⁰⁴ to improve the achievable resolution. Specifically, for long‐term imaging and following the embryonic development of mice over time, adaptive optics have been instrumental by being able to adapt to the growing and changing specimen. ⁸⁴ However, the optimisation of the wavefront for best correction is often slow, which can lead to both phototoxicity and photobleaching of the specimen. Adaptive optics can also be applied to OPT and the Mesolens. For the Mesolens, however, the deformable elements would need to be huge to be compatible with the optics of the Mesolens. This problem may be solved by working with the astronomy community, who originally developed adaptive optical elements for long‐range imaging applications. ¹⁰²

5.6. Correction in OPT

For OPT, different approaches for correcting scattering and absorption have been implemented. In particular, an optical attenuation map generated via transmission can be used to inform the reconstruction of emission images and, thus, account for non‐uniform absorption. ¹⁰⁵ Moreover, in fluorescence OPT, specific illumination schemes using parallelised semi‐confocal line illumination and detection have been developed to discriminate against scattered photons. ¹⁰⁶

5.7. OPTiSPIM

For correcting absorption in light‐sheet imaging, the OPTiSPIM ⁶³ is an interesting approach. The device is a combined OPT and light‐sheet system. The idea is to measure the absorption using the transmitted light OPT. From that, a 3D mask is generated to correct the intensities of the light‐sheet data. This is remarkable given that corrections come from a measurement‐based system, and the combination of techniques is used to create a sample‐based correction of the data. Scattering in the transmitted light OPT is interpreted as attenuation of the signal, so that the correction of the SPIM data will compensate for the loss of signal in this respect. However, scattering will also lead to less crisp images in the OPT. Although there is no way to distinguish between absorption and scattering in this approach, artefacts can be corrected and image quality improved. This approach can be considered in some ways analogous to the corrections used in medical imaging, where a CT image is used for correcting Singlephoton Emission Computerised Tomography (SPECT) or PET images.

6. STRATEGIES FROM MEDICAL IMAGING

Given the similarity of correction between the OPTiSPIM and medical imaging approaches, we thought it worthwhile to look closer at how correction is carried out in medical imaging, and the underlying foundations. Next to corrections, the potential for connecting mesoscale and medical imaging for multimodal imaging has been highlighted recently. ¹⁰⁷

Tomography, the generation of cross‐sectional images via the acquisition of projection data of a penetrating wave/particle, is well established in a range of medical imaging techniques that have now found application at the mesoscale. Conceptually, emission tomography is similar to fluorescence microscopy in the sense that images are created from photons that are emitted from the sample. Then, projection data are used to reconstruct the image, which is corrected for both attenuation and scattering. Alternatively, attenuation correction is incorporated into iterative reconstruction techniques to allow the generation of images that reflect the true geometry and activity of the source. Importantly, the resulting quantitative images are not biased due to depth‐dependent effects, that is, the attenuation of high‐energy photons as they pass through the body to the detector or their mispositioning due to scatter.

In PET and SPECT, attenuation and scattering are part of the same physical process, that is, the (inelastic) Compton scattering of high‐energy photons as they pass through the body (photoelectric absorption is limited at the photon energies for PET and most SPECT isotopes). Thus, attenuation is a loss of signal as a consequence of the inelastic interaction, while scattering seemingly leads to an increase due to erroneous coincidence detection ¹⁰⁸ (although at energies below 15 keV Rayleigh scattering is dominant in tissue, the angles of deflection are small, and so, in this case, scattered and primary photons are not separated). Without correction, attenuation leads to images where the centre of the object has less apparent activity than the edges, which are overestimated (Figure 4A), and scattering leads to an increase in events towards the centre of the object (Figure 4C).

FIGURE 4.

Reconstructed images from a cylinder containing a uniform activity, with corresponding line profiles. (A) Reconstructed PET image without attenuation or correction for scattering. (B) Attenuation map derived from CT image of the same cylinder. (C) Reconstructed PET image corrected for attenuation but uncorrected for scattering effects. (D) Final corrected PET image. Adapted from Ref. (109)

In PET, correction for attenuation has the largest impact and is made based on an attenuation map of the object, created via a transmission source or by scaling the information from a CT image to the photon energy of the emitter. Due to the unique emission of (almost) coincident photons, attenuation for any line of response can be readily established, and individual projections can then be corrected before image reconstruction (Figure 4C). In SPECT, the situation is more complex due to the lack of these coincident photons. Thus, information from the attenuation map is used as part of the iterative reconstruction of the radioactive image.

Without correction for scattering, uniform density images will demonstrate higher apparent counts towards the centre of the imaged object (Figure 4C). Light scattering is the most difficult phenomenon to correct for in positron and single‐photon emission tomography, involving the incorporation of modelled or measured effects of scattering in an iterative reconstruction. ¹¹⁰ , ¹¹¹ There are two general approaches: the measurement of scattering at a separate energy window(s) set narrowly around the expected peak that can then be scaled and subtracted from the detected peak or modelling approaches involving the calculation of the scattering, for example, based on Klein‐Nishina equations or experimental values. In both cases, the derived terms for the scattering are then incorporated into the image reconstruction. Although attenuation maps created via CT are widely used in preclinical emission tomography, the advent of PET‐Magnetic Resonance Imaging (PET‐MRI) has renewed efforts at correcting for attenuation directly, using only emission data.

OPT, where optically cleared samples are employed so that scattering is reduced, can be considered analogous to transmission (CT) or emission (PET, SPECT) imaging depending on the mode. However, it is useful here to note the differences between the implementation of tomographic approaches in both domains. In (biomedical) emission tomography, quantitative imaging is established as a goal, and corrections, albeit implemented in a range of ways, are considered standard. To this end, clinical (and increasingly preclinical) sites routinely assess the quantitative performance of their instruments using defined standards and phantoms, for example, National Electrical Manufacturers Association (NEMA) NU2, NEMA NU4. As well as allowing comparison between instruments and enabling the standardisation of best practice, ¹¹² phantoms facilitate the assessment of novel or more complex experimental setups by allowing a robust assessment of any effects on image quality and quantification (for example, multiple animal scanning in preclinical studies). ¹¹³ Consequently, reconstruction algorithms used for OPT are also discussed that include scattering. ¹⁰⁶ , ¹¹⁴ , ¹¹⁵

7. PERSPECTIVES

7.1. Future use of optical phantoms

In medical and preclinical imaging, so‐called imaging phantoms, or just phantoms, are commonly used to evaluate, tune and calibrate the imaging process. Phantoms simulate the imaging properties of human tissue. They are used as standard test objects to validate the performance of imaging devices and often are part of a quality control procedure. Typically, they contain recognisable features that can be used for evaluation ¹¹⁶ and can mimic a tissue's optical, thermal and mechanical properties.

With phantoms being used as test objects for a specific imaging modality, they need to be tailor‐made for that imaging process. The advent of 3D printing offers new possibilities for microscopy in general ¹¹⁷ and specifically to create solid or hydrogel tissue phantoms for a multitude of imaging modalities with relative ease. Fabrication allows the optical properties of phantoms to be controlled through the incorporation of scattering and absorbing media. These phantoms and their print files can then easily be shared.

This opportunity has already been picked up in medical imaging, where printed phantoms have been investigated to improve microCT, X‐ray imaging, and OCT, ¹¹⁸ and similar approaches have been applied for diffuse optical imaging ¹¹⁹ as well as MRI and CT ¹²⁰ and quantitative phase imaging. ¹²¹ The printing process offers an opportunity, specifically, as the resolution of the prints can be in the same range as that of optical mesoscale imaging.

Emerging bioengineering technologies such as 3D bioprinting can create well‐defined 3D tissue structures that can be tailored using a variety of extracellular matrix (ECM) components, including collagen. This enables the design of multilayered tissue phantoms with specific physical characteristics, such as different densities or fibrous content. 3D bioprinted optical tissue phantoms can be printed with fiducial markers deposited within them during the printing process, allowing the assessment of light at different depths and positions within the volume. They can accurately mimic the optical properties of biological tissues for mesoscale imaging device calibration, comparison, validation, and even training for the detection of specific structures. However, 3D bioprinted tissue phantoms are not yet widely adopted in optical imaging. ¹²²

We believe that phantoms can be used for optical mesoscale imaging to tackle scattering and absorption, similarly to their use in medical imaging. The growing availability, combined with the versatility of 3D bioprinting, can provide multiple approaches for phantoms in optical mesoscale imaging. Consequently, we believe that akin to medical imaging corrections based on phantoms combined with elevated theoretical, and machine learning approaches, or similar strategies could help optical mesoscale imaging become more quantitative. We expect that with more use cases and the ease of creating phantoms, more and more dedicated phantoms will emerge for specific purposes, such as resolution targets, phantoms to calibrate reconstructions and distortions, as well as to allow for registration between modalities.

7.2. Analysis and future use of AI

For analysis of mesoscale data, the size of the volume imaged poses a burden, and often new tools that are fully 3D capable and able to work through the big volumes are required. ¹²³ , ¹²⁴ Increasingly, Graphics Processing Units (GPU) ¹²⁵ and machine learning tasks are being used to classify pixels and voxels ¹²⁶ , ¹²⁷ in ‘big data’. However, the advent of Artificial Intelligence (AI) is leading to more complex Deep Learning (DL) tools being employed for mesoscale data.

In the field of AI different implementations of machine learning exist, including DL. For DL, the idea is that an artificial neural network learns to approximate ground truth to carry out analysis tasks. ¹²⁸ Different obstacles are in the way of doing this efficiently. On the one hand, training data and ground truth can be challenging to obtain. Often the idea of setting up an AI analysis is to replace tedious and labour‐intensive interpretation of the data by humans. For that purpose, enough data analysed and annotated by humans need to be at hand, and the data need to be similar to the new data that are going to be analysed. ¹²⁹ Furthermore, the process of training a deep neural network involves determining the values of the vast number of network parameters using an iterative algorithm and multiple passes over all the training data. This requires considerable computational resources and can take up to days on high‐end machines. However, once the network is trained, analysing similar data is relatively straightforward computationally.

For analysing mesoscale data with DL and other AI strategies, new problems arise. The data sets themselves are rather large, which means that the computationally intensive training task might be exacerbated even more. Smaller subsets of the mesoscale data could be used for training. However, for DL algorithms to work, the samples given to the network for prediction need to come from the same statistical distribution as the samples used during training and therefore need to be representative. As discussed before, inhomogeneities in the data set stemming from the non‐perfect clearing, scattering, and absorption lead to inhomogeneities in the data quality across the sample. Consequently, the size of the training data set will ultimately be a balance between faster training of the model and better generalisation and predictive power of the trained network.

The creation of a DL analysis tool that is very flexible regarding the input and allows reliable detection of features, like (for example) Cellpose, is a great asset. ¹³⁰ However, this flexibility comes at the price of very extensive training.

By themselves, the discussed shading and scattering artefacts in mesoscale imaging could be ameliorated using DL strategies. Specifically, for the shading, for example by blood vessels, the artefacts could be detected and the missing information could be filled in by DL approaches. This particular problem is described by image in‐painting, which is the task of realistically completing an image that has missing pixels. Multiple solutions for photographic scenarios have been described. ¹³¹ , ¹³² , ¹³³ We expect that respective solutions for mesoscale imaging will become available soon, as some have already been applied to material microscopy. ¹³⁴

Next to absorption and shading, scattering could be tackled computationally, ¹³⁵ and interestingly, simulation approaches have been used to guide adaptive optics. ¹³⁶ Given the computational difficulty and cost of generating rapid physical corrections, there is also increasing use of AI and DL to correct emission data, ¹³⁷ especially in novel hybrid systems ¹³⁸ that could also be applied to optical mesoscale imaging.

With multiple groups working on tasks to analyse microscopy and mesoscale data, in the future, strategies like the Model Zoo (https://modelzoo.co/) that contain implementations of already trained networks can help with reproducibility, benchmarking, and finding ready‐made solutions.

In medical imaging, AI finds broad usage ranging from patient organisation ¹³⁹ to diagnostics, but two aspects stick out: dosage reduction and diagnostics. For diagnostics, the question boils down to image segmentation, identifying features and classifying objects. ¹⁴⁰ This is in line with microscopy, where AI and related techniques have found broad use for image segmentation. ¹⁴¹ , ¹⁴² , ¹⁴³ Likewise, we expect this to become increasingly relevant for mesoscale imaging. ¹⁴⁴

Additionally, for medical imaging, dosage reduction takes an important role. ¹⁴⁵ OPT can take advantage of these CT strategies from medical imaging, such as compressed sensing and DL approaches, to reconstruct 3D volumes from fewer projection images to reduce the light dose and accelerate 3D image data acquisition. This approach was recently exemplified for in vivo OPT imaging of zebrafish. ¹⁴⁶ Similarly, the use of DL for image restoration, denoising and enhancement of resolution in microscopy applications is promising ¹³⁶ , ¹⁴⁷ and can enable faster and more gentle imaging in general and promote live imaging. Related strategies have already been applied to light‐sheet data for isotropic volume imaging. ¹⁴⁸

Although full quantification in medical imaging is ultimately driven by patient benefit, consideration and correction for factors that affect even semi‐quantification of mesoscale microscopy can be considered relevant in the determination of biological structures and functions. Despite their successes, DL and AI are not free from artefacts, and there is a danger that present information is overinterpreted and that the DL networks reproduce only the learned content.

A specific mesoscale challenge for DL and AI approaches could be to take care of specific optical limitations that are difficult to deal with in the real world, like distortions and aberrations.

8. CONCLUSIONS

The most important parameter that affects the application of mesoscale imaging techniques is the nature of the sample. By themselves, sample handling, mounting and related issues can be very tricky, and so far, we have seen many studies on optical developments. However, a challenge will be to tackle imaging, and specifically sample handling, to leverage the novel optical approaches for many different biological questions. In the future, we need more robust systems that work for different types of samples and allow us to tackle biological questions in a quantitative manner. In this context, the problem may easily be hidden in detail. Light‐sheet microscopy has been described as well suited for neuroscience imaging. ¹⁴⁹ , ¹⁵⁰ This is mostly because zebrafish brains can be imaged alive and clearing of brain tissue works well. Therefore, neurodevelopment can be imaged without many problems. However, for neurodegenerative diseases, aged animals need to be imaged, and clearing of old brains is more difficult, likely due to the deposition of lipofuscin and other factors. This makes mesoscale imaging much more challenging for Alzheimer's disease research. ²⁴

Compared to optical imaging, the medical imaging community has been dealing for far longer with the difficulties of extracting quantitative data from images of large samples. Elaborate procedures for estimating tissue attenuation and means for compensation are well established in the medical imaging field, while we are currently missing this in optical imaging to some extent. The creation of specific phantoms and better characterisation of the systems could be a way to tackle this. In addition, direct ways to estimate the attenuation from the sample in place might be required. For that, building on strategies like OPTiSPIM and, for example, combining it with adaptive optics can be a way to develop this further.

Consequently, we need new developments for optical mesoscale imaging, and we already see that developments for the light‐sheet world are recapitulated in mesoscale light‐sheet imaging. ⁹³ , ¹⁵¹ The successful approaches of the Mesolens to overcome the century‐old limit of the eyes as the determinant of the field of view demonstrated this impressively. In the future, we can expect that all different permutations of configurations with multiple illuminations and readout arms will also manifest on the mesoscale, and we need to combine them with different readout and analysis strategies to make optical mesoscale imaging more quantitative.

Munck, S. , Cawthorne, C. , Escamilla‐Ayala, A. , Kerstens, A. , Gabarre, S. , Wesencraft, K. , Battistella, E. , Craig, R. , Reynaud, E. G. , Swoger, J. , & McConnell, G. (2022). Challenges and advances in optical 3D mesoscale imaging. Journal of Microscopy, 286, 201–219. 10.1111/jmi.13109