Single molecule orientation and localization microscopy

Abstract

Single Molecule Localization Microscopy (SMLM) offers enhanced spatial resolution in optical microscopy, providing detailed insights into the spatial organization of proteins in cells at the nanoscale. Over the past decade, SMLM has progressively incorporated the capability to retrieve the orientations of single molecules using their polarized dipolar emission pattern. This Review explores recent advancements in Single Molecule Orientation and Localization Microscopy (SMOLM), which yields super-resolved images of molecular 3D orientations, wobble, and 3D positions. This advancement opens possibilities to explore the nanoscale organization and conformation of biological molecules as well as to monitor and design local 3D optical fields in nanophotonics. The Review covers the principles of SMOLM, discusses recent advances and applications in biology and photonics, and finally highlights exciting future directions and challenges in the field.

Introduction

Polarization-sensitive imaging has a rich history in optics and photonics.^1,2 The polarization of the optical field is fundamental to near-field optics and nanophotonics³ and generating optical contrast using absorption, scattering,⁴ and linear and nonlinear radiation.⁵ Soon after single molecules (SMs) were first detected optically via absorption⁶ and fluorescence,⁷ researchers leveraged their dipole-like absorption and radiation behaviour to measure molecular orientations.^5,8 Precise measurements of molecular orientations and conformations are particularly important in biological, chemical, nano-, and material science applications, where molecular movements and interactions are central to how matter is organized, how cells respond to external stimuli and maintain biological function, how molecular processes evolve over time, and how to control light with matter.^3,9 Combined with the development of single-molecule localization microscopy (SMLM),¹⁰ super-resolved polarization microscopy now has the power to image molecular orientations and positions with resolution beyond the diffraction limit; we term this technology single-molecule orientation and localization microscopy (SMOLM).^5,11 Over the last decade, SMOLM has progressively impacted our understanding of the organization of biomolecules at the nanoscale in cells and in vitro; SMOLM complements ultra-resolution methods such as electron microscopy (EM) and scanning tunnelling microscopy (STM) with added capabilities such as live cell imaging, labelling specificity, and large fields of view. Additionally, SMOLM yields enhanced spatial resolution not only for resolving fundamental processes such as Förster Resonance Energy Transfer (FRET) or complex photophysics, but also for imaging nanoscale electromagnetic phenomena in plasmonic or dielectric photonics. In short, molecular orientation is today as fundamental as spatial position for understanding complex nanoscale processes in biological and chemical systems, as well as in nanophotonics.

SMOLM synergistically leverages complementary capabilities of fluorophores, optical hardware, and computational methods to reconstruct multi-dimensional images of molecules and materials with nanoscale precision. An illustration of the benefits of SMOLM over ensemble polarized measurements is given in Fig. 1a in the context of biological imaging, which is also applicable to nanoscale structural imaging. To access structural information in biology, SMOLM requires fluorophores to exhibit a certain degree of rigidity when attached to biomolecules of interest (see SMOLM in biological imaging) (Fig. 1b). While ensemble measurements average out both a protein’s orientational organization over space and a label’s orientational flexibility over time, called “wobble” (Fig. 1c), SMOLM gives access to orientation and flexibility separately, enabling direct structural imaging of molecular dynamics. SMOLM faces however the difficult issue of solving a multi-parameter estimation problem: super-resolved localization in 3D, orientation in 3D, and wobble need to be measured simultaneously from images containing millions of isolated SMs that exhibit low signal to noise (SNR), likely with background present. SMOLM has thus required not only the design of dedicated optical hardware to encode both spatial and orientation information (Fig. 1d), but also modern algorithms to produce robust and precise estimates of the orientations and positions of fluorophores (Fig. 1e–j). The goal of SMOLM is finally to provide insights into the orientational behaviour of SMs in complex systems that could not be assessed by an ensemble measurement (Fig. 1k–m). In this review, we briefly describe the principles of polarization-resolved microscopy of single fluorophores. We then highlight seminal recent advancements in experimental methods and image processing and analysis, focusing on the essential aspects of the most practical approaches. Readers can refer to previous reviews in the field for complementary information on earlier works that paved the way to SMOLM and more details of various techniques.^5,11–13 The following sections review recent impacts of SMOLM in biology and photonics, and we finally give an outlook of the field and its challenges.

Fig. 1 |. Principles of single-molecule orientation localization microscopy (SMOLM).

a, A self-assembled peptide/protein fibre (grey) is labelled with fluorophores (red ellipses) using a linker (yellow). b, Individual peptide/protein domain, linker, and fluorophore (AlexaFluor 488 shown) exhibiting a dipole radiation pattern (red rings). c, A dipole lies at a position ($x,y,z$) with mean orientation ($\xi ,\eta$) (red arrow) and wobble represented as a solid angle $\Omega$ or cone angle $\delta$ (red cone). d, General microscope architecture for SMOLM: Polarized illumination beam (green, ${\mathbf{E}}_{\text{ex}}$), fluorescence (red), objective lens (OL), dichroic mirror (DM), phase mask (PM, pixOL³⁰ shown here), polarizing beamsplitter (PBS, optionally complemented with waveplates, see Table 1), tube lens (TL), image plane (IP). e, Molecules of different positions, orientation, and wobble: i, $(h=0,\eta =\frac{\pi }{2},\xi =\frac{\pi }{6},\Omega =0)$; ii, $(h=0,\eta =\frac{\pi }{3},\xi =\frac{\pi }{6},\Omega =0)$; iii, $(h=0,\eta =\frac{\pi }{3},\xi =\frac{\pi }{6},\Omega =\frac{2\pi }{3})$; iv, $\left(h=200\text{nm},\eta =\frac{\pi }{3},\xi =\frac{\pi }{6},\Omega =0\right)$. f-j, Images of molecules in e from f, unpolarized microscope, g, pixOL,³⁰ h, polarized vortex,³² j,CHIDO ($c=\pi$).³³ (red and blue rows) orthogonally polarized imaging channels. k, Top-left, Fluorescence image of a fixed cell labelled with phalloidin-AF568. Top-right, ensemble polarimetry (zoomed region of white square in top-left image) showing line segments oriented along the mean in-plane dipole orientation measured at each pixel, coloured using ensemble orientational order. Bottom: Similar cells measured with SMOLM, where line segments are coloured using molecule wobble. l, SMOLM imaging of fibres formed by (left) KFE8 and (right) amyloid-beta 42($\text{A}\beta 42$). Line segments are parallel to Nile red’s in-plane orientation, coloured using tilt relative to the fibre backbone. Insets, SMLM. m, SMOLM imaging of Nile red within spherical lipid bilayers of radii (left to right) 150 nm, 350 nm, and $1\mu \text{m}$, with line segments coloured using polar angle. Inset, Cross-sections coloured using azimuthal angle. Bin size,l, 20×20 nm². Colour bars, f-j, normalized intensity; k,l, fluorophore orientation (deg) relative to fibre axis; m, polar angle $\eta$ (deg); m, inset, azimuthal angle $\xi$. Scale bars, f-j, 500 nm; k, $10\mu \text{m}$ (top-left), $2\mu \text{m}$ (top-right), 800 nm (bottom); l, 100 nm; m, 500 nm. Panels adapted with permission from: a, ref.⁵¹ under a Creative Commons license CC BY 4.0; b,k, ref.⁵, Optica Publishing Group; l, ref.⁶⁵, American Chemical Society; m, ref.³⁶, Springer Nature Limited.

Physical principles of SMOLM

SMOLM builds upon single molecule-based super-resolved fluorescence microscopy, recognized by the Nobel Prize in Chemistry 2014. In SMLM,¹⁰ millions of individual well-separated point spread functions (PSFs), or images of single fluorophores, are localized and accumulated to reconstruct an image of the structure of interest with effective resolutions of ~10 nm. In the last decade, researchers have exploited novel illumination strategies, imaging system architectures, and fluorophores’ sensitivity to chemical environments to dramatically improve spatial resolution to <1 nm,^14,15 image at greater depths,^16,17 and decipher biomolecular functions within cells.¹⁰

In contrast, SMOLM utilizes fluorophores’ sensitivity to light polarization to measure its orientation. First, a fluorophore’s probability of absorbing a photon depends upon the molecule’s absorption transition dipole ${\mathbf{\mu }}_{\text{a}}$ and the polarization of the excitation light ${\mathbf{E}}_{\text{ex}}$, i.e., $P_{\text{a}}\propto {\left|{\mathbf{E}}_{\text{ex}}\cdot {\mathbf{\mu }}_{\text{a}}\right|}^2$. Since the amount of fluorescence emitted is linearly proportional to $P_{\text{a}}$ below saturation, one can measure fluorescence in response to a rotating incident linear polarization to calculate fluorophore’s orientation in 2D^8,18 or in 3D using more complex schemes.^19–21 Modulating illumination polarizations has recently been used to induce blinking, thanks to a complementary polarized depletion beam that further increases photoselection contrast.²² It has also been used to improve drastically spatial resolution at low temperature where dipoles are fixed.²³ Nevertheless these approaches require sequential measurements.

Second, most fluorophores are well approximated as radiating linear electric dipoles $\mathbf{\mu }=\left({\mu }_x,{\mu }_y,{\mu }_z\right)=(\text{sin}\eta \text{cos}\xi ,\text{sin}\eta \text{sin}\xi ,\text{cos}\eta )$ (Fig. 1c), which emit light in a toroidal pattern (Fig. 1d).^11,24 One may measure 3D molecular orientation by defocusing the microscope by $~1\mu \text{m}$ when using high-numerical-aperture (NA>1) objective lenses^5,11,24 at the cost of over-extended PSFs that lead to poor signal-to-noise ratios and makes spatial location delicate to assess, especially for dim fluorophores. In contrast, due to the vectorial nature of dipole emission, the focused image of a SM forms a PSF whose shape also depends on a dipole’s orientation, especially for fixed dipoles observed using high NAs (Fig. 1f).^5,11,24 A major breakthrough in SMOLM has been to take advantage of phase masks, polarization optics, and optimization algorithms to engineer this dipole PSF, which we call the dipole-spread function (DSF), to enable sensitive, simultaneous measurements of molecular orientation and position, especially in the presence of wobble and sub-micrometre defocus.¹²

Accurate SMOLM imaging requires therefore a detailed understanding of how microscopes form images of radiating dipoles (Fig. 1e–j), which has been extensively modelled.^11,24–28 A molecule with (emission dipole) orientation $\mathbf{\mu }$ located at a position $\mathbf{r}$ produces an intensity DSF $I(x,y;\mathbf{r},\mathbf{\mu })$ at the image plane that is linearly dependent on the second-order moment matrix $M={⟨\mathit{\mu }\left(t\right)\mathit{\mu }(t{)}^{†}⟩}_T$, where $†$ denotes transpose conjugation and $⟨\cdot {⟩}_T$ an average over the integration time $T$ (Box 1). The second moments $M$ depend on both the dipole’s mean orientation angle ($\xi ,\eta$) and its wobble over time, represented as a solid angle $\Omega$ or cone aperture angle $\delta$ (Fig. 1c). Any DSF may be decomposed conveniently into a linear superposition of DSF basis images (Fig. 2a,b) with weights that are simply the elements of $M$ (Box 1). This model yields direct insights into whether any particular SMOLM method is likely to exhibit poor sensitivity to some $M_{ij}$ parameters, e.g. if the corresponding basis elements are vanishing.

Box 1 |. Orientation models, rotational diffusion, and SMOLM instrument design.

SMOLM senses polarized fluorescence emitted by single molecules. A molecule located at $\mathbf{r}=(x,y,h)$ with an emission dipole $\mathbf{\mu }$ produces an electric field ${\mathbf{E}}_{\text{BFP}}(u,v;\mathbf{r},\mathbf{\mu })$ in the microscope’s back focal plane (BFP), and after the tube lens, forms an intensity image given by^11,24

$$I\left(x,y;\mathit{r},\mathit{\mu }\right)={\left|ℱ\left\{J\left(u,v\right){\mathbf{E}}_{\text{BFP}}\left(u,v;\mathbf{r},\mathbf{\mu }\right)\right\}\right|}^2,$$

which is termed the dipole-spread function (DSF). $ℱ\{\cdot \}$ is the two-dimensional Fourier transform over space $(u,v).J(u,v)$ represents any additional phase masks or polarization elements placed in the BFP to modulate the microscope’s DSF.

Second-order transition dipole moments.

The intensity DSF $I(x,y;\mathbf{r},\mathbf{m})$ can be expressed as a linear combination of basis images $B_{ij}(x,y;\mathit{r})$, with $i,j\in \{x,y,z\}$, and the dipole orientation second-order moments $\mathbf{m}$ averaged over the integration time $T$ of the camera, such that

$$I(x,y;\mathbf{r},\mathbf{m})=N\mathbf{B}(x,y;\mathbf{r})\cdot \mathbf{m},$$

where $\mathbf{m}=\left(m_{xx},m_{yy},m_{zz},m_{xy},m_{xz},m_{yz}\right)$ is defined as

$$m_{ij}=\frac{1}{T}{\int }_{t=0}^T{\mu }_i\left(t\right){\mu }_j\left(t\right)\text{d}t=\frac{1}{2\pi }{\iint }_{S}f(\theta ,\varphi ){\mu }_i(\theta ,\varphi ){\mu }_j(\theta ,\varphi )\text{sin}\theta \text{d}\theta \text{d}\varphi ,$$

$N$ is the number of fluorescence photons detected by the imaging system, and $†$ denotes transpose conjugation. (The second-moment vector $\mathbf{m}$ may be easily rearranged into a Hermitian matrix $M={⟨\mathbf{\mu }\left(t\right)\mathbf{\mu }(t{)}^{†}⟩}_T$.²⁵) The dipole moments $\mathbf{\mu }$ are real quantities for fluorescent molecules, and $f(\theta ,\varphi )$ is the angular distribution of dipole orientations on the unit hemisphere $S$. The second moments $\mathit{m}$ contain information on the mean orientation angle ($\xi ,\eta$) of $f(\theta ,\varphi )$ and on the wobble angle $\Omega$, i.e., the angular width of $f(\theta ,\varphi )$. Importantly, the spatioangular bandlimit imposed by far-field dipolar imaging⁹⁸ projects the orientation trajectory on the unit hemisphere into a 5-dimensional space characterized by $\mathbf{m}$. Therefore, dipoles sharing identical mean orientations but different degrees of wobble (panel $\mathbf{a}$) exhibit different second-moment vectors $\mathbf{m}$ (panel $\mathbf{b}$).

To design a sensitive SMOLM instrument, one may examine its basis images $B_{ij}(x,y;\mathbf{r})$—its intensity response to each second moment $m_{ij}$ (Fig. 2a–d). Images $B_{ij}$ that exhibit excellent contrast, compactness, and orthogonality produce fluorescence images that enable sensitive discrimination between different mean orientations ($\xi ,\eta$) and degrees of wobble $\Omega$.

Stokes-Gell-Mann parameters.

The 6 parameters $m_{ij}$ exhibit a similarity with the elements of the polarization matrix in the context of non-paraxial light.² Therefore, if a dipole produces elliptically polarized light, the second moments $M$ become complex and can be expressed as 9 normalized Stokes-Gell-Mann parameters, $s_n=\frac{\sqrt{3}}{2}\text{Tr}\left(M{\Theta }_n\right)$, with ${\Theta }_n$ representing the Gell-Mann matrices and $n\in \left\{\mathrm{11,12,21,22,23,31,32,33}\right\}.$²⁸ A DSF can be similarly written as a basis decomposition over the Stokes coefficients $\mathbf{s}=\left(s_{11},\dots ,s_{33}\right)$.^28,33

Order parameters and slow rotation.

The DSF has also been expressed as linear combinations of the order parameters $g_n=\iint f(\theta ,\varphi )P_n\left(\text{cos}\theta \right)\text{sin}\theta d\theta d\varphi$, with $P_n\left(\text{cos}\theta \right)$ representing the $n$th order Legendre polynomial.²⁶ This formalism explicitly describes the effect of rotational motion, in which the DSF is a combination of a fast diffusion DSF and a slow diffusion DSF, with weight coefficients given by the relative strength of the rotational diffusion time ${\tau }_r$ and the fluorescence lifetime ${\tau }_f$.²⁶ While fast rotation leads to a dependence on ($g_0,g_2$) only, slow rotational motions lead to a higher-order dependence on the incident excitation field polarization ${\mathbf{E}}_{\text{ex}}$ and ($g_0,g_2,g_4$).

Quantifying SMOLM measurement performance.

If $𝒫(x,y;\mathbf{p})$ is the probability that a photon is present at ($x,y$) on the detector from a dipole with position and orientation parameters $\mathbf{p}$, then the Fisher information (FI) matrix $𝒥\left(\mathbf{p}\right)$ is a measure of the sensitivity of the DSF to $\mathbf{p}$, given by^12,33,42

$$\left[𝒥\right(\mathbf{p}){]}_{ij}\approx N\sum _{x,y}\frac{\frac{\partial }{\partial p_i}𝒫(x,y;\mathbf{p})\frac{\partial }{\partial p_j}𝒫(x,y;\mathbf{p})}{𝒫(x,y;\mathbf{p})}$$

where $N$ is the number of measured photons summing over all camera pixels and polarization projections. Since the microscope DSF is proportional to this probability, its decomposition into basis components $B_{ij}(x,y;\mathbf{r})$ with weights $m_{ij}$ leads to the FI matrix elements

$$\left[𝒥\right(\mathit{m}){]}_{ij,kl}\approx N\sum _{x,y}\frac{B_{ij}(x,y;\mathbf{r})B_{kl}(x,y;\mathbf{r})}{I(x,y;\mathbf{r},\mathbf{m})}$$

The non-diagonal terms of the FI matrix quantify the covariance between estimates of distinct parameters, given by the inner product between two DSF basis components. An upper bound of the determinant of $𝒥\left(\mathbf{m}\right)$, which quantifies overall performance, is found when all its diagonal elements are equal, i.e., when basis DSFs are orthogonal. This insight enables the design of merit functions that must be maximized to achieve optimal measurement precision²⁸ and performance.³⁰

Measuring rotational diffusion using SMOLM.

a, Three orientation distributions that share an identical mean orientation $\left(\eta =\frac{\pi }{3},\xi =-\frac{\pi }{6}\right)$ but exhibit different wobble: (i, red) no wobble ($\Omega =0$), (ii, purple) anisotropic wobble within an elliptical cone with half angles of 15° and 45°, and (iii, green) isotropic wobble within a circular cone of solid angle $\Omega =\pi$. b, Second-moment amplitudes $m_{ij}$ corresponding to the orientation distributions in a. The colour code used is the same as in a.

Fig. 2 |. Measurement and performance assessment in SMOLM.

a,b, Basis images $B_{ij}$ (red and blue, orthogonal polarizations) of a, raPol³⁴ and b,CHIDO³³ ($c=\pi$) DSFs corresponding to second moments $m_{ij}$ with (left to right) $ij\in \{xx,yy,zz,xy,xz,yz\}$. c, raPol and d, CHIDO images corresponding to orientation distributions indicated by (i), (ii), (iii) in Box 1. e, Normalized average Fisher information matrix elements for measuring polar angle $\eta$, azimuthal angle $\xi$, wobble $\Omega$, and 3D position x, y, and z using (left) pixOL³⁰ and (right) CHIDO ($c=\pi /2$). f, Fisher information matrix elements for measuring the normalized Stokes-Gell-Mann parameters $\mathit{s}$ (see Box 1) using CHIDO (left: $c=\pi /2$, right: $c=2\pi$). g, Best-possible precisions (square-root of the Cramér-Rao bound) of measuring mean orientation $\mathbf{\mu }=(\text{sin}\eta \text{cos}\xi ,\text{sin}\eta \text{sin}\xi ,\text{cos}\eta )$, wobble $\Omega$, and (top-right) 2D position $x$ for molecules at a glass-water interface and (bottom-right) 3D position $\mathit{r}$ for molecules at various heights above a glass-water interface. $z_f$, focal plane position; $h$, molecule axial position. Blue, xy-polarized (xyPol); red, radially and azimuthally polarized (raPol); yellow, CHIDO($c=\pi /2$); purple, polarized vortex;³² green, pixOL; cyan, multi-view reflector (MVR).³⁶ h, (top) Precision and (bottom) accuracy of measuring 3D position $x$ and $z$, mean orientation $\mu$, and wobble $\Omega$ using MVR and a regularized maximum-likelihood estimator, calculated via Monte-Carlo simulations. Colour bars, a-d, normalized intensity; e, f, normalized Fisher information; precision (units; h, top); bias (units; h, bottom). Scale bars, 500 nm. Various numbers of signal and background photons were utilized in these evaluations; see references for details. Panels adapted with permission from: e, ref.³⁰, Optica Publishing Group; f, ref.²⁸, Optica Publishing Group; g, ref.¹¹, Cambridge Univ. Press; h, ref.³⁶, Springer Nature Limited.

In the last ten years, different strategies have been developed to measure orientations efficiently from DSF images (Fig. 2c,d). In Table 1, we distinguish (I) pure intensity measurements (i.e. integrated DSFs) from (II) encoding information into the shape of DSFs (i.e. DSF engineering). Methods in (I) are based on using different (1) excitation polarizations or (2) emission polarizations, while methods in (II) are based on the use of (1) pupil splitting, (2) phase masks, or (3) polarization masks made of spatially varying birefringence. Among these methods, DSF engineering is the most flexible for accessing both orientation and spatial location, thanks to many innovations that have been deployed over past decade and more: pupil-splitting methods (bisected and quadrated pupils²⁴ and duo/tri-spot methods²⁹) rely on phase masks made from spatial light modulators (SLM) programmed with linear phase functions. SLMs have also been used for polarized double helix,^5,11,13 pixel-wise optimized (pixOL³⁰) and vortex-shape phase masks.^31,32 Finally, birefringent polarization masks have been employed in CHIDO,³³ which relies an optics where stress is applied to provoke a radially varying birefringence, and raPol, which generates a radially and azimuthally polarized DSF using a vortex half wave plate.^34,35 The raMVR microscope extends raPol by adding multi-view reflector mirrors in both radially and azimuthally polarized imaging channels.³⁶ Each methods’ experimental implementation, advantages, limitations, and reported performance are given in Table 1.

Table 1 |. SMOLM techniques and applications.

Note that measurement precision scales as $1/\sqrt{N}$, where $N$ is the number of photons detected.

Technique		Benets	Limitations	Demonstrated applications	Angular precision (measurement conditions)
(I) Intensity-based methods	(1) Illumination polarization modulation. Uses Pockels cells or other polarization-control optics.	Relatively simple implementation, simple intensity-based orientation measurements, preserves compact DSFs	2D localization only, SM must emit stably for multiple frames, necessitates an accurate control of the illumination polarization state	DNA structure imaging18	5.8° mean 2D angular precision18 (4800 photons over 3 frames, 27 background photons per pixel, 40 ms exposure time)
	(2) Polarization splitting: 2polar, 4polar,51 xyPol, Polcam.52 Uses polarizing beamsplitters, waveplates, and/or polarizers.	Simple implementation, simple, intensity-based orientation measurements, preserves compact DSFs	2D localization only, 3D orientation and localization necessitate NA manipulation or DSF tting with known parameters (e.g. position of nominal focal plane)	Actin imaging in cells,51,52 DNA and protein lament imaging,39,47,49,50,52,66, lipid membrane imaging,52 conformational tracking of membrane proteins in vitro,59 nanorod orientation tracking94	4polar:51 <10° mean 2D angular precision (1500 photons, 60 background photons per pixel, 100 ms exposure time); Polcam:52 7.5° mean 2D angular precision (500 photons, 50–200 ms exposure times)
(II) Dipole-spread function (DSF) engineering	(1) Pupil splitting: bisected, quadrated pupils;24 duo-spot, tri-spot DSFs;29 raMVR,36 annular split.69 Uses segmented/annular mirrors or phase masks.	Robust intensity-based orientation measurements, robust to aberrations	Complex construction and alignment, enlarges DSF size	Amyloid lament imaging,36 lipid membrane imaging29,36	Tri-spot:29 ${\sigma }_{\eta }={6.0}^{\circ },{\sigma }_{\xi }={10.6}^{\circ },{\sigma }_{\Omega }=0.11\pi \text{sr}$ (1300 photons, 5 background photons/pixel, 30 ms exposure time); raMVR:36 2.0° mean 3D angular precision, 0.23 sr wobble precision (5000 photons, 40 background photons per pixel, 100 ms exposure time)
	(2) Phase mask engineering: double helix,5,11,13 pixOL (Fig. 1g),30 vortex (Fig. 1h),31,32 arrowhead46 Uses phase masks.	Customizable implementation: can balance DSF compactness with orientation-position sensitivity	Sensitivity to optical aberrations	DNA and amyloid lament imaging,31,32,65 lipid membrane imaging30,32, protein organization at interfaces of biomolecular condensates72	pixOL:30 4.1° mean 3D angular precision, 0.44 sr wobble precision (2500 photons, 3 background photons per pixel, 110 ms exposure time); polarized vortex:32${\sigma }_{\eta }={4.5}^{\circ },{\sigma }_{\xi }={7.7}^{\circ },{\sigma }_{\delta }={26.4}^{\circ }$ (510 photons, 2.3 background photons per pixel, 20–50 ms exposure times); Arrowhead:46${\sigma }_{\eta }=2^{\circ },{\sigma }_{\xi }=3^{\circ }$ (2500 photons, 25 background photons per pixel)
	(3) Polarization mask engineering: Stress engineered optics CHIDO (Fig. 1j, Fig. 2b,d),33 raPol (Fig. 2a,c).34,35 Uses polarization masks.	Customizable and exible implementation: can balance DSF compactness with orientation-position sensitivity	Sensitivity to polarized aberrations	Actin lament imaging,33 nanorod orientation tracking,35 lipid membrane imaging34	CHIDO:33 ${\sigma }_{\eta }={0.8}^{\circ },{\sigma }_{\Omega }=0.13\text{sr}$ (using 40000 photons, 1000 background photons per pixel, 0.2–1 s exposure times); raPol:34 1.5° mean 3D angular precision, ${\sigma }_{\Omega }=0.16\text{sr}$ (using 5000 photons, 30 background photons per pixel, 100 ms exposure time)

Importantly, illumination polarization can significantly impact the molecular orientations that are observed in SMOLM. If rotational diffusion times are long compared to the fluorescence lifetime, then the orientation of the emission dipole is strongly coupled to that of the absorption dipole. This effect introduces a bias^31,34,37 that can be corrected using models that consider higher-order dipole moments (Box 1).²⁶ One may also avoid photoselection effects by creating nearly isotropic vectorial illumination patterns, e.g., by using multi-angle illumination¹⁹ or sequential complementary polarization illuminations.^20,38

SMOLM implementations

Assessing performance.

We may view SMOLM imaging as a two-step process: 1) an algorithm must detect or identify molecules within noisy fluorescence images, and 2) it must estimate the orientation and position of every molecule detected within those images. To identify isolated molecules and not overlapping SM images, one may employ simple intensity thresholding, techniques from statistical signal processing such as likelihood ratio tests,³⁹ and machine learning.⁴⁰ An image-analysis algorithm has superior performance if it can robustly detect dim molecules in the presence of background autofluorescence, while also minimizing false detections.

The second step is parameter estimation, i.e., “fitting” values of molecular brightness, orientations, and positions that match the observed images (Fig. 2c,d). One particularly useful performance metric is the Cramér-Rao bound (CRB) (Box 1), which states that the variance $\text{Var}\left({\hat{p}}_i\right)$ of any unbiased estimate ${\hat{p}}_i$ is bounded by the corresponding element of the inverse of the associated Fisher information (FI) matrix⁴¹

$$\text{Var}\left({\hat{p}}_i\right)\ge {\left[{𝒥}^{-1}\left(\mathbf{p}\right)\right]}_{ii}\ge {\left(\left[𝒥\right(\mathbf{p}){]}_{ii}\right)}^{-1},$$

where $p_i$ is the $i^{\text{th}}$ parameter we wish to measure and the subscript $ii$ denotes the $i^{\text{th}}$ diagonal element of the FI matrix $𝒥\left(\mathbf{p}\right)$. Importantly, Fisher information is independent of the image analysis algorithm to be used. It also explicitly accounts for trade-offs inherent in estimating molecular position and orientation parameters simultaneously. A diagonal FI matrix, in which ${\left[{ℐ}^{-1}\left(\mathbf{p}\right)\right]}_{ii}={\left(\left[𝒥\right(\mathbf{p}){]}_{ii}\right)}^{-1}$, is optimal in the sense that all position and orientation parameters can be estimated independently with no crosstalk; nonzero off-diagonal matrix elements represent covariances that degrade overall measurement precision (Box 1, Fig. 2e).^28,30 The CRB has been used to quantify the fundamental performance limits of SMOLM.^11,42 Fundamental limits have also been derived for measuring wobble,^42–44 which is prone to imprecision and inaccuracy especially in the presence of background.²⁸

DSFs may also be decomposed into basis elements, e.g., using the Stokes-Gell-Mann parameters $\mathit{s}$,²⁸ to estimate a method’s efficiency. In this representation, the FI matrix $𝒥\left(\mathbf{s}\right)$ characterizes their mutual local orthogonality; again, large off-diagonal entries imply significant correlations between basis elements that reduce precision (Box 1, Fig. 2f).^28,30,33 The DSF basis expansion can also be used to assess robustness to aberrations and to defocus, for which approaches using polynomial expansions and pixel-wise phase retrieval have been introduced.^31,33,45

Comparing approaches.

Choosing the optimal SMOLM method for a particular application is a careful balance of maximizing detection accuracy, maximizing estimation precision and accuracy, matching the instrument’s usable depth range to the sample, and balancing hardware and software complexity with capability. Fisher information has been used to compare techniques,¹¹ and high-performance DSFs have been designed by using machine learning software libraries to optimize phase masks pixel-by-pixel to maximize Fisher information.³⁰ A recent study has shown explicitly how DSF compactness affects FI and SMOLM performance, and this insight yielded an optimal pupil-splitting scheme with a nearly diagonal FI matrix for orientation measurements.²⁸ In addition, deep learning has proven powerful for both designing SMOLM phase masks and SMOLM image processing.⁴⁶

Comparing various SMOLM approaches reveals trade-offs in spatial vs. angular parameter estimation performance, as well as precision variations when imaging molecules within thin 2D vs. thick 3D volumes (Fig. 2g). At last, to test hardware and software end-to-end, Monte Carlo simulations may be used to evaluate detection and parameter estimation algorithms on simulated noisy DSF images (Fig. 2h). In this way, both measurement precision and bias can be assessed and optimized where possible.

Considering the diverse range of DSF engineering methods (Table 1), one can finally construe that using smooth phase and/or polarization variations in the pupil plane directly leads to compact and high-performing DSFs. That is, raPOL,³⁴ polarized vortex,³² and CHIDO³³ are capable of a good compromise in (i) reaching high estimation efficiency and low inaccuracy, even within a large z depth range and in the presence of background, (ii) preserving a compact DSF size, allowing high density measurements, and (iii) keeping the experimental implementation simple. Polarization-splitting methods such as xyPol^39,47–50 and 4polar-SMOLM^51,52 (Table 1) remain popular and valuable for their extremely simple implementation and compact DSFs, providing that some capabilities can be sacrificed, such as 3D position estimation. Polarization splitting is also the closest technology that is compatible with commercial microscopy modules, thanks to polarized cameras.⁵² In light of the aforementioned trade-offs, it is unlikely that any particular DSF design can be optimal universally for all imaging scenarios, regardless of the performance metric of interest.

Image processing and analysis.

Robustly extracting position, orientation, and wobble parameters from images of dim SMs is challenging. In both SMLM and SMOLM, maximum likelihood estimation (MLE) is widely used and is accurate and precise in a variety of scenarios.^11,30,31,51 However, MLE is computationally expensive and slow for SMOLM due to the many parameters that must be estimated for each molecule.⁵² With sufficient signal, one may accelerate analysis by projecting an SM image directly onto basis DSFs (Box 1),³³ provided that this basis is orthogonal and precisely calibrated. Inspired by machine-learning methods for SMLM, deep learning has been used for measuring 2D positions and 3D orientations in SMOLM.^40,46,53 Given the immense interest and power of these techniques, we anticipate many new developments in this area, but caution is warranted due to the complexity of estimating many parameters simultaneously and the severe Poisson shot noise inherent to SMOLM.

Importantly, one needs a precise calibration of the imaging system to estimate molecule positions⁵⁴ and orientations accurately. In SMOLM, conventional scalar diffraction models are insufficient, because they ignore the vectorial nature of dipole emission and light collection using a high-NA objective lens. Recent methods use vectorial diffraction models,^{30–32,34,45} combined with nanobead measurements⁵⁵ or data from single molecules,⁵⁶ to estimate optical aberrations. Importantly, SMOLM implementations based on spatially varying polarization and phase masks, such as CHIDO³³ or raPOL,³⁴ may exhibit polarization-dependent aberrations, necessitating a retrieval of a full Jones matrix to model coupled aberrations between polarization channels.⁵⁷ At last, techniques that use separate imaging channels to measure orientation such as 4polar⁵¹ and raMVR³⁶ SMOLM are the least aberration sensitive, provided that intensity is properly estimated.

SMOLM in biological imaging

SMOLM is remarkably valuable for investigating molecular organization and building mechanistic understanding of biological phenomena. We cover major examples below.

The need for rigid fluorescent labels.

Since polarized microscopes measure the orientations of fluorescent labels and not biomolecules themselves, fluorophores must be rigidly linked to the protein of interest to ensure that they do not rotate independently of the target. This rules out the use of large linkers such as antibodies. Rigidity can be provided by the use of bifunctional rhodamines or cyanines covalently linked to pairs of cysteine residues,^58,59 small binding conjugates to actin (phalloidin, SiR-actin),⁴⁷ intercalants within DNA and amyloid aggregates,^39,47,50 and the insertion of lipid probes within lipid membranes.^29,60 In live cell imaging, reducing the length or mobility of linkers between fluorescent proteins and the protein of interest has been used to study membrane and cytoskeletal proteins.^61,62 Finally, label rigidity dramatically increases at low temperature, enabling spatial localization down to the Angstrom level thanks to fine polarization encoding.⁶³ This unprecedented level of detail was obtained thanks to a strong polarization dependence, acting as a marker for specific protein domains (Fig. 3a).

Fig. 3 |. Biological applications of SMOLM.

a, Polarized SMLM images of a ClpB hexamer protein from which different classes of images are obtained and merged to reconstruct in 3D the measured labels (red) over the protein crystal structure (right). b, SMOLM image of $\lambda$-DNA labelled with the intercalator Sytox Orange, coloured as a function of the azimuthal angle. Lower-left, schematic drawing of a plectoneme; lower-right, orientation distribution measured by SMOLM within a plectoneme region aligned along x. c, Top, Fluorescence images of optical tweezers used to align and stretch a DNA molecule densely coated with YOYO-1, using polarized excitation (ex) and emission (em) in the sample plane, indicated as _exI_em. Top pair of images, overstretching transition; middle pair, beyond the overstretching transition. Bottom, schematic of the intermediate regime. d, SMOLM image of amyloid-beta ($\text{A}\beta 42$) fibrils labelled with transient binding Nile red, coloured using azimuthal angle. e, SMOLM imaging of Nile red transiently binding to (top) $\text{A}\beta 42$ and (bottom) KFE8 fibrils. Left, Nile red binds parallel to fibrillar backbones, with its 3D orientations visualized from the fibril axis (${\mathbf{\text{u}}}_x$). Middle, SMLM image. Right, SMOLM image with line segments oriented and coloured using each fluorophore’s in-plane orientation. f, SMOLM of fixed U2OS cells labelled with phalloidin-AF568. The azimuthal angle of a single molecule is encoded in colour (left) and sticks angles (right), complemented by the wobble cone angle (middle). g, SMOLM imaging of ordered and disordered lipid domains within a ternary lipid mixture, labelled with (i) MC540 or (ii-iv) Nile red. (i) SMLM image, (ii) wobble, and (iii) mean polar angles from single molecules, and (iv) ‘phase index’ combining these two variables to emphasize the existence of ordered domains. Scale bars, a,3 nm; b, $10\mu \text{m}$; c, $5\mu \text{m}$;d, $1\mu \text{m}$; e,100 nm; f, $6.5\mu \text{m}$ (left), $1.3\mu \text{m}$ (right); g, $2\mu \text{m}$. Panels adapted with permission from: a, ref.⁶³ under a Creative Commons License CC BY 4.0; b, ref.³¹ under a Creative Commons License CC BY 4.0; c, ref.⁴⁸, AAAS; d, ref.⁴⁹, Optica Publishing Group; e, ref.⁶⁵, American Chemical Society; f, ref.⁵¹, under a Creative Commons License CC BY 4.0; g, ref.²⁹, Wiley.

DNA structures.

The conformation of DNA governs important processes in genomics, defect recognition and repair, as well as drug-DNA interactions. In vitro SMOLM imaging of dsDNA intercalants (e.g., YOYO-1, Sytox Orange) have reported local DNA architectures using excitation polarization imaging,¹⁸ polarization splitting,⁴⁷ or vortex³¹ imaging (Fig. 3b). Combined SMOLM-optical tweezer assays have shown that beyond the DNA overstretching transition, intercalants undergo a tilt of their orientation, providing fine insights into DNA mechanics (Fig. 3c).⁴⁸ SMOLM could benefit the field of mechanobiology, in which designer oligonucleotide strands confer intricate control over the orientation and rigidity of DNA structures to probe local tension in cell adhesion via FRET or quenching for instance.⁶⁴

Amyloid aggregates in vitro.

The nanoscale organization of protein aggregates, their growth and decay mechanisms, and the relation of these factors to neurodegenerative diseases are still poorly known. SMOLM has brought new insights into nanoscale structural heterogeneities of fibrils made of amyloid-$\beta$ and $\alpha$-synuclein peptides using both engineered and pathological aggregates labelled with transiently binding thioflavin T (ThT), Nile red, and Nile blue via polarized^39,49,50,52 and vortex DSFs^32,65 (Fig. 3d). Recently, SMOLM was able to image local polymorphism in the helicoidal architectures of $\beta$-sheet fibers⁶⁵ (Fig. 3e) and its evolution over time, in which a correlation between local order and fibril growth was evidenced.⁶⁶ These measurements complement other approaches such as spectrally resolved SMLM, which produces hydrophobicity maps of these structures with nanoscale precision.^67,68

Dense cellular cytoskeletons.

Imaging actin organization in cells is challenging because of the high density of F-actin filaments in the cell cytoskeleton. 2D polarization-splitting SMOLM^47,51,52 has revealed a high order with nearly parallel F-actin filaments in stress fibres (SFs) and filopodia, in line with their contractile behavior.⁵¹ In contrast to SFs, the cell cortex and lamellipodium observed at the front edge of migrating cells confirmed their more complex organization (Fig. 3f).⁵¹ This SMOLM scheme was also applied in live cells, revealing anisotropy in the tracking of F-actin clusters⁶¹.

Lipid membranes and membraneless condensates.

Nanoscale heterogeneities occur within artificial and cell membranes with fast dynamics that are delicate to assess.⁶⁹ The time-averaged orientation of single transiently bound Nile red (NR) lipid probes imaged by tri-spot SMOLM in supported lipid membranes (SLBs) has shown its sensitivity to the membrane composition and packing, evidencing a decrease in wobble in presence of saturated lipids and cholesterol (Fig. 3g).²⁹ Using fast position and orientational tracking, raPol SMOLM later quantified diffusion jump characteristics of cholesterol-rich nanodomains.³⁴ Radially and azimuthally polarized multi-view reflector (raMVR) SMOLM was also applied to study of lipid-peptide interactions in contact with amyloid aggregates and to image the organization of cell membranes.³⁶ Thus, molecular orientation measurements complement both spectral imaging of the local membrane polarity using solvatochromic probes, as well as fluorescence lifetime imaging of local viscosity and membrane tension using molecular flippers and rotors.⁶⁰

More recently, SMOLM has been applied to study biomolecular condensates, also known as membraneless organelles, which play prominent roles in many diverse cellular processes. They form when biological macromolecules such as proteins, DNA, and/or RNA undergo phase separation, and their dynamic spatial architecture and viscoelastic behaviour are of great interest. Complementing SM tracking studies,^70,71 pixOL SMOLM has revealed specific orientations of single merocyanine 540 (MC540) molecules at the interface of single-component protein condensates in vitro.⁷² Simultaneous spatial and orientational tracking of single molecules are of great interest to study interactions between biomolecules in such complex environments; however, the intrinsic low signal-to-noise conditions present significant challenges.

SMOLM in photonics

SMOLM is also applicable to the investigation and control of phenomena in nonbiological photonic materials and devices. Below, we illustrate a few examples.

Organic light-emitting diodes (OLEDs).

The control of the orientation of luminescent molecules in OLEDs is important to optimize their performance. In-plane orientations are known to enhance carrier mobility and light out-coupling,⁹ however the interaction between emitters and the host molecules is generally delicate to assess. SMOLM defocused imaging of coumarin-based molecules at low concentration in a vacuum deposited OLED has shown fabrication-dependent heterogeneous orientations within typical host materials.⁷³ This approach opens novel tools for single molecule opto-electronics⁷⁴ in general, which also apply to biosensing devices, e.g., electrostatic gate tuning in semiconducting carbon nanotubes based on the control of protein orientation.⁷⁵

2D nanomaterials.

An exciting material for controlled single photon emission is 2D hexagonal boron nitride (h-BN) crystals, whose optical properties are controllable by irradiation or through the chemisorption of organic molecules. Polarization-splitting SMOLM has found that at the solid-liquid interface, emitters align along the symmetry axis of the h-BN lattice and can be influenced by the regulation of the electrochemical environment.⁷⁶ Thus, SMOLM can be used to relate dynamic molecular conformations to photophysics, even when there are complex interactions with a 2D material.

Programmable nanomaterials.

DNA origami provides a rich platform to fabricate programmable nanomaterials that can be incorporated into photonic devices, including biosensing platforms, nanoplasmonics, and hybrid photonic systems.⁷⁷ Excitation polarization SMOLM of SMs attached to origamis have shown that their orientational mobility and orientation can be controlled by the removal of nucleotides at their vicinity, producing a preferred orientation.⁷⁸ This strategy was further extended to intercalation and stretching, using defocused imaging⁷⁹ or polarization-splitting SMOLM on cyanine dyes which are doubly linked to oligonucleotide strands with controlled inter-distances (Fig. 4a).⁸⁰ The control of dye orientations using such molecular architectures opens novel routes for optimizing coupling to resonant nanostructures and the creation of original devices such as color-encoded polarizers.⁸¹

Fig. 4 |. SMOLM in photonics.

a, DNA origami labelled with Cy5 dyes with adjusted linkage conditions to the double-helix of dsDNA with 0 and 8 bases unpaired, showing contrasted behaviours. b, DNA-PAINT of ATTO655 labelling freely diffusing ssDNA imager strands on a 100-nm gold spherical nanosphere (AuNS), covered with ssDNA docking strands. Both experimental and theoretical DSFs are shown, allowing retrieval of the 3D position of the dyes on the AuNS, projected onto the spherical NS to map the binding sites in 3D. c, Silicon core-shell nanoparticle of size 158nm illuminated by a tightly focused beam on a glass substrate, from which the scattering pattern is measured above the critical angle. Experimentally measured (left x component, right y component) and theoretical focal distributions (insets) of the magnetic transverse spin density of tightly focused azimuthally polarized beams. d, Back focal plane polarized imaging of spin-polarized light emitted by a linearly polarized dipole, realized from the scattering of a nanoparticle (AuNS, 40nm) close to a glass substrate interface. e, 3D orientation imaging of nanorods moving by Brownian motion, measured by SMOLM in darkfield illumination. SMOLM DSFs are shown at different times (insets, theoretical DSFs) together with the reconstruction of the single nanorod 3D orientation. f, Imaging of a mixture of gold nanospheres and nanorods using (A) traditional dark-field, (B) dark-field Calcite-Assisted Localization and Kinetics (CLocK) Microscopy and (C) SEM. Scale bars, b, 200 nm; c, 100 nm; e, $3\mu \text{m}$; f, $5\mu \text{m}$ (A,B), 50 nm (C); g, $2\mu \text{m}$. Panels adapted with permission from: a, ref.⁸⁰, American Chemical Society; b, ref.⁸⁶, American Chemical Society; c, ref.⁸⁹ under a Creative Commons License CC BY 4.0; d, ref.⁹³ under a Creative Commons License CC BY 4.0; e, ref.³⁵ under a Creative Commons License CC BY 4.0; f, ref.⁹⁶, American Chemical Society.

Nanoplasmonics.

Imaging SM localization together with their intensity or lifetime has great potential for biological imaging⁸² but also for mapping the local density of states at the proximity to metallic or dielectric structures.⁸³ This task is nevertheless quite delicate, not only because the coupling that governs the fluorescence intensity depends on the molecule’s orientation, but also because the recorded image used to locate the molecule can be strongly deformed by the local electromagnetic environment imposed by the metal particles (Fig. 4b).⁸⁴ SMOLM can assist the development of robust models accounting for the dipole’s orientation, as recently shown using DSF fitting^85,86 or defocused imaging.⁸⁷ This assessment is very important for the future development of optoelectronic devices based on SM junctions.⁸⁸

From fluorescence to scattering.

DSF engineering methods originally designed for fluorescence microscopy are extremely versatile and adaptable to imaging scattering particles. Using sufficiently small metal or dielectric particles whose scattering matrix is known, the assessment of scattering patterns in phase and polarization at the pupil plane of a microscope has allowed measurements of local optical fields in 3D (Fig. 4c).^89,90 Measuring vectorial scattering from high refractive index silicon particles permits one to fully characterize microscope objectives⁹¹ and to produce controlled polarized emission patterns (Fig. 4d).^92,93 In contrast to fluorescence however, scattering sources of emission are produced by complex dipoles, therefore necessitating estimating 9 Stokes parameters to account for full elliptical light polarization in 3D.²⁸ SMOLM approaches have also been combined with dark field imaging to track the orientation of anisotropic metal particles such as gold nanorods (< 100nm length) at high speed, using polarization splitting,⁹⁴ defocused imaging,⁹⁵ raPol (Fig. 4e),³⁵ or a rotating birefringent crystal (Fig. 4f).⁹⁶ Emerging applications appear today in cell imaging,⁹⁷ as well as implementations for enhanced sensitivities using interferometric scattering microscopy.⁹⁴

Future challenges

SMOLM is still a growing field whose instrumentation faces demanding hurdles. A challenge and opportunity lie in increasing the number of measured parameters, e.g., quantifying more detailed wobble distributions up to the fourth order of symmetry^26,38,98 and measuring ellipticity in the emitted light occurring from local fields or from magnetic dipole radiations in chiral nanostructures, which are of growing interest.⁹⁹ Among the important problems to address is measuring the 3D position of an emitter together with its full Stokes-Gell-Mann complex expansion, for instance from circularly polarized emitters oriented in 3D. Increasing the number of measurement parameters potentially increases their covariance and likely presents performance tradeoffs.⁴² The search for optimal SMOLM instruments in this context is then likely to employ programmable instruments augmented by machine learning that can adapt as measurements are collected and/or more complex light-matter interactions, possibly involving hybrid molecular and metallic nanostructures capable of probing or controlling local chirality.¹⁰⁰

Imaging faster dynamics is another important challenge to surpass, which is partially solved thanks to the development of event cameras,¹⁰¹ intensified CMOS cameras,¹⁰² and SPAD arrays.¹⁰³ Additionally, novel ways to encode localization information with time could benefit SMOLM.¹⁰⁴ Moreover, imaging dynamics at ~ ns time scales, close to an emitter’s fluorescence lifetime, would provide even richer information, e.g., quantification of their rotational diffusion time, which is related to an emitter’s local environment. Such capabilities would transform studies of molecular interactions in dynamic media that are hard to explore, e.g., in liquid condensates, heterogeneous lipid environments, and dense environments with heterogeneous packing. Such fast rotational dynamics requires time-resolved detection and novel measurement strategies.¹⁰³

The spectral dimension is also an important parameter to combine with orientation, since many properties of an environment can be probed through the spectral characteristics of molecules, for instance using polarity or viscosity-sensitive probes.⁶⁰ Adding spectral detection to SMLM, which could benefit spectral-SMOLM, is an active research area.^105,106 Other interesting strategies use the spectral dimension to access 3D orientation of single emitters, which exhibit polarization-dependent spectroscopic properties thanks to multiple transitions of different symmetries.^107,108

Finally, while SMOLM currently uses relatively simple illumination geometries, it is applicable to more complex schemes that provide specific interest. An example is extended depth of field imaging, e.g., multifocus microscopy (MFM), in which grating-based projection of different planes is compatible with 4polar imaging.¹⁰⁹ Another example is light field microscopy (LFM),^16,110 which can directly be coupled with polarized imaging based on pupil-splitting strategies. At last, polarization-resolved imaging beyond superficial depths is made possible by approaches such as light-sheet microscopy, which has been recently extended to polarization resolved imaging.^21,111 It is therefore likely that in a few years, SMOLM will explore molecular organization in more complex biological situations, such as tissues or live samples.