An affordable fiducial marker strategy for reliable autofocus in long‐term live microscopy

Abstract

Long‐term time‐lapse imaging of biological samples requires correcting for focal drift, which would otherwise gradually push the sample out of focus. We present a software‐based method that eliminates this time‐dependent blur using only a motorised Z‐drive, with no additional hardware. The method relies on imaging marks made on the side of the coverslip opposite to the sample. We provide a Beanshell script implementation, evaluate its performance across multiple objectives, and benchmark it against a hardware autofocus system, finding comparable results. Finally, we demonstrate its effectiveness in live imaging of growing bacterial colonies.

1. INTRODUCTION

Time‐lapse wide‐field microscopy is a widely used method to image live biological samples that are thin compared to the depth of field of the microscope objective. Examples of experimental setups when optical sectioning is not required range from small colonies of microorganisms on agarose pads, ¹ , ² to bacteria in a mother machine ³ , ⁴ or a micro‐chemostat, ⁵ to animal cells in microfluidic channels. ⁶ A common issue affecting image quality in such experiments is the variability of the relative position of the microscope objective lens and the sample due to thermal drift, sample aging, and mechanical distortion caused by pressure changes during the operation of microfluidics chips. ⁷ The resulting movement of the imaging focal plane must be corrected, otherwise the image will go out of focus. Since imaging can last many days, manual correction is not practical, and some sort of automated focus adjustment is usually required.

Most research‐grade microscopes can be equipped with a motorised stage, a motorised objective turret, or a piezo objective scanner which can move the objective in the Z axis (towards/away from the sample). Automated focus correction can be implemented either in the imaging software or as a separate hardware module. Although software‐based methods have the advantage of not requiring additional hardware, their performance suffers when the sample does not have sharp, well‐defined features or when the features change over time. Hardware‐based systems generally use techniques based on monitoring the distance between the objective and a surface of the sample with a sharp change of the refractive index; this usually coincides with the surface of a glass slide or the bottom of a microplate or a culture dish. These systems work well in many scenarios but are expensive and require compatible objectives. Moreover, hardware methods are not entirely immune to problems affecting software‐based methods, such as significant changes of the optical properties of the sample, for example, due to biological growth, air bubbles in microfluidic channels, and sample ageing.

Here we propose an ‘imPerfect Focus System’ (iPFS) – a method which uses a zero‐cost fiducial marker ⁸ associated with a simple, yet effective, software‐based method. The only non‐software requirement is a computer‐controlled hardware for moving the objective in the optical (Z) axis. Despite the name, we will demonstrate that the method can be as accurate as the hardware‐based Nikon Perfect Focus System. Our method uses a computer algorithm to track user‐made markers on one of the rigid surfaces of the sample (glass slide, cover slip, or plastic dish). Changes in the Z coordinate of this reference surface are then translated into offsets applied to imaging positions. Markers can be added using a Sharpie, or by utilising natural imperfections such as dust, grease, or crystal deposits already present on the surface of microscopic slides. This method is compatible with both transmitted‐light and fluorescence imaging, allows independent focusing across multiple XY positions (fields of view), and does not interfere with optical imaging as long as at least 10–20 s are permitted between successive measurements.

1.1. Overview of software‐based focus methods

There are many software‐based autofocus methods that rely on numerical techniques to objectively quantify image sharpness, which can then be used to bring the sample into focus. Earlier approaches estimate sharpness by applying a mathematical function that assigns a numerical ‘sharpness’ value to the image, often using filtered outputs such as when using the Laplacian or Sobel operators. Other approaches looking at the power spectrum of the image ⁹ or leverage how information is encoded in the spectral domain the Discrete Cosine Transform which is used in JPEG compression, ¹⁰ some even define sharpness inferred from how human can perceive the minimal amount of ‘Just noticeable blur’. ¹¹ However, most of these methods assume that the sample contains sufficient structural features for the sharpness metric to exhibit a clear, rapidly decaying peak away from the focal plane–in other words, a well‐defined local maximum. Additionally, the sharpness peak must be well separated from other local maxima, otherwise the algorithm may randomly jump between these maxima. These assumptions may not hold for certain sample types, such as thick biological specimens. Moreover, for such samples one may be interested in imaging a different plane to that for which the sharpness function is maximised.

The sharpness function may also be rugged, that is, exhibit many local peaks caused by the sample structure and intrinsic optical and camera noise, and while some methods claim to alleviate this problem, ¹² they are generally only applicable to samples exhibiting a well‐pronounced single global sharpness maximum.

More recent machine learning approaches replace traditional filters with deep convolutional networks, which can provide improved generalisation, ¹³ or even reconstruct a synthetic in‐focus image from a single real defocused image. ¹⁴ However, such approaches generally rely on the imaged specimen being similar to what the algorithms have been trained on, and may exhibit reduced performance if the real sample type is not adequately represented in the training dataset.

Growth presents another challenge: as a sample evolves over time, its sharpness characteristics can change significantly. For example, Escherichia coli colonies may transition from one‐dimensional to two‐dimensional structures, ¹⁵ effectively increasing sample thickness. This significantly changes the sharpness function, often breaking the two assumptions mentioned earlier (a sharp global maximum isolated from other local maxima).

1.2. Overview of hardware‐based focus methods

We discuss here two hardware‐based focus systems: Nikon's Perfect Focus System ¹⁶ and an open‐hardware alternative pgFocus. ¹⁷ Systems used by other microscope manufacturers (Zeiss, Leica, Revvity) and do‐it‐yourself alternatives generally follow similar principles to these two systems.

1.2.1. Commercial systems: Nikon Perfect Focus System 4

This hardware‐based system will serve as our reference for benchmarking. The system relies on partial reflection of infrared light at the sample's air–glass or water–glass interface. Light from an infrared LED passes through a narrow slit and is directed through the imaging objective onto the sample, where it is partially reflected back to the objective and captured by a linear CCD sensor. The position of the resulting intensity peak on the sensor indicates the distance between the objective and the reflective surface. An electronic controller maintains this distance fixed by adjusting the position of the objective along the Z axis. An additional lens inserted in the optical path of the infrared beam allows the user to introduce an offset between the reflective surface and the imaging plane, enabling imaging objects above or below the reflective surface. A detailed explanation is provided on Nikon's website. ¹⁸

The system works in real time (200 measurements/s) and its claimed accuracy is $1/3$ of the objective focal depth. ¹⁸ However, actual performance depends on both the sample being imaged and the type of objective used. With water‐ or oil‐immersion objectives, the primary reflection occurs at the glass–water interface, near the location of most biological samples. In contrast, for dry objectives the primary reflection occurs at the air–glass or air–transparent bottom interface, which can be hundreds of micrometres below the sample, depending of the substrate thickness. If the substrate thickness is uneven – a common issue for plastic‐bottom plates – manual correction may be needed independently for each field of view. The system is also optimised for substrates with a refractive index close to that of standard microscopic cover slips ($n\approx 1.5$) and may perform poorly with some plastic‐bottom plates. Moreover, certain objectives are incompatible with the PFS system. Another limitation is the lack of an open‐source API for the PFS, which makes writing custom acquisition software difficult.

1.2.2. pgFocus

The PFS and similar systems can represent a significant portion of the cost of an already expensive automated microscope. Moreover, these solutions may not perform optimally for certain applications. Consequently, open‐source alternatives have been developed. One example is pgFocus (‘pg’ stands for pretty good), which can be assembled from readily available components. The schematics and component part numbers are provided in a Git repository, ¹⁹ and the control software is available as a $\mu \mathrm{Manager}$ plugin.

In contrast to the PFS described above, this system measures reflected light arising from total internal reflection (TIR) at the interface. It requires a high numerical aperture (NA) objective and is therefore compatible only with water‐ or oil‐immersion objectives. The system has been reported to achieve higher accuracy than the PFS ²⁰ and can operate continuously during 3D (Z‐stack) acquisition, a capability essential for what it has been designed: total interference reflection fluorescence (TIRF, not supported by the PFS).

2. IMPERFECT FOCUS SYSTEM

The Imperfect Focus System (iPFS) proposed here integrates two concepts previously implemented separately in the hardware‐ and software‐based systems described above: (i) finding a rigid surface that moves with the sample and (ii) using the same camera and objective employed for imaging the sample. iPFS requires no additional hardware beyond a PC‐controllable Z stage and is compatible with any objective.

The operating principle is inspired by hardware‐based PFS: we detect a ‘rigid surface’ of the sample – typically the bottom surface of a coverslip or a plastic sample container – and adjust all imaging positions to follow its movement, compensating axial drift (Figure 1A and B). Unlike PFS, we track optically thin, high‐contrast markers on the transparent surface of the sample to monitor Z‐axis motion during imaging. Because the marker and the sample are part of the same rigid body, the detected movement of the marker can be used to offset the imaging plane, compensating for sample drift over time.

FIGURE 1.

Schematic of iPFS operation. (A) A marker on the external surface of the microscope slide is kept in focus, serving as a reference for all distances in the Z axis. (B) The algorithm maintains sample focus by tracking the marker over time. (C) The $\mu \text{Manager}$ iPFS script interface.

The markers are imaged in transmitted light (bright field or phase contrast) during pauses in sample imaging. The markers must be firmly attached to the transparent surface and exhibit sharp features such as edges or points that blur when moved out of the focal plane. We found that markers created by touching the glass slide very lightly with a marker pen work very well. However, naturally occurring marks such microscopic mineral deposits, micro‐scratches and similar quasi‐two dimensional objects can also serve as markers, provided they remain stable over time.

In what follows we shall discuss the algorithm and its proposed implementation as part of $\mu \text{Manager}$’s Acquisition engine.

2.1. Algorithm

The algorithm runs continuously during pauses in image acquisition, enhancing stability and reducing the risk of losing focus in the event of sudden sample movement.

A single iteration of the algorithm consists of three steps: (i) move the stage to the previously recorded position of the marker (i.e., move both the XY and Z stages) used by the autofocus routine, (ii) acquire a 3D image stack around the Z coordinate corresponding to the sharpest image of the marker from the previous step, and (iii) update the marker's Z coordinate to the position of the sharpest image in the stack. The final step involves calculating a ‘sharpness function’ for each image. Some possible choices of the sharpness function will be discussed in the next section. The following pseudo‐code illustrates these steps (Algorithm 1).

ALGORITHM 1. The Imperfect Focus algorithm.

$(x_0,y_0,z_0)$	▹ Initial position of the mark
${(x_i,y_i,z_i)}_{i\in [1,n]}$	▹ Sample positions to be imaged
$z_{\mathrm{range}}$	▹ scan range
$\delta z$	▹ step size
$z_{\mathrm{focus}}=z_0$	▹ We assume the mark is initially in focus
while True do
if EnoughTimeUntilNextAcquisition() then
for $z_{\mathrm{focus}}-z_{\mathrm{range}}\le z\le z_{\mathrm{focus}}+z_{\mathrm{range}}$ do
acquire image at $(x_0,t_0,z)$
compute $\mathrm{Sharpness}(z)$
$z=z+\delta z$
end for
$z_{\mathrm{focus}}=\max {\left(\mathrm{Sharpness}(z)\right)}_{[z_{\mathrm{focus}}-z_{\mathrm{range}},z_{\mathrm{focus}}+z_{\mathrm{range}}]}$
else
$\mathcal{O}=z_{\mathrm{focus}}-z_0$	▹ Compute drift
$z_0=z_{\mathrm{focus}}$	▹ Update mark position
$\{z_i\}=\{z_i+\mathcal{O}\}$	▹ Offset all imaging position
ImageAllPositions($\{x_i,y_i,z_i\}$)	▹ Actual image acquisition happens here
end if
end while

We note we use a rather naive approach to finding the maximum of the sharpness function along the Z coordinate by scanning over a range of Z values, rather than using potentially more efficient 1D minimisation routines. Despite its simplicity, this method ensures stability and convergence to the global maximum, while the number of function evaluations remains comparable to more advanced methods, given the relatively modest accuracy required (0.1 $\mu \mathrm{m}$ or less). In addition, this approach allows one to move the objective always in the same direction, reducing hysteresis in the Z‐stage drive mechanism and improving repeatability.

2.2. Choice of the sharpness function

The sharpness function takes a 2D image and returns a sharpness metric ‐ a real number that reaches its maximum when the marker is exactly in the focal plane of the objective. Many edge‐detecting filters followed by integration exhibit this property. ²¹ Other options include normalised variance, entropy of the intensity histogram ²² and machine learning based methods. ²³

Our criteria for selecting an appropriate sharpness metric were: (i) high sensitivity to defocusing when imaging marker‐pen marks, (ii) robustness against camera noise and variations in image brightness, and (iii) ease of implementation in Beanshell, the Java‐based scripting language used by $\mu \text{Manager}$. We tested several different sharpness metrics before finally selecting the 3 $\times$ 3 so‐called ‘Redondo’ filter:

$$R=\left[\begin{array}{ccc}0& 1& 0\\ -3& 0& 1\\ 0& 1& 0\end{array}\right].$$ (1)

This particular choice of filter is not critical to the performance of the algorithm; see Figure S6 for a comparison of different filters. Image sharpness is obtained by convolving the matrix (1) with the image intensity matrix $I_{x,y}$, squaring all elements, and summing them up:

$$S=\sum _{x,y}{\left(\sum _{i,j}{R}_{ij}{I}_{x+i,y+j}\right)}^2.$$ (2)

In what follows, we will refer to the numerical value of $S$ defined by the above equation as the sharpness metric, or the sharpness function.

2.3. Implementation in $\mu \text{Manager}$

To demonstrate the algorithm and evaluate its performance on real microscopy images, we implemented it as a Beanshell script for $\mu \text{Manager}$, a popular open‐source software for microscope controll. ²⁴ The script can be run directly from the Script Panel without any additional software and is fully integrated with $\mu \text{Manager}$’s Multi‐Dimensional Acquisition (MDA). After setting up the usual MDA settings (timepoints, Z‐stacks, etc.) through the usual GUI, the user must (1) ensure that the first position in the $\mu \text{Manager}$ positon list contains the marker, (2) launch the script via Beanshell, and (3) select the objective and the channel used to image the marker (see Figure 1C). The script then initialises all relevant variables, records the initial marker position, and goes idle. It wakes after each time step of live image acquisition, running continuously before returning to idle just prior to the next acquisition.

3. PERFORMANCE ASSESSMENT OF THE IMPERFECT FOCUS SYSTEM

We assessed the performance of our approach in a range of tests involving a sample slide containing microscopic ($\le 5\mu \mathrm{m}$) fluorescent beads sandwiched between a glass cover slip and an agarose pad as well as sixteen marks made on the external surface of the coverslip in an area of approx. $15\text{mm}\times 15\text{mm}$ (Figures 2A and 2B and 3A). The beads served as a model specimen for long‐term imaging. Their size was comparable to that of bacterial cells, making clusters of beads representative of small bacterial microcolonies. This setup poses a challenge for live imaging, as high‐magnification, high‐NA objectives are required, and even minor axial drift can severely degrade image quality. By using static beads instead of live bacteria, we were able to evaluate the long‐term stability of the algorithm under conditions where image quality was affected almost exclusively by axial drift, with no significant sample‐induced changes over time.

FIGURE 2.

Experimental setup used to characterise the stability of the iPFS algorithm. (A) Schematic of the setup. The 16 marker positions used to evaluate stability are shown as coloured dots. (B) Photograph of the actual sample. (C) Images of individual markers. (D) Sharpness‐maximising Z positions versus time, for all 16 markers (20$\times$ magnification). (E) Corresponding sharpness metric as a function of time. (F) Normalised probability distribution of changes in marker position, showing the deviation between consecutive measurements: $z_{\mathrm{focus}}(t+3\min )-z_{\mathrm{focus}}(t)=\mathrm{\Delta }{z}_{\mathrm{focus}}$. (G) Normalised sharpness curves for all markers at the end of the experiment.

FIGURE 3.

Imaging fluorescent beads using iPFS. (A) Experimental setup. Viewed from the objective side, we see the following: coverslip (thin dashed rectangle) with a marker (black dot), fluorescence beads (green circles, not to scale), a 1‐mm‐thick agarose slab (orange rectangle), and the glass slide (thick dashed rectangle). The sample is sealed with silicone grease (yellow rectangles). An example image of the beads in both bright field and fluorescence is also shown. (B, C) Normalised sharpness of the beads along the z‐direction (40$\times$ objective), at four different locations on the slide (curves in different colours), in fluorescence (B) and bright‐field (C). See Figure S2 for the same plot for the 20$\times$ objective. (D) iPFS maintains focus despite sample movement (20$\times$ objective). Red line: $z_{\mathrm{beads}}$, the $z$ coordinate of maximum sharpness $S_{\mathrm{beads}}(z)$ for the beads at one position. Black line: $z_{\mathrm{mark}}$, the position of the tracked marker. Green line: $z_{\mathrm{mark}}$ offset by $d(0)$, the initial difference between the marker and the beads' sharpest image. Perfect drift compensation would make the green line overlap with the red line. (E) Sharpness function $S_{\mathrm{beads}}(\delta z+z_{\mathrm{beads}}(0)+(z_{\mathrm{mark}}(t)-z_{\mathrm{mark}}(0)))$, where $\delta z$ is the offset from the corrected $z$ coordinate of the beads' sharpest image. Dark green line: offset $\mathrm{\Delta }z$ of the sharpest image (maximum of $S(\delta z)$). (F) $\mathrm{\Delta }z$ versus time for all six imaged locations. (G) Histogram of $\mathrm{\Delta }z$ for all six locations and all time points. The FWHM (black line) has been obtained from a Kernel density estimate (red line) fitted to the histogram.

3.1. Testing the ability of iPFS to track the movement of the sample

We first evaluated how effectively the algorithm could track the axial movement of the sample and how its performance varied with differences in marker shape (Figure 2C).

After mounting the sample on the XY stage and setting the microscope incubator to , all 16 markers were imaged continuously for 25 h at 3min intervals using a 20$\times$ objective. At each time point, the iPFS algorithm computed the sharpness function $S(z)$ of each marker by scanning a narrow range of $z$ values around the previously determined peak position $z_{\mathrm{focus}}$ of the sharpness function, and then updated $z_{\mathrm{focus}}$ to the new maximum of $S(z)$. Only the sharpest image for each marker was saved; no other images from the scan were recorded in this test. Because all markers were located on the same coverslip, they were expected to move together, such that their $z$ coordinates would change by the same amount in response to any sample movement.

Figure 2D–E shows the time‐series of $z_{\mathrm{focus}}(t)$ and the corresponding sharpness metric for each marker. As expected, relative changes in all $z_{\mathrm{focus}}$ coordinates were highly correlated. The sharpness metric, while varying between the markers, remained relatively stable during the experiment, aside from occasional, short‐lived fluctuations. Visual inspection of the recorded sharpest images of the markers showed that these fluctuations and the associated jumps in $z_{\mathrm{focus}}$ can be attributed to (i) sudden sample movements caused by agarose shrinking and detaching from the coverslip and (ii) out‐of‐focus background changes, such as the growth of air bubbles in the agarose.

Figure 2F shows histograms of changes in $z_{\mathrm{focus}}$ between consecutive time points for all markers. Aside from occasional sudden jumps of approximately $1\mu \mathrm{m}$, the distributions are concentrated near zero and have similar widths across all 16 markers. This demonstrates that the algorithm can track all markers equally well, even though their sharpness functions have very different widths (Figure 2G).

The small differences in marker movement visible in Figure 2E are likely due to slight bending of the coverslip caused by mechanical forces from ageing or drying agarose.

3.2. Accuracy of iPFS

Figure 2 suggests that the algorithm can reliably track the surface with $\sim \mu \mathrm{m}$ accuracy when using a 20$\times$ objective, irrespective of the markers' shape. We expect tracking accuracy to depend on the depth‐of‐field (DOF) of the objective: objectives with a shallower DOF should increase the algorithm's sensitivity, as small changes in the Z coordinate have a greater impact on image sharpness. The DOF is in general the sum of wave and geometrical contributions: ²⁵

$$d=\frac{\lambda n}{N{A}^2}+\frac{np}{M·NA},$$ (3)

where $n$ is the refractive index of the imaging medium between the lens and the coverslip, $NA$ is the numerical aperture of the objective, $\lambda$ is the wavelength, and $p$ the size of the smallest detail resolvable by the camera, equivalent to the physical pixel size of the camera's sensor. For our air objectives ($M,NA=(20,0.5)$ or $(40,0.95)$, $n=1$) (cf. Section 6.2.1), and $\lambda =550$ nm, the DOF $d\approx \lambda n/{(NA)}^2$ turns out to be $2.2\mu \mathrm{m}$ (20$\times$ objective) and $0.7\mu \mathrm{m}$ (40$\times$ objective). We compare this with the full width at half maximum (FWHM) of the distribution of algorithm's steps in Figure 2F. The obtained value $\approx 0.25\mu \mathrm{m}$ is much less than the DOF, meaning that the sample is expected to remain in focus.

3.3. Fluorescent beads as a test sample for benchmarking iPFS

To evaluate the algorithm's ability to maintain focus on an actual sample, we imaged fluorescent beads from the same preparation used in the previous section, tracking one of the 16 markers with iPFS (Figure 3A). Clusters of fluorescent beads resulted in a sharpness function with a single, narrow peak (FWHM $\approx 0.9\mu \mathrm{m}$) when imaged in fluorescence mode (Figure 3B), and a broader, double‐peaked profile in bright‐field (Figure 3C, FWHM $\approx 1.4\mu \mathrm{m}$); see Figure S5 for the average profile and FWHM estimation. The fluorescence‐based sharpness metric has a similar shape for different clusters of beads. Using a standard autofocus algorithm in bright‐field mode on this sample could result in selecting either peak depending on the initial Z position, or cause the focus to jump unpredictably between peaks during acquisition.

The narrow peak of the fluorescence beads‐derived sharpness function and its unimodal nature made the fluorescent beads well suited for benchmarking the algorithm, as even small deviations from the optimal focus caused noticeable changes in the sharpness metric. Furthermore, the peak position aligned with the sharpest image as judged visually by the authors.

Using the 20$\times$ objective and with the iPFS algorithm running in the background, we acquired 3D stacks of bead clusters at six different locations on the slide every 10 min over 40 h. The temperature in the incubator was maintained at for the entire duration of the experiment to simulate conditions typical for live‐sample imaging. The marker tracked by the algorithm was positioned slightly off‐centre relative to the bead clusters (Figure S1).

Figure 3D shows the Z positions of the sharpness maxima for the marker ($z_{\mathrm{mark}}$, black line) and the beads ($z_{\mathrm{beads}}$, red line) at one of the six imaged locations over time. Both curves exhibit an upward trend; the initial and final positions differ by about 30 $\mu \mathrm{m}$. Without iPFS, the bead images would have gone out of focus within a few hours at this magnification. If iPFS fully compensated for drift, the difference $d=z_{\mathrm{beads}}-z_{\mathrm{mark}}$ should remain constant over time and equal to the (optical) thickness of the coverslip. To assess this, we plotted $\mathrm{\Delta }z(t)=d(t)-d(0)$ in Figure 3E; perfect compensation would yield $\mathrm{\Delta }z=0$. Although $\mathrm{\Delta }z$ shows slight temporal fluctuations, it stays within the width of the beads' sharpness function $S_{\mathrm{beads}}$, indicating effective focus maintenance. The other five locations exhibited even greater stability (Figure 3F). Figure 3G shows the distribution of $\mathrm{\Delta }z$ from all locations and time points. The FWHM of this distribution (1.5 $\mu \mathrm{m}$) is comparable to theoretical DOF of the 20$\times$ objective used here (2.2 $\mu \mathrm{m}$).

We can see from all these examples that the fluorescent beads sample is a good testbed for assessing the performance of our autofocus system.

3.4. Comparison of iPFS with the hardware‐based PFS

To enable a fair comparison between hardware and software focusing systems, we performed the same set of measurements using the same objectives as previously (20$\times$ and 40$\times$). At each position and time point, we recorded Z‐stacks over a range of $\pm z_{\mathrm{range}}$ around the best focal plane so that the resulting kymographs from PFS and iPFS are directly comparable. Unlike iPFS, the test focal plane was determined by the hardware‐based focusing system rather than by applying an offset $d(t)$, meaning that each position now has its own $d(t)$. This does not change the interpretation of the curves: if the PFS system functions perfectly, the beads' sharpness profile would reach its maximum at $\mathrm{\Delta }z(t)=0$.

In Figure 4, we compare the two methods using the 20$\times$ objective by plotting $\mathrm{\Delta }z$, defined as the difference between the focus position identified by each system and the ground‐truth peak of the beads sharpness profile. iPFS displays consistently smaller deviations from this reference than PFS.

FIGURE 4.

Comparison between iPFS and PFS for the 20$\times$ objective. (A) Deviation between the position of the sharpest image of fluorescent beads and the position found by iPFS and PFS. (B) Histogram of $\mathrm{\Delta }z$ (the same quantity as the one plotted in Figure 3G), for all positions and time points.

The situation is different with the 40$\times$ objective. Its much shallower depth of field makes it more sensitive to changes of the axial position, producing more peaked sharpness functions. As shown in Figure 5, both systems exhibit smaller overall fluctuations in $\mathrm{\Delta }z$. However, the PFS performs substantially better with this objective: the sharpest planes at all positions vary only minimally, which is reflected in the narrow spread of $\mathrm{\Delta }z$ values. In contrast, iPFS shows only a modest improvement compared to its performance for the 20$\times$ objective. Although the spread of $\mathrm{\Delta }z$ is slightly reduced, the time series display several large, correlated jumps. This suggests that the sample may have undergone physical changes that iPFS was not able to compensate for.

FIGURE 5.

Comparison between iPFS and PFS for the 40$\times$ objective. (A) The deviation between the position of the sharpest image of fluorescent beads and the position found by iPFS and PFS. (B) Histogram of $\mathrm{\Delta }z$ for all positions and time points.

Table 1 summarises the results from Figures 4 and 5.

TABLE 1.

Comparison of iPFS and PFS for two objectives 20$\times$ and 40$\times$. $z_{\mathrm{range}}$ and $\delta z$ refer to the scan range and scan step size of iPFS. FWHM of $\mathrm{\Delta }z$ distribution represents the accuracy of tracking the sharpest focal plane.

System	Objective	$z_{\mathrm{range}}(\delta z)$	FWHM of $\mathrm{\Delta }z$ distribution	Focus quality
PFS	20$\times$	Not applicable	4.1 $\mu \mathrm{m}$	Poor
	40$\times$	Not applicable	0.4 $\mu \mathrm{m}$	Very good
iPFS	20$\times$	$5\mu \mathrm{m}$  (0.5 $\mu \mathrm{m}$)	1.5 $\mu \mathrm{m}$	Very good
	40$\times$	$2\mu \mathrm{m}$  (0.1 $\mu \mathrm{m}$)	0.6 $\mu \mathrm{m}$	Good

3.5. Precautions

As with any autofocus method, there are scenarios in which iPFS may not fully maintain the optimal focal plane. An illustrative example is shown in Figure 6. In this case, although iPFS continues to adjust the focus upward over time (black line), the sharpest focal plane (yellow curve) eventually reaches the edge of the configured scanning range. This suggests that the true optimum moved too fast and eventually fell out of the scan range, and that the system was operating near the limits of the chosen acquisition parameters.

FIGURE 6.

Example iPFS failure mode where the sharpest focus lies at the boundary of the scanned range. iPFS settings: $\delta z=0.1\mu \mathrm{m}$ step, $\mathrm{\Delta }{z}_{\mathrm{focus}}=1\mu \mathrm{m}$. The sharpest plane (green line for relative coordinates and solid black line for absolute coordinates) almost always lies at the edge of the Z‐stack range, which results in a defocused image.

Importantly, this represents a deliberately challenging test condition: iPFS was configured with a very fine step size $(\delta z=0.1\mu \mathrm{m})$ and a shallow scanning range $(z_{\mathrm{range}}=1\mu \mathrm{m})$, prioritising precision over adaptability. In practical use, such situations can be easily mitigated by increasing the step size to enable faster tracking, or by expanding the scanning range to accommodate larger drifts without sacrificing performance. This example therefore highlights not a limitation of the method, but the importance of parameter tuning to match the expected sample dynamics.

The performance of our method is mainly constrained by the speed with which the microscope hardware can complete the $Z$‐scan and the time it takes to complete the actual image acquisition, during which iPFS is not active. If image acquisition takes little time (e.g., only one FOV, a single plane or a small Z‐stack), the effective limit on correctable drift is set by the scan range required to locate the global maximum of the sharpness function, together with the time needed to complete this scan. Since the $Z$‐stage of modern microscopes moves at speeds of at least a few micrometres per second or faster (especially for piezo‐driven stages), the bottlenecks are the settling time of the $Z$‐stage and the communication speed between the computer, stage, and camera. For example, assuming the time of 300 ms per frame (typical for our Nikon 2 Eclipse, including all these bottlenecks), and 20 frames per $Z$‐stack of $10\mu \mathrm{m}$ height (step size $0.5\mu \mathrm{m}$), axial drift cannot be larger than $(10\mu \mathrm{m}/2)/(20\times 300\mathrm{ms})=0.83\mu \mathrm{m}$/s $=3$ mm/h, which would be an exceedingly large drift. Such a large drift is unlikely to be of thermal origin, but it could arise in microfluidic experiments in which pressure changes inside the microfluidic channel cause significant distortions of the cover slip to which the device is bonded.

The performance gets worse if image acquisition during which iPFS is disabled takes a long time due to multiple positions, large Z‐stacks, or multi‐channel imaging. When acquisition time $T$ is the limiting factor ($T\gg$ the time to acquire an iPFS Z‐stack), the maximum correctable axial drift is $z_{\mathrm{range}}/T$, where $z_{\mathrm{range}}$ is the iPFS scan range as defined in Section 2.1 (half of the total Z‐stack height). For example, if acquisition takes 1 min and the iPFS scan range is $\pm 10\mu \mathrm{m}$, the maximum permitted drift velocity is $10\mu \mathrm{m}$/min $=0.6$ mm/h.

Tracking can also fail if the marker's sharpness profile deviates from the ideal shown in Figure 2G. This is easily prevented with careful preparation:

• Avoid image overexposure when imaging the marker, as image saturation can distort the sharpness metric. Our advice is to focus on the marker, find an empty region next to it, and adjust illumination and exposure time such that the region's pixel intensity lies approximately in the middle of the range of pixel values.

• Ensure the marker remains stable over time. For example, when using oil‐immersion objectives, the immersion oil may gradually degrade the ink and eat away finer details of the marker, affecting the sharpness profile to the point where the iPFS will stop working (Figure S3). To avoid this, the marker can be put on the sample side of the coverslip to prevent marker degradation, but this can contaminate the sample so it must be done with care.

• Nevertheless, as long as it is practical, do not put the marker on the sample side of the coverslip. If for some reason this cannot be prevented, care must be taken to ensure that the mark is far away from any motile or growing objects to avoid any interference with tracking (compare Figure S4).

• Position the marker close – but not too close – to the region(s) of interest, to reduce the chance that moving objects enter the field of view and introduce competing sharpness peaks at different $z$ offsets.

• Secure the sample by firmly clamping it to prevent mechanical drift or loss of the marker from the field of view.

Regarding the possibility of contaminating immersion objectives by debris from the marker, the marks created by common marker pens can be easily removed by isopropyl alcohol, which is advocated by many microscopists as a safe way of cleaning objectives. ²⁶

Lastly, the sharpness function we used does not account for varying illumination intensity, that is, $S$ is not divided by average pixel intensity. We checked that normalising by average intensity did not improve the performance of the algorithm, presumably because illumination does not significantly change during the relatively short acquisition of the sharpness metric. If unstable illumination is a problem, normalisation should be added as follows:

$$S=\sum _{x,y}{\left(\sum _{i,j}{R}_{ij}{I}_{x+i,y+j}/I_{\mathrm{mean}}\right)}^2.$$ (4)

where $I_{\mathrm{mean}}$ is the mean pixel intensity of the image with the marker.

3.6. Possible improvements to the iPFS

While iPFS performs reliably in conditions investigated here, certain experimental scenarios may be challenging. Fortunately, with a few simple adjustments, the system can be made more robust:

3.6.1. Surface triangulation

Mechanical instabilities, such as improper clamping, can introduce motions beyond simple uniaxial translation, including sample rotation. To compensate, a single target dot can be replaced with at least three dots, enabling triangulation of the imaging plane (assuming it remains planar). This approach is implemented in the script 3_point_Z_interpolation.bsh provided in the software repository (Section Data and code availability). Thermally induced bending can also make the reference surface non‐planar. In such cases, three‐point triangulation is insufficient, and a larger set of points distributed across the deformation region is required.

3.6.2. Coupling with PFS

The iPFS and PFS are not mutually exclusive. The PFS system can struggle when the sample undergoes large displacements – for example, after long intervals between image acquisitions (e.g., to reduce photobleaching) or if the sample is temporarily removed and then reinserted. In $\mu \text{Manager}$, PFS‐based autofocus always returns the stage to the previously recorded position before activating the PFS. If significant Z‐drift has occurred in the meantime, the PFS may fail to find the interface and will switch off. In contrast, iPFS can maintain tracking during downtime by monitoring the air–glass interface where the markers are located, ensuring that the PFS starts from an already updated location.

3.6.3. XY shift

A natural extension is to correct for XY shifts in addition to the Z‐axis shift, which would account for three‐dimensional translations. This approach is effective only if the marker and sample are part of the same rigid body, which may not be true for biological samples that are not attached to the coverslip.

4. APPLICATION: IMAGING OF ESCHERICHIA COLI MICRO‐COLONIES

To demonstrate the practical utility of our system, we applied it to time‐lapse imaging of E. coli bacteria. These cells are spherocylinders of approximately 1$\mu \mathrm{m}$ in width, with an aspect ratio between 2 and 5 depending on their growth phase. Obtaining consistently sharp images is crucial for downstream analyses such as segmentation and cell tracking. We prepared an agarose slide as described in Section 3.3 but instead of fluorescent beads, we inoculated a 1 $\mu \mathrm{L}$ droplet of exponentially growing E. coli culture with a low optical density $\approx {10}^{-4}$, ensuring that primarily isolated, well‐separated cells were present at the start of imaging. Images were acquired using the same 40$\times$ objective as before after placing the sample on the microscope's XY stage and turning on the incubator (). Without autofocus, the combined effect of bacterial growth and thermal drift caused by the incubator would quickly push the sample out of focus.

To see whether iPFS would manage to keep the sample in focus, the algorithm was configured with a step size $\delta z=0.1\mu \mathrm{m}$ and a scanning range of $\mathrm{\Delta }{z}_{\mathrm{focus}}=3\mu \mathrm{m}$. To quantify the performance of iPFS, bright‐field Z‐stacks were acquired with a 1 $\mu \mathrm{m}$ step size, over a $\pm 5\mu \mathrm{m}$ range. The step size was slightly larger than the objective's depth of field ($0.7\mu \mathrm{m}$). Such stacks can support high‐accuracy segmentation using machine learning approaches. ²⁷ At each time point, a single GFP image was captured at the mid‐plane ($\mathrm{\Delta }z=0$), both to verify accurate tracking of the focal plane and because fluorescence serves as a standard imaging modality.

Figure 7A shows that the image of the bacterial colony remained sharp over the 16 h of live imaging. The sharpness metric of the marker stayed within 2% its initial value (Figure 7B). We conclude that iPFS performed well for this challenging long‐term imaging. This suggests that the iPFS may be suitable for other biological applications requiring stable, cost‐effective autofocus.

FIGURE 7.

Practical application of iPFS: imaging a growing bacterial microcolony. (A) Images (left to right) show the colony‐founding bacterium (about to divide) and the edge of the resulting colony at two later time points. Individual cells are clearly visible at the edge. Cells located farther from the edge form a multilayered structure and can no longer be distinguished. (B) The relative sharpness of the marker used by the iPFS as a function time. The sharpness changes very little, indicating good stability of iPFS.

Microcolonies of E. coli are also a good test bed for higher magnification, immersion‐based objectives. Figure S7 shows that iPFS can maintain focus on a growing bacterial colony using a 100$\times$, 1.4 NA oil immersion objective.

5. COMPARISON WITH OTHER METHODS

Compared to other approaches that use fiducial markers and software‐based tracking, ⁸ , ²⁸ our method combines several features that make it an attractive alternative:

• Our fiducial markers do not require special dyes, nanoparticles, or micropatterns; they can be created with a standard marker pen, making them essentially cost‐free.

• The markers are placed on the side of the glass slide or plastic plate opposite the sample; they do not affect the biological sample, do not need to be sterile, and are not degraded through interaction with the sample.

• The algorithm does not rely on the biological sample itself for focusing, and is therefore applicable to samples lacking sharp features or undergoing significant changes over time.

• The algorithm used to locate (and focus on) the marker is simple, fast, and reliable; it does not require a powerful CPU or GPU and can be readily implemented with basic programming and microscopy expertise.

6. MATERIALS AND METHODS

6.1. Sample preparation

6.1.1. Colloidal beads

We used readily available, water‐soluble acrylic fluorescent paint. After a 1:100 dilution in water, the suspension was centrifuged for 20 s at 3000 rpm. The supernatant, containing the smallest suspended particles, was collected by pipetting. Particle size was assessed microscopically, and most particles were significantly smaller than $2\mu \mathrm{m}$.

The beads were deposited onto a 1‐mm‐thick agarose pad (2% w/v) on a 1 mm glass slide and covered with a #1.5 coverslip ($\approx 150\mu \mathrm{m}$ thickness). A schematic and representative image of this prepared benchmark slide is shown in Figure 3A.

6.1.2. Markers

To enable reliable tracking of the glass surface, we use marks made with fine‐tipped permanent black markers. We tried various models with success such as Faber‐Castell Multimark permanent S and Q‐CONNECT KF1134 with similar results (both mentioned models have a 0.4 mm tip size). A very fine tip ensures that the mark left on the coverslip is small and does not obscure the entire field of view. The mark should be applied very gently, lifting the pen immediately upon contact with the glass.

6.2. Imaging setup

6.2.1. Microscopy

Images were acquired using an automated Nikon Eclipse Ti2‐E epi‐fluorescent microscope equipped with an automated XY stage and the Nikon Perfect Focus System version 4, and controlled by $\mu \text{Manager}$ ²⁴ version 2.0.1 (build dated 7 July 2022). To image fluorescent beads, we used an ORCA‐spark C11440‐36U (Hamamatsu, Japan). Fluorescent bacteria were imaged using a more sensitive Andor Zyla 4.2 sCMOS camera.

We used three different objectives:

• $20\times$: Nikon MRH10201 CFI Plan Fluor 20X, NA 0.50, WD 2.1 mm, Ph1 DLL

• $40\times$: Nikon MRD70470 CFI Plan Apochromat Lambda 40X, NA 0.95, WD 0.25–0.17 mm, DIC N2

• $100\times$ Nikon MRD01905 Plan Apo Lambda 100X, NA 1.45, oil, WD 0.13mm, DIC N2

Fluorescence imaging was performed using the filter cube Semrock brightline FITC 2024B‐NTE (excitation: 470–500 nm, emission: 510–540 nm). For illumination, we used CoolLED $\mathrm{pE}-{300}^{\mathrm{white}}\mathrm{SB}$ light source. Only the blue channel was enabled (emission peak $\approx 450\mathrm{nm})$ and the power was set to 1%, corresponding to $\approx$ 0.7 mW (40$\times$ objective) and 0.8 mW (20$\times$ objective) in the focal plane, as measured by Thorlabs PM16‐130 optical power meter. This was sufficient to obtain strong fluorescence from both the beads and the bacteria. We did not measure the power for the 100$\times$ objective because it was not used for fluorescence imaging.

6.2.2. Computation of the FWHM

In order to estimate the FWHM of the distributions, we replaced the discrete distribution of $\mathrm{\Delta }z$ by continuous distributions obtained using the Kernel Density Estimate method. ²⁹ We used $\sigma =0.5$ for the Gaussian kernel, which yielded good fits for all the distributions.

6.2.3. Bacterial strains

The E. coli strain used in this article is EEL13 (MG1655 pA1_gfp), genetically modified to constitutively express high levels of GFP. The strain (donated by Dr Elin Lilja) has been obtained from K‐12 MG1655 by plasmid mediated gene replacement to insert constitutively expressed GFP in intergenic region between pstS and glmS genes. Cells were grown overnight at 37 C in M9 medium supplemented with 0.2% glucose (v/v) and trace elements to $OD\ge 1$ and then diluted to $\mathrm{OD}={10}^{-4}$. The starting culture was therefore stationary.

6.3. Use of LLMs

ChatGPT was used to improve clarity, conciseness, and stylistic consistency, while preserving the original scientific meaning. No LLMs were used to analyse the data, interpret the results, or generate computer code.

AUTHOR CONTRIBUTIONS

BW conceptualised the system and wrote initial versions of the code. IDC further developed the code, designed benchmarks, performed experiments and analysed the data. Both authors contributed to the writing of the manuscript.

DATA AND CODE AVAILABILITY

The beanshell script that can be run in $\mu \mathrm{Manager}$, along with the data that was used to generate the figures of this publication is available on https://github.com/Dioscuri‐Centre/iPFS

Supporting information

Supporting Information