ATLAS: Machine learning-enhanced filament analysis for the In Vitro Motility Assay

Abstract

The In Vitro Motility Assay (IVMA) is a widely used experimental system to study the chemical and mechanical activity of myosin and other cytoskeletal motor proteins. In the IVMA, myosin molecules are bound to a glass surface and propel fluorescently labeled actin filaments across the surface, which are recorded using video fluorescence microscopy. The length and velocity of the actin filaments offer a measurement of the chemomechanical activity of the myosin motor proteins. Although the assay itself is well suited for high-throughput application, current video analysis approaches are slow, labor intensive, and subject to human bias. To address this shortfall, we introduce ATLAS, an open-source, platform independent software package that utilizes state-of-the-art machine learning algorithms to identify fluorescently labeled actin filaments and then track and analyze their motion in the IVMA. Utilizing both experimental data and a large array of simulated actomyosin motility movies, we demonstrate that ATLAS accurately and efficiently measures both the velocity and length of actin filaments across a broad range of experimental conditions.

Why it matters

The In Vitro Motility Assay is a powerful technique to study the activity of cytoskeletal motor proteins. Although the assay is well suited for high-throughput analysis, the required video analysis is cumbersome and time-consuming. To address this issue, we introduce ATLAS, which utilizes state-of-the-art machine learning algorithms to identify, track, and analyze the motion of fluorescently labeled actin filaments in the In Vitro Motility Assay. We demonstrate that this algorithm is fast and accurate across a broad range of experimental conditions.

Introduction

Video analysis to characterize the motion of molecular and cellular-scale systems has found widespread use in the study of muscle myosin and actin (1), microtubule-based cytoskeletal motors (2) and even whole cells (3). Here, we focus specifically on muscle myosin, but the approach we describe is applicable to the study of a wide range of systems. Myosin is a motor protein that converts the chemical energy of ATP hydrolysis into mechanical work by binding to actin filaments and exerting force and motion upon them in cardiac, skeletal, and smooth muscle (4). The molecular aspects of the myosin-actin interaction have been examined through a wide range of experimental approaches, including the In Vitro Motility Assay (IVMA), a widely used molecular-scale model system of muscle contraction (1,5). In the IVMA, tissue purified or recombinant myosin is used to coat the surface of a glass slide. Subsequently, fluorescently labeled actin filaments and ATP are added (Fig. 1 A and B), and the myosin propels the actin filaments along the surface. The velocity of the actin filaments is then a readout of myosin’s chemomechanical activity (6). Additionally, the length of individual actin filaments determines the number of myosin molecules able to contribute to the actin filament motion (7,8). Therefore, both actin filament velocity and length are important parameters measured in the IVMA. The IVMA has become a standard assay to probe actomyosin function and regulation, with respect to actin and myosin binding partner interactions (9,10,11,12), pathogenic mutations in sarcomeric proteins that lead to muscle disease (13,14,15,16,17), and therapeutic compounds (18,19).

Figure 1.

In vitro motility assay and the simulated ActoMYosin (SAMY) data set. (A) Illustration of In Vitro Motility Assay, including myosin molecules bound to the slide surface and a fluorescently labeled actin filament moving over the myosin surface (magenta arrow). (B) Single still frame from a motility movie with varying length fluorescently labeled actin filaments. (C) The SAMY data set. SAMY was parameterized to span a range of common experimental conditions, including a range of filament lengths and number of filaments per frame, representative tracks of low (0.1 μm/s), medium (2 μm/s) and high (8 μm/s) velocity movies (different filament tracks identified by different colors), and image quality (high, medium, and low signal/noise, SNR).

Historically, each individual actin filament was tracked and measured manually in video images (1) using a “point and click” approach. This process is incredibly time-consuming and impractical for high-throughput analysis. Moreover, manual tracking may skew the results, as an investigator may unintentionally favor specific outcomes, e.g., preferentially tracking the fastest filaments (20). Consequently, approaches to automating actin filament motion analysis exist in the form of commercial (e.g., Imaris (Oxford Instruments, Abingdon, UK), Aivia (Leica microsystems, Deerfield, IL), and DiaTrack (21)) as well as custom solutions developed by academic groups (22,23,24,25,26,27). However, detecting fluorescently labeled actin filaments in visual fields that vary significantly in image quality (signal/noise ratio (SNR)), both between and within movies, often necessitates human intervention, which becomes a rate-limiting step for fully automated IVMA analysis.

Here, we describe a fully automated routine to identify actin filaments, track their trajectories over time, and measure their lengths by taking advantage of convolutional neural networks (CNNs) in the field of computer vision. Our modular routine called "Automated Tracking of Learned Actin Structures” (ATLAS) is a combination of the following: 1) an ensemble of the popular “You Only Look Once” YOLOv5 model (28), custom-trained to identify actin filaments of diverse shapes and lengths; 2) a custom-trained, object segmentation UNET model (29) to define the contour of each actin filament for filament length determination; and 3) Deep Simple Online and Real-time Tracking (Deep SORT; (30), which assembles the actin filament tracks for subsequent velocity determination. Additionally, ATLAS recognizes when two filaments cross, and it accurately tracks the position of long, curved filaments following circuitous paths. These common occurrences can compromise both filament velocity and length measurements if not properly handled (27).

To benchmark ATLAS performance over an extensive range of imaging and experimental conditions, we used a modified version of an IVMA simulator created by (31) to generate a “Simulated ActoMyosin” (SAMY) motility data set. SAMY contains 840 unique IVMA movies that span a wide range of filament velocities, filament lengths, filaments per field (i.e., filament density), and SNRs (Fig. 1 C). ATLAS performed exceptionally well, returning actin filament velocity and length measurements within ±10% of simulated values across a wide range of conditions. Thus, ATLAS successfully incorporates machine learning techniques in the context of IVMA analysis to achieve accurate, rapid, high-throughput performance. In addition, ATLAS is easy to use and has a limited user interface. The only settings are for frame rate and pixel size calibration of the IVMA movie inputs and a checkbox for the “Maximum Track Selection” functionality, (described in the results and discussion section). ATLAS produces multiple output files per movie, ranging from a simple velocity histogram to detailed frame-by-frame coordinates and lengths for each filament, meaning that ATLAS can be utilized “off the shelf” for analysis of a broad range of IVMA experiments. Finally, as open-source software, ATLAS can be modified and extended to suit an even broader range of applications.

Materials and methods

Automated tracking of learned in vitro actin structures

Broadly, ATLAS measures actin filament velocities and lengths from the analysis of IVMA movies (Figs. 2 A–E, and S2). ATLAS relies on two custom-trained CNN routines to identify actin filaments (YOLOv5) and their shapes (UNET) within each frame of an IVMA movie. A third CNN-enhanced routine (Deep SORT) sorts the identified actin filaments into tracks. Both the filament tracks and shapes are used by custom-programmed routines within ATLAS to determine actin filament velocities and lengths. ATLAS generates multiple output files for each movie, including a movie showing filament detection and tracks (Video S1), frame-by-frame positional data for each filament detection, filament length measurements, and histograms of filament velocities. Below are detailed descriptions of the initial YOLOv5 and UNET training and validation, followed by the ATLAS routine workflow, developed to maximize the accuracy of actin filament velocity and length, based on testing ATLAS with a simulated IVMA movie data set (SAMY) that encompasses a broad range of experimental conditions.

Figure 2.

Automated tracking of learned actin structures (ATLAS) workflow. (A) ATLAS receives stacks of motility images from a movie as input. (B) Each image is fed to the YOLOv5 ensemble, which identifies actin filaments by a detection bounding box (green). (C) Each individual detection is fed to the UNET segmentation model, which returns filament masks, which are in turn used to determine filament lengths (see Fig. S2 for a comprehensive flowchart of the ATLAS routine). (D) Simultaneously, filament detections are sent to Deep SORT to assemble actin filament tracks. (E) Track data are then used to calculate root mean-square displacement (RMSD), and velocity is determined as the slope of the relationship between RMSD and time interval. (F) Sample images from the YOLOv5 and UNET training datasets, with filament contour annotations shown in yellow. (G) Branched filament detection due to filament crossings. For each pixel in the skeleton (green), the total number of neighboring pixels in the skeleton (red) is counted in both the adjacent eight-pixel neighborhood (yellow) and the outer 16-pixel neighborhood (blue). If either neighborhood has three or more occupied pixels from the filament skeleton (red), the entire filament detection is discarded as a “branched” filament. (H) Centroid correction. For filaments >2 μm in length, the centroid of the YOLOv5 filament detection (green dot) may lie far from the actual center of the filament skeleton (blue dot); therefore, the coordinates of the bounding box centroid are replaced with those of the filament center. (I) Illustration of Maximum Track Selection to ensure analysis of the maximum number of tracks that each arise from a distinct filament. Each track is identified by its start and stop. (∗) denotes the earliest frame containing the maximum number of tracks (in this case, seven). Tracks active in this frame are exclusively selected for analysis when MTS functionality is enabled.

Video S1. ATLAS Filament Tracking

Tracking of movie described in main text Figs. 3A–C. Actin filaments (4 μm long) are shown in white. Deep SORT detection bounding boxes are shown in red, and filament outlines determined by UNET are shown in yellow. Filament centers (as per their skeleton) are shown in blue. Green dots denote the centroid of the YOLOv5 bounding box, and each filament’s tracking history is shown as a thin yellow trail.

CNN training protocols

YOLOv5

YOLOv5 is a CNN-based object detection model from the "You Only Look Once" family, known for its speed and accuracy in identifying objects within images (28). Here we trained an ensemble of five YOLOv5 models to recognize actin filaments in each frame of an IVMA movie, as CNN ensembles have superior performance compared with using a single CNN model (Fig. S1 B) (32). To train and validate our YOLOv5 ensemble, we assembled two sets of actin filament ground truth annotations (i.e., reference standards for training and evaluating machine learning models) using Label Studio (33). Annotations were derived from individual IVMA movie frames, randomly sampled from experiments performed in the Warshaw and Trybus labs at the University of Vermont (see acknowledgments).

• (1)The initial training set consisted of 378 IVMA experimental movie frames (8-bit grayscale, 640 × 480 pixels, Fig. 2 F) containing 5512 actin filament “ground truth” annotations plus an additional 51 annotations from 17 SAMY movie frames, giving a total of 5563 actin filament ground truth annotations. Of these 5563 annotations, 4624 randomly sampled annotations (∼80%) were used for model training and 939 (∼20%) reserved for validating the model performance.
• (2)The expanded training set consisted of the initial set of 5563 ground truth annotations, plus an additional 2311 annotations from 159 SAMY movie frames, giving a total of 7874 actin filament ground truth annotations. Of these 7874 annotations, 6237 randomly sampled annotations (∼80%) were used for model training and 1637 (∼20%) reserved for validating the model performance.

To create our YOLOv5 ensemble, we started with a YOLOv5 model that was pretrained with the Common Objects in Context data set (34), a large, publicly available data set containing millions of annotated images of various categories. With this YOLOv5 model and its weights (i.e., parameters adjusted to maximize model predictive capacity), we then trained and validated the model (YOLO_I in Fig. S1 A) using the initial training set, described above, with the weights adjusted after each of the 100 cycles (i.e., epochs) through the complete training set (Fig. S1 A). Then, the YOLO_I weights with the greatest predictive capacity were used to seed a subsequent YOLOv5 model (YOLO_II_1, Fig. S1 A) of the ATLAS YOLOv5 ensemble, which was trained using the expanded training set for 100 epochs. The weights with the greatest predictive capacity (maximum average precision, see Supporting Material) from YOLO_II_1 were used to seed the next YOLOv5 model of the ensemble (YOLO_II_2) and so on, so that ultimately, the final ATLAS YOLOv5 ensemble consisted of five individual models (Fig. S1 A, YOLO_II_1 to YOLO_II_5). Within each model, the weights from the epoch with the best performance (as determined by mean average precision, see Supporting Material) were used in that model for the final YOLOv5 ensemble for ATLAS. All IVMA movie frames were scaled to 608 pixels across and padded with black pixels to achieve a square aspect ratio (608 × 608pixels), as required by YOLOv5.

UNET

UNET is a CNN-based model that partitions an image into specific segments (i.e., segmentation). In ATLAS, UNET transforms an image with an actin filament into a binary map so that the actin filament becomes a white object on a black background (Fig. 2 C). Training of the UNET model used 1919 actin filament ground truth annotations, randomly sampled from the expanded YOLOv5 training data set of 7874 annotations described above, with each actin filament’s perimeter manually annotated using Label Studio (Fig. 2 F; (33). Of these 1919 annotations, 1535 randomly sampled annotations (∼80%) were used for model training, and 384 (∼20%) were reserved for validating the model performance. The UNET training started with randomly initialized weights, and after 200 epochs, it reached its peak predictive capacity (Fig. S1 C). All actin filament ground truth images were scaled and padded to 160 × 80 pixels to comply with UNET’s input requirements (Fig. 2 F).

ATLAS actin filament velocity measurement

The ATLAS workflow to measure actin filament velocities in an IVMA movie (Figs. 2 A–E and S2) begins by passing each movie frame to the ensemble of pretrained YOLOv5 models. Each YOLOv5 model of the ensemble then produces a set of actin filament detections for each frame, with each detection described by a “bounding box” and a confidence score. Detections from all five YOLOv5 models were then pooled on a frame-by-frame basis. These detections were often redundant among each of the YOLOv5 models, which was resolved by filtering according to three criteria.

• (1)If any part of a detection lies within 10 pixels of the edge of the field of view, the detection was discarded, as both length and velocity measurements are compromised as filaments enter or exit the field of view. This is referred to as “edge handling” (Fig. S2).
• (2)Detections that overlap by ≥50% were considered the same filament, and only the detection with the highest confidence score was retained, an approach known as “nonmaximum suppression” (Fig. S2) (35).
• (3)Occasionally, a YOLOv5 model will only detect a small portion of a longer filament, resulting in one detection contained within a larger detection. These are not detected during nonmaximum suppression if the smaller detection is <50% the size of the larger. This is resolved by retaining only the larger detection when two detections overlap by at least 1% of their area. This is referred to as a “containment check” (Fig. S2).

Before actin filament tracks are assembled by linking detections for a given actin filament from frame to frame, ATLAS identifies and accounts for commonly encountered issues that compromise track validity. Specifically, occurrences where filaments cross and when a curved filament takes a circuitous path make accurate displacement estimates difficult. To address these issues, ATLAS uses the YOLOv5 bounding box detections to crop the original actin filament and its immediate background from the corresponding movie frame (Fig. 2 C). These image crops are then passed through the trained UNET model, which creates a binary mask of each filament, from which ATLAS produces a filament outline or contour. These contours are then converted into a skeletonized representation using the erosion methodology of (36). Filament skeletons were then used: 1) to identify when two filaments cross, 2) to determine the filament midpoint, and 3) to measure filament length, each described below.

• (1)When filaments cross paths, their images merge into a single branched object (Fig. 2 G), confounding both length and velocity measurements, the latter due to a sudden displacement introduced into a track when the object centroid “jumps” at the onset or ending of a filament crossing. ATLAS detects these crossing events by recognizing and discarding branched actin filament skeletons. Skeleton branch points are detected by examining each pixel (indicated in green in Fig. 2 G) of an actin filament skeleton. Specifically, ATLAS considers both the ring of eight immediate neighbor pixels (yellow ring in Fig. 2 G) and the adjacent ring of 16 pixels that are one pixel away from the pixel in question (blue ring in Fig. 2 G). If three or more of the pixels in either ring are occupied by the skeletonized filaments (red in Fig. 2 G), then the entire detection is discarded as a branched filament due to filaments crossing.
• (2)Curved filaments can be problematic as the centroid of their bounding box may lie far from the filament center (Fig. 2 H), resulting in a systematic underestimate of filament velocity (27). Therefore, for each filament detection with a skeletonized line length >2 μm (20 pixels in the SAMY data set), ATLAS replaces the detection bounding box centroid with the filament’s midpoint location (defined as the pixel in the middle of the skeletonized line) for that detection.
• (3)ATLAS measures actin filament length only for filament detections with skeletons that are >1 μm in length (10 pixels in the SAMY data set), as filaments shorter than this begin to approach the diffraction limits of fluorescence microscopy and require a more complex analysis methodology for accurate measurement (e.g., (37). For filaments of sufficient length, length is tabulated by summing pixels along the skeleton, with orthogonal steps between pixels counting as a whole pixel and diagonal steps counting as √2 pixels.

ATLAS then assembles filament tracks by linking detections for a given filament from frame to frame. This process relied on a CNN-enhanced tracker, Deep SORT, which uses the "Hungarian" optimization method (38) to link detections to form a track. Deep SORT also applies a Kalman filter (39) to improve the filament detections’ positional uncertainty, assuming a constant velocity model. A requirement of Deep SORT is that each filament’s detection bounding boxes must overlap by at least 30% between five successive frames to initiate a track. Tracks are then extended by linking detections in subsequent frames, allowing for a “missed detection grace period” of one frame. For accurate velocity analysis (see below), tracks must be at least 10 frames. If no tracks of at least 10 frames are found by the 20^th frame of a movie, the 30% overlap requirement was lowered to 20%. Tracks are considered terminated if no detections can be linked for two or more consecutive frames. Finally, tracks are subjected to motion and velocity analysis utilizing mean-square displacement (MSD). For each track, the MSD was calculated (Fig. 3 B) according to

$$MSD\left(n\Delta t\right)=\frac{1}{N-n}\sum _{i=1}^{N-n}\left[{\left(x_{i+n}-x_i\right)}^2+{\left(y_{i+n}-y_i\right)}^2\right]\text{,}$$

where N is the total number of frames in the track, n is the number of frames for different time intervals, Δt is the time between frames (100 ms), and x_i and y_i are the filament positions in frame i. For various types of motion, the relationship between the MSD and the time interval (nΔt) can be generalized to MSD ∼ (nΔt)^α, where α is the “diffusive exponent” (40). For each track, α is determined as the slope by linear regression in a plot of log(MSD) versus log(nΔt) (Fig. 3 B, inset). Velocity of tracks exhibiting directed motion (α > 1.0) was determined as the slope of the root mean-square displacement (RMSD) versus nΔt (Fig. 3 B).

Figure 3.

ATLAS performance on SAMY data set. (A) ATLAS analysis of a representative simulated motility movie that had 10 filaments, each 4 μm long, moving at 1 μm/s. (B) RMSD and MSD analysis of tracks from (A). (C) Distribution of ATLAS measured velocity and a mean filament length. (D) 2D heatmap of measurement error for velocity for a single velocity, across a range of filament lengths and counts. (E) Stack of 2D heatmaps of measurement error for velocity across a range of velocities, filament lengths, and counts. Stars denote conditions where ATLAS may produce inaccurate results, each discussed in detail in the main text. (F) 2D heatmap of measurement error for filament length for a single velocity, across a range of filament lengths and counts. (G) Stack of 2D heatmaps of measurement error for filament length across a range of velocities, filament lengths, and counts. (H) Fraction of conditions where ATLAS results are within 10% of parameterized values are insensitive to image quality (SNR) for velocity and show only a slight decline for filament length.

ATLAS includes an optional functionality called “Maximum Track Selection” (MTS), which selects only a subset of all tracks for subsequent filament length and velocity analysis. This functionality offers a safeguard against individual filaments contributing multiple tracks and thus being overrepresented in the final output. Additionally, MTS helps ensure that each track is truly an independent measurement for purposes of statistical analysis (see results and discussion). When enabled, ATLAS tallies the number of active tracks that exist within each frame. The first frame with the maximum number of active tracks is identified, and only those tracks that are active in that frame are selected for velocity and filament length analysis (indicated with an asterisk in Fig. 2 I). This feature is enabled by default but can be turned off by unchecking the box in the ATLAS startup window.

Simulated actomyosin In vitro motility data set

To validate and challenge ATLAS, we created an extensive actin filament motility data set using a modified version of a publicly available motility simulator (31) (Fig. 1 C). Simulation code was modified to mirror the 8-μm persistence length of actin filament trajectories according to the work of (41). To simulate IVMA movies, the simulation created filaments with randomly assigned positional starting points that travel along predetermined paths. Filaments moved along these paths at a rate determined by the parameterized velocity. We created three replicates for each condition in the SAMY data set.

Simulated IVMA movies were generated to closely mimic our experimental IVMA movies: 640 × 480 pixels, 0.1 μm/pixel, 10 frames per second, 100 frames total. Simulation parameters included the filament length (0.5, 1, 2, 4, and 8 μm), filament count per frame (10, 20, 40, 80, 160, 320, and 640) filament velocity (0.1, 0.25, 0.5, 1, 2, 4, 6, and 8 μm/s) (Fig. 1 C). Filament images were subjected to Gaussian blur (σ = 2 pixels), to simulate the point spread function of diffraction-limited fluorescence microscopy. The entire range of simulated parameters listed above resulted in 280 experimental conditions (Fig. 1 C). Movies were subjected to three levels of Gaussian noise that translated into a SNR, characterized as

$$SNR=\frac{{\left(FilamentIntensity\right)}^2}{BackgroundVariance}$$

of infinite (no noise), 5, and 2, hereafter referred to as high, medium, and low SNR (Fig. 1 C). Finally, all conditions were repeated in triplicate, resulting in the complete SAMY data set consisting of 2520 simulated movies.

All instances of the ATLAS routine were run on a Windows PC (AMD Ryzen 7 5800X 8-Core Processor with 64GB RAM and NVIDIA GeForce RTX 3080) with access to the internet. ATLAS source code, the SAMY data set, and annotations used for training and testing of YOLOv5 and UNET models are publicly available at: https://zenodo.org/records/15320358.

The included README.md file contains detailed installation instructions, a description of the user interface, and a detailed description of each of the output files generated for each movie by ATLAS.

Results and discussion

Analysis of actomyosin motility

Initially, we tested ATLAS on three replicates of a single simulated motility condition, i.e., 10 filaments per frame, each 4 μm long, moving at 1 μm/s (Fig. 3 A). The filament detection arm of ATLAS found an average of 900 ± 82 total filament detections per movie of 100 frames, meaning an average of 9 ± 1 detections per frame. For each movie, Deep SORT connected these detections into an average 15 ± 1 tracks, of which 12 ± 1 satisfied the minimum of 10 detections. Although every frame of the simulation contains 10 filaments, they are free to enter or exit the field of view, resulting in the initiation or termination of a given track, respectively. Thus, the number of reported tracks for a single movie can deviate significantly from the parametrized number of filaments. For each movie, the MTS feature identified the frame with the maximum number of co-existing tracks for that movie, which was 7 ± 2 tracks on average. The median lifetime of these tracks was 45 ± 21 frames. Across all three replicates, MSD analysis indicated that 100% of tracks had an α > 1, the criterion for directed motion, with a mean α value of 1.8 ± 0.1. Velocity was then determined by plotting RMSD versus nΔt, which had a slope of 0.93 ± 0.04 μm/s (93% ± 4% of the parameterized value) (Figs. 3 B and C). Filament lengths were measured to be 3.81 ± 0.3 μm (95% ± 8% of the parametrized value) (Fig. 3 C).

We then modified the simulation to feature shorter (0.5, 1, and 2 μm) or longer (8 μm) actin filaments and also increased the number of filaments from 10 to 20, 40, 80, 160, 320, and 640 filaments per frame, for a total of 35 experimental conditions, each simulated and analyzed in triplicate. The heatmaps shown in Figs. 3 D and F show that across the majority of conditions, ATLAS accurately (to within 10%) measures both the parameterized velocity (29 of 35 or 83%) and filament length (16 of 21 or 76% (excluding 0.5- and 1-μm filaments, see below)). As pointed out in the Materials and Methods, accurate length measurement for short (<1 μm) filaments necessitates a different analytical approach (e.g., (37)); therefore, ATLAS does not report a length for these filaments. However, filament tracks and velocities are still reported for these short filaments. An additional challenge in filament length measurement is emphasized with 1-μm-long filaments. ATLAS underestimates the length of these filaments by two to three pixels, approximately the anticipated amount inherent to pixelated images at these size scales (42). However, this amounts to an ∼20% error for these short filaments (Fig. 3 F). Another challenge to accurate filament length determination is partial detections, where the YOLOv5 ensemble detects only a portion of a filament in more crowded conditions. When these partial detections are passed to UNET, the resulting length estimate will inherently be less than the full filament length. Conversely, if a YOLOv5 detection happens to extend beyond the true extent of a filament, UNET does not necessarily return a length overestimate, as it will still determine the dimensions of the filament image contained within the YOLOv5 detection box. Thus, length overestimates are generally corrected by the ATLAS workflow, whereas underestimates are more prevalent as they are difficult to detect or correct. Together, these two phenomena tend to bias length measurements toward underestimating filament lengths.

We then expanded the simulated conditions across a broad range of velocities, from as slow as 0.1 μm/s to as fast as 8 μm/s (Figs. 3 E and G). This encompasses 280 experimental conditions. (Fig. 1 C). With increasing velocity, there was an increase in the number of conditions where ATLAS was unable to determine the length or velocity of any filaments at all (white/missing values in Figs. 3 E and G), up to as many as 9 of 35 (26%) of conditions at the fastest velocity. However, when ATLAS was able to identify and track filaments, it was able to accurately measure to within ±10%, parameterized velocity in 80% (205/255) and filament length in 80% (121/149) of conditions (Figs. 3 E and G). Across this range of velocities (spanning nearly two orders of magnitude), three “problematic” regions become apparent (stars in Fig. 3 E).

• (1)The first problematic region occurs with high densities of longer filaments (Fig. 3 E, magenta star), such that many individual filaments cannot be tracked for an adequate number of sequential frames to allow analysis. With extreme filament density, it becomes difficult to even identify single, discrete filaments at all. This is emphasized by the observation that very few analyzable tracks are identified by ATLAS (Fig. 4 C), in spite of the large number of filaments present in the field of view. These simulated conditions lie far beyond typical experimental conditions.
• (2)The second problematic regime occurs with very short filaments moving at high velocities (Fig. 3 E, orange star). In these conditions, the large frame-to-frame displacement (e.g., eight pixels/frame or 8 μm/s in the SAMY data set) prevents the 30% frame-to-frame overlap required by Deep SORT to assemble filament tracks (Fig. 4 A). These undersampled conditions are characterized by a large number of YOLOv5 filament detections, but few to no assembled tracks (empty tiles in top heatmap of Fig. 4 C). To avoid this regime, experimental conditions with faster velocities necessitate video recording at a faster frame rate to ensure that filaments move no more than 30% of their total length between frames.
• (3)Conversely, a known challenge across all tracking algorithms known as “ID switching” (43) occurs when the YOLOv5 ensemble mistakenly identifies two adjacent filaments as a single object (Fig. 4 B). Resolution of both filaments in subsequent frames may cause Deep SORT to confuse the identities of two filaments. These switching events have the effect of incorrectly inserting large displacements into a track (Fig. 4 B). The impact of this phenomenon is particularly apparent in crowded conditions with very slow movement, where these false displacements lead to inflated velocity measurements (Fig. 3 E, gray star). Although this issue is inherent to image identification algorithms, the influence on velocity measurements can be reduced by recording conditions with slow velocities using a slower frame rate.

Figure 4.

Filament tracking challenges and number of analyzable tracks. (A) Small filaments, moving at fast velocities, become untrackable as filament detection overlap (red shaded area on left) falls below cutoff values for Deep SORT. (B) Example of identity switching. Two filaments identified as ID₁ and ID₂ moving slowly between frames, t to t + 3. Their identities are switched when YOLOv5 detects the two filaments as a single object (frame t + 2). Resolution of this causes the Deep SORT track of the lower filament (ID₂) to transition from the lower filament in frame t to the upper filament in frame t + 3. In extremely slow-motion conditions, this introduces a large, incorrect displacement into this filament’s track, resulting in an inflated measured velocity. (C) Stacks of 2D heatmaps of average number of tracks per filament within each movie frame (left) and average number of tracks per filament in complete movie (right) under various conditions. (D) Example of the benefit of “Maximum Track Selection” (MTS) analysis of SAMY movies featuring 10 filaments per movie, moving at velocities ranging from 0.1 to 8 μm/s. Without MTS enabled (−MTS), the number of detected tracks increases with velocity, in spite of a constant number of filaments per frame. With MTS (+MTS), the number of tracks is consistent with the number of filaments and insensitive to velocity. Error bars denote standard deviation.

To challenge ATLAS with more realistic imaging conditions, we then added Gaussian-distributed imaging noise to the SAMY data set to reduce the image SNR, which had a negligible effect on ATLAS’ ability to analyze movies across the full spectrum of parameters (Figs. 3 H and S3). We observed a small, but statistically significant (p = 0.015) increase in the number of conditions that ATLAS was unable to analyze (white/missing values in Fig. S3) with decreasing SNR (increasing noise). However, across SAMY conditions where ATLAS was able to identify and track filaments, there was no significant difference in the accuracy (as defined by the fraction of conditions accurately analyzed to within 10% of parameterized values) for velocity or filament length (p = 0.076 and 0.46, respectively).

ATLAS was extensively tested with the simulated movies of the SAMY data set. For testing purposes, simulations can provide a “ground truth” where the correct values for filament length and velocity are known in advance, which is not the case with “real world” experimental IVMA data. Therefore, using both simulated and experimental movies spanning a range of filament lengths and velocities, we compared ATLAS to widely used automated tracking packages FAST (22) and DiaTrack (21), as well as a more labor-intensive “point and click” method with MTrackJ (25). We found that ATLAS provided velocity measurements in close agreement with all these analysis routines on both simulated and experimental movies (Fig. S4). Additionally, ATLAS’ reported filament lengths were also in close agreement with those estimated by FAST, the only other routine to provide length measurements (Fig. S4).

The number of tracks and Maximum Track Selection

An important difference between real and simulated data lies in the way that the simulated SAMY data has tightly controlled numbers of filaments in each frame. Specifically, SAMY simulations employ “periodic boundary conditions,” meaning that when a filament exits the field of view, it immediately reenters the frame from the opposite side, whereupon it is recognized as a new filament. Although this is clearly not the case with experimental movies, filaments frequently do enter and exit the field of view under most experimental conditions, resulting in more filament tracks than the average number of filaments in a frame. Similarly, missed filament detections and ATLAS’ elimination of filament crossing events both effectively break the motion of a single filament into two or more tracks. Conversely, crowded conditions impair Deep SORT’s ability to assemble filament tracks with sufficient numbers of detections to allow analysis. In extremely crowded fields, the YOLOv5 ensemble struggles to discern single, discrete filaments at all. Therefore, the resulting number of tracks may deviate significantly from the parameterized number of filaments. Regardless, we found that, across all analyzed conditions, ATLAS consistently detected fewer tracks than filaments on a frame-by-frame basis (Fig. 4 C). Additionally, the total number of tracks from a complete movie is often (but not always) more than the number of filaments per frame (Fig. 4 C), as individual filaments are frequently detected in more than one track through the movie.

The complex relationship between the number of filaments and the number of tracks reported by ATLAS may confound subsequent statistical comparisons, as many tracks are not truly “independent measurements” if they arise from the same filament. Additionally, biases in the results may arise, particularly within heterogeneous fields, where individual filaments may be under- or oversampled depending on their particular appearance and motion. For example, individual fast-moving filaments may be detected as multiple tracks as they intersect other filaments during a movie and also enter and exit the field of view more frequently. Such oversampling of fast-moving filaments may inflate the average velocity for the movie as a whole. To address these challenges, we include an optional functionality in ATLAS that we term “Maximum Track Selection” or MTS. This function identifies the frame with the maximum number of contemporaneous tracks and selects them for determination of velocity and filament length (Fig. 2 I). The rationale is that tracks that are active at the same time must be arising from distinct, separate filaments, ensuring that each filament is counted only once. The value of MTS is demonstrated in Fig. 4 D, where a simulation of 10 filaments was analyzed by ATLAS with or without MTS enabled. Without MTS, the number of detected tracks increases with increasing velocity, with each filament contributing an average of six separate tracks per movie at the fastest velocity. With MTS enabled, each filament is represented by no more than one track, regardless of velocity.

Conclusion

ATLAS achieves accurate, rapid motility analysis in a wide range of conditions and reduces the need for human involvement. We believe that ATLAS has the potential to serve the muscle and molecular motor community, and potentially other applications entailing video motion analysis, as the first, to the best of our knowledge, machine learning-enhanced motility analysis method. We demonstrate that ATLAS provides accurate measurements of filament velocity and length, comparable to established and more labor-intensive approaches. However, optimizing experimental conditions and design is still encouraged, as we identify experimental regimes (the “problematic regions” discussed above) where extreme filament densities or ill-suited frame rates lead to ATLAS providing potentially inaccurate measurements. These movies would be very difficult to accurately analyze by any approach. In conclusion, ATLAS offers significant time savings for IVMA analysis, opening the door for high-throughput applications, such as small-molecule screens. Additionally, the open-source nature means that ATLAS could even be modified to run concurrently with IVMA image acquisition, enabling real-time measurements. Therefore, it could be interesting to explore the possibility of performing real-time feedback motility experiments in the future.

Acknowledgments

The authors thank Guy Kennedy from the Instrumentation and Model Facility at the University of Vermont for his microscopy expertise, Samantha Beck-Previs for her technical expertise, the Kathy Trybus lab at the University of Vermont for sharing IVMA movies, Dr. Safwan Wshah at the University of Vermont for comments on an earlier version of the manuscript, and the entire Warshaw lab for critical discussions. This work was supported by funds from the American Heart Association (23PRE1013839 to S.D.-M.), National Institutes of Health (GM141743, HL150953 and HL163585 to D.M.W.), and a generous gift from Arnold and Mariel Goran to D.M.W.

Author contributions

S.D.-M.: conceptualization, methodology, software, formal analysis, investigation, resources, data curation, writing – original draft, writing – review and editing, visualization, and funding acquisition; D.M.W.: writing – review and editing, supervision, project administration, and funding acquisition; S.R.N.: methodology, supervision, writing – original draft, and writing – review and editing.

Declaration of interests

The authors declare no competing financial interests.

Editor: Jorg Enderlein.