Automated vision-language framework for kinematic profiling and performance diagnostics in competitive swimming

Abstract

Background

Manual kinematic analysis in swimming is labor-intensive and often lacks the immediacy required for elite training. This study presents an automated vision-language framework to deliver real-time kinematic profiling and coaching diagnostics.

Methods

A robust object detection pipeline using YOLOv11 was developed, incorporating a splash-injection strategy to handle aquatic occlusion. Kinematic metrics (velocity, distance) were extracted via homography transformation. To automate pedagogical feedback, the DeepSeek-V3 Large Language Model was integrated to interpret these metrics and generate structured coaching reports.

Results

The proposed method achieved a mean Average Precision (mAP@0.5) of 94.64% in dynamic water conditions. The system successfully tracked swimmers despite turbulence and accurately synthesized quantitative data into natural language assessments of pacing and fatigue.

Conclusions

This open-source framework significantly reduces the manual burden of performance analysis. By combining computer vision with automated reporting, it offers a scalable, objective tool for daily swim training and technical evaluation.

Introduction

Elite competitive swimming demands rigorous quantification of athletic performance, where optimization of stroke mechanics and velocity distribution is paramount for success [1]. While traditional timing methods provide aggregate results, they fail to capture the intra-race kinematic details necessary for technical refinement [2, 3]. Consequently, the domain of sports analytics has increasingly turned towards computer vision to extract high-fidelity motion data without the interference of wearable sensors [4, 5], supported by recent progress in video-based player detection and tracking pipelines [6, 7]. However, the aquatic environment introduces specific physical properties, such as light refraction and water splash occlusion. These factors significantly affect the accuracy of visual recognition algorithms [8]. Currently, the application of computer vision in such high-turbulence environments remains a challenging task, particularly regarding the consistent tracking of athletes and the interpretation of kinematic data [9].

Historically, wearable technologies have become an important tool in swimming performance analysis due to their ability to provide precise real-time data. For instance, Slopecki et al. [10] used a single inertial measurement unit (IMU) placed on the sacrum to assess stroke mechanics in elite Paralympic swimmers, introducing a velocity variation score with an accuracy of 85%. Similarly, Li et al. [11] combined multiple IMUs with deep learning models to track body movements, achieving 92% accuracy in segmenting swimming phases. However, researchers note that these wearable systems are based on invasive methods, and the additional sensors may disrupt the swimmer's hydrodynamics and performance, which has led to the exploration of non-invasive video-based alternatives [12]. Video-based techniques have become more advanced with the integration of deep learning. Jayawardene and Kalansooriya [13] introduced a method leveraging LSTM networks to analyze freestyle posture, achieving up to 85% accuracy. Additionally, Hall et al. [14] introduced the DeepDASH method, utilizing a two-phase approach to detect swimmer heads and strokes with an F1 score of 97.5%. In the broader context of sports analytics, Yang et al. [15] provided a comprehensive survey on multi-object tracking (MOT) paradigms, and Sheng et al. [16] demonstrated the efficacy of broad learning frameworks for transforming raw visual data into tactical guidance. To specifically address challenges like mutual occlusion and individual shape variations in distant lanes, advanced algorithms such as the Confidence-enhanced ByteTrack (CBT) [17] have significantly enhanced the maintenance of athlete identities over time.

Despite these methodological advancements, the primary challenge in vision-based swimming analysis lies in maintaining robust visual representation under severe environmental interference. Unlike terrestrial scenarios, competitive swimming is dominated by dynamic hydrodynamic turbulence and complex light refraction. During high-intensity races, the rapid impact of limbs against the water surface produces substantial splashing and foaming, leading to frequent occlusions in which the athlete is partially or even fully obscured by white water. Consequently, detectors trained on canonical imagery often fail to extract discriminative cues in chaotic scenes, and downstream tracking can degrade rapidly, manifesting as identity switches or complete target loss when processing professional broadcast footage.

In addition to algorithmic difficulty, progress is constrained by the limited availability of representative, real-world data that matches deployment needs. Many existing studies are evaluated on controlled recordings or narrow viewpoints, which do not fully reflect the complexity of competition broadcasts, such as multi-lane interactions, strong perspective changes, and persistent splash-induced occlusion. Moreover, elite domestic championship footage is rarely curated into research-ready datasets, despite its direct relevance to daily training and national-team-level performance analysis. This mismatch between the data used for method development and the conditions encountered in real competitions makes it difficult to assess generalization and practical value.

Beyond visual perception and data limitations, existing swimming video analysis frameworks still have limitations in spatiotemporal granularity and semantic interpretability. Many current methods mainly provide aggregate indicators, such as lap time, stroke rate, or swimmer trajectories, but they are less effective in describing instantaneous velocity changes and segment-level pacing dynamics. As a result, important variations related to fatigue, drag, and technique degradation may be overlooked. More importantly, quantitative outputs such as coordinates, time series, or velocity curves are often difficult to translate directly into coaching decisions without further interpretation.

Recent studies in visual understanding and intelligent guidance have shown that visual data can be analyzed beyond object detection and scene recognition. For example, affective image content analysis, label-efficient emotion recognition, and personalized image perception studies indicate that semantic interpretation should consider both visual evidence and human-oriented context [18–20]. Similarly, real-time guidance systems such as PhotoHelper demonstrate that visual perception can be connected with interpretable feedback through a perception and guidance process [21]. These studies suggest that vision-based systems should not only extract measurements, but also provide user-oriented explanations.

This trend is also evident in AI-assisted sports biomechanics. Recent studies have applied computer vision and deep learning to estimate knee joint contact forces in ACL-reconstructed knees [22] and to predict ligament fatigue failure risk mechanisms [23]. These works show that AI methods can transform motion-related observations into higher-level biomechanical and risk-related interpretations. Inspired by this direction, the framework presented in this paper extends swimming video analysis from visual measurement to interpretable performance assessment. Specifically, race-derived kinematic indicators are extracted from competition videos and further converted into coach-readable summaries, providing more understandable feedback for post-race performance analysis.

To address these gaps, the present study targets robust swimmer analysis from fixed-camera or broadcast-style race videos and emphasizes both measurement reliability and interpretable feedback. The current system is mainly designed for post-race or post-session analysis, in which recorded race videos are processed to extract kinematic indicators and generate coach-readable summaries. Although the workflow has the potential to support near-real-time training feedback in future applications, real-time deployment was not the primary objective evaluated in this study. A competition-grade dataset was curated from championship meets held in China in 2023 and 2024, where athletes from the Chinese national team participated. On top of stable trajectory and kinematic extraction via advanced object detection (YOLOv11), we further introduce an LLM-driven interpretation module (DeepSeek-V3) that converts structured kinematic indicators into coach-readable assessments, such as performance level characterization, phase-specific strengths and weaknesses, and plausible technical or tactical explanations. In this way, the proposed framework not only improves robustness under turbulence and occlusion, but also closes the loop from raw vision measurements to semantic, training-relevant analysis.

The main contributions of this paper are as follows. First, the framework is evaluated using competition swimming videos that include realistic visual challenges such as splash occlusion, water turbulence, lane-dependent visibility differences, and lighting variation. Second, a splash-injection augmentation strategy is applied to improve swimmer detection robustness under aquatic occlusion. Third, the framework integrates homography-based kinematic extraction with a large language model to produce coach-readable summaries from measured velocity and segment-level indicators. Compared with existing swimming video analytics systems such as DeepDASH, which primarily address swimmer detection, tracking, and stroke-event analysis, the proposed framework provides a complete workflow from visual measurement to interpretable performance reporting.

Method

Method overview

The proposed framework achieves non-contact, end-to-end analysis of competitive swimming performance by integrating computer vision, kinematic modeling, and large language models into a unified pipeline, as illustrated in Fig. 1, raw broadcast videos are first sampled into frames and processed by a splash-injection augmentation module that overlays stochastic water-foam textures, so that the detector is trained to remain effective under severe occlusion; the augmented frames are then fed into a YOLOv11-based detection network, which localizes the swimmer in each frame and assigns the corresponding stroke-phase label; the resulting sequence of bounding-box centers is mapped from pixel space to real-world pool coordinates through a calibrated homography, and instantaneous forward velocity along the lane is computed and temporally smoothed to obtain a stable meter-level velocity profile; finally, the stroke-phase timeline and velocity sequence are serialized into a structured textual representation and provided as input to a trained DeepSeek large language model, which performs semantic reasoning over the kinematic patterns and generates human-readable coaching reports.

Fig. 1.

Method overview

Splash-injection data augmentation

In real-world competitive swimming environments, the visual clarity of swimmers is frequently compromised by intense water turbulence and splashes. These factors lead to domain shifts that degrade the performance of detectors trained on ideal datasets. To construct a robust training set while maximizing annotation efficiency, a sparse temporal sampling strategy is implemented. Given that adjacent video frames contain highly redundant visual information, the entire video sequence is not annotated. Instead, one keyframe is extracted and manually labeled every 30 frames. This sampling interval corresponds to approximately one second and ensures that the dataset covers a diverse range of stroke phases ranging from entry to recovery without imposing excessive manual labor costs.

However, a dataset composed solely of clean frames is insufficient for training a detector to handle severe occlusion. To address this limitation, a Splash-Injection Augmentation technique is introduced. Water turbulence is mathematically simulated using Perlin noise to generate a synthetic splash mask denoted as . In the implementation, the Perlin-noise field was generated with a spatial scale randomly sampled from 32–96 pixels, the number of octaves selected from , persistence sampled from 0.45–0.65, and lacunarity sampled from 1.8–2.4. The noise map was then normalized to [0, 1] and thresholded using a random threshold between 0.55 and 0.75 to obtain irregular foam-like regions. To avoid unrealistic sharp boundaries, a Gaussian blur with a kernel size randomly selected from 3–9 pixels was applied to produce a soft splash mask. For a given annotated keyframe, the augmented frame is synthesized via a weighted blending operation defined as [24]:

1

where is the annotated keyframe, is the augmented frame, denotes the synthetic foam texture, is the generated soft splash mask, and ⊙ denotes pixel-wise multiplication. The injection intensity coefficient α was sampled from a uniform distribution, , to simulate different levels of splash occlusion, ranging from mild turbulence to severe white-water interference. This process forces the network to learn robust features that are invariant to surface occlusion, and the resulting augmented dataset serves as the standardized input for the subsequent object detection module.

Swimmer detection

Competitive swimming analysis demands a detection framework that achieves an optimal balance between inference speed and localization precision. Two-stage detectors like Faster R-CNN provide high accuracy but typically suffer from computational latency, which makes them unsuitable for real-time broadcast processing. Conversely, earlier single-stage models often struggle with small-object detection in wide-angle pool views. Therefore, the YOLOv11 architecture is deployed, which integrates an efficient backbone with an anchor-free detection head. This choice is motivated by the superior capability of YOLOv11 in handling multi-scale targets and its rapid inference speed of over 60 FPS. This ensures that the method can capture transient stroke phases without frame dropping. The network optimization is governed by a composite loss function designed to penalize bounding box misalignment and classification errors simultaneously. The total loss can be expressed by [25]:

2

Specifically, the Complete Intersection over Union (CIoU) Loss is utilized for the bounding box regression. Unlike standard IoU, CIoU accounts for the overlap area, center point distance, and aspect ratio consistency. This is particularly critical for swimming because the aspect ratio of the athlete changes rapidly during stroke cycles. The classification branch simultaneously predicts the stroke style, such as Freestyle or Breaststroke, which provides essential context for the subsequent kinematic analysis. In this study, only freestyle events were analyzed. Quantitative evaluation of stroke classification performance is not reported, but the architecture supports multi-stroke prediction for future datasets.

Feature extraction and loss function design

Accurate localization of stroke phases is critical yet challenging due to the rapid motion blur inherent in swimming. To address this, the YOLOv11 architecture is deployed, which processes the augmented frame to localize the swimmer. The network backbone is enhanced with C3k2 blocks for efficient feature extraction and a C2PSA (Cross-Stage Partial Spatial Attention) module to focus on anatomical keypoints obscured by water. The attention-refined feature map F is computed from an input feature map as [26]:

3

where σ denotes the Sigmoid activation function. The spatial attention mechanism is formulated as , where Q,K,V are the query, key, and value projections derived from the CSP branches.

To optimize the regression accuracy of the predicted bounding box, the network minimizes a composite loss function comprising classification loss, geometric regression loss, and distribution loss, which can be expressed by [27]:

4

Where the Complete Intersection over Union Loss is employed is denoted as . This loss term minimizes the normalized distance between the center points of the predicted box and the ground truth box while penalizing aspect ratio discrepancies. Furthermore, to handle the boundary uncertainty caused by splashes, the Distribution Focal Loss can be defined as:

5

where y is the target distance from the anchor center to the box edge, and and are the softmax probabilities at the discretized locations closest to y. The network finally outputs the predicted bounding box which contains the coordinate information required for the subsequent kinematic analysis.

Spatial mapping and kinematic metric extraction

Raw pixel coordinates lack physical significance due to perspective distortion in camera views. To quantify athletic performance, the 2D pixel coordinates derived from the detection stage are transformed into 3D real-world coordinates denoted as and , using a pre-calibrated homography Matrix H. The mapping from the image plane to the physical pool plane is defined as [28]:

6

where the homography matrix H is computed by matching four key points in the image space to the standard dimensions of the swimming pool. is the instantaneous velocity, which can be calculated via discrete differentiation over the time interval Δ t, the calculation equation can be expressed by:

7

where the frame-by-frame detection jitter can introduce high-frequency noise into the velocity profile. To mitigate this issue, an Exponential Moving Average filter is applied to generate a smoothed velocity metric, which can be defined as:

8

where γ is the smoothing factor. The output of this stage is a continuous sequence of smoothed kinematic metrics which captures the temporal evolution of the swimmer's performance.

Large language model analysis

While kinematic data provides quantitative metrics, it lacks the qualitative context required for effective coaching. To bridge this semantic gap, a Large Language Model was leveraged to interpret the numerical data. In this study, DeepSeek-V3 was used through prompt engineering rather than task-specific fine-tuning. The kinematic sequence was first serialized into a structured textual prompt that aggregated key statistical descriptors such as distance, cumulative time, instantaneous velocity, peak velocity, and velocity stability. The prompt consisted of three main components: role definition, structured kinematic indicators, and task constraints. Specifically, the model was instructed to act as a swimming performance analyst and to generate an automated coaching-style summary based only on the measured kinematic indicators. This prompt was then fed into the DeepSeek-V3 model to generate a natural language diagnosis. The inference process is mathematically represented as:

9

where represents the frozen parameters of the pre-trained model. The final output R is an automated coaching-style summary based on kinematic indicators, which provides coach-readable information on stroke efficiency and pacing-related changes. It should be noted that the LLM-generated summary is intended to support, rather than replace, professional coach analysis. The generated text provides an interpretable description of the measured kinematic patterns and should be considered as auxiliary decision-support information for coaches and analysts. This completes the transformation from raw video to semantic insight.

Results

Experimental setup

To ensure research transparency and reproducibility, a dedicated swimming dataset was constructed using 76 self-collected, high-definition videos captured during the 2023–2026 National Swimming Championships, including the Chinese National Open Swimming Championships, National Swimming Championships, and the 15th National Games swimming events. All videos were recorded at a resolution of 1920 × 1080 pixels. The dataset includes freestyle, breaststroke, and backstroke events. Based on the available start lists, the dataset contains approximately 608 athlete entries, counted per event, and include repeated entries across competitions. Keyframes were sampled at 1 Hz from the 60-fps footage to capture kinematic continuity while reducing redundancy. Manual annotations include bounding boxes for athlete localization and categorical tags distinguishing steady-state swimming from start and turn phases. To prevent identity or environmental data leakage, the dataset was partitioned strictly at the video level. A summary of the dataset statistics, including stroke phase distribution and subset splits, is provided in Table 1.

Table 1.

Statistical profile and partition of the dataset

Dataset Metric	Statistical Value
Included Competitions	National Swimming Championships & Major 2025–2026 Competitions
Tracked Athletes	608 athlete entries (counted per event, include repeated entries across competitions)
Collected Videos	76 sequences
Annotated Keyframes	12,000 frames (sampled at 1 Hz)
Phase Distribution	Swimming: 78%/Tumble Turns & Starts: 22%
Training/Val/Test	80%/10%/10%
Stroke & Distance	Freestyle, Breaststroke, Backstroke; 50 m, 100 m, 200 m

As shown in Table 1, all computational experiments, including the training of the detection models and the inference of the kinematic profiling module, were executed on a high-performance workstation. The method was equipped with an NVIDIA GeForce RTX 4070 GPU, an Intel Core i9-13900 K processor, and 64 GB of RAM. The software implementation relied on the PyTorch deep learning framework, with the YOLOv11 model architecture deployed using the Ultralytics library. For the comparative analysis, the environment was configured to support consistent training hyperparameters across different model versions to ensure a fair assessment of detection accuracy and inference speed.

Comparative analysis of detection models

The selection of baseline models for this comparative analysis is grounded in established benchmarks within the field of real-time computer vision. According to recent surveys on object detection [29, 30], YOLOv8 is widely recognized as the industry standard for real-time applications due to its balance of stability and inference speed. Meanwhile, YOLOv10 is frequently cited in contemporary literature [31, 32] as representing the state-of-the-art in anchor-free architectures, pushing the limits of efficiency. Furthermore, previous studies in sports kinematic analysis [33, 34] consistently utilize Faster R-CNN as a reference point to contrast the high localization precision of two-stage detectors against the computational efficiency of one-stage networks. Consequently, these architectures were adopted as the primary baselines to evaluate the relative performance of the proposed framework.

Table 2 presents a quantitative comparison of representative object detectors on the competition-broadcast swimming dataset. All methods are evaluated using the same protocol, and performance is reported in terms of Precision, Recall, mAP@0.5, and mAP@0.5:0.95. All values are reported as mean ± 95% confidence interval across test video sequences. The * indicates a statistically significant improvement over the strongest baseline method for the corresponding metric, with p < 0.05. The proposed configuration achieves the strongest overall detection quality, attaining 93.84% Precision and 92.37% Recall, together with 94.64% mAP@0.5 and 76.53% mAP@0.5:0.95. This simultaneous achievement of high Precision and Recall indicates that the proposed architecture effectively balances the trade-off between suppressing false positives and maintaining target coverage, yielding a distinct performance margin over the baseline methods. In comparison, the one-stage baselines exhibit consistently lower mAP values, reflecting a lower overall efficacy in target identification and localization. Faster R-CNN demonstrates relatively high Precision but substantially lower Recall, suggesting a more conservative detection behavior that tends to miss targets in low-visibility frames. Overall, the results substantiate the advantage of the proposed detection configuration and establish a stronger foundation for subsequent tracking and kinematic measurement.

Table 2.

Experimental results of different scenarios

Method	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@0.5:0.95 (%)
YOLOv10	92.91 ± 0.62	91.07 ± 0.74	93.79 ± 0.58	75.08 ± 0.81
YOLOv8	92.13 ± 0.69	90.18 ± 0.83	92.88 ± 0.64	73.57 ± 0.92
Faster R-CNN	94.17 ± 0.57	89.06 ± 0.91	92.41 ± 0.71	71.92 ± 1.04
Ours	93.84 ± 0.54	92.37 ± 0.63*	94.64 ± 0.51*	76.53 ± 0.76*

*indicates a statistically significant improvement over the strongest baseline method for the corresponding metric (p < 0.05)

Robustness analysis

While standard computer vision metrics such as mAP provide insight into detection quality, they do not directly translate to the usability of kinematic data in sports science application. To demonstrate the method's practical value for coaching, this study evaluates the method performance using two specific quantitative metrics which are the Tracking Rate and the Absolute Temporal Error. These metrics assess the continuity of swimmer detection and the accuracy of the calculated timing data in real-world competition environments.

The Tracking Rate was defined as the percentage of successfully tracked time relative to the total race duration and was calculated as: The paper analyzed full race footage from the 2023 China 200 m Freestyle Finals and the 2024 China 200 m Freestyle Finals to calculate this metric. Tables 3 and 4 present the specific results for each lane in these two events.

Table 3.

2023 China 200m freestyle finals

Lane ID	Duration (s)	Tracking Loss (s)	Tracking Rate (%)
Lane 1	118	4	96.61
Lane 2	118	10	91.53
Lane 3	118	6	94.92
Lane 4	118	4	96.61
Lane 5	118	4	96.61
Lane 6	118	4	96.61
Lane 7	118	4	96.61
Lane 8	118	14	88.14

Table 4.

2024 China 200m freestyle finals

Lane ID	Duration (s)	Tracking Loss (s)	Tracking Rate (%)
Lane 1	117	4	96.58
Lane 2	117	4	96.58
Lane 3	117	3	97.43
Lane 4	117	3	97.43
Lane 5	117	14	88.03
Lane 6	117	20	82.90
Lane 7	117	27	76.92

As detailed in Table 3, the method demonstrated strong stability during the 2023 finals. Lanes 1, 4, 5, 6, and 7 achieved a consistent Tracking Rate of 96.61%, with the method losing the target for only 4 s over a 118-s duration. Even the most challenging edge lane, Lane 8, maintained a usable rate of 88.14%. The results from the 2024 finals in Table 4 further demonstrate the lane-dependent tracking characteristics of the proposed method. Lanes 1 through 4 maintained high tracking rates between 96.58% and 97.43%, whereas Lanes 5 through 8 showed lower tracking rates ranging from 74.35% to 88.03%. This difference may be attributed to a combination of camera angle, lane position, venue lighting variation, perspective distortion, partial visibility near the image boundary, lane-line interference, and stronger splash occlusion. These factors can reduce the visual contrast between the swimmer and the surrounding turbulent water surface, thereby increasing the probability of short-term detection loss. Nevertheless, the method still captured a substantial proportion of the race data, indicating that the extracted trajectories remained useful for segment-level kinematic profiling and post-race performance analysis.

Building upon this tracking foundation, the method's metrological precision was quantified by comparing its automated timing outputs against manual stopwatch measurements, which served as the empirical ground truth. The absolute temporal error was recorded at fixed distance intervals ranging from 15 m up to 200 m across eight swimming lanes. The error distribution heatmaps presented in Fig. 2 and Fig. 3 illustrate the magnitude of these absolute errors for distinct experimental trials, where the color intensity signifies the deviation in seconds.

Fig. 2.

Error distribution heatmaps of 2023 China 200m freestyle finals

Fig. 3.

Error distribution heatmaps of 2024 China 200m freestyle finals

As shown in Figs. 2 and 3, for the majority of the mid-lap segments, such as the 25-m to 45-m marks and the 75-m to 95-m marks, the method maintains a high degree of precision. In these steady-state swimming phases, the absolute errors predominantly remain below 0.20 s, indicating that the homography-based coordinate mapping aligns closely with physical reality when the swimmer maintains a consistent posture. However, a notable increase in error magnitude is consistently observed at the turning nodes, specifically at the 50-m, 100-m, and 150-m marks. As illustrated in the heatmaps, the temporal deviation at these points can occasionally spike to values ranging between 0.40 s and 0.97 s. This phenomenon is attributed to the complex body deformations and rapid acceleration changes associated with the tumble turn, which present challenges for precise center-of-mass localization. Despite localized deviations at the wall turns, the overall error distribution across the 200-m race indicates that the method is accurate enough for analysing pacing trends and segment-level performance changes. The system is not intended to replace official electronic timing, but the observed error range is acceptable for post-race analysis, such as identifying velocity drops, pacing changes, and performance fluctuations across different race segments.

Ablation study on splash-injection strategies

To rigorously evaluate the contribution of the proposed Splash-Injection Augmentation strategy to the method's robustness, a comparative ablation study was conducted. The primary objective was to quantify the impact of synthetic visual noise on the detector's ability to maintain tracking accuracy in turbulent aquatic environments, where traditional models often fail due to occlusion and feature distortion. The visual rationale for this strategy is illustrated in Fig. 4. Real-world competition environments as depicted in Fig. 4a are characterized by chaotic water turbulence that frequently disrupts feature extraction. To address this issue, the proposed strategy utilizes the clean and high-visibility samples shown in Fig. 4b as a baseline and procedurally superimposes synthetic visual noise to generate the augmented training data displayed in Fig. 4c

Fig. 4.

Visualization of the Splash-Injection Augmentation Strategy: a real-world competition images with chaotic water turbulence and splash occlusion; b clean, high-visibility training samples without splash augmentation; and c augmented training samples generated by superimposing synthetic splash noise

For the experimental setup, two distinct configurations were established: a baseline using the standard YOLOv11 architecture and a comparative model incorporating the Splash-Injection module. The latter superimposed procedurally generated, semi-transparent splash patterns onto training samples to simulate realistic visual interference. Both models were subsequently evaluated on an identical validation dataset featuring challenging sequences of heavy turbulence and partial swimmer submersion.

As presented in Table 5, the integration of the Splash-Injection strategy resulted in a consistent performance improvement across all key metrics. Notably, the mAP@0.5 increased by 4.17% to reach a value of 94.64%. More critically, the Recall metric improved by 3.83% which indicates a significant reduction in missed detections during phases of high turbulence. This quantitative evidence suggests that exposing the network to synthetic visual noise as visualized in Fig. 4c compels the model to prioritize global structural features, specifically the relative configuration of the head and torso. By reducing reliance on local edge cues that are frequently corrupted by water splashes, the augmented model demonstrates improved stability in maintaining the tracking continuity required for accurate kinematic profiling.

Table 5.

Impact of splash-injection augmentation

Configuration	Precision (%)	Recall (%)	mAP@0.5 (%)	mAP@0.5:0.95 (%)
YOLOv11	91.21 ± 0.78	88.54 ± 0.96	90.47 ± 0.84	72.16 ± 1.08
YOLOv11 + Splash	93.84 ± 0.54*	92.37 ± 0.63*	94.64 ± 0.51*	76.53 ± 0.76*
Improvement	+ 2.63	+ 3.83	+ 4.17	+ 4.37

*indicates a statistically significant improvement compared with the YOLOv11 baseline without Splash-Injection augmentation (p < 0.05)

Large language model report

The proposed framework detects and records the real-time velocity and kinematic metrics for each individual swimming lane. However, the subsequent analysis of these dense data streams presents a significant challenge in practical coaching applications. Traditional manual interpretation of instantaneous velocity curves and split timestamps is labor-intensive and requires substantial expertise, which often delays the provision of critical feedback to athletes. To overcome these limitations and streamline the data interpretation process, this study extends the method's capability by integrating a Large Language Model (LLM) to function as an automated semantic interpreter. The primary objective is to rapidly transform complex time-series data into a coherent, natural language diagnostic report, thereby enabling efficient identification of performance bottlenecks and specific training interventions without the latency associated with human analysis. Figure 5 presents a representative automated coaching report generated for Lane 4 during the 200 m Freestyle trial.

Fig. 5.

LLM report

To facilitate this transformation, a specialized prompt engineering protocol was implemented using the DeepSeek-V3 architecture. The inference process is governed by a structured prompt that defines three critical parameters: Role Definition, Data Serialization, and Task Constraints. First, the model is explicitly primed with the context of a "Swimming Performance Analyst" to ensure the output utilizes correct sports science terminology. Second, the continuous kinematic profiles are discretized and serialized into a textual format. Metrics including distance, cumulative time, and instantaneous velocity are extracted at fixed intervals and formatted as structured tokens. This serialization allows the model to process temporal dependencies and identify trends such as acceleration phases or fatigue-induced deceleration. Finally, strict task constraints are applied to prevent information hallucination, directing the model to focus exclusively on three diagnostic dimensions: phase analysis, pacing strategy identification, and fatigue assessment.

Figure 5 presents a representative automated report generated for a multi-lane freestyle session. The output demonstrates the method's ability to convert frame-level detection data into actionable insights. As depicted, the report identifies specific segments where performance deviations occur, such as a notable velocity drop between the 15 m and 20 m marks, and offers targeted feedback. This result confirms that integrating high-precision computer vision metrics with the reasoning power of generative AI effectively replicates the analytical depth of a professional human coach.

To further assess the practical feasibility of the proposed framework for real-time coaching applications, the paper evaluated its runtime performance on a workstation. In addition to detection accuracy, four deployment-related indicators were measured, including inference speed, average processing time per frame, end-to-end report generation latency, and GPU utilization. The results are summarized in Table 6.

Table 6.

Runtime performance of the proposed framework

Metric	Value
Inference speed	13.6 FPS
Average processing time per frame	73.4 ms
End-to-end report generation latency	71.0 s (968 images)
Average GPU utilization	85.2% (RTX4070)

As shown in Table 6, the proposed framework achieved an inference speed of 13.6 FPS on an RTX4070 GPU, with an average processing time of 73.4 ms per frame. The complete process, including swimmer detection, kinematic calculation, visualization, and report generation, required 71.0 s for 968 images. These results show that the system can complete swimming video analysis within a practical processing time and is suitable for post-race performance review and coaching assistance. The average GPU utilization was 85.2%, indicating that the system can run stably on a single consumer-level GPU. The DeepSeek-V3 module was evaluated for its ability to transform measured kinematic indicators into readable coaching-style reports. These reports are intended to help coaches understand pacing and performance trends, rather than replace professional coaching judgment.

Discussion

Principal findings

The integration of a splash-augmented YOLOv11 detector with the DeepSeek-V3 LLM facilitates automated kinematic profiling and pedagogical interpretation in competitive swimming. The vision module attained a mean Average Precision (mAP@0.5) of 94.64% and a tracking rate exceeding 96.0% in central lanes, even under hydrodynamic turbulence characterized by foam and splash occlusion. The extracted meter-level velocity fluctuations provide the spatiotemporal granularity necessary for the LLM to generate structured diagnostic reports. This automated pipeline effectively identifies pacing strategies and fatigue-induced deceleration, eliminating the latency associated with manual video annotation.

Comparison with existing methodologies

The performance of this non-invasive framework addresses limitations identified in previous sensor-based and vision-based studies. In contrast to sacrum-mounted inertial measurement units (IMUs) utilized by Slopecki et al. [10] and Li et al. [11], which introduce physical interference and alter a swimmer’s hydrodynamics [12], this vision-based method captures high-fidelity kinematic data without compromising stroke mechanics.

Compared with standard architectures such as DeepDASH [14] and baseline object detectors, the framework in this paper applies a splash-injection data augmentation strategy to enhance feature extraction at the water–air interface. Traditional models, including DeepDASH, often experience identity switches or target loss when athletes are partially obscured by splash or turbulent water. With the proposed augmentation, the framework maintains trajectory continuity and achieves a 3.83% increase in Recall and a corresponding improvement in F1 score relative to models trained only on canonical imagery. In addition, the integration of homography-based kinematic extraction with large language model-based report generation enables the framework to convert measured velocity and segment-level indicators into coach-readable performance summaries, highlighting its novelty compared with prior detection-focused methods.

Pedagogical utility and practical applications in elite training

The practical utility of the framework involves resolving the disconnect between raw numerical data and actionable coaching interventions. Traditional methodologies in swimming analytics often produce time-series metrics like instantaneous velocity curves. These outputs require manual interpretation by biomechanists to be useful. Incorporating the DeepSeek-V3 LLM facilitates the conversion of kinematic anomalies into structured pedagogical assessments.

In a 200 m freestyle performance analysis, the model detected a velocity attenuation of 0.12 m/s during the third 50-m segment spanning from 100 to 150 m. This quantitative decline correlated with a recorded increase of 0.08 s in mean stroke duration. Such metrics indicate a reduction in propulsive efficiency resulting from accumulated metabolic fatigue. The automated diagnostic module subsequently generated technical advice regarding the maintenance of stroke tempo and the optimization of turn transition velocity to mitigate mid-race deceleration.

This analytical loop implements the PRG paradigm established in prior literature [18]. It provides an objective basis for technical adjustments by removing the latency inherent in manual video review. Furthermore, reducing the analytical workload for coaching staff enables the delivery of evidence-based feedback between competition heats or training sets. The synchronization of meter-level velocity data with natural language reporting offers a scalable solution for performance monitoring in elite aquatic environments.

Limitations and future directions

Although the proposed method achieved a mAP of 94.64%, several limitations remain. The system relies on a single camera view, which restricts analysis to two-dimensional kinematics. While global metrics such as velocity and pacing can be reliably estimated, three-dimensional details like underwater depth or body roll cannot be captured. The spatial mapping assumes a fixed camera, so dynamic movements such as panning or tilting are not supported. Tracking performance can be affected by extreme perspective distortion and splash occlusion, particularly in the outermost lanes. The DeepSeek-V3 module generates reports from extracted kinematic indicators rather than directly observing swimmer motion, so feedback is limited to pacing and fatigue trends and does not provide detailed technical corrections. Prompt constraints help reduce unsupported interpretations, but potential hallucinations remain, and the reports should be regarded as supportive information for coaches rather than a replacement for professional judgment.

Future work will aim to address these constraints by integrating multi-view or pose-estimation methods to reconstruct three-dimensional motion, allowing quantification of stroke mechanics and intra-cycle velocity. The framework will also be extended to other strokes with distinct splash patterns and occlusion challenges. Additionally, future versions will explore vision-language models capable of combining kinematic data with visual frames to provide more comprehensive coaching feedback that integrates strategic and technical insights.

Conclusion

This research introduces an integrated computer vision framework designed for video-based kinematic profiling and automated coaching-style summarization in competitive swimming. By utilizing a YOLOv11 detector augmented with a splash-injection data augmentation strategy, the system maintains trajectory continuity in aquatic environments despite visual interference from surface turbulence and foam. Experimental evaluations demonstrate a mean Average Precision (mAP@0.5) of 94.64% across eight swimming lanes, ensuring reliable detection in professional broadcast footage. Kinematic metrics including instantaneous velocity and stroke distance are extracted from the validated trajectories and show high temporal synchronization with manual ground truth measurements. The subsequent integration of the DeepSeek-V3 Large Language Model facilitates the translation of these quantitative streams into structured coaching reports. This framework provides a scalable and objective methodology for enhancing performance monitoring and training optimization by bridging the gap between raw vision-based measurements and actionable pedagogical insights.

Authors’ contributions

GL and LL conceptualized the study and designed the methodology. GL and HL collected the data and performed the experiments. MZ and SZ developed the software and performed the data analysis. YW participated in the visualization and validation. GL and LL prepared the original draft. LL provided supervision and reviewed the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported in part by "Pioneer" and "Leading Goose" R&D Program of Zhejiang (No. 2024C03260); Zhejiang Provincial Higher Vocational Education Teaching Reform Project (JGCG202507GZ).

Data availability

The dataset constructed for this study is publicly available at [https://github.com/T-bjq/sw-data].

Declarations

Ethics approval and consent to participate

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Academic Ethics and Scientific Norms Committee of Taizhou University (Approval No. 2026008). All video data used in this research were obtained from public competitive swimming broadcasts and standard training sessions. The requirement for informed consent was waived because the study involved retrospective analysis of publicly available footage and non-invasive observations, with no direct intervention or compromise of athlete privacy.

Consent for publication

Not applicable. This manuscript does not contain any personal clinical details or identifying images of participants that would require written informed consent. All data were anonymized or derived from public domain broadcasts where athletes are public figures.

Competing interests

The authors declare no competing interests.