Label quality as a bottleneck for YOLO detection in all-weather autonomous driving using the MUSES dataset

Abstract

Autonomous-vehicle perception models perform well in clear daytime but degrade under adverse weather and low illumination. We quantify how annotation quality affects YOLO-based 2D detection using the forward-facing RGB subset of the MUSES dataset spanning clear, rain, fog, snow, and night. Focusing on lightweight detectors (YOLOv11n/YOLOv11s) to reflect embedded ADAS constraints, we observe systematic label noise (missing objects, mislocalized boxes, and class inconsistencies) that limits achievable accuracy. We therefore introduce a semi-automatic label-upgrading pipeline: a high-capacity teacher produces candidate labels, which are then verified and corrected by a human annotator. Under identical training settings, retraining on the cleaned annotations improves mAP@0.5 from approximately 0.13 to 0.56–0.62 across conditions, indicating that supervision quality is a major bottleneck for all-weather detection and that data-centric fixes can unlock large gains even for lightweight models.

Introduction

Autonomous driving perception has advanced rapidly, and modern detectors can achieve strong accuracy in clear, daytime scenes [1]. However, this progress does not transfer uniformly across operating conditions. In adverse weather and low illumination (e.g., fog, snow, heavy rain, dust, and night), visibility degradation and occlusion can cause missed detections, poor localization of distant objects, and false positives. Crucially, these errors arise in safety-critical situations, where reliable perception is most needed, making all-weather robustness a persistent barrier to dependable autonomy [2].

YOLO-style detectors and related real-time architectures remain popular in autonomous-driving stacks due to their accuracy–latency trade-off and deployability on embedded hardware [3–5]. Yet numerous studies report sharp performance drops under poor visibility, even with common countermeasures such as data augmentation or domain adaptation [6]. To benchmark these failure modes, adverse-weather datasets such as DAWN and ACDC were introduced to capture challenging conditions beyond clear-weather scenes [7, 8]. More recently, MUSES extended this line of work by providing a multi-sensor benchmark under uncertainty, covering diverse weather and lighting conditions with synchronized RGB, LiDAR, radar, event cameras, and IMU/GNSS [9].

While most prior work attributes robustness gaps primarily to model design or domain shift, an orthogonal factor is often implicitly assumed: annotation quality. In practice, adverse-weather data can exhibit systematic label noise that affects both training and evaluation. If label quality varies with weather or illumination, conclusions about model robustness can be confounded: a detector may appear to fail under fog or night partly because the supervision is incomplete or inconsistent. Motivated by this concern, we isolate and quantify the impact of supervision quality on adverse-weather detection by keeping the detector family and training protocol fixed.

Concretely, we study YOLO-based 2D detection on a forward-facing RGB subset of MUSES. Training lightweight models (YOLOv11n and YOLOv11s) on the original annotations yields low mAP@0.5, and a targeted annotation audit reveals severe issues, including unlabeled but clearly visible objects and misclassified or poorly aligned bounding boxes. To address this, we propose a semi-automatic label-upgrading pipeline: a high-capacity teacher (YOLOv12xL) generates candidate pseudo-labels for all images, followed by human verification and correction. Under identical training settings, retraining YOLOv11n/YOLOv11s on the cleaned annotations substantially improves detection accuracy, indicating that supervision quality is a major factor limiting performance in this setting, while architecture, domain shift, object scale, and weather severity may also contribute to the remaining errors.

The main contributions of this study are:

• Supervision audit for adverse-weather detection. We analyze the forward-facing RGB subset of MUSES and identify systematic label noise with a pronounced impact on small and distant objects under low visibility.

• Teacher-assisted, human-verified label upgrading. We introduce a practical semi-automatic pipeline in which a strong YOLOv12xL teacher proposes candidate annotations that are then verified and corrected by a human annotator, improving label quality without full relabeling.

• Quantifying the performance ceiling imposed by noisy labels. Keeping the detector family and training protocol fixed, we retrain YOLOv11n/YOLOv11s on the cleaned annotations and obtain large mAP@0.5 gains, showing that label quality can strongly influence the effectiveness of architectural or training improvements in adverse-weather settings.

Related work

This section reviews related work on autonomous-driving perception benchmarks and adverse-weather robustness with emphasis on datasets, evaluation protocols, and label quality. We then position our contribution in the context of label noise and relabeling for adverse weather object detection.

Adverse-weather robustness

Early advances in autonomous-driving perception were largely enabled by large-scale benchmarks collected primarily under standard visibility. The KITTI vision benchmark suite [10] established widely used tasks for road-scene understanding and object detection. Cityscapes [11] advanced urban scene perception by providing 5k finely annotated images and an additional 20k coarsely labeled images, supporting systematic evaluation of semantic and instance-level methods. To broaden domain diversity beyond a single geographic setting, Mapillary Vistas [12] released 25k high-resolution street-level images with fine-grained labels. In parallel, BDD100K [13] scaled driving data to large video collections and enabled heterogeneous multi-task learning. Despite their impact, these benchmarks remain dominated by normal conditions and provide limited coverage of severe adverse weather and nighttime scenes, motivating robustness-oriented datasets and evaluation protocols that consider not only domain shift and model capacity but also the reliability of ground-truth supervision.

To explicitly target adverse visibility, ACDC [8] covers fog, nighttime, rain, and snow, and provides uncertainty masks for regions where annotation is inherently ambiguous. WildDash [14] further emphasizes hazard-aware evaluation by curating challenging failure cases to stress-test perception systems under visual hazards. More recently, WildDash 2 [15] highlighted the importance of harmonized label policies and standardized evaluation protocols across driving datasets, showing that robustness conclusions can depend as much on annotation consistency and test design as on model architecture.

Multi-sensor datasets and the role of annotation quality

While many robustness benchmarks focus on single-camera inputs, real vehicles often rely on multi-sensor suites, and adverse conditions can make purely visual annotation inherently ambiguous. MUSES [9] is a multi-sensor driving benchmark with synchronized cameras, LiDAR, radar, event cameras, and IMU/GNSS, together with 2D panoptic annotations. Importantly, MUSES adopts a two-stage annotation process: an initial labeling stage (H1) and a refined stage (H2) supported by multi-sensor evidence and additional quality control. Their analysis shows that annotation policy strongly affects label completeness and evaluation outcomes: H2 contains roughly more instances than H1, and they report an 11.1% drop in PQ from 58.0% (H1) to 46.9% (H2), reflecting the impact of a stricter and more complete supervision protocol. They further report that evaluation outcomes and downstream performance are sensitive to the annotation stage, indicating that supervision quality can directly limit achievable accuracy under challenging conditions [9]. Beyond driving-specific benchmarks, several studies have shown that object-detection datasets frequently contain missing, mislocalized, or inconsistent annotations, and that auditing and correcting such errors can yield substantial gains without changing the underlying architecture. To move beyond aggregate mAP and better attribute performance changes, TIDE decomposes detection errors into interpretable categories (e.g., localization, classification, missed detections, and duplicates), enabling a more precise separation of supervision-induced effects from model limitations [16–18].

Positioning of our work

In contrast to work that emphasizes new architectures or weather-specific modules, this study shows that annotation quality can explain a substantial portion of the apparent performance gap in all-weather detection, while other factors such as domain shift, object scale, class imbalance, and weather severity may also contribute. Using the same dataset (MUSES), lightweight models (YOLOv11n and YOLOv11s), and the same training configuration, we show that relabeling yields gains of tens of mAP@0.5 points across conditions. This highlights the importance of revisiting dataset supervision when benchmarking adverse-weather perception and provides a label-audited baseline for future architectural or multi-modal extensions.

Dataset and label analysis

MUSES adverse-weather subset

All experiments use a subset of the MUSES dataset, a recent multi-sensor benchmark designed for autonomous driving under uncertainty [9]. MUSES provides synchronized RGB cameras, radar, LiDAR, an event camera, and IMU/GNSS across diverse weather and lighting conditions, including clear day, clear night, fog, rain, and snow. In this study, we use only the forward-facing RGB camera and formulate the task as 2D object detection. This setting reflects common ADAS and autonomous-driving deployments, while still exposing detectors to the visibility degradations that MUSES was designed to capture.

We evaluate seven safety-relevant classes: bicycle, bus, car, motorcycle, person, train, and truck. These categories include both conventional vehicles and vulnerable road users (VRUs), which are particularly challenging to detect when objects are small, distant, or partially occluded under adverse weather and low illumination.

Original subset and effective labeled data

The provided RGB-camera subset contains approximately 2,450 images, organized into training, validation, and test splits. During inspection, some images were found to have missing or empty annotation files. These images were not automatically treated as empty-background samples, because some scenes contained visible traffic participants without corresponding labels. Treating such images as negative samples would introduce false-negative supervision and could affect both training and evaluation.

To avoid this issue, images and annotation files were matched by filename before training and evaluation. Images containing visible objects from the seven target classes were corrected when the objects could be verified, while images without visible target-class objects were retained as background samples. After annotation filtering and cleaning, the final cleaned subset used in the experiments contained 1,281 training images, 215 validation images, and 675 test images, as summarized in Table 1.

Table 1.

Dataset split summary after annotation filtering and cleaning

Split	Images	Label files	Empty labels	Missing labels	Annotated instances
Train	1,281	1,258	43	23	5,099
Validation	215	214	5	1	731
Test	675	675	0	0	2,534

Dataset difficulty

To analyse robustness under different visibility regimes, we group the test images into eight conditions obtained by combining four weather types with two times of day: clear, rain, fog, and snow, each under both daytime and nighttime illumination. This split-by-condition evaluation allows us to explicitly compare clear-weather versus adverse-weather performance for YOLOv11n/YOLOv11s, instead of averaging everything into a single global score.

As shown in Table 2, the cleaned test set is not perfectly balanced across weather–illumination conditions. Therefore, condition-wise results should be interpreted with this imbalance in mind, especially for low-sample conditions such as fog night.

Table 2.

Condition-wise distribution of the cleaned test annotations

Weather/illumination condition	Images	Annotated instances
Clear day	150	776
Clear night	75	270
Fog day	100	128
Fog night	21	16
Rain day	99	503
Rain night	69	272
Snow day	89	282
Snow night	72	287
Total	675	2,534

Compared with standard urban-driving benchmarks such as KITTI or Cityscapes, the MUSES subset considered here is noticeably more challenging [19]. Many objects occupy only a small number of pixels, appear at a long distance, or are partially occluded by other vehicles and roadside infrastructure. Adverse weather introduces additional degradations, such as rain streaks, fog, snowflakes, motion blur, headlight glare, and low contrast in night scenes. In combination, these factors make it much harder for cars, trucks, buses, and especially pedestrians, cyclists, and motorcyclists to distinguish themselves from the background.

Qualitative inspection indicates that these effects are most severe in foggy-night and snowy-night, where vehicles are barely visible even to a human observer. As shown in Sect. 5, this results in a significant performance gap between clear-weather and adverse-weather conditions, despite using modern YOLO detectors [8, 9, 19].

Methodology

Our objective is to quantify the extent to which annotation quality limits all-weather object detection on MUSES and to demonstrate that systematic relabeling can unlock substantial performance gains, even for lightweight YOLO models. Figure 1 provides an overview of the proposed five-step pipeline, which spans baseline training with noisy supervision, teacher-driven pseudo-labeling, human verification, and retraining on cleaned annotations.

Fig. 1.

Overview of the proposed semi-automatic relabeling pipeline. Original noisy annotations are used for baseline training, followed by teacher-based pseudo-labeling with YOLOv12xL, human verification, and retraining on cleaned annotations

Task and model setup

We formulate the problem as 2D object detection using the forward-facing RGB camera of MUSES and evaluate seven traffic-participant classes: bicycle, bus, car, motorcycle, person, train, and truck. To reflect deployment-oriented constraints, we study two lightweight detectors from the YOLOv11 family: YOLOv11n (2.6M parameters, 6.3 GFLOPs) and YOLOv11s (9.4M parameters, 21.3 GFLOPs).

Figure 2 shows representative RGB images from MUSES that illustrate the visual diversity encountered in real-world driving. The examples highlight illumination variation, ranging from clear daytime scenes to low-light nighttime scenarios. Such variation affects visibility, contrast, and object appearance, and poses additional challenges for object detection models. Including these conditions is essential for evaluating robustness under realistic driving environments.

Fig. 2.

Representative RGB images from the MUSES dataset illustrating clear and low-illumination driving conditions

The experiments start from the original YOLO-format annotations provided with the RGB subset of MUSES, which nominally contains approximately 2,450 images split into 1,500 training, 250 validation, and 700 test samples. In practice, however, the effective supervision is weaker: a non-negligible fraction of images lack label files entirely or contain only partial annotations, particularly under adverse weather and low-light conditions. This discrepancy motivates a careful examination of annotation quality rather than model capacity alone. Figure 3 shows a representative RGB image from the MUSES dataset, illustrating the visual complexity of real-world driving scenes that must be handled by perception systems under adverse conditions.

Fig. 3.

Representative forward-facing RGB image from the MUSES dataset under challenging real-world driving conditions, with example detected vehicle boxes illustrating low-contrast and occlusion challenges

Step 2: baseline training on original annotations

YOLOv11n and YOLOv11s were initialized using pretrained Ultralytics weights and trained under the same configuration. Both models were trained for 200 epochs with 640 640 input resolution and batch size 32. The Ultralytics automatic optimizer setting was used with an initial learning rate of 0.01, final learning-rate factor of 0.01, momentum of 0.937, weight decay of 0.0005, and 3 warmup epochs. Default YOLO augmentations were used, including HSV augmentation, translation, scaling, horizontal flipping, mosaic augmentation, RandAugment, and random erasing. Validation was performed on the held-out validation split during training, and final results were reported on the held-out test split using precision, recall, mAP@0.5, and mAP@0.5:0.95. The same protocol was used for both original-annotation and cleaned-annotation experiments to ensure a controlled comparison.

Step 3: diagnosing annotation noise

To identify the causes of poor baseline performance, we manually inspected test images spanning all weather and illumination regimes. We observed four recurring annotation issues. First, missing-instance errors occur when visible traffic participants, especially small or distant vehicles, pedestrians, cyclists, or motorcycles, are not annotated. Second, localization errors occur when bounding boxes are shifted, too small, truncate part of the object, or include excessive background. These errors are often exacerbated by low contrast, motion blur, glare, rain, snow, or low illumination. Third, class-consistency errors occur when visually similar categories are confused, such as car versus bus or bicycle versus motorcycle. Fourth, some images contain visible objects but have missing or empty annotation files, which can incorrectly treat object-containing scenes as background images if not handled carefully. These annotation problems are more frequent under adverse weather and low illumination, where object boundaries are difficult to verify, leading to incomplete and unreliable supervision during baseline training and evaluation.

Figure 4 provides a qualitative example of annotation correction in a low-illumination night scene. The original annotations contain unreliable or poorly localized boxes, while the cleaned annotations retain only visually reliable objects after teacher-assisted relabeling and human verification.

Fig. 4.

Qualitative example of annotation cleaning in a night-time driving scene. The original annotations contain unreliable or mislocalized boxes under low illumination, while the cleaned annotations remove uncertain labels and retain more reliable object annotations after teacher-assisted relabeling and human verification

This example illustrates that annotation cleaning improves label reliability not only by adding missing objects, but also by removing uncertain or poorly localized annotations when visual evidence is insufficient.

Step 4: teacher-based pseudo-labeling with human verification

To upgrade the annotations without full manual re-annotation, we applied a high-capacity teacher detector, YOLOv12xL, to all images in the selected subset to generate candidate bounding boxes and class labels. YOLOv12xL was selected because it provides higher detection capacity than the lightweight YOLOv11n and YOLOv11s student models used in the final experiments. The teacher predictions were used only as candidate annotations and were not treated as final ground truth. Because no custom confidence threshold was specified during teacher prediction, candidate annotations were generated using the default Ultralytics prediction confidence threshold and then manually verified before being used as cleaned annotations.

To reduce dependence on the teacher model alone and mitigate potential teacher bias, all pseudo-labels were reviewed by a single human annotator. Correct detections were retained, bounding-box boundaries were refined, incorrect class labels were corrected, false-positive detections were removed, and missed but clearly visible objects were added manually. Ambiguous objects with very low visibility, severe occlusion, or unclear class identity were handled conservatively and excluded unless sufficient visual evidence was available. The verification process was guided by three quality-control criteria: label completeness, localization accuracy, and class consistency. Detailed annotation timing and the exact proportion of accepted versus manually corrected boxes were not recorded during verification. However, the revised pipeline description clarifies the main types of manual intervention, including box refinement, class correction, false-positive removal, and manual addition of missed visible objects.

Step 5: cleaned subset construction and retraining

After verification, we applied basic quality checks and retained image–label pairs with valid annotations for the seven target classes. The resulting cleaned subset contains 1,281 training images, 215 validation images, and 675 test images. The cleaned annotations were used for all three splits: training, validation, and testing. Thus, the performance improvement after relabeling reflects two effects. First, the models were trained with more complete and consistent supervision. Second, evaluation was performed using cleaner ground-truth labels, which provides a more reliable estimate of detection performance.

Crucially, YOLOv11n and YOLOv11s were retrained on the cleaned subset using the same architecture, optimization strategy, and training schedule as in the baseline experiments. Therefore, the main experimental variable was annotation quality, and the performance changes observed in Sect. 5 can be largely attributed to improved supervision quality rather than changes in the model architecture or training protocol.

Experimental results

This section evaluates how annotation quality affects all-weather 2D object detection on MUSES using two lightweight YOLOv11 variants: YOLOv11n and YOLOv11s. To test whether adverse-condition performance is limited by model capacity or by noisy supervision, we compare models trained on the original annotations with models trained on the cleaned-annotation subset under identical settings. We report overall results across weather–illumination conditions, training dynamics, and per-condition performance before and after relabeling.

Overall impact of relabeling

Table 3 summarizes YOLOv11n and YOLOv11s results on the original noisy annotations and the cleaned-annotation subset, averaged over all eight weather–illumination conditions.

Table 3.

Overall detection performance of YOLOv11n and YOLOv11s using original annotations versus cleaned annotations

Label version	Model	Precision (P)	Recall (R)	mAP@0.5	mAP@0.5:0.95
Original annotations	YOLOv11n	0.281	0.139	0.130	0.084
Original annotations	YOLOv11s	0.209	0.133	0.127	0.085
Cleaned annotations	YOLOv11n	0.604	0.559	0.557	0.375
Cleaned annotations	YOLOv11s	0.658	0.584	0.620	0.428

Values are averaged over the eight weather–illumination conditions

Under the original supervision, both models achieve very low accuracy (mAP@0.5 0.13 and mAP@0.5:0.95 0.085). The recall values (0.133–0.139) indicate that a large fraction of annotated objects are not recovered, consistent with supervision that is incomplete or internally inconsistent. In this regime, increasing the capacity from YOLOv11n to YOLOv11s does not produce measurable benefits, suggesting that learning is primarily limited by the quality of the training signal rather than representational power.

After training and evaluation with cleaned annotations, both precision and recall increase sharply, and this translates into large gains in average precision. YOLOv11n improves from 0.130 to 0.557 mAP@0.5 and YOLOv11s improves from 0.127 to 0.620 mAP@0.5, with corresponding improvements in mAP@0.5:0.95 (0.084 0.375 for YOLOv11n; 0.085 0.428 for YOLOv11s). The substantial increase in recall (to 0.559–0.584) suggests improved label completeness and object coverage, while the rise in precision (to 0.604–0.658) indicates more reliable localization and fewer confusions under corrected supervision. Overall, Table 3 shows that annotation quality imposes a major performance limitation on this MUSES subset: once supervision is corrected, lightweight YOLO models recover strong all-weather accuracy, and additional capacity becomes beneficial only after label noise is mitigated.

Training dynamics on original labels

Figure 5 reports validation mAP@0.5 and mAP@0.5:0.95 across epochs for YOLOv11n and YOLOv11s trained on the original annotations. Both models achieve rapid early gains but then plateau at low accuracy, with mAP@0.5 saturating around 0.25–0.30 and exhibiting noticeable oscillations across epochs. This behavior is consistent with a supervision-induced performance ceiling: the models learn from the subset of instances that are consistently annotated, while additional training yields limited improvements when labels are incomplete or inconsistent. The absence of a clear advantage for the higher-capacity YOLOv11s further suggests that scaling model capacity alone cannot overcome the limitations imposed by noisy supervision.

Fig. 5.

Training curves on the original MUSES annotations for YOLOv11n and YOLOv11s. The plotted metrics correspond to mAP@0.5 and mAP@0.5:0.95 across training epochs. Both models improve early but plateau at relatively low validation accuracy, indicating a supervision-induced performance limitation under noisy annotations

Per-condition performance before relabeling

Table 4 reports condition-wise performance across the eight weather–illumination regimes (clear/fog/rain/snow day/night) when both training and evaluation rely on the original annotations. Across all conditions, performance is low and dominated by limited recall, indicating that a large fraction of ground-truth instances are not recovered. This is consistent with supervision that is incomplete or unreliable for small, distant, and low-contrast objects—a failure mode that is expected to be amplified in night-time and adverse-weather scenes where annotation is inherently more difficult.

Table 4.

Per-condition detection performance of YOLOv11n and YOLOv11s on the original noisy MUSES subset before relabeling

Weather/time	Model	Precision (P)	Recall (R)	mAP@0.5	mAP@0.5:0.95
All (full test set)	YOLOv11n	0.281	0.139	0.130	0.084
	YOLOv11s	0.209	0.133	0.127	0.085
Clear day	YOLOv11n	0.141	0.129	0.136	0.095
	YOLOv11s	0.139	0.157	0.145	0.105
Clear night	YOLOv11n	0.260	0.100	0.106	0.070
	YOLOv11s	0.129	0.106	0.114	0.081
Fog day	YOLOv11n	0.555	0.137	0.146	0.109
	YOLOv11s	0.357	0.136	0.147	0.117
Fog night	YOLOv11n	0.330	0.312	0.188	0.078
	YOLOv11s	0.394	0.188	0.131	0.045
Rain day	YOLOv11n	0.130	0.110	0.121	0.080
	YOLOv11s	0.132	0.117	0.123	0.085
Rain night	YOLOv11n	0.402	0.079	0.092	0.058
	YOLOv11s	0.240	0.108	0.101	0.067
Snow day	YOLOv11n	0.156	0.141	0.132	0.096
	YOLOv11s	0.145	0.122	0.133	0.098
Snow night	YOLOv11n	0.273	0.106	0.117	0.084
	YOLOv11s	0.136	0.131	0.121	0.086

Metrics are reported on held-out test sets for each weather/illumination condition

Two additional patterns are noteworthy. First, the results do not exhibit a stable capacity trend: YOLOv11s does not consistently outperform YOLOv11n, suggesting that architectural scaling provides limited benefit when the supervision signal is noisy. Second, apparent condition-to-condition differences should be interpreted cautiously because label noise can confound robustness comparisons. For example, foggy daytime yields slightly higher mAP than clear daytime for both models, despite lower visibility. A plausible explanation is that fog reduces long-range background detail and decreases the number of visible instances, producing simpler scenes dominated by nearer, larger objects that are more likely to be annotated consistently. In contrast, clear daytime scenes contain more objects across a wider scale range, making reported performance more sensitive to missing instances and mislocalized boxes.

Overall, Table 4 suggests that the observed degradation under the original labels reflects not only intrinsic visibility challenges but also a supervision-induced ceiling that obscures the true robustness characteristics of the detectors.

Impact of relabeling on training dynamics

Figure 6 reports validation mAP@0.5 and mAP@0.5:0.95 across epochs after retraining YOLOv11n and YOLOv11s on the cleaned annotations. In contrast to the original-label regime (Fig. 5), both models exhibit substantially higher accuracy and more stable learning behavior. Validation mAP@0.5 rises into the 0.55–0.62 range and mAP@0.5:0.95 approaches 0.40, indicating improvements not only at loose IoU thresholds but also under stricter localization requirements. The trajectories are noticeably smoother, with reduced oscillations and no early saturation at low mAP, consistent with a stronger and more internally consistent supervision signal. Because the detector family, optimization configuration, and training schedule are held fixed between the original-label and cleaned-label experiments, the shift in both convergence level and stability can be attributed directly to improved annotation quality rather than architectural changes or hyperparameter tuning.

Fig. 6.

Training curves after training on cleaned annotations for YOLOv11n and YOLOv11s. The plotted metrics correspond to mAP@0.5 and mAP@0.5:0.95 across training epochs. Compared with the original-annotation setting, the cleaned annotations produce higher and more stable validation performance

Per-condition performance after relabeling

Performance increases across all weather–illumination regimes after relabeling (Table 5), indicating that the original results were largely constrained by supervision quality rather than detector capacity. On the full test set, mAP@0.5 reaches 0.541 for YOLOv11n and 0.622 for YOLOv11s, and the corresponding mAP@0.5:0.95 values (0.370 and 0.439) confirm that the gains persist under stricter IoU thresholds, i.e., improved localization quality in addition to higher detection rates.

Table 5.

Per-condition detection performance of YOLOv11n and YOLOv11s on the cleaned MUSES subset after relabeling

Weather/time	Model	Precision (P)	Recall (R)	mAP@0.5	mAP@0.5:0.95
All (full test set)	YOLOv11n	0.578	0.533	0.541	0.370
	YOLOv11s	0.700	0.551	0.622	0.439
Clear day	YOLOv11n	0.561	0.551	0.562	0.377
	YOLOv11s	0.728	0.622	0.658	0.462
Clear night	YOLOv11n	0.720	0.703	0.734	0.490
	YOLOv11s	0.792	0.612	0.698	0.493
Fog day	YOLOv11n	0.656	0.535	0.603	0.428
	YOLOv11s	0.698	0.764	0.760	0.519
Fog night	YOLOv11n	0.344	0.625	0.409	0.190
	YOLOv11s	0.342	0.500	0.311	0.155
Rain day	YOLOv11n	0.699	0.521	0.604	0.411
	YOLOv11s	0.814	0.553	0.656	0.498
Rain night	YOLOv11n	0.668	0.528	0.607	0.436
	YOLOv11s	0.681	0.580	0.665	0.474
Snow day	YOLOv11n	0.526	0.592	0.492	0.357
	YOLOv11s	0.689	0.436	0.506	0.374
Snow night	YOLOv11n	0.659	0.416	0.443	0.308
	YOLOv11s	0.518	0.603	0.658	0.447

Metrics are reported on the held-out test sets for each weather/illumination condition

Condition-wise, clear and rain scenarios yield the highest accuracy (e.g., YOLOv11s attains 0.658 mAP@0.5 in clear day and 0.665 in rain night), whereas fog night remains the most challenging regime (0.409 for YOLOv11n and 0.311 for YOLOv11s). This failure mode is consistent with compounded visibility degradation (low illumination + reduced contrast), where object boundaries become ambiguous and long-range cues are suppressed. Importantly, unlike the noisy-label regime, YOLOv11s now improves over YOLOv11n in most conditions, suggesting that model capacity becomes beneficial once annotation noise is mitigated and the supervision signal is reliable.

Figure 7 summarizes the mAP@0.5 gains before and after relabeling between conditions. Improvements are consistent across all regimes, highlighting that label upgrading benefits both nominal and adverse settings rather than shifting performance from one condition to another. Remaining gaps are concentrated in the most extreme visibility regimes (especially fog night), suggesting that after supervision is corrected, the remaining errors increasingly reflect intrinsic perceptual ambiguity, residual label noise, and RGB-only model limitations rather than missing or inconsistent labels alone.

Fig. 7.

Comparison of mAP@0.5 before and after annotation cleaning across weather–illumination conditions. Cleaned annotations improve performance across all conditions, while fog-night remains the most challenging setting

These results demonstrate that annotation quality is a major factor limiting all-weather object detection performance on this MUSES subset. Once reliable labels are available, lightweight detectors achieve strong and stable performance. The remaining failure cases appear to be driven largely by intrinsic visual ambiguity, residual label noise, and RGB-only model limitations rather than insufficient model capacity alone.

Discussion

The experiments demonstrate that annotation quality is a major factor affecting all-weather object detection performance on the MUSES subset. When training on the original labels, both YOLOv11n and YOLOv11s converge in terms of losses but achieve very low test accuracy, with mAP@0.5 close to 0.13. This mismatch indicates that the models are not only limited by capacity or optimization; rather, they are also learning from a supervision signal that is incomplete and inconsistent. In particular, missing boxes and partial labels create a training objective that may implicitly reward predicting fewer objects, especially for easier categories such as large cars, while penalizing correct detections that do not appear in the ground truth. Consequently, increasing model size does not yield a clear benefit under noisy supervision, since both models are constrained by the same label-quality limitations.

Existing noise-handling methods for object detection include robust loss functions, sample reweighting, pseudo-label filtering, and post-hoc error analysis tools. These approaches can reduce the effect of noisy annotations during training or help identify problematic labels. In this study, we focus on a data-centric annotation-cleaning strategy rather than comparing multiple noise-robust training methods. This choice was made to isolate the effect of annotation quality under a fixed detector family and training protocol. Future work could compare the proposed cleaned-annotation pipeline with noise-robust training strategies to determine whether both approaches provide complementary gains.

After applying teacher-assisted pseudo-labeling followed by human verification, the same architectures and training recipe produce substantially higher and more meaningful performance. Averaged over all eight conditions, mAP@0.5 rises from approximately 0.13 to 0.56 for YOLOv11n and to 0.62 for YOLOv11s, with similarly large gains in mAP@0.5:0.95. These improvements suggest that the cleaned annotations better reflect the true detection task by providing more complete coverage of traffic participants and tighter bounding boxes. This enables the models to generalize more effectively across weather and illumination conditions rather than overfitting to annotation artifacts. Importantly, once annotations are cleaned, the advantage of YOLOv11s becomes more visible: the small model consistently outperforms YOLOv11n, showing that additional model capacity becomes more useful when the supervision signal is reliable.

The per-condition analysis provides additional insight into the remaining failure modes. Fog-night remains the most challenging split even after annotation cleaning. The remaining errors in fog-night and snow-night conditions are likely caused by a combination of visual ambiguity, residual label noise, and model limitations. Visual ambiguity is a major factor because low illumination, reduced contrast, glare, fog, and snow obscure object boundaries and make small or distant objects difficult to distinguish from the background. Residual label noise may also remain because some objects are only partially visible or too ambiguous to annotate reliably, even during human verification. Finally, the models used in this study rely only on RGB images, which limits their ability to recover objects under severe visibility degradation. Snow-night also shows a larger performance spread between the two models, suggesting that higher-capacity backbones can better handle complex textures and background clutter, but still require stronger cues than RGB alone.

Although the results show that annotation quality is a major bottleneck in this MUSES subset, it should not be interpreted as the only factor limiting all-weather detection performance. Other factors, including domain shift, class imbalance, object scale, low illumination, and weather severity, can also contribute to performance degradation. In addition, the MUSES splits are not perfectly balanced across weather–illumination conditions, and the effective number of labeled images varies across conditions. Therefore, condition-wise results should be interpreted with this imbalance in mind. These findings suggest that improving supervision quality substantially reduces one important source of error, while further improvements in extreme conditions may require model-centric improvements, domain-specific augmentation, exposure normalization, or additional sensing modalities such as thermal or radar. Future work could also extend the proposed pipeline using large multimodal models and image-quality assessment methods. Such models could help identify visually ambiguous frames, compare image quality across weather and illumination conditions, and prioritize samples that require human annotation review. In particular, CLIP-based or magnitude-aware image-quality assessment methods may support no-reference evaluation of low-contrast, foggy, snowy, or night-time scenes where object boundaries are difficult to verify from RGB images alone.

Conclusion

This study demonstrates that, in the studied adverse-weather driving subset, annotation quality can be a major factor limiting object detection performance. When trained on the original MUSES RGB subset annotations, YOLOv11n and YOLOv11s plateau at very low accuracy despite stable optimization, which is consistent with learning under incomplete and inconsistent supervision. After label upgrading through teacher-assisted pseudo-labeling followed by human verification, the same architectures achieve large and consistent gains across weather and illumination conditions. This result shows that model capacity becomes beneficial only when supervision is reliable. More broadly, robustness conclusions drawn from adverse-weather datasets can be confounded by label noise that varies systematically with visibility. Future work should therefore combine model-centric advances with standardized, auditable data-centric protocols, including error decomposition and uncertainty-aware labeling, to produce dependable all-weather perception evaluations.

Author contributions

Ms. Zreiqat is the Ph.D. student of Dr. Rawashdeh. This research is funded by a NIST grant (60NANB24D227) under Dr. Nguyen (PI) and Dr. Rawashdeh (Co-PI). Ms. Zreiqat performed the research under the supervision of Dr. Rawashdeh and Dr. Nguyen.

Data availability

The data used in this study are based on the MUSES dataset, which is available from the official MUSES project website: https://muses.vision.ee.ethz.ch/, subject to its access conditions. The processed labels and derived annotations generated and analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.