Impact of AI Assistance on Radiologist Accuracy for Lung Nodule Detection on Chest CT

Abstract

Background: Accurate identification of pulmonary nodules as early-stage lung cancers is crucial to decrease the number of deaths and illnesses caused by lung cancer. Artificial Intelligence has the potential to enhance diagnostic accuracy and specificity in detecting lung cancer. Methods: Chest CT scanning from 300 patients with an age range between 40 and 80 years old were analysed comparing the pulmonary nodules detection rate (number of lung nodules) between AI-assisted reading, non-AI-assisted reading and the AI-system report standalone. Detected nodules, missed nodules (false negatives), and false-positive findings were analysed. Results: AI-assisted radiologists missed significantly fewer nodules (p<0.001) and achieved an almost perfect correlation (r~1.00) with expert reference values, reducing the mean absolute error (MAE) from 9.78 to 0.46. Additionally, AI increased detection sensitivity from 60% to 98% and reduced false negatives from 3,083 to 145, optimizing both diagnostic accuracy and efficiency. Conclusions: AI-assisted reading has shown to be beneficial in the detection of lung nodules compared to relying solely on radiologist observation. This suggests that an AI-powered system for evaluating lung nodules has the potential to become a valuable assistant tool in clinical practice. By combining the skills of radiologists with AI assistance, a new approach may emerge, leading to enhanced detection of lung nodules and encouraging the integration of AI in lung cancer screening initiatives

Introduction

Lung cancer is the top cause of cancer deaths for both men and women globally. However, a comparison based on economic development reveals that lung cancer deaths among women in industrialized nations are higher compared to those in developing countries.

Additionally, among women in developing countries, breast cancer deaths outnumber lung cancer deaths [1, 2].

Despite a rise in the one-year survival rate for lung cancer over recent years, the overall 5-year survival rate remains low under 20%.

One of the main reasons for this is that many patients are already in advanced stages of the disease when they are first diagnosed [3].

Studies have shown that up to 55% of lung cancer patients are found to have distant disease upon initial diagnosis, resulting in a predicted 5-year survival rate of only 5% [4] Tobacco smoking is the primary risk factor for lung cancer.

Even though the number of smokers is decreasing in many countries, controlling tobacco use alone is not enough to decrease lung cancer deaths [5].

Accurate identification of pulmonary nodules as early-stage lung cancers is crucial to decrease the number of deaths and illnesses caused by lung cancer.

The National Lung Screening Trial (NLST) found that using low-dose computed tomography (LDCT) to screen for lung cancer in high-risk individuals resulted in a 20% reduction in lung cancer deaths compared to chest radiograph (CXR) screening [6].

This has been supported by several randomized trials in Europe. As a result, some clinical guidelines in the US recommend LDCT screening for high-risk individuals.

However, LDCT screening also has potential harm such as high false positive rates, radiation exposure and economic costs [7, 8, 9, 10].

Early-stage lung cancer is often detected as a lung nodule on Computed Tomography (CT) scans.

However, implementing high-quality lung cancer screening in a clinical setting can be difficult.

In real-world scenarios, multiple clinicians with varying levels of experience read the images.

Some lung nodules may be overlooked due to their appearance or due to observer errors such as fatigue, distraction, or difficulty distinguishing nodules among many on the image [8, 11, 12, 13, 14].

Chest radiography is a commonly ordered imaging test, often the first examination used when a patient has symptoms of lung cancer [15].

However, the accuracy of human interpretation of chest radiographs in diagnosing lung cancer is low, with reported sensitivity ranging from 36-84%. Studies have shown that 19-26% of lung tumours are not detected during the first interpretation of a chest radiograph. Due to the limitations of chest radiography, CT has become a more commonly used diagnostic and screening tool for lung cancer [3, 16, 17, 18, 19].

The NLST has also shown that CT screening, specifically LDCT, can improve mortality rates in high-risk patients [6].

However, while CT is better at detecting pulmonary nodules, its specificity for lung cancer diagnosis is still somewhat low.

Additionally, human interpretation of CT images can be subjective and lead to high variability in detecting and diagnosing lung cancer [18, 20].

Furthermore, although not serving as a definitive diagnosis, there has been an increasing integration of techniques like 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) into regular medical practice in recent years that can offer valuable supplementary insights into the functional aspects of abnormalities.

More recently, the combination of PET and CT imaging has significantly improved the exploration of lung cancer cases, enabling more precise outlining of areas exhibiting heightened tracer absorption.

This approach has aided radiologists in sidestepping the technical challenges associated with separately conducting PET and CT scans, which had previously led to notable distortions in the results. Additionally, PET/CT has demonstrated its accuracy in evaluating solitary pulmonary nodules (SPNs) and in determining the stage of lung cancer. It achieves this by enhancing the identification of metastatic conditions, guiding treatment decisions, and facilitating predictions about clinical outcomes [21].

One way to address the limitations of chest radiography and CT is using artificial intelligence (AI) which can improve diagnostic accuracy and specificity of lung cancer detection. AI algorithms use machine learning to analyse a great number of medical images and identify specific features. This allows them to identify target lesions with greater accuracy.

Recently, many studies have been conducted using Convolutional Neural Networks (CNN) to improve the detection, segmentation, and classification of lung diseases [11].

Several studies have proposed the use of deep learning architectures to improve the detection and classification of lung nodules, with promising results. Rebouças Filho et al. proposed a 3D adaptive crisp active contour method (3D ACACM) to segment lung images, which had an average F-score of 99% for 40 chest CT scans [22].

They also used an optimum path forest classifier to automatically identify three lung diseases in CT images, with an F-score of 95% [23].

Nasrullah et al. proposed a deep 3D customized mixed link network (CMixNet) architecture and achieved a sensitivity of 94% and a specificity of 91% in detecting and classifying lung nodules, which can reduce misdiagnosis and false positives in lung cancer screening [22].

Ciompi et al. proposed a deep learning system based on multi-stream and multi-scale CNNs, which can classify lung nodules into 6 image feature categories, and its performance was comparable to human observers. In recent years, with the availability of large datasets and the increasing demand in clinical settings, AI technology has rapidly progressed in lung cancer screening. AI-based systems have achieved detection accuracy of lung nodules between 82.2-97.6% [24, 25, 26].

Some studies also suggest that AI can improve the differential diagnosis and management of lung nodules [27, 28, 29, 30].

Many previous AI research projects have focused on evaluating particular CNN techniques, either to improve their design or to retrospectively analyse their effectiveness using a particular dataset.

Objective

The objective of this study was to evaluate the impact of artificial intelligence assistance on radiologist performance in lung nodule detection on chest CT examinations.

Specifically, the study aimed to compare the number of detected and missed pulmonary nodules, diagnostic accuracy, and agreement with an expert reference standard between radiologist-only readings, AI-assisted readings, and an AI system used as a standalone reader in a real-world clinical setting.

Methods

This retrospective study included 300 patients aged between 40 and 80 years who underwent chest CT examinations over a two-month period (September-October 2024) at the Medical Imaging Department of the University of Medicine and Pharmacy of Craiova. The study population comprised 137 women and 163 men.

Both patients with a known malignancy and patients without significant prior medical history were included.

Exclusion criteria were a history of lung surgery, incomplete diagnostic reports, or the presence of major respiratory motion or beam-hardening artifacts that could compromise image interpretation.

All CT examinations were performed using a single multidetector CT system (Siemens).

Contrast-enhanced chest CT scans were acquired during a single inspiratory breath-hold, with patients in the supine position. Imaging parameters included a tube voltage of 120kV and a slice thickness of 1.5mm.

The reporting workflow consisted of multiple sequential readings. Initially, a junior radiologist (R.V.T.) generated the preliminary report, which was subsequently reviewed and approved by a board-certified radiologist (L.M.F.) based on a non-AI-assisted interpretation. Subsequently, a different radiologist (C.M.C.), blinded to the original reports, reevaluated the same CT images using an AI-based lung nodule detection system.

In the final step, a radiology trainee (M.A.E.) independently recorded the number of lung nodules detected by the radiologist alone, by the radiologist with AI assistance, and by the AI system alone, based on the respective reports.

A commercially available AI-based lung nodule assessment system developed by Mindfully Technologies SRL (trade name: Rayscape; Calea Torontalului 69, VOX Building, Building B, 6th floor, Timișoara, Timiș, Romania) was used in this study. The system automatically detects and quantifies lung nodules on chest CT scans as shown in Figure 1 and Figure 2.

Figure 1.

Data Report (AI-assisted detection and structured reporting of pulmonary nodules on chest CT). Axial chest CT image demonstrating automatic detection of a pulmonary nodule highlighted with a bounding box (left). The AI-generated interface (right) displays a structured list of detected nodules, including lesion ID (number), anatomical location (lung and lobe), size (diameter and volume), and imaging characteristics (e.g., solid appearance, suspicion level). The system also provides guideline-based follow-up recommendations according to Fleischner criteria, facilitating standardized clinical assessment.

Figure 2.

Data report (AI-assisted detection of multiple pulmonary nodules with characterization and prioritization). Axial chest CT image demonstrating automatic detection of multiple pulmonary nodules highlighted with bounding boxes (left). The AI-generated interface (right) provides a structured overview of all detected lesions, including anatomical location, size measurements, and attenuation characteristics, with emphasis on subsolid nodules. Suspicious lesions are prioritized within the list, and guideline-based follow-up recommendations according to Fleischner criteria are displayed, supporting risk stratification and clinical decision-making.

CT images were transferred directly from the scanner console to the AI server for automated processing, with an average processing time of 10-20 minutes per examination. AI results were displayed on the reading workstation, where detected nodules were marked with bounding boxes and accompanied by quantitative information, including diameter, volume, anatomical location, and follow-up recommendations based on the Fleischner Society guidelines.

During AI-assisted image analysis, the radiologist documented system errors, including false-positive findings such as fibrosis, vascular structures, vertebral osteophytes, and respiratory artifacts, as well as false-negative findings, represented by missed pulmonary nodules.

Statistical analysis was performed to evaluate differences in lung nodule detection and diagnostic performance between radiologist-only and AI-assisted readings. The paired samples t-test was used to compare continuous variables between paired groups.

The Wilcoxon signed-rank test was applied for non-parametric comparisons. Correlation with the reference standard was assessed using the Pearson correlation coefficient. Linear regression analysis was used to evaluate the relationship between detected nodules and reference values, including the coefficient of determination (R²). Diagnostic error was quantified using the mean absolute error (MAE). Sensitivity and specificity were calculated to assess diagnostic performance.

A p-value <0.05 was considered statistically significant. The reference standard (“expert panel”) consisted of a senior board-certified radiologist with extensive experience in thoracic imaging (I.A.G.), who reviewed all CT examinations and established the reference findings.

Results

The results section presents a comparative analysis of lung nodule detection performance across three reading approaches: radiologist-only interpretation, AI-assisted radiologist interpretation, and an AI system used as a standalone reader. Quantitative assessments were performed using multiple statistical methods to evaluate detection accuracy, agreement with the expert reference standard, and diagnostic error.

The following subsections detail the impact of AI assistance on missed nodules, correlation with reference values, diagnostic accuracy metrics, and overall performance.

Detection performance and missed nodules

The number of missed pulmonary nodules was significantly lower in AI-assisted readings compared to radiologist-only interpretation. This difference was confirmed by a paired samples t-test (t=5.71, p<0.001). Radiologists using AI assistance demonstrated improved detection performance, with fewer missed nodules and reduced variability across cases.

This reduction is illustrated in Figure 3, which shows a narrower distribution and lower median values for AI-assisted readings.

Figure 3.

Boxplot representation of the number of missed lung nodules per patient for radiologist-only and AI-assisted readings.

AI assistance resulted in a significant reduction in missed nodules and decreased variability across cases (p<0.001).

In terms of overall detection, the total number of lung nodules identified differed between methods, as summarized in Table 1.

Table 1.

The number of lung nodules detected by each method

Method	Total number of lung nodules detected
Radiologist	6760
Radiologist AI assisted	5811
AI system	7959
Reading Panel	5783

The AI-assisted radiologist achieved results closer to the reference standard compared to the unassisted radiologist.

Agreement with the reference standard

Agreement between radiologist findings and the reference standard was assessed through correlation and regression analysis. The unassisted radiologist showed a strong correlation with the reference panel (r=0.861), while the AI-assisted radiologist demonstrated near-perfect agreement (r≈1.00). Linear regression analysis further supported this improvement, with the slope increasing from 0.76 in radiologist-only readings to 1.01 in AI-assisted readings, and the coefficient of determination (R²) improving from 0.74 to 0.999.

These relationships are visually represented in Figure 4, where AI-assisted readings show near-complete overlap with the reference standard.

Figure 4.

Scatter plot showing the correlation between the number of lung nodules detected by the radiologist alone and by the radiologist assisted by AI, compared with the reference standard (reading panel). AI-assisted readings demonstrate near-perfect agreement with the reference standard.

Diagnostic error analysis

Diagnostic error was quantified using the mean absolute error (MAE) and further assessed using the Wilcoxon signed-rank test. The MAE decreased substantially from 9.78 in radiologist-only readings to 0.46 in AI-assisted readings, indicating a marked improvement in agreement with the reference standard.

In addition, the Wilcoxon signed-rank test showed a statistically significant difference in diagnostic errors between the two reading approaches (test statistic=13218, p<0.001), confirming that AI assistance consistently reduced discrepancies between radiologist assessments and the reference standard.

False-positive and false-negative findings

A comparative analysis of false-positive and false-negative findings is presented in Table 2.

Table 2.

False positives and negatives

Method	False positive	False negative
Radiologist	2073	3083
Radiologist AI assisted	59	145
AI system	451	398

The number of false positives decreased from 2073 in radiologist-only readings to 59 in AI-assisted readings, while false negatives were reduced from 3083 to 145.

Although the AI system alone generated a higher number of false positives (451), the combination of AI and radiologist interpretation resulted in a more balanced diagnostic performance, reducing both overdiagnosis and missed lesions.

Diagnostic performance: sensitivity and specificity

Diagnostic performance metrics are summarized in Figure 5. Sensitivity increased from 60% in radiologist-only readings to 98% in AI-assisted readings, while specificity improved from 69% to 99%.

Figure 5.

Bar chart comparison of mean absolute error (MAE) and sensitivity for radiologist-only and AI-assisted readings.

AI assistance resulted in substantially lower diagnostic error and higher sensitivity.

The AI system alone demonstrated high sensitivity (95%) but lower specificity (94%) compared to the AI-assisted radiologist. These findings indicate that AI enhances sensitivity while maintaining high specificity when used as a second reader, supporting its role as a complementary tool in clinical practice.

Workflow

The time required for the radiologist to visually detect the number of lung nodules per examination varied from a few minutes for cases with a total number of nodules less than 10 and up to 45-50 minutes for cases with a total number exceeding 100 nodules per patient.

AI-assisted reading facilitated faster identification and evaluation of nodules, particularly in cases with a high number of lesions, suggesting improved workflow efficiency and reduced reading time in complex examinations.

Discussions

The results of the present study demonstrate that the use of artificial intelligence as a second reader significantly improves lung nodule detection in routine chest CT examinations. In a cohort of 300 patients, AI assistance led to a marked reduction in missed nodules, a near-perfect agreement with the expert reference standard, and a substantial decrease in diagnostic error, as reflected by the reduction in mean absolute error and the increase in sensitivity. These findings confirm that AI-assisted reading enhances radiologist performance without replacing clinical judgment, supporting its role as an adjunctive tool in daily practice.

A key finding of this study is the significant reduction in false-negative results when AI assistance was used. Missed pulmonary nodules remain a major concern in lung cancer imaging, as delayed or missed detection may lead to diagnosis at advanced stages, where prognosis is poor [3, 4].

Previous studies have demonstrated considerable inter-reader variability in lung nodule detection, even among experienced radiologists, with a non-negligible proportion of nodules being overlooked in screening trials such as the National Lung Screening Trial (NLST) [31, 33].

In this context, our results show that AI assistance substantially reduces the number of missed nodules and increases sensitivity from 60% to 98%, bringing radiologist performance closer to the reference standard.

This improvement is consistent with prior reports suggesting that computer-aided detection systems can function effectively as a second reader rather than as a standalone diagnostic tool [35, 36].

The strong correlation observed between AI-assisted readings and the expert panel further underscores the benefit of AI integration. While the unassisted radiologist al-ready demonstrated good agreement with the reference standard, AI assistance resulted in an almost perfect correlation and eliminated the systematic underestimation observed in non-assisted readings. Similar improvements in agreement and consistency have been reported in studies evaluating deep learning-based systems for pulmonary nodule detection, which have shown detection accuracies ranging from 82.2% to 97.6% [24, 25, 26].

Our findings support these results in a real-world clinical setting, emphasizing that AI can enhance diagnostic consistency when applied to routine CT examinations rather than curated research datasets.

An important aspect highlighted by our analysis is the balance between false-positive and false-negative findings. AI systems alone tend to generate a higher number of false positives due to their high sensitivity, as also reported in previous studies [11, 37].

However, when AI output is reviewed and validated by a radiologist, false positives are markedly reduced while maintaining high sensitivity. This complementary interaction between AI and human expertise represents one of the major strengths of AI-assisted reading.

Rather than overwhelming radiologists with excessive false alarms, AI functions as a supportive filter that draws attention to potential lesions while allowing clinical judgment to exclude irrelevant findings such as vessels, fibrosis, or degenerative changes. This observation aligns with earlier research demonstrating that CAD systems are most effective when used in conjunction with radiologist interpretation [34, 35].

From a practical perspective, workflow efficiency is another important consideration.

The interpretation of chest CT scans can be time-consuming, particularly in patients with numerous pulmonary nodules, where visual assessment may require pro-longed reading times. Previous studies have reported that increasing imaging volume and case complexity contribute to radiologist fatigue and diagnostic errors [11, 31].

In this study, AI-assisted reading facilitated faster identification and review of nodules, particularly in cases with a high nodule burden, suggesting that AI integration may help optimize workflow without compromising diagnostic accuracy. These findings support the growing need for AI tools to assist radiologists in managing increasing workloads in lung cancer screening and routine clinical practice.

Despite these encouraging results, several limitations should be acknowledged. First, this was a single-centre retrospective study, which may limit the generalizability of the findings to other institutions with different patient populations or imaging protocols. Second, the study focused exclusively on nodule detection and did not include histopathological confirmation or assessment of nodule malignancy. Additionally, nodules were not stratified according to size, morphology, or attenuation characteristics, such as solid versus ground-glass nodules. As observed during image review, ground-glass nodules remain more challenging for AI systems to detect reliably, a limitation that has also been reported in previous studies [11, 38].

Future research incorporating multi-center data, morphological stratification, and outcome-based validation would provide further insight into the clinical impact of AI-assisted lung nodule detection.

Conclusions

The integration of artificial intelligence as a second reader in chest CT interpretation provides clear benefits in lung nodule detection.

AI assistance significantly reduces missed nodules and improves diagnostic accuracy, resulting in higher sensitivity and closer agreement with expert reference standards, without compromising specificity.

When combined with radiologist interpretation, AI optimizes the balance between false-positive and false-negative findings, highlighting its complementary role rather than a standalone solution.

These results support the clinical value of AI-assisted reading as a reliable tool to enhance radiologist performance in routine lung nodule assessment.

Conflict of interests

The authors declare no competing interests.