Adaptive Breast MRI Scanning Using AI

Abstract

Background

MRI protocols typically involve many imaging sequences and often require too much time.

Purpose

To simulate artificial intelligence (AI)–directed stratified scanning for screening breast MRI with various triage thresholds and evaluate its diagnostic performance against that of the full breast MRI protocol.

Materials and Methods

This retrospective reader study included consecutive contrast-enhanced screening breast MRI examinations performed between January 2013 and January 2019 at three regional cancer sites. In this simulation study, an in-house AI tool generated a suspicion score for subtraction maximum intensity projection images during a given MRI examination, and the score was used to determine whether to proceed with the full MRI protocol or end the examination early (abbreviated breast MRI [AB-MRI] protocol). Examinations with suspicion scores under the 50th percentile were read using both the AB-MRI protocol (ie, dynamic contrast-enhanced MRI scans only) and the full MRI protocol. Diagnostic performance metrics for screening with various AI triage thresholds were compared with those for screening without AI triage.

Results

Of 863 women (mean age, 52 years ± 10 [SD]; 1423 MRI examinations), 51 received a cancer diagnosis within 12 months of screening. The diagnostic performance metrics for AI-directed stratified scanning that triaged 50% of examinations to AB-MRI versus full MRI protocol scanning were as follows: sensitivity, 88.2% (45 of 51; 95% CI: 79.4, 97.1) versus 86.3% (44 of 51; 95% CI: 76.8, 95.7); specificity, 80.8% (1108 of 1372; 95% CI: 78.7, 82.8) versus 81.4% (1117 of 1372; 95% CI: 79.4, 83.5); positive predictive value 3 (ie, percent of biopsies yielding cancer), 23.6% (43 of 182; 95% CI: 17.5, 29.8) versus 24.7% (42 of 170; 95% CI: 18.2, 31.2); cancer detection rate (per 1000 examinations), 31.6 (95% CI: 22.5, 40.7) versus 30.9 (95% CI: 21.9, 39.9); and interval cancer rate (per 1000 examinations), 4.2 (95% CI: 0.9, 7.6) versus 4.9 (95% CI: 1.3, 8.6). Specificity decreased by no more than 2.7 percentage points with AI triage. There were no AI-triaged examinations for which conducting the full MRI protocol would have resulted in additional cancer detection.

Conclusion

AI-directed stratified MRI decreased simulated scan times while maintaining diagnostic performance.

© RSNA, 2025

Supplemental material is available for this article.

See also the editorial by Strand in this issue.

Summary

Image-based artificial intelligence triage could be used to make modifications to breast MRI scanning protocols in real time, ensuring that sufficient diagnostic information is obtained for each patient as quickly as possible.

Key Results

• ■ In a reader simulation study of 863 women who underwent screening breast MRI, artificial intelligence (AI)–directed scanning was used to triage 50% of 1423 breast MRI examinations to an abbreviated breast MRI (AB-MRI) protocol while maintaining diagnostic performance.

• ■ With AI-directed triage of 50% of examinations, specificity decreased by no more than 2.7 percentage points.

• ■ In no examination did triage lead to a missed cancer (ie, all cancers missed with AB-MRI were also missed with the full protocol).

Introduction

MRI enables unparalleled visualization of human anatomy and physiology without the use of ionizing radiation (1,2). However, examination times are often too long, leading to increased patient motion that degrades image quality, decreased patient throughput, and increased health care costs (3).

Advances in MRI acquisition techniques, including parallel imaging (4,5), compressed sensing (6,7), and, most recently, deep learning–driven methods (8,9), have reduced the amount of data needed to acquire and reconstruct high-quality MRI scans and thus have shortened scan times. There have also been efforts to abbreviate MRI protocols. A screening breast MRI typically includes dynamic contrast-enhanced (DCE) image acquisition as well as multiple non-DCE sequences (which can add 10 or more minutes to the acquisition time but impact image interpretation in only 3%–11% of examinations) (10,11). In abbreviated breast MRI (AB-MRI), some or all of the non-DCE sequences are skipped, yet this protocol still demonstrates near-equivalent diagnostic performance to the full protocol (12–15). However, other studies have shown that non-DCE sequences can be helpful in the screening of women who carry high-risk mutations (eg, BRCA), and imaging centers have been slow to shift to AB-MRI for screening (11).

Deep learning tools have been developed to increase the efficiency of breast imaging (16–18). For example, artificial intelligence (AI) triage tools that flag low-suspicion breast MRI examinations have been developed to enable worklist sorting (19,20). In this study, we repurposed an AI triage tool and used it not to sort the worklist or to remove low-suspicion examinations but rather to decide which images to acquire from which patients, which, to our knowledge, has not been done before.

Currently, MRI protocols (ie, MRI sequences to be run during the examination) are selected by the radiologist before examination initiation. Here, an AI-directed scanning paradigm is proposed, whereby an AI triage tool makes an on-the-fly midscan decision about whether to end the examination at that point or continue with the full protocol. A reader study was designed and performed to simulate AI-directed stratified MRI scanning with a range of triage threshold points, and the diagnostic performance of this approach was evaluated against that of the full breast MRI protocol.

Materials and Methods

Study Design and Sample

This retrospective simulation study was approved by the institutional review board at Memorial Sloan Kettering Cancer Center and complied with Health Insurance Portability and Accountability Act regulations. The requirement for informed consent was waived.

The study sample included consecutive contrast-enhanced annual screening breast MRI examinations performed at three regional cancer sites between January 2013 and January 2019. The women underwent screening breast MRI because they were designated as having high risk for breast cancer, typically defined as a greater than 20% lifetime risk. The breast MRI scans were acquired at imaging sites different from those used for the training and tuning of the previously described AI triage tool (21). These data were part of a held-out test set previously used to evaluate the tool but not previously used as part of a reader study test set.

Screening breast MRI examinations in women with a personal cancer history, any surgical history, or with breast implants were included. The American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) assessment in the original radiology report, and any associated pathologic examination results, were extracted from electronic medical records. Breast MRI examinations were considered cancer examinations if there was a breast cancer diagnosis within 1 year of follow-up and noncancer examinations if there was at least 1 year of imaging, pathologic, and/or clinical follow-up without a breast cancer diagnosis. Breast MRI scans were excluded if they lacked pathologic or follow-up information, if the examination did not include the standard MRI sequences, if the examination was performed for nonscreening indications, if the patient had a recent breast cancer diagnosis, or if the MRI scan was used in the training or validation of the AI model.

Breast MRI Acquisition

All patients underwent contrast-enhanced breast MRI on a 1.5-T or 3.0-T system (Discovery 750; GE HealthCare) with an eight- or 16-channel breast coil. A gadolinium-based contrast agent was administered at 0.1 mmol gadobutrol per kilogram body weight (Gadavist; Bayer Healthcare) at a rate of 2 mL/sec. See Table S1 for the MRI acquisition parameters. The MRI examinations in this reader study and those used to develop the AI triage tool used the same parameters (21).

Reader Study

The AI tool was applied to generate a suspicion score for each breast MRI. The tool generates a suspicion score from a subtraction maximum intensity projection image from the precontrast phase and first postcontrast phase (acquired approximately 70 seconds after contrast agent injection). All examinations with an AI suspicion score below the median suspicion score (ie, 50% of examinations) were reviewed by the readers (21).

The AB-MRI protocol included fat-saturated T1-weighted precontrast and postcontrast phases, as well as maximum intensity projections and coronal and sagittal reconstructions. The full MRI protocol included these sequences as well as non-fat-saturated T1-weighted images, fat-saturated T2-weighted images, and diffusion-weighted images.

For the reader study, each examination was randomly assigned to be reviewed by three of six fellowship-trained breast imaging radiologists (L.H.K., R.L.G., D.D., K.B., M.P.H., and N.B.S., with 7–24 years of experience). Each reader first reviewed each examination using the AB-MRI protocol only (ie, DCE sequences only) and assigned an AB-MRI BI-RADS score: BI-RADS 1 (negative), BI-RADS 2 (benign), BI-RADS 3 (probably benign), BI-RADS 4 (suspicious), or BI-RADS 5 (highly suspicious). Majority voting was used to determine the final BI-RADS score for each examination; scores were grouped as follows for analyses: BI-RADS 1 or 2, BI-RADS 3, and BI-RADS 4 or 5. The readers were blinded to the AI suspicion score and radiology report but were permitted to review prior examinations and associated reports.

Then the readers were unblinded to the non-DCE sequences and provided with the full set of MRI scans for each examination. The readers reviewed each examination with the full MRI protocol and assigned a full MRI BI-RADS score.

Simulation of AI-directed Stratified Scanning

With AI-directed scanning, examinations categorized by the AI tool as “low suspicion” (ie, under the triage threshold) undergo AB-MRI instead of the full MRI protocol, whereas examinations categorized by the AI tool as “high suspicion” undergo the full MRI protocol (Fig 1). Reader study data were used to simulate such AI-directed stratified scanning at triage thresholds from 0% to 50%. For a given triage cutoff, examinations with an AI suspicion score below that cutoff value were assigned the reader AB-MRI BI-RADS score, and examinations with an AI suspicion score above the cutoff value but below the median value were assigned the reader full MRI BI-RADS score. Examinations with an AI suspicion score above the median suspicion score were not reviewed by any readers and were assigned the BI-RADS score from the original radiology report. For example, at 25% triage, examinations in the first quartile of suspicion score were assigned the reader AB-MRI BI-RADS score, examinations in the second quartile were assigned the reader full MRI BI-RADS score, and examinations in the third and fourth quartiles were not part of the reader study and were assigned the radiology report BI-RADS score. The diagnostic performance when no examinations were triaged to AB-MRI was compared with that when 25% or 50% of examinations were triaged to AB-MRI.

Figure 1:

Study overview comparing (A) the conventional scanning paradigm, where all examinations undergo the full MRI protocol, and (B) the artificial intelligence (AI)–directed stratified scanning paradigm, where an AI tool triages examinations to an abbreviated breast MRI protocol based on the level of suspicion from initial MRI scans.

Statistical Analysis

Diagnostic performance metrics were calculated with 95% CIs (ie, calculated as estimate ± 1.96 × standard error of the estimate) for conventional (ie, full MR protocol for all patients) and AI-directed breast MRI. Metrics included sensitivity, specificity, accuracy, positive predictive value 3 (ie, percent of biopsies yielding cancer), negative predictive value, cancer detection rate, interval cancer rate, and negative biopsy rate. Interval cancers were defined as cancers that were diagnosed within 12 months of a negative screening MRI examination. The maximum performance gap was determined by comparing the performance metric for conventional scanning with the lower CI of the performance metric for AI-directed scanning at the 25% and 50% triage thresholds. Subgroup analysis was performed for cancer examinations and noncancer examinations to further demonstrate the effects of adaptive scanning with AI triage. All analyses were conducted using RStudio (version 2023.09.0 + 463; Posit) and overseen by one author (V.S.).

Results

Sample Characteristics

A total of 1423 screening breast MRI examinations from 863 women (mean age, 52 years ± 10 [SD]) were included in this study (Fig 2), of which 84% were in women who self-reported as non-Hispanic White (Table 1). Among the sample of 1423 examinations, 51 (3.6%) were categorized as cancer examinations (histopathologic cancer diagnosis within 1 year) and 1372 (96.4%) were categorized as noncancer examinations (1 year of follow-up with no cancer diagnosis). Table S2 shows the cancer subtypes in the sample.

Figure 2:

Patient flowchart. AI = artificial intelligence, DICOM = Digital Imaging and Communications in Medicine.

Table 1:

Patient and Examination Characteristics

AI Suspicion Scores and Cancer Outcomes

An AI suspicion score was successfully generated for all examinations, with a median score of 0.73. Figure 3A illustrates the number of total examinations and the number of cancer examinations in each AI suspicion score range. In total, of the 1423 examinations, 93 (6.5%) had a suspicion score less than 0.25, of which none were cancer examinations; 256 (18.0%) had a suspicion score of 0.25–0.49, of which two (0.8%) were cancer examinations; 401 (28.2%) had a suspicion score of 0.50–0.74, of which six (1.5%) were cancer examinations; and 673 (47.3%) had a suspicion score of 0.75–1.0, of which 43 (6.4%) were cancer examinations.

Figure 3:

Performance metrics of an artificial intelligence (AI)–directed stratified scanning paradigm, with 0%–50% of cases triaged to abbreviated breast MRI based on AI suspicion score. (A) Counts of cancer examinations (black) and total examinations (gray) assigned to each AI suspicion score bracket. (B) Sensitivity, specificity, and positive predictive value 3 (PPV3, percent of biopsies yielding cancer), and (C) cancer detection rate and interval cancer rate (per 1000 examinations) for AI suspicion score thresholds (0%–50%) used to trigger triage to abbreviated breast MRI.

Diagnostic Performance of AI-directed Triage

Conventional scanning with the full MRI protocol and no AI triage yielded a sensitivity, specificity, and positive predictive value 3 of 86.3% (44 of 51; 95% CI: 76.8, 95.7), 81.4% (1117 of 1372; 95% CI: 79.4, 83.5), and 24.7% (42 of 170; 95% CI: 18.2, 31.2), respectively, with a cancer detection rate of 30.9 (95% CI: 21.9, 39.9) cancers per 1000 examinations and an interval cancer rate of 4.9 (95% CI: 1.3, 8.6) cancers per 1000 examinations. AI-directed triage of 50% of the examinations to AB-MRI yielded a sensitivity, specificity, and positive predictive value 3 of 88.2% (45 of 51; 95% CI: 79.4, 97.1), 80.8% (1108 of 1372; 95% CI: 78.7, 82.8), and 23.6% (43 of 182; 95% CI: 17.5, 29.8), respectively, with a cancer detection rate of 31.6 (95% CI: 22.5, 40.7) cancers per 1000 examinations and an interval cancer rate of 4.2 (95% CI: 0.9, 7.6) cancers per 1000 examinations. See Table 2 for performance metrics for the full protocol and adaptive scanning with 25% and 50% triage thresholds.

Table 2:

Breast MRI Diagnostic Performance Metrics

Compared with conventional scanning, AI-directed scanning with 25% triage showed a maximum decrease in specificity of 2.0 percentage points and a maximum decrease in accuracy of 2.0 percentage points. Compared with conventional scanning, AI-directed scanning with 50% triage resulted in a maximum specificity decrease of 2.7 percentage points and a maximum accuracy decrease of 2.6 percentage points. See Figure 3B and 3C for full metrics across triage cutoffs.

AI-directed Triage of Cancer Examinations

With 25% triage (ie, triage of examinations with AI suspicion score < 0.51), 356 examinations were triaged to AB-MRI; among these triaged examinations, two (0.6%) were ultimately followed by cancer diagnoses and 354 (99.4%) were not. The two cancer examinations were both given suspicious assessments (ie, BI-RADS 4) by readers based on both AB-MRI and full MRI.

With 50% triage (ie, triage of examinations with AI suspicion score < 0.73), 711 examinations were triaged to AB-MRI, of which six (0.8%) were ultimately followed by cancer diagnoses and 705 (99.2%) were not. Among the six cancer examinations, four examinations were given suspicious assessments (ie, BI-RADS 4) by readers based on both AB-MRI and full MRI, one examination was given a benign assessment (ie, BI-RADS 2) based on both AB-MRI and full MRI, and one examination was given a suspicious assessment based on AB-MRI but a benign assessment based on full MRI. Figure 4 shows cancer examinations where although the AI suspicion score triggered triage to AB-MRI, readers still rendered a suspicious assessment for the AB-MRI scan. All cancers detected with the full MRI protocol were also detected with the AB-MRI protocol. Therefore, for no examination did triage to AB-MRI lead to a missed cancer.

Figure 4:

Example cancer examinations where the artificial intelligence (AI) suspicion score prompted triage to abbreviated breast MRI (AB-MRI), but readers still assessed the AB-MRI examination as suspicious. Axial (A) dynamic contrast-enhanced (DCE) and (B) fat-saturated T2-weighted MRI scans in a 58-year-old woman with a 0.3-cm enhancing focus (arrow) in the left breast. The AI tool generated a suspicion score of 0.68 (43rd percentile of suspicion score), prompting triage at the 50% threshold. For AB-MRI, readers gave the examination a suspicious assessment, but for the full MRI protocol, readers missed the suspicious finding, possibly owing to its T2 hyperintensity. The original radiologist also gave a benign assessment, but cancer was diagnosed 6 months later via a mammogram, with calcifications indicating ductal carcinoma in situ. Axial (C) DCE and (D) fat-saturated T2-weighted MRI scans in a 51-year-old woman with a 0.3-cm enhancing focus (arrow in C) in the left breast. The AI tool generated a suspicion score of 0.42 (17th percentile), prompting triage at the 50% threshold. For both AB-MRI and full MRI, readers gave the examination a suspicious assessment, although the original radiologist report gave a benign assessment. Cancer was diagnosed 6 months later via mammography. Biopsied calcifications indicated ductal carcinoma in situ. Axial (E) DCE and (F) fat-saturated T2-weighted MRI scans in a 49-year-old woman with a 0.4-cm enhancing focus (arrow) in the left breast. The AI tool generated a suspicion score of 0.62 (37th percentile), prompting triage at the 50% threshold. The lesion was missed by readers at both AB-MRI and full MRI. The patient was diagnosed with invasive ductal carcinoma 6 months after an abnormal contrast-enhanced mammogram.

AI-directed Triage of Noncancer Examinations

Reader BI-RADS assessments using full MRI versus AB-MRI at 25% and 50% triage thresholds are shown in Figure 5. Of the 354 noncancer examinations triaged to AB-MRI with 25% triage, 4.5% (16 of 354; 95% CI: 2.4, 6.7) were assigned an AB-MRI BI-RADS score (BI-RADS 4 or 5) that would lead to unnecessary biopsy (negative biopsy rate), whereas 4.2% (15 of 354; 95% CI: 2.1, 6.3) were assigned a full MRI BI-RADS score that would lead to unnecessary biopsy. At the 50% triage threshold, 6.0% (42 of 705; 95% CI: 4.2, 7.7) of the noncancer examinations triaged to AB-MRI were assigned an AB-MRI BI-RADS score that would lead to unnecessary biopsy, and 4.4% (31 of 705; 95% CI: 2.9, 5.9) were assigned a full MRI BI-RADS score that would lead to unnecessary biopsy.

Figure 5:

Heatmaps show the Breast Imaging Reporting and Data System (BI-RADS) category differences in reader assessments between the full MRI protocol (x-axis) and adaptive protocols (y-axis) with 50% (top) and 25% (bottom) triage to abbreviated breast MRI. Heatmaps show numbers of examinations.

Figure 6 shows examples of noncancer examinations where non-DCE sequences included in the full MRI protocol helped readers “downgrade” their BI-RADS assessment from that for the AB-MRI protocol.

Figure 6:

Example noncancer examinations for which readers assessed the examination as suspicious at abbreviated breast MRI (AB-MRI) (ie, access to only the dynamic contrast-enhanced [DCE] images) but correctly assessed the examination as benign at full MRI. Axial (A) DCE and (B) fat-saturated T2-weighted MRI scans in a 69-year-old woman with an enhancing focus (arrow) in the right breast. At AB-MRI (artificial intelligence [AI] suspicion score of 0.60; 35th percentile of suspicion score), the enhancing focus was deemed suspicious by readers. The fat-saturated T2-weighted image shows prominent T2 hyperintensity, which prompted a benign assessment for the full MRI examination. Axial (C) DCE and (D) non-fat-saturated T1-weighted MRI scans in a 70-year-old woman with a focal nonmass enhancement (arrow) in the right breast. At AB-MRI (AI suspicion score of 0.40; 16th percentile), the enhancement was deemed suspicious by readers. The T1-weighted image shows an associated susceptibility artifact from a biopsy marker (indicating the location of a prior benign biopsy), yielding a benign assessment with the full MRI data. Axial (E) DCE and (F) non-fat-saturated T1-weighted MRI scans in a 47-year-old patient with an enhancing mass (arrow) in the left breast. The mass was deemed suspicious at AB-MRI (AI suspicion score of 0.38; 14th percentile). The T1-weighted image shows that this is a fat lobule, yielding a benign assessment with the full MRI data. Notably, the original radiologist nevertheless assessed the lesion as suspicious, and it was biopsied under US guidance, yielding prominent adipose tissue with fat necrosis (benign).

Discussion

This reader simulation study demonstrated that image-based artificial intelligence (AI) triage has the potential to direct 25% or even 50% of examinations to abbreviated breast MRI (AB-MRI) while achieving similar diagnostic performance to conventional scanning with the full MRI protocol. In this study, there were no AI-triaged examinations for which adding MRI sequences (ie, the full MRI protocol) would have resulted in additional cancer detection. In fact, AI-directed triage of 50% of examinations to AB-MRI led to one additional cancer being detected in our study, although the small numbers here do not allow for a statistical comparison. AI-directed triage of 50% of examinations to AB-MRI yielded a maximum specificity decrease of only 2.7 percentage points.

A critical step in the development of any AI tool is preventing harm and anticipating any consequences of the tool failing. Here, our results suggest that even in examinations where the AI triage tool might fail, AB-MRI provides sufficient information to ensure that diagnostic performance is maintained. During the reader study, all triaged cancer examinations that were assessed as suspicious with the full MRI protocol were also assessed as suspicious with AB-MRI.

AI-directed MRI protocol selection is appropriate for screening breast MRI examinations because of the specialized contribution of the different MRI sequences. In examinations where cancer is detected, it is always identified on DCE images. In current clinical practice, non-DCE images are used only for further characterization of abnormalities already detected on DCE images, to improve specificity. Therefore, AI-directed MRI protocol selection can rely on first acquiring DCE sequences and then deciding on the necessity of the remaining sequences. However, acquiring DCE sequences first is not always standard practice. Fortunately, prior work has indicated that the diagnostic quality of non-DCE sequences can be maintained after contrast agent injection (22,23).

AI-directed MRI protocols have the potential to save a considerable amount of time. If the scan time is reduced from 15 minutes to less than 5 minutes for 50% of patients, the average scan time is decreased by at least 33%. Shorter scan times decrease medical costs, which could increase MRI access for more patients, improve patient satisfaction, and lead to better image quality since patients are less likely to move in the scanner during shorter examinations (24). The literature shows that AB-MRI examinations also reduce radiologist interpretation times (12,25). In this reader study, radiologists were blinded to the AI suspicion score, although recent prospective studies of AI tools for screening mammography suggest that biasing readers with AI scores might be diagnostically beneficial (16,17). However, this practice might also carry the risk of making radiologists more similar to, and therefore less synergistic with, the AI tool.

These results suggest the potential for a paradigm shift in MRI scanning. In current practice, MRI protocols are selected before the examination begins, leading to a fixed examination duration regardless of the patient’s needs. Here we introduce an AI-directed method for adaptive MRI scanning, whereby an AI tool analyzes the initial images in real time and determines, while the patient is still in the scanner, whether to proceed with the full examination or stop image acquisition early. This approach could be used to shorten the scan time in patients who do not need the images from extra sequences. Alternatively, this approach could be applied as a safety check at centers that already offer AB-MRI, thus averting situations where the radiologist must ask the patient to return for additional MRI sequences. It also represents an early step in a broader initiative to personalize MRI acquisition on the basis of the initial data acquired during the examination. This personalization could involve AI-directed selection of MRI sequence parameters (eg, echo time and voxel size) or even real-time optimization of the k-space trajectory. Ultimately, AI could enable dynamic medical tests that acquire only the necessary and sufficient data to answer a specific clinical question for a particular patient, thus avoiding the time, cost, and exposure of acquiring extraneous data.

This study has several limitations. First, the study was conducted at a single institution, although data were pulled from three geographically distinct imaging centers under our institutional umbrella, thus increasing the diversity of the dataset and the generalizability of the results. Second, AI-directed MRI triage was simulated for examinations with suspicion scores in the lower half of the score distribution to assess the effect of triaging only up to 50% of examinations to AB-MRI. Owing to the time constraints of our readers, analyzing the effects of triaging up to 95% of examinations to AB-MRI, which would match the expected number of examinations where noncontrast sequences might be extraneous, was not possible; this topic should be explored in future work. Third, this study simulated an AI-directed MRI triage paradigm, but it did not replicate the real-time decision-making nature of the process. We are working to integrate AI triage directly into the scanner platform, which would permit “real world” evaluation, including determining how long it would take to generate a real-time AI suspicion score. Fourth, this study included patients before and after menopause and with and without a personal history of cancer, as well as those with surgical histories and breast implants. This is reflective of a real-world screening cohort, and in future work, we will perform subgroup analysis across these populations.

This simulation study suggests that artificial intelligence (AI)–directed triage of breast MRI examinations to an abbreviated protocol has the potential to decrease scan time without compromising diagnostic performance. These findings herald a new era of AI-directed MRI scanning, where the acquisition plan is not static but changes in real time, as information is acquired and analyzed.