Use of “Diagnostic Yield” in Imaging Research Reports: Results from Articles Published in Two General Radiology Journals

Abstract

Objective

“Diagnostic yield,” also referred to as the detection rate, is a parameter positioned between diagnostic accuracy and diagnosis-related patient outcomes in research studies that assess diagnostic tests. Unfamiliarity with the term may lead to incorrect usage and delivery of information. Herein, we evaluate the level of proper use of the term “diagnostic yield” and its related parameters in articles published in Radiology and Korean Journal of Radiology (KJR).

Materials and Methods

Potentially relevant articles published since 2012 in these journals were identified using MEDLINE and PubMed Central databases. The initial search yielded 239 articles. We evaluated whether the correct definition and study setting of “diagnostic yield” or “detection rate” were used and whether the articles also reported companion parameters for false-positive results. We calculated the proportion of articles that correctly used these parameters and evaluated whether the proportion increased with time (2012–2016 vs. 2017–2022).

Results

Among 39 eligible articles (19 from Radiology and 20 from KJR), 17 (43.6%; 11 from Radiology and 6 from KJR) correctly defined “diagnostic yield” or “detection rate.” The remaining 22 articles used “diagnostic yield” or “detection rate” with incorrect meanings such as “diagnostic performance” or “sensitivity.” The proportion of correctly used diagnostic terms was higher in the studies published in Radiology than in those published in KJR (57.9% vs. 30.0%). The proportion improved with time in Radiology (33.3% vs. 80.0%), whereas no improvement was observed in KJR over time (33.3% vs. 27.3%). The proportion of studies reporting companion parameters was similar between journals (72.7% vs. 66.7%), and no considerable improvement was observed over time.

Conclusion

Overall, a minority of articles accurately used “diagnostic yield” or “detection rate.” Incorrect usage of the terms was more frequent without improvement over time in KJR than in Radiology. Therefore, improvements are required in the use and reporting of these parameters.

INTRODUCTION

A majority of studies in radiology focus on diagnostic accuracy (sensitivity and specificity) for evaluating the clinical effectiveness of diagnostic tests [1]. This is based on the assumption that improving diagnostic accuracy inevitably results in improvements in health outcomes. However, this assumption is not always true as diagnostic accuracy is only one of several factors that affect the clinical effectiveness of a diagnostic test, and the potential benefit of high accuracy may be nullified by other clinical factors [2]. Therefore, the best way to determine the clinical usefulness of a diagnostic test is to evaluate whether patients who undergo a test have better clinical outcomes than those who do not [2,3]. A randomized trial is an ideal study design for this purpose; however, conducting clinical trials for a diagnostic test is complex [4].

Studies on “diagnostic yield” can bridge the gap between diagnostic accuracy and clinical outcome studies [5]. “Diagnostic yield” may also be referred to as “detection rate” and is defined as the number of disease-positive patients detected by a diagnostic test divided by the total cohort size. For example, Kim et al. [6] demonstrated a comparable diagnostic yield of computed tomography (CT) colonography and optical colonoscopy for screening advanced neoplasia. In this study, the diagnostic yield was defined as the number of patients in whom the target lesions were detected using CT colonography and who were subsequently proven to be true-positives through the use of reference standards divided by the total number of patients undergoing CT colonography. In another example, Tu et al. [7] reported a low diagnostic yield of cranial CT for patients presenting with psychiatric symptoms and questioned the routine use of imaging in this cohort. “Diagnostic yield” is a parameter that is positioned between diagnostic accuracy and diagnosis-related patient outcomes in studies of diagnostic tests [3,8]. Diagnostic yield studies have focused on the effects of test results on clinical decisions [2,3,9]. These studies target diagnostic cohorts with a particular indication for a test and evaluate how often the test result is abnormal, or which group of patients receive the most or least benefit from the test. Additionally, the parameter can be used in studies in which the true disease condition status is only available for test positives, as in screening tests (Fig. 1). A high diagnostic yield of a test does not guarantee its clinical usefulness because there could be a large number of false-positive cases. Therefore, parameters indicating the magnitude of false-positive results should be reported in addition to diagnostic yield. A well-known example of this parameter is the false referral rate, defined as the number of patients with false-positives created by a test divided by the total cohort size.

Fig. 1. Schematic diagram of a study setting in which diagnostic yield (or detection rate) and false referral rate are used and the contingency table reconstructed from the study setting.

As illustrated in the figure, diagnostic yield and false referral rate can be obtained even if reference standard information is not available for test-negative patients, which often occurs in screening test research. FN = false-negative, FP = false-positive, TN = true-negative, TP = true-positive

Although diagnostic yield is a well-established parameter as a diagnostic research endpoint, relative unfamiliarity with the term when compared to sensitivity or specificity may lead to incorrect usage and incorrect delivery of information. For example, recently published articles in the journals Radiology and Korean Journal of Radiology (KJR) used “diagnostic yield” interchangeably with “diagnostic performance” in studies in which sensitivity and specificity were available [10,11]. In this regard, we investigated how properly diagnostic yield and related parameters have been used in articles published in Radiology and KJR during the last 10 years.

MATERIALS AND METHODS

Search Strategy

We searched the MEDLINE and PubMed Central databases for potentially relevant articles that reported specific diagnostic terms (i.e., diagnostic yield or detection rate) published in the two peer-reviewed journals, Radiology and KJR. The search terms were (“Diagnostic yield” OR “detection rate” OR “true positive rate”) AND (“Radiology”[Journal]) in MEDLINE, and (“Diagnostic yield” OR “detection rate” OR “true positive rate”) AND (“Korean Journal of Radiology”[Journal]) in PubMed Central. The “true-positive rate” was added to the search terms for a more sensitive literature search. The search was limited to articles published since 2012. The search date was May 15th 2022, and 239 records were identified (110 from Radiology; 129 from KJR).

Study Eligibility Criteria and Selection Process

Articles were included if they met the following criteria: 1) used either “diagnostic yield” or “detection rate” and 2) explicitly documented index tests and reference standards. The exclusion criteria were as follows: 1) articles on breast cancer, 2) articles that were not screening or diagnostic imaging studies (e.g., technical consideration or biopsy study), 3) articles with unavailable index tests or reference standards, 4) articles based on animal or phantom models, 5) reviews, editorials, letters to the editor, or special reports. Breast cancer studies were excluded because “Cancer Detection Rate (number of true-positives per 1000 screened)” is a well-established term in the field of breast radiology. Moreover, breast cancer articles comprised nearly 30% (73/239) of all studies; this may introduce biases in evaluating the general quality of published studies assessing the usage of the terms. Additionally, articles that correctly reported the “true-positive rate” with a meaning of sensitivity were further excluded. Two reviewers independently evaluated the eligibility of 239 articles. Disagreements were resolved by consensus and discussion with a third reviewer.

Data Extraction

For each article, the parameters used for true-positives (diagnostic yield or detection rate) and false-positives (false referral rate or any other term describing false-positive results) were extracted. We evaluated whether these terms were used in appropriate study settings. Additionally, the following data were extracted: name of the first author, year of publication, imaging purpose (screening vs. diagnostic study), imaging modality, imaging target, study design (prospective or retrospective cohort study, case-control study, or clinical trial), whether a comparison with other diagnostic tests was performed, and whether a subgroup analysis was conducted. Data extraction was independently performed by two reviewers.

Data Analysis

The primary outcome was the proportion of articles correctly reporting “diagnostic yield” or “detection rate” in appropriate study settings. The secondary outcome was the proportion of articles reporting companion parameters to describe the magnitude of the false-positive results. These parameters encompass various terms including “false referral rate,” “false-positive rate,” “false-positive finding,” and “false-positive case.” The incorrect use of the terms in the articles was reviewed for their intended meaning, and the study settings in which the terms were used were determined. The proportion of correct uses of diagnostic terms was compared between the two journals. Additionally, we evaluated whether there were differences in the proportions according to publication date (2012–2016 vs. 2017–2022). As this study intended to obtain descriptive statistics, formal statistical comparisons were not performed.

RESULTS

Characteristics of the Included Articles with Correct Reporting of the Terms

Our search terms initially yielded 239 records (110 from Radiology and 129 from KJR) from the MEDLINE and PubMed Central databases. After screening 239 records, 168 were excluded and 71 were thoroughly reviewed. Further 32 articles were then excluded [12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43]. Finally, 39 articles (19 from Radiology and 20 from KJR) were included in the analysis [10,11,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80]. Figure 2 illustrates the study-selection process. The detailed procedure is available in Supplement.

Fig. 2. Flowchart showing the article selection process.

Of the 39 included articles, 17 (43.6%; 11 from Radiology and 6 from KJR) articles reported “diagnostic yield” or “detection rate” in appropriate clinical settings [45,47,48,49,50,53,63,64,65,66,67,68,69,71,72,73,77]. The detailed characteristics of the 17 articles are shown in Supplementary Table 1. Table 1 and Figure 3 summarize the characteristics of these articles. Briefly, 64.7% (11 of 17) of the articles adopted “diagnostic yield” [45,47,48,49,53,64,65,67,68,72,73], and 35.3% (6 of 17) adopted “detection rate” [50,63,66,69,71,77]. Companion parameters for describing false-positive results were reported in 12 articles (70.6%, or 12 of 17) [47,49,50,53,64,65,66,67,68,71,73,77]. Fifteen articles were diagnostic cohort studies [81] and two were clinical trials with cohort sizes ranging between 86 and 29138 patients (median: 524). Other characteristics (including imaging purpose, imaging modality, imaging target, and whether a subgroup analysis was performed) are provided in Supplement.

Table 1. Summary of Studies that Used the Diagnostic Terms Correctly.

Journals		Number of Articles (%)
		Radiology	KJR
All articles		11	6
Parameter used for TP
	Diagnostic yield	6 (54.5)	5 (83.3)
	Detection rate	5 (45.5)	1 (16.7)
Companion parameter used for FP*
	Reported	8 (72.7)	4 (66.7)
	Not reported	3 (27.3)	2 (33.3)
Comparison between diagnostic modalities†
	Performed	5 (45.5)	2 (33.3)
	Not performed	6 (54.5)	4 (66.7)
Subgroup analysis‡
	Performed	4 (36.4)	2 (33.3)
	Not performed	7 (63.6)	4 (66.7)

^*This encompasses various terms, such as false referral rate, FP rate, FP cases, and FP findings, ^†Diagnostic yields or detection rates of index tests were compared with other imaging modalities. Among them, two studies also reported the added values of index tests [64,66], ^‡Two studies used logistic regression analyses to identify factors affecting diagnostic yields. Three other studies performed subgroup analyses to calculate diagnostic yields according to patient clinical parameters or tumor cell type. The remaining study involved a reader performance test from a subset of the included patients. FP = false-positive, TP = true-positive

Fig. 3. Summary charts of the included articles according to (A) study setting, (B) imaging purpose, (C) imaging target, and (D) imaging modality.

“Others” in (B) include screening of Crohn’s disease recurrence, detection of the pyramidal lobe, and detection of the epileptogenic focus.

Characteristics of the Included Studies Using the Terms Incorrectly

Of the 39 articles, 22 incorrectly used “diagnostic yield” or “detection rate.” “Diagnostic yield” was used incorrectly as “diagnostic performance” in two studies [10,11], while “detection rate” was incorrectly used as “sensitivity” in 20 [44,46,51,52,54,55,56,57,58,59,60,61,62,70,74,75,76,78,79,80]. All of the articles were diagnostic accuracy studies. The study settings were different from those in the 17 articles that used correct diagnostic terms and were classified into four categories (Fig. 4). A detailed description of these study settings is provided in Supplement.

Fig. 4. Study methods of diagnostic accuracy articles in which “diagnostic yield” and “detection rate” were misused (n = 22 studies).

“Diagnostic yield” was used as diagnostic performance or sensitivity, and “detection rate,” as sensitivity in all these studies. A. Only disease-positive cohorts were recruited; thus, only sensitivity could be calculated (n = 6). B. A specific imaging modality was used as a reference standard, and the performance of index imaging study was evaluated (in a mainly per-lesion analysis) (n = 9). C. Classic diagnostic cohort study in which all individuals with positive and negative test results underwent a gold-standard confirmatory test (n = 5). D. Case-control study (n = 2). FN = false-negative, FP = false-positive, TN = true-negative, TP = true-positive

Examples of Articles

Hwang et al. [64] and Kim et al. [65] studies were good examples of using “diagnostic yield” and “false referral rate”. Additionally, Hwang et al. [64] compared the diagnostic yield of computer-aided detection (CAD)-assisted chest radiography for detecting lung metastasis with that of conventional chest radiography in a cohort of 1521 outpatients. Diagnostic yield was calculated as the “number of radiographs with true-positive results/total number of radiographs,” and false referral rate was calculated as the “number of radiographs with false-positive results/total number of radiographs.” They demonstrated an improved diagnostic yield of CAD-assisted interpretation without increasing the false referral rate, thereby demonstrating the added value of CAD during patient care. Kim et al. [65] investigated the diagnostic yield of staging brain magnetic resonance imaging (MRI) in 1712 patients with lung cancer. The study calculated the diagnostic yield of brain MRI according to clinical cancer stage and demonstrated a low diagnostic yield in clinical stage Ia.

Proportion of Studies Using Correct Terms and Usage Trends according to Publication Date

Table 2 summarizes the proportion of studies that correctly used these terms. The proportion was higher in studies published in Radiology than in those published in KJR (57.9% vs. 30.0%). The proportion of studies reporting companion parameters was similar between the journals (72.7% vs. 66.7%). The false referral rate was used in only four articles (23.5%, 4 of 17) [64,65,66,68]. Improvements in using the terms correctly were observed with time in the studies published in Radiology (33.3% [3 of 9 articles from 2012–2016] vs. 80.0% [8 of 10 from 2017–2022]). However, there was no improvement in the studies published in KJR (33.3% [3 of 9] vs. 27.3% [3 of 11]). There was no considerable improvement in reporting companion parameters for either Radiology (66.7% [2 of 3] vs. 75.0% [6 of 8]) or KJR (100.0% [3 of 3] vs. 33.3% [1 of 3]).

Table 2. Proportion of Studies that Used the Diagnostic Terms Correctly.

	Proportions of the Studies
	Radiology	KJR
Studies that used the correct definition of “diagnostic yield” or “detection rate”, %	57.9 (11/19)	30.0 (6/20)
Studies that reported companion parameter, %*	72.7 (8/11)	66.7 (4/6)

Values denote proportion. Data in parentheses indicate the number of studies that satisfied a specific condition/total number of studies. ^*Proportions of studies reporting “false-positive” were calculated from the articles that used the correct definition of “diagnostic yield” or “detection rate.”

DISCUSSION

In this study, we demonstrated that only a small proportion of studies used the correct definition of “diagnostic yield” or “detection rate” in the appropriate study settings. Additionally, the companion parameters for false-positive results were suboptimal. Although the correct use of diagnostic terms improved with time in Radiology, no improvement was observed in KJR.

“Diagnostic yield” is a parameter that is positioned between diagnostic accuracy and diagnosis-related patient outcomes. Despite the well-known concept of “diagnostic yield” in the medical field, the term is infrequently used in radiology journals. This is mainly because the main focus of radiologists is on diagnostic accuracy (sensitivity and specificity). In this study, we reestablished the definitions of diagnostic yield and false referral rates as follows:

Diagnostic yield (detection rate): number of true-positives/total study cohort

False referral rate: number of false-positives/total study cohort

As the definition implies, “diagnostic yield” studies evaluate the frequency with which an index test can detect a target disease in a diagnostic cohort with a particular test indication. These studies focused on the effect of test results on clinical decisions, such as a change in the treatment plan. “Diagnostic yield” conveys a more clinically meaningful concept rather than simply presenting the performance of a diagnostic test represented by “sensitivity” and “specificity.” For example, if screening test A showed a higher diagnostic yield than screening test B, one can interpret that the screening test may further improve patient outcomes by detecting more diseases in a given study cohort. Hwang et al. [64] provided a good example where an additional benefit of the CAD system was shown in interpreting pulmonary metastasis surveillance by demonstrating an increase in diagnostic yield compared to visual interpretation alone (0.86% vs. 0.32%). The other article also focused on the added value of the index test to that of the conventional screening test [66].

Another characteristic of “diagnostic yield” is that it is used in studies where the true disease condition status is only available for test positives, such as a screening test study. For example, two screening articles published in JAMA differentiated the diagnostic yield from diagnostic accuracy [82,83]. One of these articles mentioned that the “diagnostic yield” for pancreatic cancer surveillance was pooled instead of “diagnostic accuracy,” as sensitivity and specificity could not be determined [82]. This is because individuals screened negative did not undergo a confirmatory test. Moreover, “diagnostic yield” is only calculable in a diagnostic cohort study, and a large number of patients are often required. In our study, 17 articles using the correct definition were either diagnostic cohort studies or clinical trials, with a median cohort size of 524 patients. Finally, since the clinical impact of “diagnostic yield” goes beyond “diagnostic accuracy,” diagnostic accuracy of an index test should be sufficiently evaluated beforehand to perform “diagnostic yield” studies. Therefore, the diagnostic accuracy of the index tests used in our study has been well-established in multiple prior articles.

Among the various terms for describing the magnitude of false-positive results, we prefer using “false referral rate” as a companion parameter to “diagnostic yield.” Reporting “false referral rates” is crucial because patients with false-positive test results undergo unnecessary confirmatory tests that are often invasive. Thus, enhancing “diagnostic yield” while keeping the “false referral rate” low is a prerequisite for a good imaging test. In a study by Hwang et al. [64], CAD-assisted chest radiography improved the diagnostic yield for detecting lung metastasis without increasing the false referral rate compared to that of conventional radiography. There is no universal rule for interpreting the level of false referral rate as high or low. This should be set individually, considering the diagnostic yield of an index test, disease prevalence, accessibility of confirmatory tests, etc. The best way to evaluate whether the diagnostic yield and false referral rate are clinically acceptable is to perform additional cost–benefit analyses. However, this is often not feasible in a single study, and none of the included articles conducted a cost–benefit analysis. A less rigorous method is to compare the diagnostic yield and false referral rate of an index test with those of conventional imaging, as performed in seven of the included studies [50,53,63,64,66,67,71].

When compared with Radiology, studies published in KJR misused diagnostic terms more frequently (57.9% vs. 30.0%). Moreover, unlike in Radiology, there was no improvement in the correct use of diagnostic terms over time (2012–2016, 33.3% vs. 2017–2022, 27.3%). Our results suggest the need for further quality improvements in diagnostic yield studies to be published in the KJR.

Nearly 60% (22 of 39) of the included articles incorrectly defined “diagnostic yield” or “detection rate.” In all 22 studies, “diagnostic yield” was used incorrectly as “diagnostic performance,” and “detection rate” was used as “sensitivity.” All of the articles that did so were diagnostic accuracy studies. Among them, six only recruited patients with true disease conditions and evaluated the lesion detection performance of an index test, which was described as the “detection rate” (Fig. 4). However, technically speaking, this outcome should be stated as “sensitivity.” In this case, we suggest using the phrase “sensitivity for detection” rather than “detection rate” to reduce confusion.

Our study has a few limitations. First, we reviewed articles from only two journals, Radiology, and the KJR, which may raise concerns regarding the generalizability of our results. However, they were chosen for being the most frequently cited radiology journals that provide wide coverage of imaging topics that can help improve human health. Therefore, both journals can serve as representative publications. Second, none of the included studies contained a cost-benefit analysis, and only seven included a comparison with conventional imaging. Therefore, the evaluation of the clinical usefulness of an index test based on diagnostic yield and false referral rate has been arbitrary.

In conclusion, a minority of articles correctly used the terms, “diagnostic yield” and “detection rate.” Incorrect use of the terms was more frequent in KJR without improvement over time than was in Radiology. Additionally, the companion parameters for false-positive results were suboptimal. Therefore, improvements are required in the use and reporting of these parameters.

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Supplement

The Supplement is available with this article at https://doi.org/10.3348/kjr.2022.0741.