Per lesion analysis is misleading

We read with interest the randomised controlled trial by Häußinger and co‐workers¹ which compared autofluorescence bronchoscopy (AFB) plus white light bronchoscopy (WLB) with WLB alone for detecting precancerous lesions. The authors stratified their patients into four different risk groups before randomisation. They also excluded from analysis biopsy samples taken from or next to visible tumours. Their major findings suggested that WLB plus AFB was significantly superior to WLB alone for detecting precancerous lesions.

While we appreciate the clinical significance of their major findings, we found the per lesion analysis adopted in the paper misleading for evaluating the sensitivity, specificity, and predictive values of AFB plus WLB. They obtained biopsy tissue from all suspicious areas and at least two areas of non‐suspicious appearance in each subject.¹ Thus, each study subject contributed an arbitrary number of biopsy samples which might also be dependent on each other when they were taken from the same subject. Other investigators also adopted a similar approach in a loose manner.²,³,⁴ Apart from causing confusion, a per lesion analysis does not inform clinical decision. It may also partly explain the high variability of sensitivity and specificity in different studies.¹

Sensitivity, specificity, and predictive values are clinically relevant because they inform us how well a test will perform in certain clinical contexts. The preferred approach for ascertaining these parameters is therefore a per subject analysis in which each subject is labelled as either test positive or test negative and the test status is matched against the representative histological result of the subject's biopsy. Study subjects should also be representative of those encountered in a typical clinical scenario.

To illustrate the potential flaw in a per lesion analysis, let us vary the number of biopsy samples taken arbitrarily from non‐suspicious sites in both arms (WLB plus AFB arm versus WLB alone) of the quoted study¹ without changing negative predictive values and the number of biopsy samples from suspicious sites (table 1). When the number of non‐suspicious biopsy samples is doubled or tripled, the sensitivity, specificity and prevalence in each arm change accordingly. The sensitivity of WLB plus AFB relative to that of WLB alone also changes from 1.42 (95% CI 0.94 to 2.15) to 1.72 (95% CI 1.04 to 2.83) and 1.94 (95% CI 1.13 to 3.33), respectively. Likewise, the prevalence of preinvasive lesions detected by WLB plus AFB relative to that detected by WLB alone changes from 1.61 (95% CI 0.93 to 2.79) to 1.37 (95% CI 0.84 to 2.22) and 1.23 (95% CI 0.79 to 1.90), respectively. Thus, a per lesion approach could generate different sets of arbitrary values according to an arbitrary change in the number of biopsy samples taken from non‐suspicious areas.

Table 1 Effects of varying the number of samples from non‐suspicious areas in a per lesion analysis.

Diagnostic test	Biopsy results		Sensitivity (%)	Specificity (%)	Prevalence (%)
	Positive	Negative
WLB+AFB
Test positive	28*	623*
Test negative
Original	6*	874*	82.3*	58.4*	2.2*
2× Original	6×2	874×2	70.0	73.7	1.7
3× Original	6×3	874×3	60.9	80.8	1.4
WLB alone
Test positive	11*	514*
Test negative
Original	8*	843*	57.9*	62.1*	1.4*
2× Original	8×2	843×2	40.7	76.6	1.2
3× Original	8×3	843×3	31.4	83.1	1.1

WLB, white light bronchoscopy; AFB, autofluorescence bronchoscopy.

*Figures as reported in the study by Häußinger et al.[1]