The accuracy of colposcopic grading for detection of high grade cervical intraepithelial neoplasia

Abstract

Objective

To relate aspects of online colposcopic image assessment to the diagnosis of grades 2 and 3 cervical intraepithelial neoplasia (CIN2+).

Methods

To simulate colposcopic assessment, digitized cervical images were obtained at enrollment after acetic acid application from 919 women referred for equivocal or minor cytologic abnormalities into the ASCUS-LSIL Triage Study. For each, two randomly assigned evaluators from a pool of 20 colposcopists assessed images using a standardized tool on-line. We calculated the accuracy of these assessments for predicting histologic CIN2+ over the two years of study. For validation, a subset of on-line results was compared to same-day enrollment colposcopic assessments.

Results

Identifying any acetowhite lesion in images yielded high sensitivity: 93% of women with CIN2+ had at least one acetowhite lesion. However, 74% of women without CIN2+ also had acetowhitening, regardless of human papillomavirus (HPV) status. The sensitivity for CIN2+ of an on-line colpophotographic assessment of high grade disease was 39%. The sensitivity for CIN2+ of a high grade diagnosis by Reid Index scoring was 30%, and individual Reid Index component scores had similar levels of sensitivity and specificity. The performance of on-line assessment was not meaningfully different from that of same-day enrollment colposcopy, suggesting that these approaches have similar utility.

Conclusion

Finding acetowhite lesions identifies women with CIN2+, but using subtler colposcopic characteristics to grade lesions is insensitive. All acetowhite lesions should be assessed with biopsy to maximize sensitivity of colposcopic diagnosis with good specificity.

Introduction

Cervical cancer prevention in the developed world currently relies on cytology screening and treatment of high grade cervical intraepithelial neoplasia (CIN2 or 3, CIN2+), a cancer precursor. However, cytologic screening fails to identify women with cancer precursors with sufficient specificity to justify treatment of all cytologically abnormal women. The problem with specificity is especially acute when screening is supplemented by testing for the human papillomavirus (HPV), the etiologic agent of cervical cancer, to improve sensitivity. Current approaches to cervical cancer prevention interpose colposcopy as a triage test to define better which women need treatment. Optimal use of colposcopy requires that it improve specificity by identifying women with CIN2+ without substantial loss of sensitivity.

The accuracy of colposcopy has been increasingly questioned. Studies of loop excision after colposcopy have identified women with CIN2+ and cancer missed colposcopically (1). Biopsy of colposcopically normal areas may reveal unsuspected CIN2+ (2). Colposcopic lesion grade may predict histology poorly (3, 4). Women with negative colposcopy remain at substantial risk for subsequent detection of CIN2+, suggesting that lesions were missed (5). In the Atypical Squamous Cells of Undetermined Significance/Low Grade Squamous Intraepithelial Lesion Triage Study (ALTS), only 53% of women found to have CIN3 over two years of follow-up were identified at intake colposcopy, though most missed lesions were small and presumably early in their natural history and so at low risk of imminent progression to invasive cancer (6). Our group has shown recently that interobserver agreement among experienced colposcopists is moderate to poor for critical components of colposcopic assessment, including lesion grade, lesion characteristics, and even the presence of a lesion (7, 8). If colposcopy is inaccurate, then prevention algorithms may need to change.

We set out to estimate the accuracy of colposcopy in the identification of CIN2+ using on-line assessment by experienced colposcopists of digitized cervical images obtained from women enrolled in ALTS. In addition to assessing the accuracy of colposcopy overall among women referred for atypical squamous cells of undetermined significance (ASCUS) or low grade squamous intraepithelial lesion (LSIL), we attempted to estimate the utility for CIN2+ detection of components of the commonly used Reid Index system for colposcopic grading (9). Finally, we compared the accuracy of assessments of digitized images to results of actual colposcopy performed on the same day to ensure that conclusions based on image assessment were valid.

Methods

Details of ALTS have been previously described (10). Briefly, 5,060 women were enrolled between November 1996 and December 1998 at four clinical settings: Magee-Women's Hospital of the University of Pittsburgh Medical Center Health System (Pittsburgh, PA), the University of Oklahoma (Oklahoma City, OK), the University of Alabama (Birmingham, AL), and the University of Washington (Seattle, WA). Eligibility required a cytological report of ASCUS or LSIL. The study involved randomization to management using immediate colposcopy, triage using HPV DNA testing, and serial cytology and was approved by local institutional review boards and in accordance with the U.S. Department of Health and Human Services standards. Each woman signed a written informed consent at enrollment before randomization. Samples were obtained for liquid-based cytology and for HPV testing. Questionnaire data were collected. Once all samples were collected, the cervix was washed with 5% acetic acid for one minute and two Cervigrams® (National Testing Laboratories Worldwide, Fenton, MO) were taken. These images, when digitized, were the basis of the image assessments in this study. For women randomized to the Immediate Colposcopy arm of ALTS, the enrollment examination was followed by a conventional colposcopy examination. Colposcopic impressions were recorded after real-time colposcopic assessment, and biopsies were obtained when indicated. Women in ALTS were followed with semi-annual visits for two years.

Women were tested for carcinogenic HPV using the Hybrid Capture 2 assay (Qiagen, Gaithersburg, MD). Women also were tested for at least 27 HPV genotypes using a line blot assay (Roche Molecular Systems, Alameda CA) (11). HPV genotypes 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68 were considered carcinogenic.

The histologic outcome was defined as the highest grade lesion identified at initial colposcopy or during subsequent follow up. Grading was adjudicated by a pathology quality control group as previously described (6). Women with less than CIN2 were subcategorized according to the HPV type found at entry as having HPV 16, other carcinogenic types, only noncarcinogenic types, or no HPV.

Image selection has also been described (7). We selected 1,000 enrollment cervical images based on initial Cervigram diagnosis, including all women with images rated as high grade or cancer (n = 117), a 10% random sample of all negatives (n = 278), and a random sample (about one-fourth) of all images read as atypical or low grade (n = 605) to achieve a total of 1000 images. This sample size was chosen as the practical limit for the collaborating colposcopist reviewers. After excluding 21 lost and 40 poor quality images, the final sample included 939 images. Demographic characteristics of these women also have been published (7). Twenty images were evaluated by all evaluators. The remaining 919 images were randomly assigned so that each had 2 evaluators. Each evaluator reviewed 112 images, and each shared the same number of images with each one of the peer colposcopists. The images were assigned in a stratified randomization sequence to ensure that each evaluator viewed comparable images. These included 668 (73%) cases of CIN1 or negative histology (<CIN2), 83 (9%) with CIN2, and 168 (18%) with CIN3+. We did not distinguish between CIN1 and negative histology because of poor reproducibility in doing so (12), and the lack of importance of this distinction in ALTS in predicting risk of CIN3 in ALTS follow-up (13). After excluding assessments not done because evaluators considered images uninterpretable, 1789 interpretations remained. For an unbiased analysis comparing the performance of on-line image assessment to that of live colposcopic impression performed the same day, we used 299 images from the Immediate Colposcopy arm of the study, yielding 582 evaluations after excluding those few images in this subgroup considered uninterpretable by some on-line evaluators.

After digitization and compression (14), images were evaluated on-line using software developed by the National Institutes of Health (15). The evaluators were 20 colposcopists (12 general gynecologists and 8 gynecologist-oncologists) with at least 10 years of experience in colposcopy and prior research in cervical cancer prevention. Evaluators were not provided clinical data and were unaware of others' responses.

During evaluation, performed between January and April 2006, evaluators first determined whether images were evaluable. For evaluable images, evaluators determined whether acetowhite lesions were present and whether lesions were completely evaluable. Each lesion was then scored for color, vascularity, and margins according common definitions similar to those described by Reid and Scalzi (3, 9) that were provided by the prompting software. Finally, each lesion was assessed for the presence of atypical blood vessels. Summed scores were created from the colposcopic components. To correspond with conventional scoring for modified Reid Index (3), scores for color and margin were added to the higher score for the two vascular components (mosaicism or punctation). When multiple lesions were present, the highest score for each variable was recorded, which tended to increase the total score compared with the alternative of taking the score of the worst lesion. Using this alternative analytic approach, with slightly lower total scores, did not alter the conclusions. When an evaluator considered an acetowhite lesion to be absent, all grading scores were considered to be zero. Images with missing evaluations were excluded from final analysis.

All proportions are estimated by logistic regression or multinomial regression with an identity link within SAS 9.1.3 (SAS Institute Inc., Cary, NC). All statistical tests and 95% confidence intervals account for the fact that the same image was evaluated multiple times by different evaluators by fitting each model using Generalized Estimating Equations (GEE) with an independence working correlation matrix to produce robust empirical standard errors (16).

To assess whether live colposcopic impression and on-line assessment of static Cervigrams yield equivalent information about final histology, we fitted an ordinal logistic regression model (17) comparing the predictive value of these two techniques for worsening state of final histology. To minimize confusion, “colposcopy” and “colposcopic impression” referred in this model to real-time, dynamic colposcopic assessment of a live patient, while “image assessment” or “on-line assessment” refers to assessment of static colpophotographs over the internet. Covariates included colposcopic impression, on-line assessment, and age. The model yielded odds ratios (ORs) associating each technique with worsening grade of final histology ranked as follows: <CIN2 and negative for HPV, <CIN2 and positive only for noncarcinogenic HPV, <CIN2 and positive for carcinogenic HPV types other than HPV 16, <CIN2 and positive for HPV 16, CIN2 or CIN3+. We conducted a 3 degree of freedom test for equality of the two ORs (for colposcopic impression and for on-line assessment) for equivalent grade of assessment, jointly across all 3 possible non-normal assessments. For colposcopy, these 3 were atypical, low grade, or high grade, while for on-line assessment they were metaplasia, low grade, or high grade.

We conducted ancillary analyses regarding the sensitivity of acetowhitening, on-line global assessment of lesion severity, and Reid Index. We restricted the disease definition to the most stringent one available in ALTS: cases of histologically-confirmed CIN3+ diagnosed during the 2 years of the study for which the apparently causative HPV type was already present at enrollment. This group is most likely to represent prevalent CIN3+, even if not identified by colposcopy at enrollment.

Results

Characteristics of women who contributed images to the study are listed in Table 1. The median age was 24 years (range 18-73 years). We first compared the accuracy of the identification in the on-line image of an acetowhite lesion for the diagnosis of CIN2+ during up to two years of follow-up (Table 2). Acetowhite lesions were identified in 1421 (79%, 95% C.I. 74%, 85%) reviewed images. Using the identification of any acetowhite lesion as a diagnostic cutpoint had a sensitivity for disease diagnosed over the subsequent two years that was 94% (95% C.I. 90%, 98%) for CIN3+ and 93% (88%, 97%) for CIN2+. The corresponding specificity (finding no acetowhitening when no CIN2+ is present) was low: 67% (95% C.I. 60%, 75%) of HPV negative women without CIN2+ had acetowhitening, as did 74% (95% C.I. 68%, 81%) of those without CIN2+ regardless of HPV and 75% (95% C.I. 70%, 82%) of women in whom CIN3+ was not found, for a specificity for less than CIN3 of only 24% (95% C.I. 18%, 30%). Although the stratified sampling of images from the total ALTS population prevented direct and simple calculation of predictive values, the great majority of women with acetowhite lesions did not have CIN2+.

Table 1.

Characteristics of 919 women contributing images.

	N	%*
Age (years)
<20	132	14
20-29	577	63
30-39	142	15
>39	68	7
Parity
0	422	46
1-2	398	43
>2	98	11
Unknown	1
Ethnicity
White/Hispanic	24	3
White/Non-Hispanic	579	63
African-American	256	28
Asian/Pacific Islander	36	4
Other/Unknown	24	2
Referral Pap
ASCUS†	564	61
LSIL‡	355	39
Study site
Alabama	268	29
Oklahoma	190	21
Pennsylvania	163	18
Washington	298	32
Histology
<CIN2§/HPV‖ negative	145	16
<CIN2/nononcogenic HPV positive	108	12
<CIN2/oncogenic HPV positive	313	34
<CIN2/HPV 16 positive	102	11
CIN2	83	9
CIN3	165	18
Cancer	3	<1

^*Some percentage columns do not add to 100% because of rounding.

^†Atypical squamous cells of undetermined significance

^‡Low grade squamous intraepithelial lesion

^§Cervical intraepithelial neoplasia

^‖Human papillomavirus. Categories of HPV are exclusive, such that women with nononcogenic HPV did not have oncogenic HPV and women with oncogenic HPV did not have the most oncogenic genotype, HPV 16.

Table 2.

On-line identification of an acetowhite lesion and the development of cervical intraepithelial neoplasia (CIN) within two years of study enrollment. Results given as N (%).

	<CIN2/HPV-1	<CIN2/HPV+	CIN2	CIN3	Total
No lesion	145 (33)	186 (22)	16 (10)	21 (6)	368
Lesion seen	298 (67)	664 (78)	149 (90)	310 (94)	1421
Total	443 (100)	850 (100)	165 (100)	331 (100)	1789

¹Detection of carcinogenic human papillomavirus at enrollment.

We next calculated chi-square statistics for all comparisons between disease outcome and on-line colposcopic features, including the presence of an acetowhite lesion, overall assessment, color, margin, punctation, mosaic, and derived Reid Index score. All were highly significant (P < 0.0001), suggesting that these factors reflect underlying biologic differences. However, correlation may not mean clinical utility if the variation is substantial. To estimate the utility of these factors, we next calculated the accuracy of global on-line colpophotographic assessment in the diagnosis of subsequent CIN3+ and CIN2+ (Table 3). The sensitivity of a global on-line assessment using a diagnostic threshold of high grade disease was only 43% for CIN3+ and 39% for CIN2+. For each component of the Reid Index system, including color, margin, punctation, and mosaic, the sensitivity of a high Reid component score for CIN3+ and for CIN2+ ranged between 9-24%. The specificity of a low Reid Index score for each component ranged between 39-69% for CIN3+ and between 41-70% for CIN2+. Results for summed Reid Index scores were similar (Table 4). The sensitivity of a high grade result by summed Reid Index score was 32% (95% C.I. 25%, 39%) for CIN3+ and 30% (95% C.I. 23%, 37%) for CIN2+. The specificity of a negative or low grade (0-3) Reid Index score was 87% (95% C.I. 84%, 90%) for all women without CIN2+.

Table 3.

On-line global assessment of Cervigrams assessed on line and the development of cervical intraepithelial neoplasia (CIN) within two years of study enrollment. (P < 0.0001) Results given as N (%). Abbreviations as in Table 1.

			Histology
		<CIN2/HPV	>CIN2/HPV	CIN2	CIN3	Total
On-line	Negative	136 (31)	161 (19)	13 (8)	11 (3)	321
Assessment	Metaplasia	118 (27)	191 (22)	39 (24)	45 (14)	393
	Low grade	140 (32)	347 (41)	63 (38)	132 (40)	682
	High grade	49 (11)	151 (18)	50 (30)	143 (43)	393
	Total	443 (100)	850 (100)	165 (100)	331 (100)	1789

Table 4.

Correlations between an on-line assessment of modified Reid Index score of negative or low grade disease (scores 0-3) and high grade disease (scores 4-6) against the development of cervical intraepithelial neoplasia (CIN) within two years of study enrollment. P < 0.0001. Results given as N (%), with results for all HPV types aggregated for brevity. Abbreviations as in Table 1.

		<CIN2/HPV-	<CIN2/HPV+	CIN2	CIN3	Total
Reid Index	0-3	399 (90)	730 (86)	124 (75)	225 (68)	1478
Score	4-6	44 (10)	120 (14)	41 (25)	106 (32)	311
	Total	443 (100)	850 (100)	165 (100)	331 (100)	1789

Assessment of the accuracy of colposcopy using static images has been criticized because real-time colposcopy allows skilled examiners to assess dynamic acetowhitening and fading as well as to manipulate the cervix (18). In order to assess the possible impact that static image assessment had on colposcopic diagnosis, we compared overall assessment derived on-line by our study group with impressions recorded at the time of intake colposcopy in the Immediate Colposcopy arm of ALTS. There were 582 reviews of images from 299 women with both a same-day colposcopic impression and a cervical image evaluated on-line. In this subgroup, intake Pap results were ASCUS in 167 (56%) and LSIL in 132 (44%). Diagnostic outcomes after two years of follow-up included CIN3+ in 54 women (18%), CIN2 in 37 (12%), <CIN2 with HPV16 in 33 (11%), <CIN2 with other oncogenic HPV in 96 (32%), <CIN2 with nononcogenic HPV in 27 (9%), and <CIN2 without HPV in 52 (17%). On-line evaluations and the corresponding same-day colposcopic impressions are presented in Table 5. Differences in the distribution of diagnoses between the two modalities were significant (P < 0.0001) because many colposcopic impressions were low-grade rather than atypical while metaplasia was more commonly diagnosed on line. More importantly, however, Table 5 shows that the proportions of the total that were interpreted as normal (13% vs 16% and high grade (15% vs 20%), were statistically (P = 0.02) but not substantially different between same-day colposcopy and on-line assessment, respectively.

Table 5.

On-line global assessment of Cervigrams assessed on line vs. Same-day colposcopic assessment (P<0.0001). Results given as N (%, 95% C.I.). Abbreviations as in Table 1.

			Assessment Type
		Same-day	Online
	Negative	39 (13, 9-17)	95 (16, 12-21)
Assessment	Metaplasia/Atypical	30 (10, 4-16)	141 (24, 18-31)
	Low grade	185 (62, 55-68)	229 (39, 35-44)
	High grade	45 (15, 10-20)	117 (20, 16-24)
	Total	299 (100)	582 (100)

The ordinal logistic regression model yielded ORs as measures of association of the same-day impression or on-line assessment with ordered categories of disease severity. Failure to find differences in the ORs for each equivalent impression/assessment category demonstrated that that colposcopic impression and on-line assessment were similar in their ability to predict the severity of final diagnostic category. For high grade vs negative, the ORs of association between impression/assessment category and increasing disease severity were strong and statistically significant for both methods: 5.4 (95% C.I. 2.3, 12.9) for same-day impression and 4.3 (95% C.I. 2.4, 7.9) for on-line assessment (P for equality = 0.63). These ORs can be interpreted as meaning that a woman with a colposcopic impression or high grade on-line assessment, for example, was respectively 5.4 or 4.3 times more likely to have a more serious disease outcome than one with a negative impression/assessment. For low-grade vs negative the ORs of associations were weaker but still substantial: 1.8 for same-day impression and 1.8 for on-line assessment (P for equality = 0.94). For metaplasia/atypia vs negative the ORs of association with increasing disease severity were virtually null: 1.3 for same-day impression and 1.2 for on-line assessment (P for equality = 0.81); these near-null values suggest that the distinction between metaplasia or atypia and normal is not clinically important. Of importance for our research, the colposcopy and on-line assessments were not significantly different from each other at each level for predicting the severity of disease during follow-up (P = 0.95 by 3 degree of freedom test that each pair of ORs is equal when considered as a set).

We addressed the possibility that some cases of CIN2+ or even CIN3+ diagnosed during the 2 years of ALTS follow-up might have been incident and not diagnosable by colposcopy. In other words, the lesions might have been due to HPV infections from after enrollment, and might not have been present at enrollment when the images were taken and same-day colposcopy was performed. Therefore, in an ancillary analysis, we considered only the 295 images from those women with CIN3+ whose enrollment specimen contained the same carcinogenic type found at the time of CIN3+ diagnosis. The results were almost identical to the main analysis. Of the 295 images from this most stringent case group, 96% (95% C.I. 93%, 99%) were judged to have at least 1 acetowhite lesion at enrollment. The percentage with high-grade on-line assessments was 45.1% (95% C.I. 37.3%, 52.9%). The sensitivity of a Reid Index of 4-6 was low: 33% (95% C.I. 25%, 40%). Choosing lower thresholds for a “positive” Reid Index did not yield improved accuracy because many of the stringently-defined cases had a total score of 1.

Discussion

Our results show that the identification of an acetowhite lesion is a highly sensitive indicator for the subsequent identification of CIN2+. However, the specificity of this finding appears to be low, and directed biopsy is required to guide subsequent therapy, as most women with acetowhite lesions do not have CIN2+. In addition, the sensitivity of more detailed colposcopic analysis is substantially lower. Clinical colposcopic impression and the modified Reid Index components that contribute to it, including color, margin, vascularity, and total score, do not discriminate between acetowhite lesions that harbor CIN2+ and those that do not.

There are several clinical implications to these observations. First, CIN2+ in women with ASCUS and LSIL cytology presents as acetowhite lesions visible colposcopically. Although others have found a high yield to random biopsy in largely unscreened women (2), this suggests that biopsy of colposcopically normal tissue in U.S. women with ASCUS or LSIL cytology is likely to increase yield minimally when acetowhite lesions have been assessed comprehensively. However, formally determining the utility of random biopsy requires prospective studies such as one we have initiated. Second, sensitive detection of CIN2+ requires biopsy of acetowhite lesions even when the colposcopic impression is low grade or metaplasia. Third, Reid Index scores cannot be used to differentiate low from high grade lesions in women with borderline cytology, nor can Reid Index scores distinguish a subgroup of women at sufficiently low risk for CIN2+ to be followed without biopsy or treatment. Patient management cannot be based on colposcopic impression unless confirmed by biopsy. Finally, in a patient with multiple acetowhite lesions, Reid Index scoring should not be used to restrict biopsy number to those areas that appear most suspicious colposcopically. Lesions that were missed colposcopically in ALTS were generally subtle and small (19).

As recently reviewed (20), these results are broadly consistent with prior work. In ALTS, the sensitivity of initial colposcopy for the subsequent development of CIN3 was only 53% (6). However, results were better when more biopsies were taken, and results were worst for the most experienced clinicians, who may have used lesion characteristics to forego biopsy of early high grade lesions (21). Some women with negative or low grade colposcopy findings after abnormal Pap tests have CIN2+ identified at loop excision (1, 22) or later develop cervical disease, suggesting that significant lesions were missed or underestimated colposcopically (23). Prior work has also shown that using lesion characteristics to define lesion grade is problematic, since experienced colposcopists disagree over how terms should be applied (8) and how grading should be derived from observed images (7). Our results also extend the findings of Sideri and colleagues, who showed that interobserver correlations and sensitivity are best when the identification of an acetowhite lesion rather than detailed colposcopic grading is the trigger for biopsy (4).

Our study was limited to women with antecedent ASCUS or LSIL cytology reports. The Reid Index and similar grading systems may still be useful in women with high grade SIL, who may present with larger or more colposcopically advanced lesions. In fact, colposcopic grading systems were developed soon after the wide dissemination of cytology screening and prior to the development of current cytology classification systems, when many women undergoing colposcopy had more advanced disease. However, at present most cytologic abnormalities requiring colposcopy are ASC or LSIL, and most CIN2+ is found among women with these cytology results (24), so our results should apply to current colposcopic practice.

Previous work from our study group has been criticized for using static images (16). Colposcopy involves longitudinal assessment of acetowhite changes, and in vivo assessment may be more accurate since it allows observation of the time course of onset and fading of acetowhitening. Colposcopy and on-line assessment differ in their distribution of diagnoses, but this occurs mostly in how the two modalities distinguish low grade from atypical or metaplastic changes. We found no clinically meaningful difference between same-day impression and on-line assessment for predicting final histology. However, the ORs for same-day colposcopic impressions were equal to or slightly larger than the ORs for on-line assessment. If colposcopy is more accurate than assessment of still images in identifying CIN2+, a far larger study will be required to substantiate the small, but possibly clinically irrelevant, differences.

Our findings have particular relevance for education, since most colposcopy teaching depends on projection of static images. The lack of significant difference between the ability of real time colposcopy and static image assessment to identify CIN2+ suggests that use of static images should continue. In fact, ASCCP and staff from the National Institutes of Health have been collaborating to develop on-line teaching and evaluation tools. The significant association between Reid Index components such as vascularity, margin, color, and total score indicate that these visual characteristics have biologic significance and should be included in curricula. However, learners should be instructed that these factors are too inaccurate to be used in clinical management, including determining biopsy site or the need for treatment without histologic confirmation.

The role of colposcopy in the prevention of cervical cancer continues to evolve. As the threshold for abnormal screening results has shifted from Papanicolaou Class III cytology to mild dysplasia to ASCUS and now to detection of persistent oncogenic HPV in the face of normal cytology, the task of identifying increasingly subtle preinvasive lesions has become more difficult. Until better strategies are developed to find CIN2+ in women with borderline changes, biopsy of all acetowhite lesions will yield the greatest sensitivity for detecting cervical precancer. Toward this end, we are now studying a protocol that attempts to balance sensitivity against patient discomfort and costs.