Accuracy of colposcopy in the Swedish screening program

Abstract

Introduction

Sensitivity and specificity of colposcopy vary greatly between studies and efficacy in clinical studies seldom corresponds with effectiveness in a real‐life setting. It is unclear whether colposcopists’ experience affects assessment; studies show divergent results. The study's objective was to investigate the accuracy of colposcopies in the Swedish screening program, the variability in colposcopists’ assessments and whether degree of experience affects accuracy in a routine setting.

Material and methods

Cross‐sectional register study. All colposcopic assessments with a concomitant histopathological sample from women aged at least 18 years, performed between 1999 and September 2020 in Sweden. The main outcome measure was accuracy. The accuracy of colposcopic assessments was calculated as overall agreement with linked biopsies, with three outcomes: Normal vs Atypical, Normal vs Low‐Grade Atypical vs High‐Grade Atypical, and Non‐High‐Grade Atypical vs High‐Grade Atypical. A time‐trend analysis was performed. The accuracy of identifiable colposcopists related to experience was analyzed.

Results

In total, 82 289 colposcopic assessments with linked biopsies were included for analysis of the outcome Normal vs Atypical; average accuracy was 63%. Overrating colposcopic findings was four times more common than underrating. No time trend in accuracy was noted during the study period. Accuracy in distinguishing High‐Grade from Non‐High‐Grade lesions was better: 76%. Among identifiable colposcopists, overall accuracy was 67%. Some had significantly better accuracy than others, but no correlation with experience was found.

Conclusions

Colposcopy, including in a referral setting, has low accuracy in distinguishing Normal from Atypical. Increased experience alone does not lead to improvement. This is supported by the substantial differences in performance between colposcopists.

Colposcopy in a routine setting had low accuracy in distinguishing Normal from Atypical epithelium but improved for distinguishing High‐Grade. Increased experience alone did not improve accuracy. Overrating colposcopic findings and nonidentification of High‐Grade lesions underline the need for biopsies.

Abbreviations

CI

confidence interval

CIN

cervical intraepithelial neoplasia

HPV

human papillomavirus

HSIL

High‐Grade squamous intraepithelial lesion

RCT

randomized controlled trial

Key message.

Colposcopy in a routine setting had low accuracy in distinguishing Normal from Atypical epithelium but improved for distinguishing High‐Grade. Increased experience alone did not improve accuracy. Overrating colposcopic findings and nonidentification of High‐Grade lesions underline the need for biopsies.

1. INTRODUCTION

Colposcopy is an essential part of the screening program aimed at detecting precancerous lesions and cancer in the cervix. The precision of this assessment is crucial, as it determines whether and when biopsies should be taken. Colposcopy performance has been intensively studied, and sensitivity, specificity and predictive values vary greatly, often due to different ways of measuring results. ¹ In a meta‐analysis with data from 1960–1996, addressing the sensitivity and specificity of colposcopy and aimed at identifying High‐Grade intraepithelial lesions (HSIL) and cancer, the average sensitivity was 85%, the average specificity was 69% and the corresponding area under the curve was 0.82. ² In Swedescreen, a randomized controlled trial (RCT) of human papillomavirus (HPV) testing in a screening program, the sensitivity of colposcopy categorized as atypical for distinguishing cervical intraepithelial neoplasia (CIN)2⁺ was 83%. ³ In contrast, an American RCT only found 54% sensitivity in diagnosing CIN3. ⁴ Several other studies have also reported low sensitivity, and authors have suggested that biopsies also be taken from colposcopically normal areas. ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ Many studies suffer from verification bias, as the outcome depends on the biopsies from the very lesions identified by colposcopy. Moreover, histopathological analysis, the gold standard, has been found to have only moderate interobserver reproducibility. ¹⁴

The issue of whether experience or professional background affects precision is debated. Several studies have shown similar sensitivity regardless of medical training background, and specially trained nurses, general gynecologists, junior gynecologists and colposcopists have been shown to perform equally well. ⁵ , ¹⁵ , ¹⁶ Staff with a shorter education or less professional experience, such as nurses and general gynecologists, are prone to overestimate colposcopic findings and tend to compensate with multiple biopsies. ⁵ , ¹⁵ In contrast, an RCT showed that senior colposcopists performed better in detecting HSIL compared with junior colleagues. ¹⁷ A retrospective German study yielded similar findings: the respective accuracy of colposcopists with <2, 2–10 and > 10 years of experience in detecting HSIL was 87%, 85.3% and 90.2%, respectively. ¹⁸ Studies on interobserver agreement have reported low reproducibility. ¹⁹ , ²⁰

The aim of this study was to investigate the accuracy of colposcopies in the Swedish screening program, the development of accuracy over time, the variability in assessment between colposcopists and how professional experience affects colposcopic accuracy in a routine setting.

2. MATERIAL AND METHODS

2.1. Data sources

We performed a cross‐sectional study using data from the Process Registry in the Swedish National Cervical Screening Registry. This registry contains data on cervical cytology, HPV and histopathology. It also contains colposcopic assessment reports for women in the screening program. ²¹ The Process Registry currently covers 85% of the Swedish population but colposcopy data coverage is lower and unevenly distributed, with most of the data coming from the Western Healthcare Region of Sweden. All laboratories in the affiliated regions deliver their complete histopathology, cytology and HPV data to the registry. ²¹ Data on histopathology specimen type are lacking, but most samples are derived from punch biopsies, and a minority from excisions. During the study period there has not been any recommendation in Swedish guidelines on whether biopsies should be taken when colposcopies were assessed as normal or on number of biopsies at abnormal colposcopy.

Colposcopies have been registered by the cytology form since 1999, initially with the categories Normal – Atypical – unsatisfactory. In 2008, a personal ID associated with the respective clinics was introduced by some laboratories. In 2012, this workplace‐related ID was converted into a user‐unique ID in parts of Western Sweden. This enables better follow‐up if colposcopists change workplace. Since 2015, the Normal – Atypical categorization has gradually been replaced by a three‐grade scale: Normal, Low‐Grade Atypical and High‐Grade Atypical. From this time forwards, Swedescore (0–10) and transformations zone type (1–3) have also been recorded in the registry. Manual registration of colposcopies, not necessitating a cytology form, was enabled in 2017. Since the start 1999, registration has successively expanded from one university laboratory in Western Sweden to be included in national guidelines since 2017. In 2020, regions outside the Western Region supplied 39% of the colposcopy data.

2.2. Study population

We retrieved data on all colposcopic assessments, registered from 1999 to September 2020 in women aged at least 18 years, which could be linked to a histopathology report within (±) 21 days. These colposcopies were performed by doctors as part of clinical routines in hospitals and gynecologic outpatient clinics, public and private, mainly in the Western Healthcare Region of Sweden. Reasons for colposcopy were abnormal cytology, in recent years in conjunction with positive HPV tests or follow‐up of previous abnormalities, which was the less common indication. In cases of multiple discrepant histopathology diagnoses, the most severe was applied, according to the hierarchical classification in the registry. Colposcopies assessed as unsatisfactory and/or with transformation zone type 3 were excluded.

2.3. Statistical analyses

We performed five analyses on different subpopulations from the registry:

• Using all the data, Normal vs Atypical colposcopies with histopathological outcome were compared. In this analysis, the subset of colposcopic assessments registered as Low‐Grade and High‐Grade were merged into Atypical.

• These data were used for assessing accuracy over time with a time‐trend analysis.

• Accuracy of the three‐grade scale (see above) using an arbitrarily chosen scale of 0–1. One point was given for a perfect match and 0.5 point was given for a one‐step mismatch, eg High‐Grade colposcopic assessment and Low‐Grade histopathology. No match was given 0 points, eg Normal colposcopy and High‐Grade histopathology. In this analysis, Low‐Grade histopathology was equivalent to CIN1 and koilocytosis, and High‐Grade histopathology was defined as CIN2⁺ (CIN2, CIN3, adenocarcinoma in situ [AIS] or cervical cancer).

• Dichotomized accuracy of colposcopic assessment of High‐Grade vs Non‐High‐Grade (Low‐Grade or Normal) was compared with the histopathology with the corresponding classification.

• Data with identifiable colposcopists enabled comparison of accuracy between colposcopists; accuracy related to degree of experience, calculated as number of registered colposcopies; and individual colposcopists’ change in accuracy with increasing experience. Only colposcopists with ≥20 registered examinations in total were included in these analyses.

The basis for the analysis of colposcopies, as well as for analysis of the individual colposcopist's performance, comprised a linked histopathology sample. As mentioned above, each colposcopy was registered with the colposcopist's ID number. Accuracy was calculated as overall agreement. Differences in accuracy between colposcopists were calculated with the chi‐square test. Pearson's correlation coefficient was used for calculations of accuracy related to experience, calculated as the total number of registered colposcopies per colposcopist. Linear regression was applied for analysis of change of accuracy with increasing experience in individual colposcopists. Change of accuracy with increasing experience for each colposcopist was also calculated for the three‐grade scale and for High‐Grade vs Non‐High‐Grade. Statistical analysis was performed with R.4.0.4.

2.4. Ethics statement

The Swedish Ethical Review Authority (reference number 2020‐02271) approved the study on July 2, 2020.

3. RESULTS

The registry contained 82 289 colposcopic assessments linked to histopathological samples, performed during 1999–2020 and with outcome Normal or Atypical (Figure 1), which constituted the basis for comparison. Average accuracy was 63% (Table 1). Colposcopic findings were overrated four times more often than they were underrated. No time trend in accuracy was noted during the study period (Figure 2).

FIGURE 1.

Overview of the study. Numerals (1, 2, etc.) show what population is used for analysis described in the Material and methods section.

TABLE 1.

Accuracy of 82 289 colposcopies assessing Normal vs Atypical colposcopy.

Histology→	Benign		Atypical
↓Colposcopy	n	%	n	%
Normal	11 076	13%	6143	7%
Atypical	24 280	30%	40 790	50%

Note: Agreement in green cells. Non‐agreement in red cells. Overall agreement 63% (95% CI 0.63–0.63).

FIGURE 2.

Accuracy for assessing Normal vs Atypical in 82 289 colposcopies over time. Brown area is 95% CI.

During the study period, 504 identified colposcopists performed a total of 24 362 registered colposcopies. Of these, 22 937 were performed by 143 colposcopists with ≥20 previously registered examinations. Overall accuracy among the latter group for outcome Normal vs Atypical was 67% (data not shown). Some colposcopists had significantly better accuracy than others (P < 0.001) and there was wide variability in performance (Figure 3). However, when accuracy was related to experience, calculated as the total number of colposcopies per colposcopist, no correlation was found (Figure 3) (correlation coefficient = 0.0024, P = 0.98). Moreover, there was no effect of increasing experience per colposcopist when the accuracy of every added colposcopy was compared with the colposcopist's previous results (regression coefficient = −0.00016) (Figure 3). There was no effect of increased experience when it came to the three‐grade scale or for the High‐Grade vs Non‐High‐Grade comparison (data not shown).

FIGURE 3.

Accuracy for assessing Normal vs Atypical of 24 561 colposcopies performed by the 143 identifiable colposcopists with more than 20 previously registered colposcopies. Overall agreement = 0.68 (max = 0.97, min = 0.38, 2 SD = 0.21). Difference in accuracy between colposcopists: chi‐square test = 734.48 (P < 0.0001). Comparison of accuracy between colposcopists related to experience, calculated as number of registered colposcopies: Pearson's correlation coefficient = 0.0024 (P = 0.9786; dotted regression line in the diagram). Change of accuracy with increasing experience per colposcopist: regression coefficient = −0.00016.

There were records of 18 266 colposcopies with three‐grade scale assessments; accuracy was 71% (Table 2). In the case of the three‐grade scale, overrating was twice as common as underrating. In all, 45% (8676/18 266) of the observations were totally accurate, whereas half (8449/18 266) deviated one step. Only 6% (1141/18 266) were clearly inaccurate and deviated two steps. When High‐Grade or Non‐High‐Grade colposcopic assessment data were linked to histopathology ≤CIN1 or CIN2⁺ data, 76% accuracy was found (Table 3). In contrast to the previous comparisons, underrating of colposcopic findings was slightly more common.

TABLE 2.

Accuracy of 18 266 colposcopies categorized as Normal, Low‐Grade Atypical or High‐Grade Atypical.

Histology→	Benign			CIN1			CIN2+
↓Colposcopy	n	%	Weight	n	%	Weight	n	%	Weight
Normal	2705	15%	1	1002	5%	0.5	389	2%	0
Low‐Grade	4244	23%	0.5	3855	21%	1	2253	12%	0.5
High‐Grade	752	4%	0	950	5%	0.5	2116	12%	1

Note: Full agreement in green cells. Intermediate agreement in yellow cells. Non‐agreement in red cells. Overall weighted agreement 71% (95% CI 0.70–0.71).

TABLE 3.

Accuracy of 18 266 colposcopies categorized as High‐Grade Atypical or Non‐High‐Grade Atypical.

Histology→	≤CIN1		CIN2+
↓Colposcopy	n	%	n	%
Non‐High‐Grade	11 806	65%	2642	14%
High‐Grade	1702	9%	2116	12%

Note: Agreement in green cells. Non‐agreement in red cells. Overall agreement 76% (95% CI 0.75–0.76).

4. DISCUSSION

In this study comprising 82 000 colposcopies in Sweden performed in 1999–2020, accuracy in distinguishing Normal from Atypical epithelium was found to be 63%. Overrating was four times more common than underrating. The accuracy did not change over the study period. Just over 24 000 colposcopies were attributed to an identifiable colposcopist. In this group, two‐thirds of colposcopic diagnoses were correct, but colposcopists with many registered examinations performed no better than those with few registered examinations. Furthermore, no improvement in individual colposcopists’ accuracy could be found with increasing experience. Colposcopic accuracy improved to 71% with the three‐grade scale (Normal, Low‐grade atypical and High‐Grade Atypical) and reached 76% in distinguishing histopathological High‐Grade from Non‐High‐Grade lesions.

In distinguishing Normal from Atypical, the main weakness was colposcopic overrating, which was four time more common than underrating. However, when High‐Grade lesions were colposcopically distinguished from Non‐High‐Grade lesions, overrating was less common and underrating was more of a problem. This has clinical implications, due to the potential risk related to abstaining from biopsy. However, the finding that only 8% of High‐Grade lesions were considered colposcopically normal (Table 2) is somewhat reassuring.

To our knowledge, there is only one previous study with comparable population‐based data, ²² comprising 84 244 patients examined by 37 colposcopists. That study found 79% accuracy in distinguishing benign from atypical lesions. The corresponding accuracy in our data was only 63%. The screening and colposcopy program in the British Columbia province of Canada is well known for its high professional standard and its results can be regarded as a benchmark. Compared with the Swedish program, the number of colposcopists involved was much lower, indicating the benefit of expertise.

Accuracy was measured for three different outcomes in our study: Normal vs Atypical, Normal vs Low‐Grade Atypical vs High‐Grade Atypical and Non‐High‐Grade Atypical vs High‐Grade Atypical. The latter two exhibited superior performance. This is in accordance with the study from British Columbia, which demonstrated that the predictive accuracy of colposcopy increased with advancing severity of assessed disease. ³ Similar findings were described in an Australian study in which higher sensitivity in detecting CIN2⁺ was found for High‐Grade referral smears than for low‐grade referral smears, ¹⁶ as well as in our own recent study on routine colposcopic performance with Swedescore. ²¹ The low accuracy of colposcopy for distinguishing Normal from Atypical (63%) and the high proportion of overrating concurs with the known low sensitivity for colposcopy as a screening method. ²³ The fair ability of colposcopy, in this study, to distinguish High‐Grade from other findings is clinically important, as the main purpose of colposcopy is to identify High‐Grade lesions from which to take biopsies. It should be noted that the majority of CIN2⁺ cases were assessed as Low‐Grade (Tables 2 and 3), but no more than 389/4756 (8%) of the CIN2⁺ cases were assessed as normal. Underrating was more common (14%) than overrating (9%). These signs of rather low performance of colposcopy in the Swedish program, support taking biopsies liberally, as well as adopting a realistic view of what colposcopy can detect and miss.

Our data demonstrated that some colposcopists performed significantly better than others but we failed to find any effect of experience or any effect of additional experience based on the performance of each added colposcopy for the individual colposcopist. Results were not altered when the outcomes High‐Grade vs Non‐High‐Grade and Normal vs Low‐Grade vs High‐Grade were analyzed (data not shown). It is thus still unclear what characterizes more skilled colposcopists. Are some simply more gifted at pattern recognition? This hypothesis is supported by the previous finding that superior performance may be a result of complex integration of learning. ²⁴ Some researchers argue that an inverse relation may exist between duration of professional experience and quality of care. ²⁵ Furthermore, a large Swedish population‐based study in urology found no association between surgeon volume and surgical outcome. The authors emphasized the importance of individual feedback on results. ²⁶

Are our findings thus an effect of the lack of continuing education, so that initial mistakes and misinterpretations are simply repeated and not corrected over the years? Our results concur with previous smaller studies ⁵ , ¹⁵ , ¹⁶ in which neither professional background nor learning curve had any effect. However, the contrary was shown in an Italian RCT, ¹⁷ comparing juniors and seniors, in which a maximum of two biopsies per colposcopy were taken. Inexperienced colposcopists often take more biopsies, increasing sensitivity, which may explain the dissimilarity between these studies. ¹⁶ In a retrospective German study, with similar findings to the Italian RCT, less experienced colposcopists had a higher detection rate than more experienced colleagues. ¹⁸ Our study lacks data on actual colposcopic education and monitored training, and our findings do not contradict the need for formal training, certification and re‐certification of colposcopists. There is a lack of prospective studies investigating whether training improves accuracy. However, in the impressively managed screening program in British Columbia ²² and among experts, ²¹ accuracy is substantially higher, indicating substantial potential for improvement of the Swedish program, a finding that probably is relevant for a large part of colposcopy services, globally.

Our findings could suggest a need for formalized training programs for colposcopists and conferences with discussions around colposcopy cases and images, as well as registry‐based quality assurance of individual colposcopists’ performance. We found no trend in accuracy during this century, which is a disappointment, since educational endeavors have increased since 2015, albeit to a relatively small extent.

There is clearly a need for quality assurance of colposcopy, which is ongoing in different countries as well as within the European Federation for Colposcopy (EFC). Accuracy, the number of correct assessments divided by all assessments, is easy to calculate and understand. However, there is no agreement concerning the criteria with which to define accuracy. We suggest, with some caution, that distinguishing High‐Grade lesions, which should normally be treated, from those that should not, is the most important parameter. Indeed, these results were the best in this study, but the tendency to overrate rather than underrate and the large number of unidentified High‐Grade lesions show the imperfection of colposcopy in the Swedish program and the need to take biopsies, including when lesions are assessed as low‐grade or, in many instances, as benign.

There is a need for research on background factors that can explain differences in accuracy across colposcopists. Furthermore, there is a need to establish whether conventional education and training programs improve colposcopic performance, as our results indicate that this is questionable.

This is, to our knowledge, the first study based on routine colposcopies in a screening program outside British Columbia. ²² The strengths of our study are the population‐based data, the large number of colposcopies and colposcopists and the time period of more than two decades.

This study also has several limitations. There may have been information bias. Despite national data retrieval, a majority of the identified colposcopies and colposcopists were from the Western Healthcare Region of Sweden, highlighting deficiencies in nationwide colposcopy registration. Moreover, not all colposcopies were linked to individual colposcopists by a unique ID, which may have led to underestimation of experience. There is certainly a verification bias, as the outcome was a histopathological diagnosis of samples from lesions identified by the method studied in most cases. Another selection bias derives from the inclusion criterion of a histopathological sample that could be unequivocally linked to the colposcopy. Thus, the most normal‐appearing examinations, where colposcopists might have abstained from biopsy, are not represented in this study.

5. CONCLUSION

Colposcopy, even in a referral setting, has low accuracy in distinguishing Normal from Atypical epithelium. Accuracy improves in identifying High‐Grade lesions. Increasing experience alone does not correlate with increased accuracy. However, this does not rule out the importance of education and training, supported by the considerable differences in performance between colposcopists. Our study supports taking biopsies at routine colposcopy.

AUTHOR CONTRIBUTIONS

EA and BS conceived and designed the study and formulated the research questions. FD reviewed medical records and matched colposcopists’ IDs. CZ was responsible for data management and statistical analysis. EA, BS, CZ and FD analyzed the data. EA, BS and FD drafted the paper, which was revised and approved by all authors.

FUNDING INFORMATION

This study was partially financed with a grant from the Hjalmar Svensson Foundation (grant number HJSV2020068), and with a LUA/ALF national grant (grant number 11315).

CONFLICT OF INTEREST STATEMENT

None.

Alfonzo E, Zhang C, Daneshpip F, Strander B. Accuracy of colposcopy in the Swedish screening program. Acta Obstet Gynecol Scand. 2023;102:549‐555. doi: 10.1111/aogs.14538