Abstract
Objective
The aim of this study was to find out whether ThinPrep Integrated Imager (Hologic Inc.) screening is non-inferior to manual screening in the detection of cervical lesion.
Study Design
For a total of 4,011 ThinPrep Pap test specimens stained by ThinPrep staining, manual screening (Manual arm) and ThinPrep Integrated Imager screening (Imager arm) were performed so as not to be screened by the same cytotechnologist, and the sensitivity and specificity in the detection of cervical lesion were compared using McNemar's test.
Results
The sensitivity to detect CIN1 or more squamous cell abnormalities or glandular abnormalities was 91.67% (= 374/408, 95% confidence interval [CI]: 88.44–94.08%) for the Manual arm and 92.40% (= 377/408, 95% CI: 89.28–94.70%) for the Imager arm, and the specificity was 88.87% (= 3,113/3,503, 95% CI: 87.77–89.88%) for the Manual arm and 89.55% (= 3,137/3,503, 95% CI: 88.48–90.54%) for the Imager arm. The differences in sensitivity and in specificity, respectively, were 0.74% (95% CI: −3.14–4.61%, McNemar's test, p = 0.8041) and 0.69% (95% CI: −0.13–1.50%, McNemar's test, p = 0.1125). About the equality of sensitivity and specificity between the 2 methods, 95% CIs of the difference between sensitivity and specificity are in the clinical equivalence range of ±5%, so the Imager arm is non-inferior to the Manual arm.
Conclusion
The Imager arm was confirmed to have an equivalent and non-inferior capacity in the detection of cervical lesions compared with the Manual arm, suggesting that its practical application in cervical cytology tests is highly possible.
Keywords: Liquid-based cytology, ThinPrep Pap test, Cervical screening, Automated cytology screening
Introduction
As cytology test is a morphological examination different from other clinical laboratory tests, quality control in cytology tests is characterized by multiple control points, including the specimen preparation process, microscopic screening of specimens by cytotechnologists, and total assessment by an expert cytopathologist to achieve a final diagnosis. In particular, quality control in microscopic screening is considered to be most important, but no specific methods and procedures have actually been established.
Cervical cytology test (Pap test) is the most widely used among all cytology tests. According to conventional cervical cytology, cytotechnologists should examine direct smear specimens by manually performing microscopy for whole cells on the surface of a smeared slide glass. In the recent 2 decades, a groundbreaking cytological specimen preparation technique called liquid-based cytology (LBC) has been replacing conventional cytology in many countries, and the use of an automatic screening support device exploiting computer image analysis technology has also been implemented [1]. ThinPrep Imaging System (Hologic Inc.), one of the automatic screening support devices, is reported to be superior to manual screening for the detection of cervical lesions [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] and effective in reducing screening time [13, 14].
The aim of this study was to compare cytological results of ThinPrep Pap test specimens with manual screening (Manual arm) and using a ThinPrep Integrated Imager (I2, Hologic Inc.), which is the new stand-alone model of ThinPrep Imaging System, screening (Imager arm) to confirm that the Imager arm is at least equivalent to the Manual arm. In other words, it means the Imager arm is non-inferior to the Manual arm.
Materials and Methods
Between September 2011 and November 2012, a total of 4,011 cases were collected for the study from the routine clinical volume at the participants' hospitals, Keio University Hospital, St. Marianna University Hospital, Tokyo Dental College Ichikawa General Hospital, and NTT Medical Center Tokyo. All women patients agreed with the IRB approval procedure of each hospital.
Bloom-type brush (Cervex-Brush, Rovers) was used as the cell collection device, and the collected cells were fix-preserved following the recommended procedure by the ThinPrep® Pap test (Hologic Inc.), and rinsed in a vial containing preservative solution (ThinPrep® PreservCyt Solution, Hologic Inc.). The cell specimen collected in the vial was sent to the Setagaya Ward Health Center Clinical Laboratory Department, and the specimen preparation process was carried out using a ThinPrep® 5000 processor (Hologic Inc.). The prepared slide glasses were stained with ThinPrep® stain (Hologic Inc.), which is the most suitable stain for I2.
All cytotechnologists of the Setagaya Health Center Clinical Laboratory Department had already been trained with a special curriculum of ThinPrep® Pap test morphology training. Stained specimens were evaluated by both the Manual arm and Imager arm using I2 according to the Bethesda System for Reporting Cervical Cytology [15].
Regarding the microscopic screening performance, the Manual arm and Imager arm for 1 specimen were sorted out so that they would not be performed by the same cytotechnologist. Both arms were performed with all clinical information hidden. Moreover, randomization with double blinding was done so that the results of each group were separately managed.
All specimens diagnosed cytologically as ASC and AGC were classified as suspected-positive according to the Bethesda System for Reporting Cervical Cytology, or specimens that were more frequently diagnosed as positive were double-checked by the expert cytopathologist of the relevant cell sampling site to obtain the final diagnosis. Specimens diagnosed as NILM, which is classified as negative, were triple-checked by some cytotechnologists from external institutions who had undergone ThinPrep Pap test morphology training and the final result determined.
Among the subject cases that had been diagnosed histologically by biopsy at each site, such histological diagnosis was compared with each final cytological result. Cases with no CIN1 or more higher squamous cell abnormalities or glandular abnormalities confirmed by the histological diagnosis on biopsy and cases that had not undergone biopsy for histological diagnosis and were determined as negative by triple-check were defined as negative for cervical lesion.
On the other hand, those cases with CIN1 or more severe squamous cell abnormalities or glandular abnormalities confirmed by histological diagnosis were defined as positive for cervical lesion. For each of the groups defined as negative and positive, respectively, sensitivity and specificity in cytological evaluation were calculated and compared between the Manual arm and Imager arm. The inferiority/non-inferiority of the Manual arm versus the Imager arm was assessed using McNemar's test [16].
Results
Cytological diagnosis of each group obtained from 4,011 cases is as follows. In the Manual arm, diagnosis was unsatisfactory, 26 cases; NILM, 3,213 cases; ASC-US, 196 cases; ASC-H, 69 cases; LSIL, 242 cases; HSIL, 230 cases; SCC, 9 cases; AGC, 21 cases; and adenocarcinoma, 5 cases. In the Imager arm, diagnosis was unsatisfactory, 8 cases; cannot be analyzed, 73 cases; NILM, 3,188 cases; ASC-US, 168 cases; ASC-H, 45 cases; LSIL, 231 cases; HSIL, 269 cases; SCC, 7 cases; AGC, 18 cases; and adenocarcinoma, 4 cases (Table 1).
Table 1.
Cytological diagnosis of each group
| Manual arm, n | Imager arm, n | |
|---|---|---|
| Unsatisfactory | 26 | 8 |
| Unreadable | 0 | 73 |
| NILM | 3,213 | 3,188 |
| ASC-US | 196 | 168 |
| ASC-H | 69 | 45 |
| LSIL | 242 | 231 |
| HSIL | 230 | 269 |
| SCC | 9 | 7 |
| AGC | 21 | 18 |
| AIS | 0 | 0 |
| Adeno-Ca | 5 | 4 |
| Total | 4,011 | 4,011 |
The results were unsatisfactory for specimens obtained from a total of 30 cases, which were determined by both arms in 4 cases, by the Imager arm only in 4 cases, and by the Manual arm only in 22 cases. There were 73 cases that could not be analyzed in the Imager arm; 3 of those cases had unsatisfactory results as determined by the Manual arm. Among the 3,911 cases, 30 cases with unsatisfactory results and 70 cases that could not be analyzed (obtained by subtracting 3 unsatisfactory cases from 73 cases that could not be analyzed) were excluded; from a total of 4,011 cases, 3,503 cases were defined as negative for cervical abnormalities (negative group) and 408 cases were defined as positive for cervical abnormalities (positive group).
When defining NILM as negative and ASC-US/AGC or more as positive by cytological diagnosis, the cytology results obtained in the negative group were classified as negative by both arms in 3,020 cases, positive by both arms in 273 cases, positive by the Imager arm and negative by the Manual arm in 93 cases, and negative by the Imager arm and positive by the Manual arm in 117 cases. The test statistics was 2.519 with p = 0.1125 in the negative group. The cytology results obtained in the positive group were classified as negative by both arms in 0 cases, positive by both arms in 343 cases, positive by the Imager arm and negative by the Manual arm in 34 cases, and negative by the Imager arm and positive by the Manual arm in 31 cases. The test statistics was 0.06153 with p = 0.8041 in the positive group. The sensitivity to detect CIN1 or more squamous cell abnormalities or glandular abnormalities was 91.67% (= 374/408, 95% confidence interval [CI]: 88.44–94.08%) for the Manual arm and 92.40% (= 377/408, 95% CI: 89.28–94.70%) for the Imager arm, and the specificity was 88.87% (= 3,113/3,503, 95% CI: 87.77–89.88%) for the Manual arm and 89.55% (= 3,137/3,503, 95% CI: 88.48–90.54%) for the Imager arm. The negative predictive value was 98.90% for the Manual arm and 99.00% for the Imager arm (Table 2). For reference, Table 3 shows a comparison of the sensitivity, specificity, and negative predictive value of each group in lesion detection by histological diagnosis results.
Table 2.
Ability to detect CIN1 or more severe abnormalities of each group
| Manual arm | Imager arm | ||||||
|---|---|---|---|---|---|---|---|
| cytological diagnosis | histological diagnosis | cytological diagnosis | histological diagnosis | ||||
| negative, n | CIN1 ↑, n | total, n | negative, n | CIN1 ↑, n | total, n | ||
| NILM | 3,179 | 34 | 3,213 | NILM | 3,157 | 31 | 3,188 |
| ASC-US↑ | 392 | 380 | 772 | ASC-US↑ | 360 | 382 | 742 |
| Total | 3,571 | 414 | 3,985 | Total | 3,517 | 413 | 3,930 |
| Sensitivity, % | 91.8 | Sensitivity, % | 92.5 | ||||
| Specificity, % | 89.0 | Specificity, % | 89.8 | ||||
| Negative predictive value, % | 98.9 | Negative predictive value, % | 99.0 |
Table 3.
Comparison of detection ability by histological diagnosis result in each group
| Histological diagnosis | Group | Sensitivity, % | Specificity, % | Negative predictive value, % |
|---|---|---|---|---|
| CIN1 ↑ | Manual arm | 91.8 | 89.0 | 98.9 |
| Imager arm | 92.5 | 89.8 | 99.0 | |
| CIN2 ↑ | Manual arm | 92.5 | 85.9 | 99.4 |
| Imager arm | 94.0 | 86.5 | 99.5 | |
| CIN3 ↑ | Manual arm | 92.7 | 83.5 | 99.7 |
| Imager arm | 94.0 | 84.1 | 99.7 |
The differences in sensitivity and specificity, respectively, were 0.74% (95% CI: −3.14–4.61%, McNemar's test, p = 0.8041) and 0.69% (95% CI: −0.13–1.50%, McNemar's test, p = 0.1125) (Table 4). From these results, no statistically significant difference in sensitivity and specificity was found between the Manual arm and the Imager arm. Regarding the equality of sensitivity and specificity between the 2 methods, 95% CIs of the difference between sensitivity and specificity are in the clinical equivalence range of ±5%, so the Imager arm was comparable in sensitivity and specificity with the Manual arm. In other words, it became clear that the Imager arm is non-inferior to the Manual arm (Fig. 1).
Table 4.
Concordance rate of the negative and positive groups
| Negative group: 3,503 cases | Positive group: 408 cases | ||||
|---|---|---|---|---|---|
| Imager | Imager | ||||
| positive, n | negative, n | positive, n | negative, n | ||
|
Manual Positive, n Negative, n |
273 93 |
117 3,020 |
Manual Positive, n Negative, n |
343 34 |
31 0 |
| Concordance rate | 94.0% | Concordance rate | 84.1% | ||
| Test statistic = 2.519 | p = 0.1125 | Test statistic = 0.06153 | p = 0.8041 | ||
| Imager specificity: 89.55% (= 3,020/3,137, 95% CI: 88.48–90.54%) | Imager sensitivity: 92.40% (= 377/408, 95% CI: 89.28–94.70%) | ||||
| Manual specificity: 88.87% (= 3,113/3,503, 95% CI: 87.77–89.88%) | Manual sensitivity: 91.67% (= 374/408, 95% CI: 88.44–94.08%) | ||||
| Differences in specificity: 0.69% (95% CI: −0.13 to 1.50%) | Differences in sensitivity: 0.74% (95% CI: −3.14 to 4.61%) | ||||
| McNemar's test p = 0.1125 | McNemar's test p = 0.8041 | ||||
| CI, confidence interval. | |||||
Fig. 1.
McNemar's test result. Δ: clinically significant difference level (equivalence/non-inferiority margin) is set at ±5%.
Discussion
The Imager arm led to a smaller number of unsatisfactory and ASC (ASC-US and ASC-H) results and a larger number of HSIL results than the Manual arm. I2 has a unique function of automatically selecting 22 fields of view, including cells with dark nuclear chromatin and a large-diameter nucleus by computer image analysis technology, and the selected fields of view are microscopically examined one by one using an automatic stage linked with the computer [17].
Therefore, using I2, cytotechnologists can concentrate on observing cells found in the selected field of view by exempting the so-called screening work of finding cells with differences in stainability and size. Since cellular atypia is also evaluated during observation of cells found in the selected field of view, it can clearly count the number of appearing cells and clearly distinguish the atypism of individual cells. Therefore, according to these results, the number of unsatisfactory and ASC interpretation decreased. Moreover, especially when atypical squamous cells are detected, low-grade atypia and high-grade atypia of the detected cells can be more selectively distinguished from each other due to focus on the selected field of view using I2, and these effects may have contributed to the increase in HSIL results in the Imager arm. In addition, the Imager arm was associated not only with equivalent sensitivity and specificity in the detection of CIN1 or more severe lesions but also with equivalent negative predictive values compared with the Manual arm. This suggests its usefulness in accurate management of specimens classified as negative, for example, in double-check after manual screening to confirm the negative result. Also, it is interesting that the Imager arm was equivalent or superior to the Manual arm in all of the sensitivity, specificity, and negative predictive values even when the histological diagnosis results were CIN2 or severe, or CIN3 or severe.
On the other hand, 73 cases (1.8%) were unreadable by the instrument. Image analysis by I2 consists of image processing to separate the nucleus from the cytoplasm based on the analysis of differences in stainability. If there is over-staining of the cytoplasm caused by, for example, bacterial infection, it becomes impossible to distinguish between the nucleus and the cytoplasm, thus leading to an unreadable result. Also, since the function to analyze the slide glass is by recognizing the edge on the long side of the slide glass, if the cover glass is not parallel to the slide glass, then the analysis cannot be performed, thus leading also to an unreadable result. These are problems peculiar to the equipment associated with image analysis of the I2 device, and measures to address these must be identified in the future.
The use of LBC, which has been developing in all countries, mainly in the USA, is gradually spreading in Japan too. Computer-assisted automatic screening support systems are considered to have higher applicability in standardized LBC specimens, so its use is expected to increase in the future. Under such circumstances, the study of the ThinPrep Imaging System has only seen reports from Korea [12] so far in the Asia-Pacific region, but this study is the first report that has been studied in earnest in Japan in the Asia-Pacific region. Computer-assisted automatic screening support system, especially the ThinPrep Imaging System and ThinPrep Integrated Imager, is highly expected to become an effective new tool in the future for cervical cancer screening with various issues in modern-day Japan [18, 19, 20, 21].
Conclusion
The Imager arm was found to have an equivalent and non-inferior capacity in the detection of cervical lesions compared with the Manual arm, suggesting that its practical application in cervical cytology tests is highly possible. Since the equivalence in lesion detection compared with the conventional method was confirmed, as a secondary effect, cervical cytology testing using this I2 was considered as contributing to the efficiency and standardization of microscopic work.
Statement of Ethics
This study was approved by the Research Ethics Committee of the Keio University School of Medicine, St. Marianna University School of Medicine, Tokyo Dental College Ichikawa General Hospital, and NTT Medical Center Tokyo. Women who agreed to participate in the study signed an informed consent form. All information regarding the research participants were kept confidential.
Disclosure Statement
This study was supported by Hologic Japan Inc. (Tokyo, Japan) according to the multicenter collaborative research agreement.
References
- 1.Birdson GG. Automated screening of cervical cytology specimens. Hum Pathol. 1996;27((5)):468–81. doi: 10.1016/s0046-8177(96)90090-8. [DOI] [PubMed] [Google Scholar]
- 2.Bishop JW, Kaufman RH, Taylor DA. Multicenter comparison of manual and automated screening of AutoCyte gynecologic applications. Acta Cytol. 1999;43:34–8. doi: 10.1159/000330866. [DOI] [PubMed] [Google Scholar]
- 3.Wilbur DC, Parker EM, Foti JA. Location-guided screening of liquid-based cervical cytology specimens: a potential improvement in accuracy and productivity is demonstrated in a preclinical feasibility trial. Am J Clin Pathol. 2002;118((3)):399–407. doi: 10.1309/7LRF-DU8Q-8H1W-N7T4. [DOI] [PubMed] [Google Scholar]
- 4.Hutchinson M, Lapen D, Werneke S, Bracco D, Linder J, Brahm C, et al. Cancer detection with the ThinPrep imaging system (abstr) Acta Cytol. 2003;47:8. [Google Scholar]
- 5.Biscotti CV, Dawson AE, Dziura B, Galup L, Darragh T, Rahemtulla A, et al. Assisted primary screening using the automated ThinPrep Imaging System. Am J Clin Pathol. 2005;123((2)):281–7. [PubMed] [Google Scholar]
- 6.Dziura B, Quinn S, Richard K. Performance of an imaging system vs. manual screening in the detection of squamous intraepithelial lesions of the uterine cervix. Acta Cytol. 2006;50((3)):309–11. doi: 10.1159/000325959. [DOI] [PubMed] [Google Scholar]
- 7.Davey E, d'Assuncao J, Irwig L, Macaskill P, Chan SF, Richards A, et al. Accuracy of reading liquid based cytology slides using the ThinPrep Imager compared with conventional cytology: prospective study. BMJ. 2007;335((7609)):31–8. doi: 10.1136/bmj.39219.645475.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Roberts JM, Thurloe JK, Bowditch RC, Hyne SG, Greenberg M, Clarke JM, et al. A three-armed trial of the ThinPrep Imaging System. Diagn Cytopathol. 2007;35((2)):96–102. doi: 10.1002/dc.20600. [DOI] [PubMed] [Google Scholar]
- 9.Miller FS, Nagel LE, Kenny-Moynihan MB. Implementation of the ThinPrep Imaging System in a high-volume metropolitan laboratory. Diagn Cytopathol. 2007;35((4)):213–7. doi: 10.1002/dc.20627. [DOI] [PubMed] [Google Scholar]
- 10.Chivukula M, Saad RS, Elishaev E, White S, Mauser N, Dabbs DJ. Introduction of the Thin Prep Imaging System (TIS): experience in a high volume academic practice. CytoJournal. 2007;4:6. doi: 10.1186/1742-6413-4-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Halford JA, Batty T, Boost T, Duhig J, Hall J, Lee C, et al. Comparison of the sensitivity of conventional cytology and the ThinPrep Imaging System for 1,083 biopsy confirmed high-grade squamous lesions. Diagn Cytopathol. 2010;38((5)):318–26. doi: 10.1002/dc.21199. [DOI] [PubMed] [Google Scholar]
- 12.Ha SY, Lee YK, Oh YL. Effectiveness of the ThinPrep Imaging System in the detection of abnormal cervicovaginal cytology: a practical experience in Korea. Acta Cytol. 2013;57((2)):159–63. doi: 10.1159/000345103. [DOI] [PubMed] [Google Scholar]
- 13.Davey E, Irwig L, Macaskill P, Chan SF, D'Assuncao J, Richards A, et al. Cervical cytology reading times: a comparison between thinprep imager and conventional methods. Diagn Cytopathol. 2007;35((9)):550–4. doi: 10.1002/dc.20689. [DOI] [PubMed] [Google Scholar]
- 14.Boost T. A comparison of screening times between the ThinPrep Imager and conventional cytology. Diagn Cytopathol. 2009;37((9)):661–4. doi: 10.1002/dc.21069. [DOI] [PubMed] [Google Scholar]
- 15.Solomon D, Nayar R. The Bethesda System for reporting cervical cytology: definitions, criteria, and explanatory notes. 2nd ed. New York: Springer-Verlag; 2004. [Google Scholar]
- 16.McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12((2)):153–7. doi: 10.1007/BF02295996. [DOI] [PubMed] [Google Scholar]
- 17.ThinPrep imaging system operation summary and clinical information. Malborough: Hologic Inc; 2009. pp. p.1–13. [Google Scholar]
- 18.Utada M, Chernyavskiy P, Lee WJ, Franceschi S, Sauvaget C, de Gonzalez AB, et al. Increasing risk of uterine cervical cancer among young Japanese women: comparison of incidence trends in Japan, South Korea and Japanese-Americans between 1985 and 2012. Int J Cancer. 2019;144((9)):2144–52. doi: 10.1002/ijc.32014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Konno R, Shin HR, Kim YT, Song YS, Sasagawa T, Inoue M, et al. Human papillomavirus infection and cervical cancer prevention in Japan and Korea. Vaccine. 2008;26((Suppl 12)):M30–42. doi: 10.1016/j.vaccine.2008.05.006. [DOI] [PubMed] [Google Scholar]
- 20.Kurokawa T, Onuma T, Shinagawa A, Chino Y, Kobayashi M, Yoshida Y. The ideal strategy for cervical cancer screening in Japan: result from the Fukui Cervical Cancer Screening Study. Cytopathology. 2018;29((4)):361–7. doi: 10.1111/cyt.12576. [DOI] [PubMed] [Google Scholar]
- 21.Sugiyama Y, Sasaki H, Komatsu K, Yabushita R, Oda M, Yanoh K, et al. A multi-institutional feasibility study on the use of automated screening systems for quality control rescreening of cervical cytology. Acta Cytol. 2016;60((5)):451–7. doi: 10.1159/000449499. [DOI] [PubMed] [Google Scholar]