Improving diagnostic capability for HPV disease internationally within the NIH-NIAID-Division of AIDS Clinical Trial Networks

Abstract

Objectives

To evaluate an external quality assurance (EQA) program for the laboratory diagnosis of human papillomavirus (HPV) disease that was established to improve international research capability within the Division of AIDS at the National Institute of Allergy and Infectious Disease–supported Adult AIDS Clinical Trials Group network.

Methods

A three-component EQA scheme was devised comprising assessments of diagnostic accuracy of cytotechnologists and pathologists using available EQA packages, review of quality and accuracy of clinical slides from local sites by an outside expert, and HPV DNA detection using the commercially available HPV test kit.

Results

Seven laboratories and 17 pathologists in Africa, India, and South America participated. EQA scores were suboptimal for standard packages in three of seven laboratories. There was good agreement between the local laboratory and the central reader 70% of the time (90% confidence interval, 42%-98%). Performance on the College of American Pathologists’ HPV DNA testing panel was successful in all laboratories tested.

Conclusions

The prequalifying EQA round identified correctable issues that will improve the laboratory diagnosis of HPV related cervical disease at the international sites and will provide a mechanism for ongoing education and continuous quality improvement.

Cervical cancer is the third most common cancer in women globally, and most of the burden occurs in resource-limited settings. These areas have a disproportionate mortality: the age-specific mortality rates for women in resource-limited countries are at least triple those of women in resource-rich environments. Human immunodeficiency virus (HIV)–infected women bear a greater burden of disease than uninfected women, and the high mortality rates from cervical cancer in the younger age groups in resource-limited countries may reflect the burden of HIV disease.^1-5 In much of Africa, cervical cancer is the leading cancer cause of mortality for women.⁶

Cervical diagnostic testing allows for early diagnosis and treatment of precursor lesions. Cytology screening programs have had a substantial impact on mortality in those countries where access to regular screening is available and where there is an organized approach to cervical cancer prevention. Suspicious cytology usually requires histologic confirmation, but the approach is imperfect—a single test using standard cytology detects high-grade squamous intraepithelial lesions between 55% and 85% of the time.^7-9 When abnormalities occur, colposcopy-directed biopsy is the gold standard against which cervical screening modalities are measured, but the sensitivity of colposcopy-directed biopsy for cervical intraepithelial neoplasia stage 3 (CIN3) varies from 50% to 65%. It is presumed that the sensitivity for the detection of CIN2 is lower.¹⁰

Countries with limited resources have turned to alternative screening strategies because of the substantial clinical and pathologic infrastructure required for cytology-based screening for cervical cancer prevention. Visual inspection methods using either acetic acid (VIA) to find suspicious lesions have been adopted by a number of countries; these methods have a range of reported sensitivity for high-grade lesions of 64% to 90% and a specificity of 75% to 83% compared with biopsy in research settings where practitioners are heavily supervised. Results are operator dependent, and inter-rater reliability is quite variable.^11,12 VIA and HPV testing have been shown to reduce the incidence of cervical cancer and its mortality. ^{13, 14, 15} Human papillomavirus (HPV) DNA is being incorporated in screening in the in the United States,^16,17to lengthen the cytologic screening interval, but the technology is attractive in resource-limited settings because it may allow for better triage of women with the highest risk of clinical disease. The test has been proposed as a primary screen for cervical disease, and its sensitivity and specificity for detecting CIN2 have been compared with the histologic gold standard, with good sensitivity (91.9%) and poor specificity (51.4%).^3,11 HPV testing with cryotherapy and visual inspection with cryotherapy have been evaluated in a large clinical trial of both HIV-infected and uninfected women in South Africa, and both methods are effective in preventing CIN2+.¹⁸ Importantly, when cytology and histopathology results are not optimal, the results of HPV DNA testing cannot be validated accurately.

AIDS Clinical Trial Group (ACTG) 5282 is a multicenter randomized phase II trial that compares screening with HPV testing with a cytology-based strategy (NCT01315353). The study will screen 700 women and randomize 280 HIV-infected women to compare cumulative CIN2 or greater rates using the two different strategies. It is being conducted at ACTG sites in Africa (Botswana, Malawi, South Africa, Zambia, and Zimbabwe), Haiti, India, and Peru, where screen-and-treat strategies are being considered by the national programs.

The Division of AIDS (DAIDS) at the National Institute of Allergy and Infectious Disease conducted an audit of histopathology capabilities in 2009-2010 at potential sites to assess the abilities of the laboratories to support research in this area. Significant concerns were raised about the ability of potential participating sites to perform the required evaluations in cytology, histopathology, and HPV virology. Specimen preparation was an issue; specimen recordkeeping, including requisition and report tracking, were also problematic at various laboratories. It was unclear whether local pathologists with large clinical workloads, which include a wide range of anatomic specimens, would be able to perform the cervical cytology and histology to the standard required for this protocol.

External quality assurance (EQA) is an accepted method for ensuring the validity of test results and comparability across different sites, and there are robust EQA programs in cervical cytology.¹⁹ Histology-based EQA programs are less developed and occur in the context of continuing professional development or re-certification of pathologists. Histologic diagnosis often relies on clinical judgment, and inter-rater reliability testing is performed for few clinical entities. Available schemes for both cytology and histology are costly, time-consuming, and not implemented in most countries where A5282 is being conducted. Where voluntary schemes are in place, there is often little or no oversight of adherence.^20-28

In contrast, there is an expectation that virology performance will be monitored. The use of automated viral DNA and RNA testing means that the development of specific standards for performance is now less complex. Assay-related internal quality assurance is commonly undertaken. Many tests include an internal standard in each test-well, and the manufacturer of the assay provides assay controls.²⁹

A comprehensive EQA program was developed to improve research capacity and clinical care and to support the laboratories during the conduct of A5282. This article reports the prestudy assessments and activities.

Materials and Methods

All international ACTG sites were invited to participate in the A5282 protocol. Each site was asked to identify a pathology laboratory and a pathologist; DAIDS visited the laboratories and considered the capacity proposed by the site using an inspection tool developed for the purpose. A three-part EQA scheme to support HPV-related trials in the ACTG was developed to assess cytopathology (Papanicolaou smear), histology (biopsy), and HPV virology (Abbott m2000 Real Time High-Risk HPV DNA assay; Abbott Molecular Wiesbaden Germany). Both specimen preparation and diagnostic accuracy were assessed. Scoring schemes were developed to standardize and quantify results. Data were collected and tabulated by Patient Safety Monitoring in International Laboratories (SMILE), a DAIDS contractor charged with laboratory monitoring.

Cytology Proficiency Testing (Scheme I)

Participation by cytologists and supervising pathologists in an internationally recognized EQA program was required (eg, College of American Pathologists [CAP] or Royal College of Pathologists Australia [RCPA]) and performance was tabulated. Pathologists who scored greater than 90% were allowed to participate in the protocol and followed the schedule of quality assurance (QA) activities recommended by the testing program. Those pathologists who scored between 80% and 90% could participate but were required to take part in remediation activities. Pathologists who scored less than 80% could not participate in the protocol until remediation was complete. Remediation was tailored to the site but always included the specific web-based educational activities provided by either CAP or RCPA.

Cytology and Histology Specimen Adequacy and Concurrence (Scheme II)

An independent expert (P.M.M.) with significant expertise in the evaluation of cervical pathology in HIV-infected women undertook review of locally collected and prepared specimens. Local guidelines for the collection and preparation of specimens were used rather than introducing potentially unsustainable new practices. Laboratories submitted conventional (ie, not liquid-based) cytology slides from five women, preferably HIV infected; at least two slides were to have abnormal cellular results. The central reviewer assessed specimen adequacy and smear preparation and offered a diagnosis based on the Bethesda classification. Discrepancies were adjudicated by a third reader. The agreement between local and central interpretation of cytology specimens was scored in two ways. First, the cytologic results were dichotomized as normal or abnormal (atypical cells of uncertain significance or worse), and the number of concordant specimens was counted. An acceptable result was defined as having four or more concordant results on five separate specimens. In addition, a scoring system, the “modified cytology score,” for discrepant results was developed in which clinically important discordant results were weighted more heavily. A modified cytology score of 60% or greater was acceptable for protocol participation, and less than 60% required remediation prior to interpreting study specimens. Scores between 60% and 75% required remediation, but participation was allowed. The scoring schemes are presented in Tables 2 and 3.

Table 2.

Cytology Grading

Central	Local
	Normal	ASC-US	LSIL	HSIL
Normal	Concordant	Discordant	Discordant	Discordant
ASC-US	Discordant	Concordant	Concordant	Concordant
LSIL	Discordant	Concordant	Concordant	Concordant
HSIL	Discordant	Concordant	Concordant	Concordant

ASC-US, atypical squamous cells of undetermined significance; HSIL, high-grade squamous intraepithelial lesions; LSIL, low-grade squamous intraepithelial lesions.

Table 3.

Cytology Scoring^a

Central	Local
	NILM	ASC-US	LSIL	ASC-H/HSIL	Invasive Cancer
Unsatisfactory	0	0	0	0	0
NILM	20	10	0	0	0
ASCUS	10	20	20	15	10
LSIL	0	20	20	10	10
ASC-H/HSIL	−10	10	10	20	10
Invasive cancer	−10	0	0	10	20

ASC-H, atypical squamous cells—cannot exclude high-grade squamous intraepithelial lesions; ASC-US, atypical squamous cells of undetermined significance; HSIL, high-grade squamous intraepithelial lesions; LSIL, low-grade squamous intraepithelial lesions; NILM, negative for intraepithelial lesions or malignancy.

^aCytology scoring reflects clinically important differences.

The histology assessment was similar to the cytology assessment described above. One histology slide from each of five separate colposcopies, in which one or more cervical biopsy specimens were obtained, was submitted along with the corresponding site-interpreted cervical cytology result. At least two of the five cases were to have had CIN2 or greater on histology. The central expert assessed slides for adequacy of specimen collection and slide preparation and submitted a diagnosis based on commonly accepted criteria. The local interpretation was categorized as concordant versus discordant as above and was given a weighted score reflecting clinically important differences, referred to as the “modified histology score.” A score of 80% was considered acceptable and allowed protocol participation. Pathologists who scored less than 80% were not allowed to participate until they obtained a passing score.

All laboratories were provided the results of the central review, and specific feedback was given. All laboratories are required to participate in EQA activities and provide study slides for interpretation every six months. For those laboratories that did not achieve acceptable scores, remediation plans were developed that were tailored to the particular site and could include being asked to submit more samples for the prequalifying round and ongoing remediation with more frequent on-study EQA. When discrepancies occurred, corrections with specific teaching points were provided to the sites. An example is provided in Fig.1.

Figure 1.

A Local result: atypical squamous cells of undetermined significance/inflammation. Central result: high-grade squamous intraepithelial lesion (HSIL). Although this smear is degenerate and poorly stained, parabasal and metaplastic cells with a very high nuclear cytoplasmic ratio are noted (arrow). At the thin arrow is a syncitia of squamous cells, with ill-defined cell borders, showing HSIL. The nuclear borders are irregular and the chromatin is hyperchromatic. These cytologic features are consistent with HSIL. At the thick arrow is koilocyte (low-grade squamous intraepithelial lesion)–intermediate squamous cells with a well-defined perinuclear halo and enlarged nucleus. Image 1B and C: Metaplastic cells, some of which have a higher nuclear: cytoplasmic ratio with irregular, hyperchromatic nuclei (arrow) most in keeping with atypical squamous cells- cannot exclude HSIL (ASC-H) although the local result was negative. Image 1D: A syncitium of squamous cells with ill-defined cell borders and enlarged, hyperchromatic nuclei with irregular nuclear borders and occasional longitudinal grooves (arrow) consistent with HSIL, although the local result was negative.

HPV DNA Testing (Scheme III)

The DNA test used for HPV testing was the Abbott m2000 Real Time High-Risk HPV DNA assay (Abbott Molecular), which tests 14 high-risk HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68). The test is an automated real-time polymerase chain reaction (PCR) assay with internal controls. The DAIDS-sponsored Virology Quality Assurance (VQA) program was asked to develop an external quality control validation panel that was prepared from dilutions of the Abbott High-Risk HPV DNA kit.

The DAIDS-sponsored VQA program developed a validation panel. For the panel, Abbott provided multiple vials of HPV-positive and HPV-negative kit controls and HPV Cervicollect medium as a diluent. C33A cells (HPV-negative cervical epithelial cells) were added to the diluent and to the HPV-positive and HPV-negative controls to achieve a final concentration of approximately 100,000 cells/mL. The diluent (with cells) was then used to dilute the HPV-positive kit control 1:5, 1:10, or 1:20. Over four runs, each laboratory tested the negative control approximately eight times, the positive control (undiluted) eight times, the positive control (1:5) 10 times, the positive control (1:10) 11 times, and the positive control (1:20) 11 times.

Positive and negative controls were provided to each site laboratory. The validation panel was pretested at two different reference laboratories: the Abbott m2000 High-Risk HPV DNA assay was tested at Abbott Molecular Delkenheim Laboratory in Wiesbaden, Germany, and the Roche Linear Array HPV Test (Roche molecular systems, Indianapolis IN, USA) and a liquid bead microassay based on Luminex technology (Luminex Austin TX, USA for HPV genotyping³⁰.y (Austin Tx USA) at the University of Washington, Seattle.

Results

Proficiency Testing (Scheme I)

Seven laboratories and 16 pathologists in Africa, India, and South America participated in the program. Sixteen pathologists from five laboratories used the CAP program purchased by SMILE, and two sites used the RCPA program already in use at their site. Sixteen pathologists at five sites used the CAP program, and two sites subscribed to the RCPA program and submitted consensus results in which multiple pathologists participated. Eight pathologists from five sites had scores of greater than 90%. Five pathologists participating in CAP required remediation but were allowed to participate (scores 80%-90%), and three required substantial remediation, one of whom was ultimately successful. Of the two laboratories using the RCPA scheme, one was fully successful and the second was allowed to participate with remediation.

Central Review (Scheme II)

Cervical smears and biopsy specimens were prepared according to local protocols and were not standardized. Details on the individual laboratories from the prequalifying rounds of EQA are presented in table 6.

Table 6.

Laboratory Scores: Scheme II

	Cytology			Histology
Laboratory No.	Specimen Adequacy/Quality	% Agreement	Modified Score, %	Specimen Adequacy/Quality	% Agreement	Modified Score, %
1	Acceptable	100	90	Specimen collection: 4/5 specimens cautery effect/lack of epithelium; specimen processing: 4/5 suboptimal staining	40	70
2	5/5 poor staining and cellularity	60	30	1/5 insufficient tissue	80	70
3	Acceptable	80	65	2/5 insufficient tissue; repeat: acceptable	40; repeat: 80	50; repeat: 90
4	Acceptable/minor issues	60; repeat: 80	25; repeat: 65	Acceptable	100	100
5	Acceptable/minor issues	0; repeat 1: 40 (failing); repeat 2: 80 (passing)	15; repeat 1: 20; repeat 2: 80	Acceptable	100	80
6	Acceptable	80	90	Acceptable	100	90
7	Acceptable	100	100	Acceptable	80	80

Cytology

Conventional cytology was used since liquid-based smears were not available at the sites. Eleven (31%) of 35 slides examined had problems with smear processing such as suboptimal fixation and air-drying artifact. In six (17%) instances, poor cellularity was noted, and there were insufficient cells for diagnosis in one case. The diagnoses at two sites had good concordance with the central reviewer, two had borderline concordance, and three required remediation. When categorized as normal versus abnormal, overall, there was agreement between the central reader and the local laboratory 71% of the time (90% confidence interval, 42%-98%). Slides with discordant results between the central reviewer and the site had a third reader who concurred with the central reviewer in every instance. The mean modified cytology score for all laboratories was 60%, reflecting suboptimal performance by three laboratories. One laboratory was considered to have borderline performance.

Histology

Nine (26%) of 35 histologic specimens had significant issues with sample collection, including cautery artifact and/or absence of the transformation zone. Nine (26%) of 35 specimens were suboptimally processed. For 28 (80%) of 35 slides, there was concordance between the local and central reviewer. The aggregate mean modified histology score was 81%. Three laboratories had complete concordance with the central reviewer, two had acceptable scores, and two required further remediation.

Virology (Scheme III)

Nine laboratories participated over a two-year period, and all were successful. Specimens were shipped from laboratory 2 to another site for HPV DNA testing. See table 7.

Table 7.

Scheme III Virology Results^a

	2011			2012
Laboratory No.	A	B	C	A	B	C
1	No data	No data	No data	No data	100
2 (sent specimens to a different laboratory)	No data	No data	No data	No data	No data	No data
3	No data	No data	No data	No data	100	100
4	No data	No data	100	100	100
5	No data	No data	No data	No data	100
6	No data	100	100	100	100
7	No data	100	100	100	100

Values are presented as percentages.

Remediation

Improvement plans were specifically tailored to the site and were separate for EQA schemes I and II. For scheme I, among those sites participating in the CAP Pap proficiency testing (PT) program, only one laboratory did not require remediation; however, during the follow-up round of EQA, this laboratory achieved only provisional approval to participate in the protocol, thus emphasizing the requirement for continuous quality improvement as part of protocol execution. The CAP program provides web-based educational activities, and all pathologists with scores less than 90% were required to participate in this program. Additional intensive training activities were provided to selected sites by the central reviewer. Of the sites participating in the RCPA program, one site required remediation. Pathologists who scored less than 80% were not allowed to participate, and those with scores between 80% and 90% could participate as long as remediation occurred.

For scheme II, no issues were identified in two of the seven laboratories. Four of those requiring remediation needed technical support for collection and staining procedures. In all five laboratories requiring remediation for scheme II, there were significant diagnostic discrepancies between the site and the central reader; these pathologists were provided specific and individual feedback,. During the conduct of the study, ongoing EQA will be performed, with these sites having intensive oversight. The remediation plans for cytology and histology are presented in Table 8.

Table 8.

Remediation Requirements

Laboratory No.	Remediation Cytology	Remediation Histology
1	Review cytology collection procedures	Loop excision procedures and staining protocol reviewed
		Reagents identified and used
		Submit five additional slides
2	Collection and staining techniques reviewed	Remediation plan required
	Submit five additional slides	Plan not fully articulated, will include quarterly follow-up
	Protocol team to send illustrative pictures and additional feedback
3	Submit five additional slides	Review procedure for sectioning tissue blocks
	Quarterly follow-up	Submit five additional slides
		Protocol team to give additional feedback.
4	Submit five additional slides	None required
	Protocol team to send illustrative pictures and additional feedback
	Quarterly follow-up
5	Submit five additional slides	None required, although poor staining noted and histology staining protocols shared
	Protocol team to send illustrative pictures	Digital photographs shared
	Quarterly follow-up	Site visit being considered
6	No issues	No issues
7	No issues	None required
	None required	Protocol team recommends that “insufficient biopsy material” be communicated to the site with a plan for rebiopsy
		One sample with insufficient specimen, communicated to site

Discussion

To our knowledge, this is the first report of a comprehensive assessment of protocol prequalifying EQA for the investigation of HPV-related disease at selected ACTG international clinical trial sites. Our approach of certifying research site laboratories, rather than using a central diagnostic laboratory for making the diagnosis of HPV-related cervical disease, allows for in-study performance monitoring and is a powerful educational tool for strengthening HPV cytologic and histopathologic diagnostics. The EQA is an ongoing and regular process that will continue for the duration of the study, and it is expected that gains made for the A5282 clinical trial will be translated into improved clinical diagnostics at the trial sites.

This pre-study certification program identified important factors that compromised specimen evaluation before the specimen reached the laboratory. For example, a laboratory with poorly preserved biopsy specimens was noted to be using formalin that had expired. A faulty colposcope and suboptimal biopsy equipment resulted in poorly directed biopsies at another site. Specimen preparation was not standardized, and there were common, easily modified errors in specimen preparation, such as length of fixation and standardized formalin that improved slide quality. Other resources for collection and processing of specimens were identified and provided to improve the existing processes. This included expert advice on biopsy collection as well as the purchase of new colposcopes for a number of sites. In some instances, referral patterns were established for both clinical and pathologic consultation. Although these modifications occurred within the context of the study, these improvements will likely have a significant impact on clinical care at the study sites.

Accurate diagnosis is essential for the conduct of clinical trials. There is an expectation that laboratory results will be comparable and reproducible within and between laboratories. A common approach in cancer and cancer prevention trials is to verify each diagnosis with a central review. Our analysis of the prequalifying EQA program has emphasized that in our environment, central review would need to occur in near real time with discordant results being adjudicated and transmitted to the patient to make clinical management decisions. This would present significant logistic and methodologic difficulties for the protocol team. Most important, however, the team and sponsors felt strongly about the need to build or improve capacity at each site for both research and clinical care. For this reason, laboratories rather than clinical diagnoses were certified. It is expected that a significant number of specimens will have central review during the EQA process; this is being monitored by both the protocol team and the data safety monitoring board. It is likely that we will be able to verify the wisdom of the approach of monitoring laboratories rather than diagnoses both during the trial and in aggregate at the end of the trial. The EQA process is a powerful educational tool and complements other ongoing quality improvement efforts in the ACTG Network Laboratory program. We expect that substantial gains in the laboratory diagnosis of HPV-related cervical disease will continue to be made with other simple interventions.

Finally, cervical disease is the most common HPV-related cancer in HIV-infected individuals, but there is a growing literature suggesting that HPV-related cancers at other anatomic sites may become important. For example, a strong association between HIV and anal cancer has been documented in the United States, with case rates in HIV-infected men 45 to 170 times that of age- and risk-matched populations.^31, As the life expectancy for HIV-infected individuals improves because of more universal access to effective antiretroviral therapy, it is likely that these cancers will become more important.³² It is therefore prudent to ensure that the clinical and laboratory infrastructure is in place for accurate diagnosis and treatment of these conditions.

Table 1.

Program Description

Scheme	Description
I	Cytology proficiency testing (procured commercially)
II	Assessment of clinical material from each site
III	Virology (human papillomavirus DNA testing)

Table 4.

Histology Grading^a

Central	Local
	Normal	CIN1 (Low-Grade CIN)	CIN2 (High-Grade CIN)	CIN3 (High Grade)
Normal	Concordant	Concordant	Discordant	Discordant
Low-grade CIN (CIN1)	Concordant	Concordant	Discordant	Discordant
High-grade CIN (CIN2)	Discordant	Discordant	Concordant	Concordant
High-grade CIN (CIN3)	Discordant	Discordant	Concordant	Concordant

CIN, cervical intraepithelial neoplasia.

^aNote that a reading called “normal” and “CIN1” was considered concordant, reflecting the understanding of the pathophysiology. Modified grading schemes: The schemes were developed to reflect gradations in interpretation and to attach greater importance to clinically important diagnoses. There were separate schemes for cytology and histology, and pathologists and cytotechnologists have different scoring grids.

Table 5.

Histology Scoring

Central	Local
	NILM	CIN1 (Low-Grade CIN)	CIN2/3 (High-Grade CIN)	Invasive Cancer
NILM	20	10	−10	−10
CIN1	10	20	10	−10
CIN2/3	−10	10	20	0
Invasive cancer	−10	−10	0	20

CIN, cervical intraepithelial neoplasia; NILM, negative for intraepithelial lesions or malignancy.