Abstract
Introduction:
The introduction of the protein-specific antigen (PSA) test in care means that prostate cancer (PCa) is being detected earlier and more frequently. The result of increased screening using PSA, digital rectal examination and awareness of prostate was an increase in the number of men with low risk cancers. Active surveillance has become a viable alternative to immediate treatment with surgery, radiation and other forms of localized treatment. Evidence suggests that there is no significant difference in mortality rates between AS and surgery. In addition, patients may potentially delay other complications associated with surgery, radiation or deprivation therapy.
Methods:
This quality improvement study using a delivery system science framework describes the electronic identification of AS candidates given an evidence-based identification criteria based upon national guidelines and clinical judgement. The study population (n=649) was drawn from health records of all patients who received a prostate biopsy (n=1731) at Intermountain Healthcare from 1/1/2013 to 12/31/2014. Automated and manual abstraction was performed. Receiver operating characteristic (ROC) analysis was used to compare AS-eligible patients to the actual treatment received to identify potential care improvement opportunities.
Among those with complete data, 24.7% of this population (n=160) met “AS-eligible” criteria. 39.1% of the population had not received surgery, radiation or androgen deprivation therapy and were considered as being treated using an AS approach. 9% of AS-eligible patients did not receive AS; 27% of patients who did not meet AS-eligible criteria received AS. Estimated guideline adherence measured using area under the curve was 0.70 (95% CI: 0.66–0.73). Modest variation in criteria parameters for identifying AS-eligible patients did not significantly change estimated adherence levels.
Conclusion:
Implementation of evidence-based criteria for detection of AS candidates is feasible using electronic health record data and provides a reasonable basis for delivery system evaluation of practice patterns and for quality improvement.
Keywords: Cancer, Population Health, Quality Measurement
Introduction
Approximately 80 percent of prostate cancer (PCa) detected today is slow growing and unlikely to spread.1 Despite this, the majority of men with prostate cancer receive immediate treatment with radiation or surgery.2 These therapies introduce complications including impotence and urinary incontinence.3 The introduction of the prostate-specific antigen (PSA) test and other developments in care mean that PCa is being detected earlier and more frequently, leading to potential overdiagnosis and excess invasive treatment.4 Earlier diagnosis of often low-risk cancer increases the viability of an active surveillance (AS) strategy as an alternative to immediate radiation therapy, surgery, or androgen deprivation therapy (ADT).5
An AS strategy is designed for men who have low or very low risk of clinically localized cancer.6 For eligible patients, AS involves monitoring the disease over time through prostate-specific antigen (PSA) testing, digital rectal exam, and biopsies. The patient remains on AS unless active, meaningful change in the course of the disease requires that the patient pursue active treatment options.7 While no definitive criteria exist for measuring a change in disease course, increasing PSA score, increased aggressiveness of the disease as defined by an increased Gleason score, and increased volume defined by repeat biopsy showing more cores positive than previously detected are common evaluation criteria.8 New modalities, including genetic biomarkers are also being explored as predictors of disease progression and aggressiveness.9,10
Evidence suggests that there is no significant difference in mortality rates between AS and surgery though study follow-up is limited.11–13 In addition, patients who select AS may potentially delay or avoid other complications associated with surgery, radiation, or ADT including sexual dysfunction and incontinence.14 These benefits are offset by the potential for increased anxiety, progression of the disease beyond being curable, and reduced quality of life, though limited studies on these effects exist among AS patients.8 Between 20–30 percent of patients will go on to more invasive treatment within two to three years.15 Evidence suggests that anxiety—and not biochemical progression or other clinical indicators—may be the ultimate reason patients elect invasive treatment.15
Overall, given the potential benefits, the 2011 United States National Institutes of Health (NIH) Consensus and State-of-the-Science Conference concluded that AS should be offered to patients with low-risk PCa. This was reaffirmed in 2014 in National Comprehensive Cancer Network (NCCN) guidelines including AS for very-low and low-risk candidates.6 Evidence is scant on methods for assessing health system–level adherence to NCCN guidelines as a basis for practice evaluation and quality improvement. This quality improvement study uses an evidence-based detection criteria to describe the electronic identification of AS candidates. Implementation of an evidence-based criteria for detection of AS candidates can provide a reasonable basis for evaluation of practice patterns and for quality improvement.
Methods
A retrospective cohort study design was used to evaluate AS eligibility. The study population (n=649) was drawn from health records of all patients who received a prostate biopsy (n=1731) at Intermountain Healthcare (IH) from January 1, 2013 to December 31, 2014. Follow-up treatment was measured through June 30, 2015. Final patient study cohort selection is summarized in Figure 1.
Figure 1.
Final Patient Study Cohort Selection
No broad agreement exists on a standard for identifying patients with insignificant or low-risk PCa though measurement criteria are quite similar. Primary measurement factors include clinical stage, PSA, PSA density, PSA velocity, Gleason pattern score, and number of positive cores.9 Patient selection criteria is based upon evidence-based protocols consistent with national guidelines including patient age, recent PSA test value, Gleason score, and positive cores.2,6 Criteria to identify candidates for surveillance was based upon the criteria noted in Table 1.
Table 1.
Intermountain Healthcare Selection Criteria for Active Surveillance (AS) Candidates
| CRITERIA | STANDARD |
|---|---|
| Patient age | >55 years of age |
| Prostate-Specific Antigen (PSA) test value from most recent lab | <10 ng/mL |
| Gleason score | <7 on each side |
| Cores containing adenocarcinoma | 3 or fewer on each side |
Notes: Prostate-specific antigen (PSA) is a protein produced by prostate gland cells. The PSA test measures PSA in a man’s blood. Results are reported in nanograms of PSA per milliliter (ng/mL)
Source: National Canter Institute, 2012.
Three primary data sources were used to identify the population and were drawn from the Enterprise Clinical Data Warehouse (EDW). Pathology tables documented the results of all biopsies including Gleason Score and positive cores. Lab tables included the most recent PSA results prior to the biopsy date. Patient demographic and other health care characteristics were drawn from the patient medical record. Automated abstraction was performed using both structured and unstructured data extraction methods to capture selection criteria by patient. PSA results and patient demographic data were pulled from structured tables in the EDW. The pathology tables were drawn from the Intermountain Pathology system PowerPath, an externally vended product from Sunquest. Pathology results are stored in the PowerPath system and imported into the Intermountain EDW in both structured and unstructured (text field) formats. Unstructured data is stored in the EDW as a “character large object” (CLOB) data type. Extraction of unstructured data from the CLOB required utilization of pattern matching methods to identify desired data elements. Specific patterns that were detected in the unstructured data included “Primary Pattern,” “Secondary Pattern,” “Total Gleason Score,” “Total All Cores,” and “Total Cores” including variations in the above patterns. Further manual abstraction from the medical record was necessary to ensure complete identification of selection criteria for the study population.
Actual treatment received was extracted for patients with a cancer diagnosis and complete data (n=649) using a combination of the International Classification of Disease Ninth Edition (ICD-9) diagnosis (185) and procedure codes (605, 606, 1742) to identify subsequent prostatectomy or other related surgery. Patients receiving radiation therapy subsequent to the diagnosis were identified using ICD-9 diagnosis code (185) and transaction-level charge codes for radiation therapy. Patients receiving surgical ADT were identified from the ICD-9 orchiectomy procedure codes (62.30, 62.40–62.42). Patients receiving pharmacological ADT were identified based upon prescribed medications, including luteinizing hormone-releasing hormone (LHRH) agonists, LHRH antagonists, antiandrogens, and androgen-synthesis inhibitors. The presence of a subsequent surgery, radiation, or ADT was measured through June 30, 2015—providing all patients with a minimum of six months from biopsy date to identify more invasive procedures performed subsequent to the study period. For all patients not receiving surgery or radiation (n=258), a manual chart review was conducted to confirm actual patient treatment. Manual health-record abstraction was conducted, using Intermountain’s electronic medical record system, by two trained research assistants within the Institute for Healthcare Leadership. A subset of manually abstracted data was tested for accuracy by comparing manually abstracted data against the patient medical record. No significant differences were noted.
Descriptive statistical methods were used to establish baseline characteristics of the population. To assess the effect of variation in criteria on measured adherence levels, receiver operating characteristic (ROC) analysis was used to evaluate guideline adherence by comparing the results of the use of detection criteria (test result) with the actual treatment received (actual result or gold standard). Sensitivity analysis was conducted to understand the effect of changes to the detection criteria on estimated adherence levels by comparing the “area under the curve” (AUC) of various scenarios. All statistical analysis was performed using Stata statistical software (version 13.0; StataCorp, College Station, Texas).
Results
Baseline characteristics of the study population are noted in Table 2.
Table 2.
Baseline Characteristics of the Study Population
| N=649 | % TOTAL | ||
|---|---|---|---|
| Age | Mean | 65.99 | |
| Median | 66.00 | ||
| SD | 8.94 | ||
| Min | 42.00 | ||
| Max | 98.00 | ||
| Age category | <=45 | 5 | 0.77% |
| 46–50 | 22 | 3.39% | |
| 51–55 | 60 | 9.24% | |
| 56–60 | 85 | 13.10% | |
| 61–65 | 128 | 19.72% | |
| 66–70 | 152 | 23.42% | |
| 71–75 | 113 | 17.41% | |
| >75 | 84 | 12.94% | |
| Gleason score | Mean | 6.84 | |
| Median | 7.00 | ||
| SD | 0.99 | ||
| Min | 1.00 | ||
| Max | 10.00 | ||
| Gleason category | 5 or less | 6 | 0.92% |
| 6 | 242 | 37.29% | |
| 7 | 300 | 46.22% | |
| 8 | 38 | 5.86% | |
| 9 | 62 | 9.55% | |
| 10 | 1 | 0.15% | |
| PSA score | Mean | 27.40 | |
| Median | 6.54 | ||
| SD | 248.24 | ||
| Min | 0.01 | ||
| Max | 6107.00 | ||
| PSA category* | <=5 | 197 | 30.35% |
| >5–10 | 263 | 40.52% | |
| >10–15 | 69 | 10.63% | |
| >15–20 | 35 | 5.39% | |
| >20 | 77 | 11.86% | |
| Unknown | 8 | 1.23% |
Notes:
Patients without a PSA test result may still have a cancer diagnosis and may have been excluded as an AS candidate based upon age or Gleason criteria.
Excluding patients with a PSA score>100 (n=15), the mean PSA was 10.45 (SD 13.03).
Of the final study population, 24.7 percent (n=160) met the criteria for AS and were considered “AS eligible.” And, 39.1% of the population had not received surgery, radiation, or ADT and were considered as being treated using an AS approach. Overall estimated adherence measured using AUC was 0.70 (95 percent CI: 0.66–0.73). In the study sample, 36 patients who met the eligibility criteria for AS had surgery, radiation, or ADT during the study period (false positive rate = 0.09). Separately, 130 patients (false negative rate of 0.51) who did not meet eligibility criteria for AS received AS treatment.
Changes in identification criteria including adjustments for age and PSA score led to an increase in observed AUC but differences were not significant, as noted in Table 3. Adjusting to include patients over age 75 as receiving some form of observational treatment (no surgery, radiation, or ADT), the false positive rate increased to 0.19 (77/395). The false negative rate fell to 0.37 (95/254) but remained high.
Table 3.
Adherence Results for Active Surveillance (AS) by Eligibility Criteria
| SCENARIO # | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| MEASURES | BASELINE AS ELIGIBILITY CRITERIA | BASELINE CRITERIA, EXCLUDING PATIENTS >75 | BASELINE CRITERIA, WITH PATIENTS >75 AS AS | BASELINE CRITERIA, WITH PATIENTS >75 AS AS; PATIENTS 51–55; PSA<=20 | BASELINE CRITERIA, WITH PATIENTS >75 AS AS; PATIENTS 51–55; NO PSA | BASELINE CRITERIA, WITH PATIENTS >75 AS AS; PATIENTS 51–55; PSA<=20; GLEASON <8 |
| True positive | 124 | 116 | 159 | 175 | 176 | 194 |
| False positive | 36 | 36 | 77 | 90 | 91 | 135 |
| False negative | 130 | 95 | 95 | 79 | 78 | 60 |
| True negative | 359 | 318 | 318 | 305 | 304 | 260 |
| Total population | 649 | 565 | 649 | 649 | 649 | 649 |
| % AS eligible | 24.7% | 26.9% | 36.4% | 40.8% | 41.1% | 50.7% |
| % treated with AS | 39.1% | 37.3% | 39.1% | 39.1% | 39.1% | 39.1% |
| Sensitivity | 0.49 | 0.55 | 0.63 | 0.69 | 0.69 | 0.76 |
| Specificity | 0.91 | 0.90 | 0.81 | 0.77 | 0.77 | 0.66 |
| Accuracy | 0.74 | 0.77 | 0.73 | 0.74 | 0.74 | 0.70 |
| False positive rate | 0.09 | 0.10 | 0.19 | 0.23 | 0.23 | 0.34 |
| False negative rate | 0.51 | 0.45 | 0.37 | 0.31 | 0.31 | 0.24 |
| PPV | 0.78 | 0.46 | 0.67 | 0.66 | 0.66 | 0.59 |
| NPV | 0.73 | 0.77 | 0.77 | 0.79 | 0.80 | 0.81 |
| Area under curve | 0.70 | 0.72 | 0.72 | 0.73 | 0.73 | 0.72 |
| 95% CI Range-LL | 0.66 | 0.69 | 0.68 | 0.70 | 0.70 | 0.68 |
| 95% CI Range-UL | 0.73 | 0.76 | 0.75 | 0.77 | 0.77 | 0.75 |
A limited review of characteristics of patients 75 years old and younger who did not accept recommended treatment examined mean age, PSA score, and Gleason score. Eligible patients who did not elect AS had similar mean PSA- and Gleason scores but were significantly younger. Patients not eligible for AS who elected AS were older with significantly higher mean PSA scores and significantly lower mean Gleason score as noted in Table 4.
Table 4.
Preliminary Characteristics of Patients Not Electing Treatment Consistent with Active Surveillance (AS) Guidelines—Patients 75 Years Old and Younger
| AMONG PATIENTS AS ELIGIBLE | AMONG PATIENTS NOT AS ELIGIBLE | |||||
|---|---|---|---|---|---|---|
| ELECT AS | NOT ELECT AS | SIG* (P<.05) | ELECT AS | NOT ELECT AS | SIG* (P<.05) | |
| Patient count | 116 | 36 | 95 | 318 | ||
| Mean Age (SD) | 66.2 (5.5) | 62.8 (4.2) | <.001 | 64.5 (8.1) | 62.9 (7.6) | 0.07 |
| Mean PSA (PSA<25) (SD) | 5.2 (2.4) | 5.4 (2.2) | 0.70 | 8.7 (4.7) | 7.4 (4.6) | 0.03 |
| Mean Gleason Score (SD) | 6.0 (0.0) | 6.0 (0.2) | 0.07 | 6.7 (0.9) | 7.2 (0.8) | <.001 |
Note:
Two-sided t-test comparison; significant at p<.05.
Discussion
Electronic identification of potential AS candidates using electronic health record data for AS program evaluation is promising, though challenges remain. Comparing eligible AS patients against actual treatment received provides a basis for understanding the current state of application of AS guidelines within an integrated health care system. This analysis provides a basis for evaluating the effect improvements may have in the adoption of AS protocols, including such things as the use of decision aid tools to promote AS, and impacts on the quality and cost of prostate care.
One in four patients (25 percent) with prostate cancer in the study population were identified as AS eligible. Estimates of the percentage of men eligible for AS are difficult to pinpoint and range from 7.0 percent to 69.1 percent in PSA-screened American men undergoing radical prostatectomy.16 A separate United States–based prostate cancer registry suggests 36 percent of men with PCa are low risk.17 Variation is driven by several factors including the restrictiveness of eligibility criteria used to identify candidates and differences in population characteristics.
The actual AS treatment level during the study period was estimated at 39.1 percent, which is higher relative to current national estimates of the number of men that elect AS of about 10 percent.2 Several factors may explain this finding. IH has been recommending AS in men with low risk prostate cancer for over eight years, with patients increasingly accepting AS as an option. Another factor that might impact this result is that information regarding subsequent surgery or radiation was performed outside our care delivery network and may not be complete. Extending the lag period to determine actual treatment by six months was designed to ensure that patients diagnosed near the end of the study period had complete information. However, patients receiving follow-up surgery, radiation, or ADT outside the Intermountain system may not be fully captured.
Overall 78 percent of patients who met AS criteria received AS treatment. Several factors may explain this result. Misclassification of insignificant PCa using more common AS-selection criteria ranges from 22 percent to 33 percent in other studies.16 This suggests that the majority of false positive and false negative results are due to inherent misclassification risk in the criteria. In our study, altering criteria standards did not meaningfully change these results. Another principal driver may result from the preference-sensitive nature of prostate treatment including the bias toward traditional invasive approaches to care. However, given this, we would anticipate a relatively high level of false-positive results (patients eligible for AS who elect more invasive treatments). The false positive level using the baseline eligibility criteria was 9 percent.
False negative results had a greater impact. This includes patient not eligible for AS that received AS. Even after relaxing criteria for age to include patients ages 51–55 and to include higher PSA levels, the false negative rate remained relatively high at 31 percent. This may be explained in part if a patient was originally diagnosed at IH but received follow-up care from another health system.
Age stratification remains an important consideration in the implementation criteria for detection of eligible AS patients. Given that PCa is slow growing, PCa is very common in men over age 75. As a result, watchful waiting is recommended for patients in this age category with no invasive procedures or further PSA testing recommended. While both AS and watchful waiting are observational treatment strategies, they represent two meaningfully different treatment approaches. For measurement purposes, baseline results were presented both excluding patients >75 years old as not eligible for AS and including them as observation candidates. Examining these older patients as observation eligible allows for evaluation of practice patterns in this age cohort. For example, in this study sample, 28 patients over the age of 75 received either surgery or radiation treatment. Age criteria can also be useful in examining patients who may be receiving unnecessary PSA testing.
In terms of data collection, while a single factor could rule out an AS candidate, all criteria were needed to effectively rule in a patient as an AS candidate. Of the 811 patients receiving a positive prostate biopsy during the study, 162 were excluded from the final study due to missing or incomplete data. The most common missing data was a PSA test value preceding the biopsy date. The primary reason for missing PSA test data is that the physician may have noted that the patient had an elevated PSA test performed previously but did not include a specific PSA value in the electronic medical record. This would generally result if the PSA was obtained from a non-IH facility. Clear identification of the count of positive cores was also difficult given how the pathology results are reported. Despite a standard reporting system, pathologists report final diagnosis in several different ways that may exclude information on positive cores. A pathologist may report that a certain percentage of the total samples taken was positive, but not provide a breakdown by cores. Other approaches include identifying a proportion of chips taken that are positive, identifying adenocarcinoma length, or reporting the aggregate right or left result only as cancerous without specifying core results.
Conclusion
Implementation of an evidence-based criteria for detection of AS candidates is feasible and provides a reasonable basis for evaluating practice patterns and for quality improvement. Further study is needed to understand variation in practice patterns for treatment of PCa in an integrated health care system, including understanding barriers to implementation of treatment guidelines. Improvements in detection criteria should continue to be evaluated including the use of new molecular biomarkers that may have an impact on the AS recommendations and acceptance in the ages 50–70 years old patient population.10
Footnotes
Disciplines
Bioinformatics | Medicine and Health Sciences
References
- 1.Epstein JI, Walsh PC, Carmichael M, Brendler CB. Pathologic and clinical findings to predict tumor extant of nonpalpable (stage I1c) PCa. JAMA. 1994;271(5):368–374. [PubMed] [Google Scholar]
- 2.Ganz P, Barry J, Burke W, et al. National Institutes of Health State of the Science Conference: Role of AS in the management of men with localized PCa. Annals of Internal Medicine. 2012;156:591–595. doi: 10.7326/0003-4819-156-8-201204170-00401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ganz PA, Barry JM, Burke W, Col NF, Corso PS, Dodson E, Hammond ME, Kogan BA, Lynch CF, Newcomer L, Seifter EJ, Tooze JA, Viswanath K, Wessells H. National Institutes of Health State-of-the-Science Conference Statement: Role of AS in the Management of Men With Localized PCa. NIH Consens State Sci Statements. 2011 Dec 5–7;28(1):1–27. [PubMed] [Google Scholar]
- 4.Etzioni R, Penson DF, Legler JM, et al. Overdiagnosis due to prostate-specific antigen screening: lessons from US prostate cancer incidence trends. J Natl Cancer Inst. 2002;271:368–374. doi: 10.1093/jnci/94.13.981. [DOI] [PubMed] [Google Scholar]
- 5.Dall’Era MA, Albertson PC, Bangma C, et al. AS for prostate cancer: a systematic review for the literature. Eur Urol. 2012;62:976–983. doi: 10.1016/j.eururo.2012.05.072. [DOI] [PubMed] [Google Scholar]
- 6. National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines in Oncology _PCa. Version 2 - 2014. Available at www.nccn.org/patients/.
- 7.Lund L, Svolgaard N, Poulsen MH. Prostate cancer: a review of active surveillance. Research and Reports in Urology. 2015;6:107–112. doi: 10.2147/RRU.S41653. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bastian PJ, Carter BH, Bjartell A, Seitz M, Sanislaus P, Montorsi F, Stief CG, Schroder F. Insignificant PCa and AS: from definition to clinical implications. Eur Urol. 2009;55(6):1321–1330. doi: 10.1016/j.eururo.2009.02.028. [DOI] [PubMed] [Google Scholar]
- 9.Bangma CH, Valdagni R, Carroll PR, et al. Active surveillance for low-risk prostate cancer: developments to date. European Urology. 2015;67:646–648. doi: 10.1016/j.eururo.2014.11.004. [DOI] [PubMed] [Google Scholar]
- 10.Ross-Adams H, Lamb AD, Dunning MJ, et al. Integration of copy number and transcriptomics provides risk stratification in prostate cancer: A discover and validation cohort study. EBioMedicine. 2015 doi: 10.1016/j.ebiol.2015.07.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Klotz L. AS with selective delayed intervention for favorable risk PCa. Utol Oncol. 2006;24:46–50. doi: 10.1016/j.urolonc.2005.07.002. [DOI] [PubMed] [Google Scholar]
- 12.Carter HB, Ketterman A, Warlick C, et al. Expectant management of PCa with curative intent: an update of the Johns Hopkins experience. J Urol. 2007;178:2359–64. doi: 10.1016/j.juro.2007.08.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Dall’Era MA, Konety BR, Cowan JE, et al. AS for the management of PCa in a contemporary cohort. Cancer. 2008;112:2664–70. doi: 10.1002/cncr.23502. [DOI] [PubMed] [Google Scholar]
- 14.Sanda MG, Dunn RI, Michalski J, et al. Quality of life and satisfaction with outcome among prostate-cancer survivors. NEJM. 2008;358:1250–61. doi: 10.1056/NEJMoa074311. [DOI] [PubMed] [Google Scholar]
- 15.Latini DM, Hart SL, Knight SJ, et al. The relationship between anxiety and time to treatment for patients with PCa on surveillance. J Urol. 2007;178:826–31. doi: 10.1016/j.juro.2007.05.039. [DOI] [PubMed] [Google Scholar]
- 16.Iremashivili V, Palez L, Manohoran M, Jorda M, Rosenberg DL, Soloway MS. Pathologic prostate cancer characteristics in patients eligible for AS: a head to head comparison of contemporary protocols. Eur Urol. 2012;62:462–468. doi: 10.1016/j.eururo.2012.03.011. [DOI] [PubMed] [Google Scholar]
- 17.Dall’Era MA, Cooperberg MR, Chan JM, et al. Active surveillance for early stage prostate cancer: review of the current literature. Cancer. 2008;112:1650–1559. doi: 10.1002/cncr.23373. [DOI] [PubMed] [Google Scholar]