Intended Versus Inferred Care After PET Performed for Initial Staging in the National Oncologic PET Registry

Abstract

Rationale

The National Oncologic PET Registry (NOPR) collected data on intended management before and after PET in cancer patients. We have previously reported that PET was associated with a change in intended management of about one-third of patients and was consistent across cancer types. It is uncertain if intended management plans reflect the actual care these patients received. One approach to assess actual care received is using administrative claims in order to categorize the type and timing of clinical services.

Methods

2006-2008 NOPR data were linked to Medicare claims for consenting patients age ≥65 years having an initial staging PET for bladder, ovary, pancreas, small cell lung (SCL), or stomach cancers. We determined the 60-day agreement between claims-inferred care and NOPR treatment plans.

Results

4,661 patients were assessed of whom 30-52% had metastatic disease. Planned treatments were about two-thirds mono-therapy, of which 46% was systemic therapy only and one-third combinations. Claims paid by 60 days confirmed the NOPR plan of any systemic therapy, radiotherapy or surgery in 79.3%, 64.7% and 63.6% respectively. Single-mode plans were much more often confirmed: systemic therapy in >85% of patients with ovary, pancreas, and SCL cancers and surgery in >73% of those with bladder, pancreas, and stomach cancers. Intended combination treatments had claims for both in only 28% of patients receiving surgery-based combinations and in 55% receiving chemo-radiotherapy. About 90% of patients with NOPR-planned systemic therapy had evaluation/management claims from a medical oncologist. Age <75 years was associated more often with confirmation of chemotherapy, less often for radiotherapy, but not with confirmation of surgery. Performance status or comorbidity did not explain confirmation rates within action categories, but confirmation rates were higher if the referrer specialized in the planned treatment.

Conclusion

Claims confirmations of NOPR intent for initial staging were widely variable but were higher than previously reported for restaging PET, suggesting that measuring change in intended management is a reasonable method to assess the impact diagnostic tests have on actual care.

Introduction

Optimal care for a patient with a newly diagnosed solid tumor has evolved into a complex, intertwined series of steps often involving multispecialty teams. Advanced medical imaging has an important role in establishing the initial stage and usually precedes definitive treatment.(1)

Since 2001, PET, alone or integrated with computed tomography (hereinafter together referred to as PET), using ¹⁸F-fluorodeoxyglucose has been a service covered by Medicare (and most commercial insurers) for the initial staging of patients with non-small cell lung, colorectal, esophageal and head and neck cancers, as well as lymphoma and melanoma, and it's use has grown rapidly.(2)(3) However, at that time, PET remained a non-covered service for most less-common cancer types.

In 2005, CMS announced a novel coverage mechanism, “coverage with evidence development (CED)”, for the otherwise non-covered cancer types and indications in conjunction with prospective data collection within a registry. (4) In response to the CED requirements, the National Oncologic PET Registry (NOPR) was created and opened for accrual in 2006.(5) We have previously reported results of the impact of PET on intended management by collecting prospective questionnaire data before and after PET stratified by testing indication and by cancer type.(6, 7)

Limitations of the NOPR data include the following: (1) change in planned management is only a surrogate for actual health outcomes and (2) the care actually delivered is not documented.(8, 9) While there are numerous studies of adherence to oral cancer therapies, there are few series that address the concordance between recommended actions and actual care initiated. (10, 11) If the concept of “change in intended management” should be carried forward and applied to future assessments of new imaging techniques, this decision is likely to influenced by evidence of confirmation of actual actions and/or outcomes.

One common approach is to use and analyze administrative claims in order to document the patterns of use and timing of various clinical services. Oncology has been the subject of wide-ranging evaluations using SEER (Surveillance, Epidemiology, and End Results) cancer registry data and Medicare claims.(12)

In this report, we linked NOPR participant's individual identifiers with their Medicare claims. The analytic approach to this linked data-set were stratified by the clinical indication for PET: initial staging, restaging or treatment monitoring. Here we report on intended plan initiation following PET used for staging of newly diagnosed cancer. We have previously reported results for restaging that found moderate agreement between intended and claims-inferred care.(13) We hypothesized that confirmation rates would be higher than for restaging since prior treatments would not be a factor. In addition, we assessed patterns of delayed treatment and secondary decision points for combination therapies.

Patients and Methods

NOPR is a prospective data registry (ClinicalTrials.gov #NCT00868582); its operational details, human subject protection procedures, and PET's impact on intended management were previously reported.(6, 14) In brief, the PET facility collects referring physician responses on pre-PET and post-PET forms. The pre-PET form collects the testing indication, cancer type, working stage, performance status, and the referring physician's plans if PET were not available. After PET, the referrer records an estimate of the patient's tumor stage and management plan in light of the PET findings.

Claims Linkage

We linked NOPR data from December 2006 thorough 2008 for consenting participants to their Medicare claim files by matching individual identifiers (DOB, SSN, and gender). We limited our analysis to the seven most frequent cancer types within NOPR. As previously noted, PET was a covered service for more common cancer types, such as non-small cell lung and colorectal cancers. We stratified our analyses by testing indication (initial staging, treatment monitoring or restaging). In addressing initial staging, we excluded kidney and prostate cancer patients because of our prior work showing poor detection of oral chemotherapy claims for kidney cancer and explicit non-coverage of initial staging PET for prostate cancer in CMS' 2009 National Coverage Decision.(15)

The analysis dataset of 4,661 was determined after the following exclusions: patients who were age less than 65 years (9.0%); who were HMO participants (8.0%); or for whom we were unsuccessful in linking identifiers (1.5%), if the registry and claim dates for PET differed by >7 days (2.0%), or if the post-PET plan was “other treatment(s)” or additional imaging” because of their low frequencies (1.5% and 3.0%, respectively).

Management categories

Post-PET categories assessed were watching, biopsy, and treatment. Treatment categories were systemic-therapy (chemotherapy or immunotherapy), radiotherapy, or surgery alone or in combination. For combination therapies, NOPR did not record their sequence or concurrence.

Claims definitions and time frame

Supplemental Table 1 documents how we used administrative coding data to classify claims into inferred-care categories.

While prompt treatment for newly diagnosed cancer is optimal, defining acceptable time windows from either initial diagnosis to primary surgical therapy or from surgical therapy to beginning adjuvant therapy is more problematic.(16-18) Median time from diagnosis to surgery or starting neoadjuvant therapy in gastric and pancreas cancer in 2003-2006 were between 20 to 25 days.(17) In a SEER-Medicare analysis from 1999, 81% of patients started adjuvant therapy within 60 days of their surgery.(18) A recent report from the National Comprehensive Cancer Network found an average time of 12 weeks from diagnosis to start of adjuvant chemotherapy in early stage breast cancer.(16) Since we did not have the NOPR participant's date of cancer diagnosis, we used our best estimate of 60 days post-PET for the claims-inferred action window. We also explored an extension of the time-window to 180 days.

Summary stage

The referring physician's impression of the patient's summary stage was abstracted from the post-PET form.

Comorbidity

We determined the Klabunde comorbidity index—derived from inpatient and outpatient physician claims in the 12 months preceding the PET date (which approximates the date of diagnosis)—using a publically available SAS algorithm.(19)

Specialties

The CMS provider part-B taxonomy codes for physician specialty were used to categorize referring providers. If no specialty or non-physician coding was found, then specialty was coded as “other”.

Statistical analysis

The initial analyses treated the claims-inferred care as the reference standard to calculate measures of agreement between treatment plans and claims-inferred actual management, including positive predictive value (PPV), raw agreement, and kappa (chance-adjusted agreement).(20) For the treatments (systemic therapy, radiotherapy, or surgery), agreement was defined as claims for that type action within the interval without considering other treatments; for plans with two or rarely three different therapy modes, we defined agreement as at least one claim paid for two types. The measures of agreement were computed separately by cancer type and compared using chi-squared tests.

To assess the effect of patient, cancer and provider factors in predicting agreement, separate logistic regression models were fit for patients with plans for any systemic therapy, radiotherapy, or surgery only. The outcome was the indicator of agreement with that element of the plan. Calculations were done with PROC LOGIST in Linux SAS version 9.2. This analysis plan was similar to that used in our preceding work done assessing NOPR PET for restaging.(13)

Results

Clinical characteristics

Table 1 summarizes the cancer type, age, performance status, comorbidity scores, referring physician specialty, and pre-PET plan for our 4,661-patient cohort.

Table 1.

NOPR cohort clinical characteristics for PET imaging performed for initial staging.

	Combined	Bladder	Ovary	Cancer type Pancreas	SCL*	Stomach
Patients in cohort, n	4,661	1,178	309	1,250	752	1,172
Age, mean in years (inter-quartile range)	74.8 (72-77)	76.4 (71-81)	73.0 (68-77)	73.7 (68-78)	73.6 (69-77)	75.7 (70-81)
ECOG performance status (%)
0, asymptomatic	26.6	32.8	28.2	23.4	19.7	27.7
1, Symptomatic, fully ambulatory	58.9	53.2	60.2	62.8	62.5	57.8
2 or higher	14.5	14.0	11.7	13.8	17.8	14.4
Post-PET summary stage (%)
No residual disease	6.8	9.0	14.9	4.5	0.9	8.8
Local only	30.3	36.9	20.7	30.5	14.4	36.2
Regional (or nodal) diseases	19.8	16.3	17.2	19.4	27.0	19.7
Metastatic disease, single site	11.5	8.2	8.7	15.5	13.4	10.0
Metastatic disease, multiple sites	28.0	26.5	33.3	26.7	42.2	20.2
Not recorded	3.6	3.1	5.2	3.4	2.1	5.1
Comorbidity index score (%)
0	36.3	36.3	55.3	37.8	25.0	37.0
1 or 2	46.9	42.3	36.6	49.3	56.1	45.8
≥3	16.8	21.4	8.1	13.0	18.9	17.2
Referring physician specialty (%)
Medical Oncology	55.1	55.9	57.0	54.9	54.9	54.2
Internal Medicine †	11.2	5.2	5.8	9.8	19.8	14.4
Radiation Oncology	9.0	12.5	1.3	11.5	11.0	3.5
Surgery	7.2	0.9	2.9	10.3	2.5	14.3
Urology	3.5	13.7	0.3	0.1	0.0	0.1
Gynecology/Gynecologic Oncology	1.5	0.3	19.7	0.2	0.0	0.2
Other	12.5	11.5	12.9	13.1	11.7	13.3
60-day post-PET mortality(%)	10.0	6.8	4.8	12.6	12.8	10.0
Pre-PET plan, %
Watching	1.9	2.4	3.2	1.7	1.0	2.0
Additional imaging	36.2	36.2	40.8	32.1	41.6	36.0
Biopsy	8.0	9.5	6.8	9.6	4.8	7.2
Treatment	53.9	52.0	49.2	56.6	52.7	54.9

^*SCL: small cell lung

^†Internaedcne sum of all subspecialties otherthan hematologyand oncology

The summary stage distribution varied across cancer types, and sometimes differed from national SEER averages, suggesting that the NOPR patients were selectively referred for PET.(21) When patients judged to have no evidence of residual disease (e.g., if PET was done after a primary resection) are combined with those judged to have only local disease, NOPR patients with ovarian, pancreas and stomach cancers had modestly higher rates (35-45%) of local disease than their SEER cancer-specific averages. Metastatic disease at initial staging was judged present much more commonly for bladder cancer (35%) than in SEER. Sixty-day mortality overall was 10% ranging from 5% in ovarian to 13% in SCL and pancreas cancers.

Referring physicians were identified as medical oncologists in slightly more than half of patients, were >10% surgeons in pancreas and stomach cancer patients, and >10% radiation oncologists in bladder, pancreas and SCL cases.

NOPR plan

After PET, staging appeared to be clinically complete in 90% (that is, only 10% had plans for imaging or biopsy). Excluding that 10% of patients, the intended plan was new or additional treatment in 82.9% of patients (range 79-91%) and watching in 9.8% (Table 2). Modest frequency variations by cancer type were noted in the planned treatment (of any type), watching, or biopsy.

Table 2.

NOPR intended management plan after PET for initial staging by cancer type.

NOPR intended management	All	%	Bladder	%	Ovary	%	Cancer type Pancreas	%	SCL*	%	stomach	%
Patients	4,661		1,178		309		1,250		752		1,172
Watching	457	9.8	149	12.6	41	13.5	99	7.9	38	5.1	130	11.1
Biopsy	341	7.3	101	8.6	24	0.6	90	7.2	28	3.7	98	8.4
Treatment	3,864	82.9	928	78.8	245	79.3	1061	84.9	686	91.2	944	80.6
Treatment details†
Systemic therapy only	1,767	45.7	387	41.7	169	69.0	530	50.0	349	50.9	332	35.2
Radiotherapy only	250	6.5	111	12.0	6	2.4	64	6.0	41	6.0	28	3.0
Surgery only	554	14.3	129	13.9	15	6.1	135	12.7	11	1.6	264	28.0
Combinationtherapies	1,293	33.4	301	32.4	55	22.4	332	31.3	285	41.5	320	33.9
Plans with systemic therapy	3,030	78.4	676	72.8	223	91.0	854	80.5	629	91.7	648	68.6
Plans with radiotherapy	1,293	33.5	337	36.3	23	9.4	362	34.1	321	46.8	250	26.5
Plans with surgery	1,031	26.7	248	26.7	57	23.3	242	22.8	26	3.8	458	48.5

^†Plans with therapy are the sum of plans with single or combination therapy

^*SCL: small cell lung

Planned treatments were single modality in about two-thirds of patients and combination therapies in one-third. The most common treatment plan was systemic therapy alone (45.7%, range 35-69%). Almost all combination-therapy patients had plans including systemic therapy, so that 78.4% (range 69-92%) had systemic-therapy plans in their intended care. Radiotherapy was planned for one-third of patients, usually in combination with chemotherapy or surgery. Surgery plans were equally split between surgery only or surgery plus chemotherapy or radiotherapy. Surgery was planned most frequently in stomach cancer (48%) as opposed to 20% of bladder, ovarian and pancreas cancer patients.

60-day agreement

Table 3 shows the agreement measures by cancer type for NOPR-intended treatments—any systemic therapy, radiotherapy or surgery—and claims-inferred actions at 60 days.

Table 3.

Agreement measurements by type of intended therapy at 60 days after PET.

Post-PET planned therapy and cancer type	NOPR Plan, n	PPV†(%)	95% C.I.	Raw agreement(%)	95% C.I.	Kappa	95% C.I.
Any Systemic Therapy
All	3,030	79.3	77.9-80.8	74.1	72.7-75.4	0.40	0.37-0.43
Bladder	676	79.0	75.9-82.1	73.0	75.6-79.0	0.42	0.36-0.48
Ovarian	223	86.1	81.6-90.6	82.5	78.1-86.9	0.52	0.41-0.64
Pancreas	854	79.3	76.6-82.0	72.8	70.3-75.4	0.33	0.27-0.39
SCL	629	89.5	87.1-91.9	83.8	81.2-86.5	0.34	0.24-0.43
Stomach	648	67.6	64.0-71.2	67.6	64.8-70.4	0.34	0.29-0.40
Any Radiation
All	1,293	64.7	62.1-67.3	80.8	79.6-82.0	0.53	0.51-0.56
Bladder	337	68.8	63.9-73.8	83.3	81.1-85.5	0.60	0.55-0.65
Ovarian	23	30.4	11.6-49.2	91.6	88.4-94.8	0.32	0.12-0.53
Pancreas	362	70.2	65.5-74.9	82.8	80.6-84.9	0.59	0.54-0.64
SCL	321	69.2	64.1-74.2	72.5	69.3-75.8	0.44	0.38-0.51
Stomach	250	48.8	42.6-55.0	79.0	76.5-81.4	0.39	0.32-0.45
Any Surgery
All	1,031	63.6	60.7-66.6	77.9	76.7-79.2	0.43	0.40-0.46
Bladder	248	56.9	50.7-63.0	75.8	73.2-78.3	0.36	0.30-0.42
Ovarian	57	52.6	39.7-65.6	75.9	69.6-79.3	0.31	0.19-0.44
Pancreas	242	59.9	53.7-66.1	77.8	74.1-78.8	0.39	0.33-0.45
SCL	26	50.0	30.8-69.2	85.4	82.6-87.7	0.15	0.06-0.24
Stomach	458	71.4	67.3-75.5	75.7	70.3-75.4	0.50	0.45-0.56

^*SCL:small cell lung

^†PPV: positive predictive value

Agreement: sum of true positive + true negatives

For any systemic therapy, the combined PPV was 79.3% ranging by type >86% in ovarian and SCL, >79% in bladder, and only 67% in stomach cancer. For plans of any radiotherapy, the combined PPV was 64.7% with only modest differences between cancer types. Plans for any surgery had a PPV of 71.2% in stomach cancer and ranged between 50-60% in the other cancer types.

In aggregate, raw agreement between NOPR plans and claims at 60 days was 74.1% for any systemic therapy, 80.8% any radiotherapy, and 77.9% any surgery. Kappas were 0.40 for systemic therapy, 0.53 radiotherapy and 0.43 surgery.

In addition to imperfect PPVs, raw agreement was impacted by claims likely capturing a second decision point. For example, in patients with intended surgery-only plans, there were claims for both surgery and systemic therapy in 19% of bladder, 30% stomach and 38% pancreas cancer patients, likely reflecting appropriate care after surgical staging. Conversely, many systemic-therapy-only plans likely also represented neoadjuvant intent with a deferred surgery decision. In pancreas cancer, 20% (n=105) patients with chemotherapy-only plans had surgery and chemotherapy claims at 60 days.

Single vs. combination therapy and time window

Table 4 shows results of extending the time frame to 180 days for claims confirmation of mono- and combination therapies. The PPVs for chemotherapy only in pancreas, ovarian and SLC patients were noticeably high (85-91%) at 60 days and extending the time window to 180 days increased PPVs by 1-3%. Radiotherapy was most frequently planned together with chemotherapy. In these cases, claims for both treatment types at 60 days were only 47-62%; however, extending the time window to 180 days increased the confirmation index by 7-16%. Cases having surgery-only plans had surgical claims at 60 days in 80% of stomach, and 74% of bladder or pancreas cancers. By 180 days, surgery claims increased an additional 16% in bladder and 6% in pancreas or stomach cancer patients.

Table 4.

Impact of time interval post-PET on claims confirmation (PPV) of post-PET plan.

Cancer type and treatment categories in Post-PET Plan	Patients, n	Claims confirmed at 60 days, %	Claims confirmed at 180 days, %
Bladder
Chemo/immune only	387	81.1	84.2
Chemo+RT	182	62.1	69.2
Surgery only	129	73.6	90.7
Surgery+(chemo or RT)		23.5	60.5
Pancreas
Chemo only	530	85.3	88.7
Chemo+RT	225	53.8	69.8
Surgery only	135	74.1	80.0
Surgery+(chemo or RT)	107	36.4	51.4
Stomach
Chemo only	332	69.6	78.2
Chemo+RT	126	47.6	61.9
Surgery only	264	80.3	86.4
Surgery+(chemo or RT)	194	31.4	60.8
Ovarian
Chemo only	169	90.5	93.5
SCL
Chemo only	349	88.8	89.7
Chemo+RT	270	62.6	73.0

Excluded plans with<100 patients.

Chemo: chemotherapy

Immune: immunotherapy

PPV: positive predictive value

RT: radiotherapy

SCL: small cell lung

The lowest rates and widest variation with time were found in confirming claims with surgery plus other treatments. By 60 days, claims for surgery plus chemotherapy or radiotherapy ranged from 23% to 36% in bladder, stomach and pancreas cancers. However, by 180 days, combined claims increased to 51%-61%.

Although extending action windows to 180 days tended to increase agreement with explicitly planned actions, it had the opposite effects for unplanned actions: by 180 days, more actions occur that were absent in the NOPR plan. For instance, the status of “no chemotherapy” by plan matched to claims 71.6% over 30 days, but this dropped to 42.5% over 180 days. Correspondingly, the kappa statistic declined from 0.42 to 0.29, from a fair to a poor level of agreement.

Specialist claims

Using the provider specialty codes, we explored if patients had evaluation and management (E&M) claims from the specialist associated with the intended treatment, even if therapy claims were not found. This was true for medical oncology and chemotherapy.

Figure 1 shows that within 30 days most patients with chemotherapy planned had E&M medical oncologist claims—even in the absence of paid chemotherapy claims. Overall, E&M oncologist claims were found in 87% at 30 days, rising to 92% at 60 days. Almost all SCL patients having medical oncologist E&M claims also had chemotherapy claims. In contrast, 24% of stomach cancer patients with chemotherapy plans had medical oncologist visits but no detected chemotherapy claims by 60 days.

Figure 1.

Frequency of chemotherapy claims and evaluation and management (E&M) claims by cancer type among subjects for whom the NOPR intended-management plan was chemotherapy.*

*We excluded ovarian cancer since gynecologic oncologists in addition to medical oncologists often give systemic therapies.

E&M: Evaluation and management

SCL: Small cell lung

Predictors

The results when we assessed the impact of patient (age, performance status, comorbidity), cancer (type, summary stage) and referring provider (specialty, pre-PET plan) factors in a logistic regression model in predicting agreement by treatment type at 60 days are summarized in Table 5 (with full-details in Supplemental Table 2).

Table 5.

Factors associated with agreement of treatment plan and claims for treatment.

Factors	Chemotherapy (n=3,030) Agreement	p-value	Radiotherapy (n=1,293) Agreement R	p-value	Surgery (n=1,031) Agreement	p-value
Unadjusted	79.3		64.7		63.6
Adjusted Agreement Range†0
Age ≤ or > 75 years*	75.5-84.7	<.0001	65.4-71.7	0.007	64.8- 66.2	0.846
Performance status*	77.2-82.6	0.14	66.0-69.8	0.744	57.7-68.8	0.019
Comorbidity*	80.0-81.9	0.604	65.3-71.5	0.194	61.7-68.4	0.197
Cancer type*	69.2-87.1	<0.0001	30.9-77.9	<0.0001	57.2-72.5	<0.0001
Stage*	70.5-84.0	0.001	47.6-73.2	<0.0001	43.8-72.8	<0.0001
Specialty*	70.2-80.9	0.001	25.7-89.4	<.0001	57.3-82.4	<0.0001
Pre-PET plan*	74.0-83.4	0.178	67.8-83.7	0.803	60.0-72.8	0.065

^†Logistic regression was used to assess factors affecting the proportion of individual plans agreeing with subsequent claims. Factor categories assessed were: age (< 75, >75 years), performance status (0,1, >2), comorbidity (0,1 or 2, 3+), cancer type (bladder, ovary, pancreas, small cell lung, stomach), provider specialty (medical oncology, gynecology, internal medicine, radiation oncology, surgery, and other) and Pre-PET plan (image, biopsy, treatment, watch). Based on the logistic regression model, adjusted percent agreements were calculated for each category and the range calculated.

Complete details of individual factor agreements are available in online supplemental Table 2.

Patient age ≤ 75 years was associated with higher claims agreement for chemotherapy, slightly lower agreement for radiotherapy, and not associated for surgery.

The impact of stage varied by treatment type. Patients having local disease with plans for surgery had claims confirmation in 73% vs. 53% in nodal/regional disease. Radiotherapy agreement rates were similar for local and nodal/regional disease at 72% and were markedly less commonly planned or confirmed for metastatic disease. Claims confirming chemotherapy plans did not differ among patients with nodal/regional, single-site, or multi-site metastasis at about 82-84% but were lower with local disease (76%).

If the referring physician was the specialist providing the treatment type, there was much higher concordance for surgery or radiotherapy plans but minimal differences for medical oncologists and chemotherapy.

Discussion

The NOPR management plan reflects the physician's intentions shortly after receiving the PET report, and not necessarily the patient's agreement to the plan. Agreement between claims-inferred management (assuming these are complete and accurate) and the NOPR plan may be affected by patient treatment preferences revealed in subsequent consultations and/or the physician's level of commitment to his/her plan. Unexpected shifts in a patient's symptoms and clinical course could also impact the intended plans' feasibility. This analysis of five cancer types when PET was used as part of initial staging found claims-validated monotherapy plans in over three-quarters of patients within 60 days. However, for combination therapy, a 60-day window was too short to fully capture clinical actions inferred from claims analysis. One relevant NOPR data limitation was that the post-PET questionnaire did not document whether combination therapy plans were to be concurrent or sequential. Therefore, for sequential plans, it is likely that the decision to give the anticipated second therapy encompassed a further decision point dependent on reassessment. For combined chemoradiotherapy plans, claims for both therapies increased by over 10% using a 180-day vs. 60-day window. A second decision point was even more likely with surgery-based combination therapies where the chemotherapy sequencing was not specified (neoadjuvant vs. adjuvant).

While this study uses as its starting point the unverified NOPR plan data and claims-inferred actions as the actual care, it is notable that there is no “gold standard” for the study design under which maximum concordance between “plan” and “execution” occurs, and how high that concordance is. A more ideal design might involve a structured interview with each physician after PET once a care plan is established with the patient, documenting nuances of the plan, and a manual chart or electronic record review. Additionally, one might also collect information on patient perspective—after initial post-PET clinical consultation.

Outside oncology, there are many examples in surgical randomized trials showing that actual delivery rates of randomly assigned therapy are often substantially different than intention-to-treat. For example, 2 years after treatment assignment for lumbar spinal stenosis, 43% had surgery after being assigned non-surgical treatment and only 67% completed assigned surgery. (22) In coronary artery disease trials, crossover from medical therapy to surgical intervention occurs in 10-20% of patients and surgical refusal in 2-10%.(23, 24) Cross-over in abdominal aortic aneurysm trials are even higher.(25) Thus, even if our plan and execution data points were obtained by more resource-intensive methodologies, it is unlikely that very high (e.g., >90%) agreement would occur.

There is a substantial, growing literature addressing the desire of cancer patients for information and participation in decision making in their care.(26)(27) These studies have predominantly addressed preference-based primary therapy decisions in breast and prostate cancer. In general, older patients and those with metastatic cancers deferred more to physician recommendations.(26, 28) To the best of our knowledge, few studies address discrepancies between physician plan, patient preference, and actual therapy received, factors that may explain some observed differences in intended vs. claims-inferred management. One notable report is CanCORS, which assessed over 5,000 newly diagnosed lung or colorectal cancer patients, one-fifth presenting with metastatic disease; 24% of CanCORS patients did not receive the treatment discussed.(26, 28)

Agreement studies within cancer registries focus on either registry classification accuracy when an independent record re-abstraction(29) is made or when registry treatments are compared to claims-inferred care. Since 2000, numerous SEER-Medicare linkage reports have assessed claims completeness in identifying chemotherapy type,(30-32) surgery,(33) and radiation.(34) Most studies did not focus on the claims' temporal accuracy. Since 2004, SEER guidelines require gathering all information through completion of first-course surgery or available within 4 months of diagnosis in the absence of disease progression.(29) Our analysis shows that a 60-day window is likely too narrow for assessing planned treatment completion, particularly in patients with less-aggressive, local disease and for combination therapies. However, lengthening this window may also result in “false positives” if claims are made for reasons not closely associated with the original post-PET plan (e.g., “unplanned” treatment at 3-6 months for unexpectedly aggressive metastatic disease).

Two recent reports provide some context for our findings. In a comparison of Medicare administrative claims against clinical trial data (as a reference standard), Lamont et al. assessed patient cohorts from six CALGB first-line chemotherapy trials in breast, colorectal and lung cancer.(30) They found that 78% of claims correctly identified the drugs given and the correct treatment schedule. In a second study, Lund et al. reported NCI Patterns of Care studies in four cancer types where SEER initial treatment plans were directly validated against hospital and outpatient records that were re-abstracted for chemotherapy.(35) They found that the claim sensitivities were very time dependent, ranging from 36%-50% at two months to 84%-96% at six months.

Our analysis was not powered to, and did not attempt to, stratify patients by stage within each cancer type and we did not assess if either the planned or claims-inferred management in each patient agreed with authoritative clinical practice guidelines. Our findings that about 90% of patients with an intended plan including chemotherapy had E&M (office visit) claims from a medical oncologist within 60 days even in the absence of chemotherapy claims suggests appropriate consultation for informed decision making was available. That is, patients who did not receive chemotherapy did not fail to see an oncologist.

The above concerns should not divert attention from the primary clinical endpoint and findings that NOPR was designed to measure—does PET as part of initial cancer staging lead to a change in (intended) clinical management? We have previously reported that during the first two years of NOPR data collection, PET was associated with a 40% change in intended management if additional imaging was a post-PET option or a 13% change if no benefit was assumed in cases where additional imaging was chosen. Using a similar design to that of the NOPR, Scott and colleagues from Australia have reported series involving smaller patient numbers (range 71-129) and centers (n=3 to 6) but with detailed medical record follow-up to assess intended vs. actual management. (36-38) They found that PET (without CT) used for initial staging of head and neck and esophageal cancer and lymphoma was associated with a 34-38% change in intended management and that the agreement at 3 to 6 months between the post-PET plan and actual care ranged from 53% in esophageal cancer to 74% in lymphoma and head and neck cancer. Our results from a national registry with little control over center-to-center data quality are quite similar.

We have previously noted that PET-associated changes in intended management would not always be beneficial (since PET has both false-negative and false-positive results).(14) Moreover, changes in intended actions have a presumed, but uncertain, relationship to more tangible health outcomes, such as progression-free or overall survival.(39) However, given the above limitations, we were gratified to observe that claims at 60 days confirmed chemotherapy plans in 90% of SCL cancer patients, and surgery in 80% with stomach cancer; assuming that in most cases PET results were accurate and accepted contemporary treatment is beneficial.

If future evaluations of new imaging techniques are initiated under the CMS CED policy, there are lessons to be learned from the NOPR experience. For example, the type and extent of prior imaging or treatment are likely to impact the agreements rates between intended and actual care. While refinements in granular detail of intended management plans (e.g., specific drugs, sequencing for combination therapies) warrant consideration, our results suggest that the principal future change should be at least one additional follow-up data collection point—to confirm the actual initiation or cessation of treatment or changes in the use of relevant diagnostic studies. Innovative study designs that allow for investigation of the patient's role as a factor in planned vs. received care at preference-sensitive decisions points should be piloted. We hypothesize that differences in treatment urgency and acceptability likely explain discordance more than inaccurate assessment of physicians' intended plans. The inability to make inferences about the impact of diagnostic imaging on patient outcomes outside a randomized trial will remain. A modest greater time investment and energy of all parties (patients, referring physicians, interpreting physicians, and registry analysts) as well as an expanded informed consent will be needed to strengthen the value of diagnostic registries for comparative effectiveness research.'

Supplementary Material

Supplemental Table 1. Codes used to identify Medicare management. The Current Procedural Terminology (CPT©) codes, the Healthcare Common Procedure Coding System (HCPCS) codes and International Classification of Diseases (ICD-9) codes for outpatient care used to classify claims into inferred-care categories are listed below. Our preliminary analysis found numerous surgical procedures for complications (e.g., chest tube insertion) rather than directed at the patients cancer. Therefore, we used anticipated surgical CPT codes and anesthesia claims after excluding eye, central vascular access, gastrointestinal endoscopy, and conscious sedation. Chemotherapy and immunotherapy were inferred from professional claims, HCPCS codes for hospital-based chemotherapy, and drug J-codes (but oral chemotherapy codes were not assessed, since Medicare Part D claims were not included in the analysis). Radiotherapy codes included all common techniques. These systemic-therapy and radiotherapy definitions were similar to those used in SEER-Medicare analyses.(34, 40)

Supplemental Table 2. Factors associated with agreement of treatment plan and claims for treatment.

SCL: small cell lung

NED: no evidence of disease

Onc.: Oncology

Gyn.: Gynecology