Do Survivorship Care Plans Impact Patients’ Evaluations of Care? A Randomized Evaluation with Gynecologic Oncology Patients

Abstract

OBJECTIVE

Despite recommendations from the Institute of Medicine regarding survivorship care plan (SCP) delivery to cancer patients, there have been few health service outcome evaluations thus far.

METHODS

Gynecologic cancer survivors who were up to one-year post-treatment could participate in an evaluation assessing the health services provided and their perceptions of quality of care. A randomized, nested, cross-sectional design was used in a large group gynecologic oncology practice. Half (n=3) of physicians were randomized to provide and discuss a SCP to patients during a follow-up visit, and the other half did not. Following their visits, all patients were informed that a health services evaluation of the practice was being conducted. Interested patients completed an anonymous 26-item survey assessing administrative, clinical, and educational health services, helpfulness of written materials, and perceptions of quality of care.

RESULTS

Of the 121 survivors surveyed, 64 received SCPs and 57 were in the no-SCP condition. As a validity check, one question asked about educational materials received during the visit with an expected significant difference noted between conditions (X²=5.513, p=.019; more SCP patients reported receiving materials). However, there were no differences between conditions when patients rated health services (Fs>.37) or helpfulness of materials and perceptions of care (Fs>.19).

CONCLUSIONS

Gynecologic oncology patients providing ratings of health services and satisfaction with care provided equivalent evaluations, regardless if they had/had not received a SCP from the physician. Thus, the need remains for further evaluations of SCPs if they are to be vehicles for improving health service outcomes.

Introduction

There are an estimated 12 million cancer survivors in the US [1], making cancer survivorship care a more salient issue in the last decade. The Institute of Medicine’s (IOM) From Cancer Patient to Cancer Survivor: Lost in Transition [2] provides recommendations to significantly enhance patients’ care as they transition to follow-up care. One key recommendation states that patients completing primary treatment should be provided with a survivorship care plan (SCP), incorporating a summary of treatment received and follow-up care recommendations. SCPs are thought to have many benefits, including increasing patients’ satisfaction with and perceptions of quality of care [2]. However, few studies have examined the impact of SCPs on patient outcomes.

SCPs rate favorably amongst individuals (i.e., patients, physicians, and/or nurses) asked to evaluate the concept or an exemplar care plan [3–5] (see also a recent review [6]) and those who have received them [7–8]. Since 2004, only three studies have more closely met the IOM’s goal of evaluating “the impact and costs associate with…SCPs [2]^(p5).” The method for two single group studies was to provide SCPs to patients and/or primary care physicians and then evaluate adherence to screening recommendations. Blaauwbroek et al. [9] reported 83% adherence rate to recommended screenings in sample of 69 primary care physicians treating 70 childhood cancer survivors. Oeffinger et al. [10] reported 41% adherence to screenings recommended to 62 Hodgkin Lymphoma patients. Unfortunately, control groups were not included nor was base rate adherence data provided for either study. The single randomized study was conducted in Canada with patients coming from nine tertiary cancer centers transitioning patients from follow-up care with their oncologists to primary care physicians (PCPs) [11]. Breast cancer survivors (N=408) were randomized to receive standard care (discharge visit and a discharge letter sent to the PCP) or standard care plus provision of a SCP. Analyses showed no group differences between survivors who did and did not receive a SCP on measures of patients’ satisfaction with care, continuity of care ratings, psychological (cancer-related stress, negative mood) outcomes or quality of life [11].

Thus, the available evaluation of health service outcomes for SCPs comes from two single group pre-post studies and one randomized trial. No study of which we are aware has examined the impact of receiving a SCP on patients’ perceptions of quality of care (i.e., the subjective perception that the quality of health care received matches the patients’ expectations [12]). Similar but distinct from “patient satisfaction”, perceptions of quality of care are routinely measured as part of health care organizations’ focus on quality assurance. Assessing perceptions of quality of care is a useful way to alert health care providers to patients’ needs and concerns and to identify potential areas of improvement [13] and are particularly beneficial when implementing and evaluating new health care practice initiatives.

Focus of the present investigation

This randomized trial compared ratings of perceptions of quality of care and health service outcomes from gynecologic cancer survivors. In a large group practice, physicians were randomized, with half providing a SCP to their patients during the visit and the others not providing a SCP to their patients. Patients completed a self-report measure of perceptions of quality of care after their physician visit. It was hypothesized that significantly higher positive perceptions of quality of care would be found among the patients receiving a SCP.

Methods

Procedures

From March thru October 2010, the Gynecologic Oncology division at a NCI designated Comprehensive Cancer Center in the Midwest conducted an evaluation of survivorship care plans. This was done to study the impact of SCP distribution on patients’ evaluations of the quality of care and health services. All gynecologic cancer survivors who were within one year post-treatment and had a follow-up care appointment during that time were solicited for the evaluation. A randomized nested design was used; physicians in the practice were randomized (i.e. did not self-select) to either provide their patients with a SCP (physicians 1–3) or not (no-SCP; physicians 4–6), with patients nested within these two groups.

After the appointment, patients met with the practice medical secretary (who was blind to study condition) for follow-up appointment scheduling. She informed eligible patients of the opportunity to complete an anonymous survey evaluating the services of the clinic. They received a written description of the program evaluation and a survey. Patients were told that completion and return of the survey was voluntary and confidential. Interested patients were directed to a private space in the clinic and instructed to return the completed survey to the reception desk in the sealed envelope provided. Patients requesting to complete the survey at home were given an addressed, postage-paid envelope for survey return. Of the 222 surveys distributed, 121 (55%) were returned. As is routine with program evaluations, no personal health information was collected. The local Institutional Review Board considered the research exempt with no informed consent necessary as no PHI was collected.

Physician participants

At the time of the program evaluation, all physicians had completed gynecologic oncology fellowship training. Physicians participating in the SCP condition had been employed with the practice for an average of 11 years and physicians participating in the no-SCP condition were employed an average of 7 years. Physicians were not given a copy of, or any specific education, regarding the content of the evaluative measure. Physicians were aware that the evaluation was taking place and those distributing care plans were given instructions on SCP delivery to ensure standardization amongst delivering physicians.

Patient participants

Data collection was anonymous; only age [mean=60 years, SD=13, range: 24–89) and general disease/treatment information was obtained. Participants reported the following disease sites: n=65 (54%) endometrial, n=35 (29%) ovarian, n=16 (13%) cervical, and n=5 (4%) vaginal. The majority had received surgery (98%) and chemotherapy (52%) with fewer receiving radiation therapy (17%). Of those who had received chemotherapy or radiation therapy, only 2 patients reported being in treatment at the time of participation. General characteristics of the clinic population are predominantly 91% Caucasian (6% African-American, 1% Asian, 2% other) and 57% living in a rural area (39% urban, 5% from out of state).

Conditions

Standard Care (no-SCP)

Patients in the no-SCP condition received standard care which was receipt of a medical examination and surveillance of recurrence and any treatment-related morbidities by the physician, referral to other healthcare professionals as needed, and availability of patient education materials from nurses and/or the Patient Resource Center located in the clinic lobby.

Standard Care plus Survivorship Care Plan (SCP)

Individualized SCP documents were created by a research assistant. A SCP had two components. The 3–4 page treatment summary was created manually using information located throughout the patient’s electronic medical record. The treatment summary described a patient’s diagnosis and treatment details: disease site and stage, and as applicable, surgical treatment received, type and total dosages of any chemo- or hormonal therapies received, site and total dose of radiation therapy, report of any additional hospitalizations or significant toxicities that had occurred thus far, and a brief medical history. The care plan portion, created using the online LIVESTRONG Care Plan software [14], was individualized using data from the treatment summary. Information on the following topics was provided: late effects of treatments received, cancer screening recommendations, healthy lifestyle information, common psychosocial concerns, and general tips for cancer prevention, among others. Both the treatment summary and care plan were provided in a folder, along with copies of CT scans, lab results, and pathology reports, relevant to the diagnosis and treatment.

In addition to standard care, SCP physicians distributed the folder during the follow-up appointment, reviewed the SCP, and encouraged the patient to share it with other healthcare professionals, such as her PCP. Patients were also encouraged to review the information and to contact their physician if they had questions or concerns at any later point.

Outcomes

The investigators queried survivorship directors at comprehensive cancer centers for relevant surveys as validated measures for evaluating the clinical utility and impact of SCPs were not found at time of the program evaluation (January 2010). Of the centers contacted, the Jonsson Comprehensive Cancer Center at UCLA shared an 8-item perceptions of quality of care survey, with items rated on a 5-point Likert scale of 1=Poor to 5=Excellent, in use with gynecologic cancer patients. The items were generally relevant but did not contain ones regarding SCPs (i.e. educational services or provision of written materials). Therefore, the investigators used an iterative process for item refinement and development.

First, a 7-point, rather than a 5-point, Likert scoring scale was used with the aim to increase sensitivity and achieve a wider range of item scores. Second, the UCLA items were adapted (see Table 1) with 6 of the 8 items used. Third, nine new items were written with content specific to the information provided in the SCPs (e.g., strategies to monitor/manage short and long term side effects of cancer, information on risks factors for second cancers, a personalized description of the treatments received). Fourth, the entire scale was reviewed and modified by another psychologist familiar with the needs and concerns of gynecologic cancer patients. The last review by the multidisciplinary team of investigators resulted in a 26-item scale.

Table 1.

Items composing the area scales for the Patient’s Perceived Quality of Care survey.

Area	Item Description
Administrative Services: Please select a rating from 1 = Very Poor to 7 = Excellent for each of the following items based on today’s visit to the clinic.
	Appointment scheduling*
	Locating clinic/checking in, checking out, parking, etc.*
Clinical Services: Please select a rating from 1 = Very Poor to 7 = Excellent for each of the following items based on today’s visit to the clinic.
	Your medical concerns*
	Your psychological and/or social concerns*
	Your sexual well-being
	Your health risk factors, like smoking or weight
	Your need for a referral to another medical provider
Educational Services: Please select a rating from 1 = Very Poor to 7 = Excellent for each of the following items based on today’s visit to the clinic. Mark “None Received” if no materials were received.
	Your cancer and the treatment you have received
	The short term effects of your cancer treatment
	Strategies to monitor or manage the short term effects of cancer treatment
	The long term effects of your cancer treatment*
	Strategies to monitor or manage the long term effects of cancer treatment*
	Strategies for reducing risk, such as changing health habits or behaviors
	Other resources available to you at the clinic or at the medical center
Helpfulness of Written Materials: Please select a rating from 1 = Very Poor to 7 = Excellent for the helpfulness of written materials (care plans, booklets, etc.) you have received from this clinic (or mark “None Received” if no materials were received) regarding…
	Your cancer
	Cancer treatments you have had
	Coordinating your own care
	Risk of a second cancer
	Fatigue (as applicable)
	Surgical procedures and side effects (as applicable)
	Chemotherapy procedures and side effects (as applicable)
	Radiation procedures and side effects (as applicable)
	Other treatment procedures and side effects (as applicable)

^*Item adapted from Perceptions of Quality of Care survey provided by the Jonsson Comprehensive Cancer Center at UCLA.

The items assess quality of care in health services and helpfulness of written materials, as well as likelihood of recommending the clinic and overall experience (see Table 1). Items were grouped by content into four scales: Administrative Services (2 questions), Clinical Services (5 questions), Educational Services (7 questions), and Helpfulness of Written Materials (9 questions). Patients rate their most recent (today’s) clinical appointment using the 7-point Likert scale (1=Very Poor to 7=Excellent). The same scale was used for 16 items assessing educational services and helpfulness of written materials, but patients also had the option of indicating 0=None received.

A scale score was the sum of the respective items divided by scale’s number of items so that scale scores ranged from 1 to 7. Items rated as 0=None received in the Educational Services and Helpfulness of Written Materials scales were excluded from scale score calculation. Internal consistencies(α) were: 0.85 for Clinical Services, 0.87 for Educational Services, and 0.75 for Helpfulness of Written Materials. The Administrative Services scale contained two items and the correlation between the two items was r=.325, p< .001, an acceptable estimate for a two-item scale [15]. Lastly, patients evaluated their overall experience at the appointment (using the same scale) and the likelihood of recommending the clinic to others on two items. Response choices range from 1=Not at all Likely to 7=Extremely Likely.

Analytic Strategy

For ANOVA analyses, a priori estimates indicated that to detect a medium effect size of .5 with alpha = .05 and two groups, 120 individuals would be needed to detect an effect at 80% power. Therefore, all gynecologic cancer survivors were given a copy of the survey until the minimum number of 120 surveys was returned. As expected, the randomization of physicians resulted in approximately equal numbers of patients (ns of 64 vs. 57) between the respective conditions.

Analyses were conducted using a mixed factorial analysis of variance (ANOVA) model, with Condition (SCP versus no-SCP), and Physician nested within Condition as a between-subjects factor. Of primary interest was the main effect for Condition, with the effects for Physician nested within Condition further examined in planned post-hoc analyses using the Fisher’s Least Significant Difference (LSD) test. Missing data was considered to be missing at random (MAR) and ignored in analyses as MAR data does not impact inference from statistical analyses. A single item in the Helpfulness of Written Materials scale asking about “Your Cancer” was recoded dichotomously as 0=None received and 1=Education/Materials received and analyzed individually using a chi-square analysis as a validity check for the patient assignment to Condition.

Results

Preliminary Analyses

No significant differences were found between the SCP and no-SCP conditions on patient age, cancer site, or receipt of surgery, chemotherapy, or radiation therapy (ps>.29). Regarding the validity check item, chi-square analyses indicated a significant difference between conditions (X²=5.513, p=.019), as expected, with the SCP patients more likely to indicate that written educational materials were received (97%) than the no-SCP patients (84%).

Primary Analyses

Analyses were conducted for the four scales (see Table 2 for descriptive data and results of mixed factorial ANOVAs by Condition). For the Administrative Services scale, no main effect of Condition was found, nor was Physician nested within Condition significant [F(4, 115)=1.698, p=.155]. Similar results were found for the Clinical Services scale [Physician nested within Condition: F(4, 115)=.613, p=.654], the Helpfulness of Written Materials scale [Physician nested within Condition: F(4, 115)=1.052, p=.384], and the item, “How likely are you to recommend this clinic?” [Physician nested within Condition: F(4, 115)=.351, p=.843].

Table 2.

Descriptive data and mixed factorial analysis of variance (ANOVA) results for Condition, comparing Patients’ Ratings of Health Services and Perceptions of Quality of Care between patients who received a survivorship care plan (SCP) and those who did not (no-SCP).

Area Scales	SCP Mean (SD)	No-SCP Mean (SD)	F	Sig	df
Administrative Services	6.41 (0.81)	6.33 (0.75)	.205	.670	1, 4
Range	4 – 7	4 – 7
Clinical Services	6.38 (0.89)	6.28 (0.85)	.991	.368	1, 4
Range	2 – 7	3 – 7
Educational Services	6.33 (0.84)	6.44 (0.57)	.566	.492	1, 4
Range	4 – 7	5 – 7
Helpfulness of Written Materials	6.28 (0.96)	6.18 (0.85)	.549	.495	1, 4
Range	3 – 7	4 – 7
Overall Experience	6.44 (1.18)	6.61 (0.73)	.490	.521	1, 4
Range	1 – 7	4 – 7
Likelihood to Recommend	6.87 (0.38)	6.80 (0.48)	2.188	.192	1, 4
Range	5 – 7	5 – 7

Note: Responses of 6 = Very Good and 7 = Excellent. Total possible range for the scales is 1 – 7. Physician was nested within Conditions in the analyses.

For the Educational Services scale, there were no differences by Condition; however, there was a main effect for Physician nested within Condition [F(4, 115)=2.941, p=.024]. Scores for all physicians were high (>5 on a scale of 1–7); however, one physician in the SCP condition was rated significantly lower (mean=5.834, SE=.181) than all other physicians (means range, 6.369–6.625, ps<.03). Similarly, for the “Rate your overall experience today” item, no mean differences were noted by Condition. However, there was a main effect for Physician nested within Condition [F(4, 115)=3.018, p=.021]. Again, all ratings were high. The same physician noted above was significantly lower (mean=5.765, SE=.234) than the ratings of all other physicians (means range from 6.474–6.731, ps<.03).

Discussion

An outpatient clinic program evaluation was conducted, assessing patient-reported quality of care from gynecologic oncology survivors. Half of clinic physicians from a large group practice were randomized to give their patients individualized survivorship care plans whereas the others did not. The patients, in general, had high regard for the health services provided irrespective of the SCP addition.

This research was designed in a manner to insure internal validity and conducted in a manner to maximize external validity. Regarding internal validity, the findings for the validity check item revealed data as predicted, i.e., the groups significantly differed in the reporting of the receipt of educational materials from the clinic. Importantly, however, this difference also confirms that the sample size was sufficient to detect differences between conditions should they exist in other domains. Surveys were completed anonymously and completed in a private location to encourage “true” responses. Our use of a 7-point Likert scale rather than a 5-point scale was done in an attempt to spread the scores, but the average responses still clustered between the two highest possible marks.

Regarding external validity, physicians (and therefore patients) were assigned either the SCP or the no-SCP group. The distribution method, physicians describing the SCP document to patients and encouraging them to review the materials carefully, was similar to current practices for other physician delivery of education materials and interventions (e.g. smoking cessation) and to procedures of other clinics. In addition, results would likely generalize to clinical community practices as the oncology clinic was located off-site from the university hospital and functioned more similarly to a private practice with faculty physicians (rather than residents or fellows) providing the care. Limitations of the current study include sampling of a homogeneous group of gynecologic cancer patients and not collecting other outcome measures. Despite these limitations, we feel that these results would replicate across disease sites and across outcome measures as they are in line with the null effects from the larger trial of Grunfeld and colleagues (2011).

Our primary findings---no differences between conditions--- are like those of the only other experimental data currently available from Grunfeld et al. [11]. As described above, they reported no differences in distress, quality of life, and patient satisfaction for breast cancer patients who did or did not receive a SCP. Whereas we had physicians deliver and show the contents of the SCP folder to a patient, they used a procedure in which the SCP was given to patients during a 30 minute educational session with a nurse, which yielded no additional benefit on patient-reported distress or health service outcomes. In the same issue in which the Grunfeld et al. [11] study appeared, there was, interestingly, an accompanying editorial. In it, Smith and Snyder [16] proffered three hypotheses for the lack of differences between the groups in Grunfeld et al. [11]. One, breast cancer survivors receive better preventive and comorbid condition care than patients having other cancers and thus, a SCP did not provide value added. Two, the outcomes used were not sensitive to the effects of SCP. Three, the study was conducted in Canada where an emphasis on primary care follow-up of oncology patients means that there are already practices in place to facilitate communication between patients and providers and a SCP may be unnecessary.

We considered these hypotheses in terms of their relevance for the present study. Regarding the first, we sampled gynecologic cancer patients. As a group, their experience of the psychological and sexual sequelae of cancer treatments are considerable [17], but their plight is unknown by most and tailored resources for them are substantially less. Regarding the second hypothesis, the outcomes used here are those they recommended, i.e., ones specific to the content of the SCP. And third, this US study was done at a time and in a climate of increasing attention to cancer survivorship in general and SCPs in particular in this country.

Studies such as ours and that of Grunfeld et al. [11] may be producing true null effects for reasons other than those noted above. One possibility might be that patients do not know what they do not know. That is, patients who do not receive a survivorship care plan may be unaware that anything is “missing.” Data would suggest that few patients are actually receiving one, with less than 10% of breast and gynecologic cancer patients receiving one in recent surveys [8, 18]. More advocacy work to the general population may be necessary to educate patients about the potential benefits of SCPs.

Begging the question of the utility of SCPs, the slow pace of nationwide distribution/adoption of SCPs is due in part to the barriers for doing so. Providers cite insufficient time, personnel, and resources to create and distribute care plans [19, 20]. The lack of reimbursement for SCP preparation means that oncology practices must absorb the cost of their production. For the current study, SCP preparation by a trained individual with a baccalaureate education (similar to reports of others preparing SCPs [20]) was an average of 90 minutes (range of 60 – 120 minutes depending on case complexity) at an estimated cost of $19/hour for labor or $28.50 per patient. Currently, the primary contributor to SCP preparation time is the volume and complexity of treatment documentation (e.g., multiple provider notes, dates and dosages of treatments, etc.). (Because of this, some centers are attempting to “build” the treatment summary portion of SCPs via templates in the electronic medical record that are populated with the relevant information automatically.) Alternatively, it has been suggested that SCPs will be cost saving via impacting the under- or over-use of health services [16]. Obtaining such evidence would require a design of lengthy follow-up and cost analysis.

Thus far, the utility of the SCP in improving the quality of care remains unclear. More advocacy work directed to cancer survivors regarding the potential utility of SCPs is necessary. Only then might we see a shift in patients’ ratings of health services, particularly in those patients who do not receive a SCP. Preparation and distribution of SCPs require significant time, effort, and cost. Utilizing technological advances [such as electronic medical records (EMRs)] may decrease the effort required for preparation. At this time, the need for SCP evaluation research remains. Future research should consider collecting information regarding patients’ review and use of SCPs.

Research Highlights.

• Oncologists were randomized to provide/not provide survivorship care plans (SCPs) to gynecologic oncology patients at follow-up.

• Patients rated quality of care similarly regardless if they had/had not received an SCP during recent follow-up visit.

• The need remains for further evaluations of SCPs if they are to be vehicles for improving patient-reported outcomes.