Florida Initiative for Quality Cancer Care: Changes in Psychosocial Quality of Care Indicators Over a 3-Year Interval

Findings suggest that more intensive efforts than audit and feedback will be required to improve the quality of psychosocial care, and that greater recognition of problems with emotional well-being may tax the ability of practices to link patients with appropriate services.

Abstract

Purpose:

Identifying and addressing psychosocial concerns is increasingly recognized as an important aspect of cancer care that needs to be improved. As part of the Florida Initiative for Quality Cancer Care, medical record reviews were conducted to evaluate cancer care, including psychosocial care, at oncology practices in Florida in 2006. Results were subsequently disseminated to the practices, and performance was reassessed at the same practices in 2009.

Methods:

Data were available for patients with colorectal, breast, and non–small-cell lung cancer first seen by a medical oncologist in 2006 (n = 1,609) and 2009 (n = 1,720) at the same 10 practice sites. Performance on each psychosocial indicator was evaluated for overall change over time and for variability in change based on practice site and cancer type.

Results:

The percentage of patients identified as having a problem in emotional well-being increased significantly over time, from 24% to 31% among those assessed (P = .002) and from 13% to 16% overall (P = .026). In contrast, there no significant changes over time in assessment of emotional well-being (53% to 51%, P = .661) or in action taken to address problems (57% to 45%, P = .098).

Conclusion:

Findings suggest more intensive efforts than audit and feedback will be required to improve the quality of psychosocial care and that greater recognition of problems with emotional well-being may tax the ability of practices to link patients with appropriate services. Systematic research is needed to identify and disseminate effective strategies for implementing routine assessment of well-being and addressing the increased demands for care this will generate.

Introduction

Institute of Medicine reports^1,2 and surveys of patients and care providers^3,4 suggest limited progress has been made in implementing recommendations that oncology practices have procedures in place to identify and assist patients who experience psychosocial problems. Efforts to date to improve the quality of psychosocial care have focused mostly on issuance of clinical practice guidelines for distress management⁵ and accreditations standards designed to foster greater patient-centered care.⁶

A complementary approach is to measure and provide feedback to practices about the quality of psychosocial care received by their patients. Research has shown that providing medical oncology practices with feedback demonstrating their poor performance on quality indicators can result in improvements over time.^7–9 The first step in pursuing this approach is to develop measurable indicators of the quality of psychosocial care. Toward this end, the American Psychosocial Oncology Society formed a workgroup in 2007. As described elsewhere,¹⁰ this effort resulted in creation of two medical record indicators considered necessary (but not sufficient) for providing quality psychosocial care: documentation that emotional well-being was assessed within 1 month of the first visit with a medical oncologist, and documentation of action taken to address an identified problem with emotional well-being or an explanation provided for why no action was taken.

As part of the Florida Initiative for Quality Cancer Care (FIQCC),¹¹ these indicators were embedded in a larger set of quality indicators and applied to the medical records of > 1,600 patients with breast, colorectal and non–small-cell cancer first seen by a medical oncologist in 2006 at 11 practice sites. As previously reported,¹² there was documentation of emotional well-being for only 52% of patients and documentation of action taken (or an explanation provided for no action) for only 58% of patients identified as having a problem with emotional well-being. Using methods described below, practices received feedback about their performance on these and other indicators and were encouraged to undertake quality improvement efforts for indicators for which performance fell below 85%. They were also informed that the same audit procedures would be repeated with patients first seen by a medical oncologist in 2009 to assess possible changes in quality of care.

The purpose of this report is to examine whether changes in performance on psychosocial care indicators occurred between the two assessments. If changes did occur, we also examined whether they were independent of other changes over time (eg, changes in payer mix). Finally, we examined whether change over time for each indicator differed across practice sites and cancer types.

Methods

Study Sites

The FIQCC was founded with 11 medical oncology practices in Florida. Eligibility for participation in FIQCC has been described previously.¹¹ The present report focuses on 10 of these practices that still met eligibility criteria and were willing to participate in the 2009 abstraction. The project received approval from the institutional review boards at each institution.

Quality of Care Indicators

Medical records were abstracted for numerous indicators of the quality of cancer care.¹¹ The present report focuses on two indicators of the quality of psychosocial care described previously¹² and assessed in 2006 and 2009: (1) there should be evidence in the medical record that the patient's current emotional well-being was assessed within 1 month of the patient's first visit with a medical oncologist; and (2) if a problem with emotional well-being was identified, there should be evidence in the patient's medical record that action was taken to address the problem or an explanation provided for why no action was taken. Measurement was operationalized by formulating questions that could be answered yes or no based on medical record review (see Appendix, online only). The current report also includes information about whether there was evidence that the patient's pain status was assessed within 1 month of the first visit with a medical oncologist (Appendix).

Medical Record Selection and Review

All patients 18 years of age or older at each practice site diagnosed with colorectal, breast, or non–small-cell lung cancer and seen for a new medical oncology consultation in calendar years 2006 or 2009 were eligible for selection, with certain disease-specific exclusions (eg, among patients with colorectal cancer, those with anal/recto-sigmoid carcinoma were excluded). To ensure similar numbers of patients of each cancer type within each practice site, the following strategy was used. After first determining the number of patients with colorectal cancer abstracted, patients with breast and non–small-cell lung cancer were randomly selected from available patients. For practice sites that abstracted 60 or fewer patients with colorectal cancer, up to 60 patients each with breast and colorectal cancer were abstracted (if possible). For practice sites that abstracted more than 60 patients with colorectal cancer, the number of patients with breast and non–small-cell lung cancer abstracted was set at the number of patients with colorectal cancer abstracted. Medical record review and quality control procedures were conducted in both 2006 and 2009 as reported previously.¹²

Disclosure of 2006 Findings and Initiation of Quality Improvement Plans

In July 2008, a conference was held with representatives from each consortium site to discuss results of the 2006 abstraction of patients with colorectal cancer. Similar conferences were held in March 2009 and February 2010 when results of the 2006 abstraction of patients with breast and non–small-cell lung cancer, respectively, became available. At each conference, results for all the quality of care indicators were presented by practice site. Findings were blinded such that each practice site could identify its own results but not those of any other site. Particular attention was given to performance indicators with less than 85% overall adherence and those quality indicators with significant variance in performance among the sites. The 2006 results for the two psychosocial quality of care indicators met both these criteria. After results were disclosed, each site representative had the opportunity to present suggestions for improvements and/or describe “what works well” in their practice. The conference concluded with the plan that the representative from each site would disclose these results to their respective practices. Each site was also encouraged to develop and implement a quality improvement plan for any performance indicators less than 85%. Site representatives were informed that the same abstraction procedures would be repeated with patients first seen in 2009 to assess possible changes in quality of care. Information obtained from site representatives indicated that three of the 10 practices initiated specific plans to improve the quality of psychosocial care before or during 2009. These plans included more systematic efforts to identify patients with problems in emotional well-being (three practices) and to identifying resources for patients with such problems (one practice).

Statistical Analysis

Statistical comparisons of patient characteristics and of performance on quality of care indicators in 2006 and 2009 were made in bivariable analyses with the Pearson χ² exact test using Monte Carlo estimation. Patient characteristics variables that differed (P < .05) between 2006 and 2009 were subsequently included in multivariable logistic regression models to determine whether the effects of time (2006 v 2009) on performance on quality of care indicators were independent of identified differences in patient mix. The interactions between practice site and time and between cancer type and time were also tested in multivariable logistic regression models to evaluate the extent to which changes in performance on quality of care indicators varied among the practice sites and by type of cancer. For each of these analyses, the P value from the type 3 analysis of interaction effects based on the Wald χ² test was reported. Firth's penalized maximum likelihood approach was used to fit the logistic regression models for small sample sizes.^13,14 A P value of .05 (two-sided) was considered significant. All analyses were conducted using SAS 9.3 (SAS Institute, Cary, NC).

Results

Patient Characteristics

A total of 3,329 patients from 10 FIQCC sites were included in this analysis (1,609 reviewed in 2006 and 1,720 reviewed in 2009). The number of patients reviewed at each site ranged from 85 to 291 in 2006 and 139 to 293 in 2009. Table 1 presents the characteristics of the 3,329 patients evaluated in this study. There were no differences between 2006 and 2009 in the patient mix with regard to age, sex, race, site of cancer, or cancer stage (Ps ≥ .109). There was a significant difference between 2006 and 2009 in the distribution of payer status types (P = .002). Compared with 2006, higher percentages of patients were self-paying, received charity care, or covered by Medicaid, and lower percentages were covered by private insurance or Medicare. Accordingly, multivariable analyses that included payer status were conducted to determine the extent to which changes over time in performance on quality indicators were affected by changes in payer mix.

Table 1.

Patient Characteristics

Variable	Total (N = 3,329)		2006 Patients (n = 1,609)		2009 Patients (n = 1,720)		P*
	No.	%	No.	%	No.	%
Age, years							.716
≤ 55	889	26.7	432	26.8	457	26.6
56-65	857	25.7	410	25.5	447	26.0
66-75	945	28.4	447	27.8	498	29.0
> 75	638	19.2	320	19.9	318	18.5
Sex							.633
Female	2,240	67.3	1,076	66.9	1,164	67.7
Male	1,089	32.7	533	33.1	556	32.3
Race/ethnicity							.126
White	2,671	80.2	1,297	80.6	1,374	79.9
Black	251	7.5	120	7.5	131	7.6
American Indian	6	0.2	0	0.0	6	0.3
Asian	32	1.0	15	0.9	17	1.0
Missing	369	11.1	177	11.0	192	11.2
Payer status							.002
Private	1,258	37.8	633	39.3	625	36.4
Medicare	1,683	50.6	818	50.8	865	50.3
Medicaid	159	4.8	60	3.7	99	5.8
Self-pay	101	3.0	40	2.5	61	3.5
Charity	98	0.29	37	2.4	61	3.5
Missing	30	0.9	21	1.3	9	0.5
Cancer site							.784
Breast	1,238	37.2	602	37.4	636	37.0
Colorectal	1,000	30.0	489	30.4	511	29.7
Lung	1,091	32.8	518	32.2	573	33.3
Cancer stage
I	757	22.7	343	21.3	414	24.1	.109
II/III	1,533	46.0	759	47.2	774	45.0
IV	862	25.9	399	24.8	463	26.9
Missing	177	5.3	108	6.7	69	4.0

^*P values were calculated using χ² test exact method with Monte Carlo estimation. Missing level excluded for P value calculation.

Changes in Performance on Pain and Emotional Well-Being Assessment Indicators

Table 2 presents the average performance rates for the emotional well-being and pain assessment quality of care indicators in 2006 and 2009 and the percent change over time. The percentage of patients in whom pain was assessed increased by 1.3% between 2006 and 2009. The magnitude of this change was not significant before (P = .079) or after (P = .078) adjusting for payer status. The percentage of patients in whom emotional well-being was assessed declined by 1.8% between 2006 and 2009. Similarly, the magnitude of this change was not significant before (P = .718) or after (P = .661) adjusting for payer status.

Table 2.

Change in Adherence (%) to Psychosocial and Pain Indicators Between 2006 and 2009

Indicator	2006 Patients		2009 Patients		% Change*	P†	P‡
	% Adherence	No./Eligible Cases	% Adherence	No./Eligible Cases
Pain assessed	86.6	1,393/1,609	87.9	1,512/1,720	1.3	.079	.078
Emotional well-being assessed	53.1	854/1,609	51.3	882/1,720	−1.8	.718	.661
Problem with emotional well-being identified	13.0	209/1,609	16.0	276/1,720	3.0	.021	.026
Problem with emotional well-being identified among those assessed	24.5	209/854	31.3	276/882	6.8	< .001	.002
Action taken for problem with emotional well-being among those assessed and identified as having a problem	57.4	120/209	45.3	125/276	−12.1	.059	.098

^*Difference in percent adherence between 2009 and 2006.

^†Logistic regression model P values for change in adherence controlling for practice site.

^‡Logistic regression model P values for change in controlling for practice site and payer status.

There was significant variability in the amount of change over time across practice sites on the pain assessment indicator (P < .001). The extent of the variability is illustrated in Figure 1A, which shows the magnitude and direction of change in performance on this indicator between 2006 and 2009 by practice site. Changes ranged from an increase of 26% at practice H to a decrease of 15% at practice J. The amount of change over time on this indicator did not vary by type of cancer (P = .83). Similarly, there was significant variability across practice sites in the amount of change over time on the emotional well-being assessment quality of care indicator (P < .001). The extent of the variability is illustrated in Figure 1B. Changes ranged from an increase of 21% at practice C to a decrease of 24% at practice D. The amount of change over time on this indicator also did not vary by type of cancer (P = .972).

Figure 1.

(A) Variability in adherence to pain assessment indicator over time across practice sites. (B) Variability in adherence to emotional well-being assessment indicator over time across practice sites. (C) Variability in documentation of problems in emotional well-being over time across practice sites.

Changes in Documentation of Problems in Emotional Well-Being

Although it is not a quality indicator per se, analyses were also conducted to determine whether documentation of problems with emotional well-being changed over time. Table 2 lists by year the prevalence of documented problems in emotional well-being and among those patients for whom there was documentation that emotional well-being was assessed (n = 854) and among all patients (N = 1,609). As shown, the prevalence of documented problems increased significantly (P ≤ .021) in both instances. Among patients in whom emotional well-being was assessed, the prevalence of documented problems increased by 6.8%, from 24.5% in 2006 to 31.3% in 2009. This increase translates into an increase of 3% among all patients, from 13.0% in 2006 to 16.0% in 2009. The same pattern of significant increases was observed in multivariable analyses that included payer status (P ≤ .026).

There was significant variability in the amount of change over time across practice sites for documentation of problems among patients in whom emotional well-being was assessed (P = .004) and among all patients (P = .02). The extent of the variability is illustrated in Figure 1C, which shows the magnitude and direction of change between 2006 and 2009 by practice site in documentation of a problem in emotional well-being among patients who were assessed. Change ranged from an increase of 26% at practice D to a decrease of 8% at practice B. It is also worth noting the extent of variability across practice sites in the percentage of assessed patients identified as having a problem in emotional well-being. In 2006 it ranged from 8% of patients in practice A to 69% of patients in practice J, and in 2009 it ranged from 10% of patients in practice A to 92% of patients in practice J. In both years, the extent of variability was significant (P < .001). The amount of change over time did not vary by type of cancer for documentation among patients in whom emotional well-being was assessed (P = .28) or among all patients (P = .67).

Changes in Performance on Action Taken Indicator

Table 2 presents the average performance rates for the action taken indicator in 2006 and 2009 and the percent change over time. Among patients identified as having a problem in emotional well-being, the percentage of patients in whom action was taken declined by 12.1% between 2006 and 2009. The magnitude of the change was not significant before (P = .059) or after (P = .098) adjusting for payer status. The variability in change over time on this indicator was not significant across practice sites (P = .258) or by cancer type (P = .920).

Discussion

Comparisons between 2006 and 2009 for the quality of psychosocial care indicators and the prevalence of problems in emotional well-being yielded three different patterns of change. First, there was no change in the percentage of patients for whom assessment of emotional well-being or assessment of pain was documented. Rates were consistently low over time for assessment of emotional well-being (53% in 2006 and 51% in 2009) and consistently high for assessment of pain (87% in 2006 and 88% in 2009). Second, despite no overall increase in the percentage of patients for whom assessment of emotional well-being was assessed, the percentage of patients documented as having a problem in emotional well-being increased significantly over time. Among those patients assessed, 7% more were identified as having a problem with emotional well-being; this translates into an absolute increase of 3% over time among all patients. Finally, there was a nonsignificant decrease of 12% over time in the percentage of patients identified as having a problem in emotional well-being for whom action was taken or an explanation provided for no action being taken. The discussion which follows seeks to explain these findings and consider their implications for future efforts to improve the quality of psychosocial care for patients with cancer.

The lack of an increase over time in assessment of emotional well-being suggests the limited impact feedback alone is likely to have on performance on this quality indicator. This conclusion is supported by reports that only three of the 10 practices undertook efforts aimed specifically at improving the assessment of well-being. Although performance rates at these three sites may have improved, the relative lack of change at several other sites and decreases of 10% or more evident at two sites resulted in no overall increase on this indicator.

Given the lack of an overall increase in assessment of emotional well-being, how could there be a 3% overall increase over time in the percentage of patients identified as having problems in emotional well-being? One possibility is that it reflects a temporal trend for an overall increase in emotional well-being problems among patients at the 10 Florida practice sites. Support for this explanation comes from results showing a change in payer mix over time that included fewer patients having private insurance and more patients being self-pay or receiving charity care. These changes likely reflect the broad US economic recession that began in 2007 and persisted through 2009, which heavily affected Florida and many of its residents.¹⁵ Although this possibility cannot be ruled out, the increase of 7% in problems in emotional well-being among patients who were assessed suggests another explanation. That is, increases in sensitivity of methods used to assess emotional well-being may have resulted in more patients being identified as having problems. As part of their quality improvement efforts, three practices reported moving from informal methods of identifying patients with problems, such as clinical interview, to formal methods, such as routine screening using the Distress Thermometer.¹⁶ Previous research suggests that informal methods often result in under-recognition of problems in emotional well-being compared with the use of validated patient-reported outcome measures to identify problems.¹⁷

The 12% decrease over time in the percentage of patients for whom action was taken for a problem in emotional well-being is disappointing but not surprising in the light of other study findings. Along these lines, it should be noted that only one of 10 sites reported quality improvement efforts aimed directly at improving performance on this indicator. This feature, combined with the increase in the percentage patients identified as having problems with emotional well-being, may be the result of problems with capacity. That is, although more patients were being identified as having problems, there appears to have been no corresponding increase in efforts to ensure they were linked to appropriate resources to address their problems. These findings are consistent with research showing that quality of life outcomes are unlikely to improve by assessment alone.¹⁸ Routine screening needs to be accompanied by algorithms that translate findings into appropriate referrals or care pathways for quality of life outcomes to be positively affected.^19,20 With regard to addressing psychosocial concerns, existing algorithms include the National Comprehensive Cancer Network guideline for distress management¹⁶ and the recently issued American Society of Clinical Oncology guideline adaptation on screening, assessment, and care of anxiety and depressive symptoms.²¹

The strength of this project include its longitudinal design, the use of in-person meetings to provide and discuss audit feedback, the care taken to ensure the accuracy of the data collected, and the focus on three common cancers. Weaknesses include the inability to conduct analyses that nest patients within specific oncologists, the lack of data on practices that did not receive feedback for comparison purposes, and the lack of formalized methodology to translate audit findings into quality improvement plans that were then actively monitored. These weaknesses point to the need to conduct formal demonstration projects that seek to identify and rigorously test different approaches to improving the quality of psychosocial care in outpatient oncology practices. This need is particularly timely given that provision of psychosocial care is now among those aspects of care evaluated by groups such as the American College of Surgeons Commission on Cancer²² and the American Society of Clinical Oncology's Quality Oncology Practice Initiative Certification Program.²³

Appendix

Psychosocial and Pain Quality of Care Indicators and Related Rating Criteria

1. Is there evidence current emotional well-being was assessed within one month of the first visit with a medical oncologist?

Documentation is sufficient if the medical chart includes:

• copy of distress, depression, or anxiety screening measure OR

• copy of form including patient self-report of distress, depression, or anxiety OR

• note with statement referring to current “coping,” “adjustment,” “distress,” “emotional,” “depression,” or “anxiety” status of patient.

If Yes, mark Yes and answer question 2

If No, mark No

2. Was a problem with emotional well-being identified within one month of the first visit with a medical oncologist?

A problem can be considered present if the medical chart includes a statement that the patient is “distressed,” “depressed,” “anxious,” or having problems with “coping,” “adjustment,” or “emotional well-being.” All patients for whom action was taken (see Question 3) are considered to have had a problem with emotional well-being.

If Yes, mark Yes and answer question 3

If No, mark No

3. Is there evidence action was taken to address the problem or an explanation provided of why no action was taken within one month of the first visit with a medical oncologist?

Documentation is sufficient if the medical chart includes one of the following:

• note describing care provided by primary oncology team for problem with “coping,” “adjustment,” “depression,”, “anxiety,” or “distress” OR

• note describing referral to another professional for care of problem with “coping,” “adjustment,” “depression,” “anxiety,” or “distress” OR

• note describing referral to mental health professional (ie, psychiatrist, psychologist, social worker, pastoral care professional, mental health counselor, or psychotherapist) OR

• note describing care provided by another professional for problem with “coping,” “adjustment,” “depression,” “anxiety,” or “distress” OR

• note describing why no action was taken to address problem with “coping,” “adjustment,” “depression,” “anxiety,” or “distress”.

If Yes, mark Yes

If No, mark No

4. Is there evidence that pain was assessed within one month of the first visit with a medical oncologist?

Documentation is sufficient if the medical chart includes a note with a statement referring to any “pain” or “discomfort”.

If Yes, mark Yes

If No, mark No

Authors' Disclosures of Potential Conflicts of Interest

Although all authors completed the disclosure declaration, the following author(s) and/or an author's immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Douglas Faig, Pfizer Honoraria: Guillermo Abesada-Terk Jr, Pfizer Research Funding: Paul B. Jacobsen, Pfizer; Ji-Hyun Lee, Pfizer; William J. Fulp, Pfizer; David Shibata, Pfizer; Christine Laronga, Pfizer; Tawee Tanvetyanon, Pfizer; Fred Schreiber, Pfizer; Richard Brown, Pfizer; Richard M. Levine, Pfizer; Thomas H. Cartwright, Pfizer; Carlos Alemany, Pfizer; Douglas Faig, Pfizer; Philip Sharp, Pfizer; Merry-Jennifer Markham, Pfizer; Mokenge Malafa, Pfizer Expert Testimony: None Patents, Royalties, and Licenses: None Other Remuneration: None

Author Contributions

Conception and design: Paul B. Jacobsen, Erin M. Siegel, David Shibata, Christine Laronga, Jhanelle E. Gray, Tawee Tanvetyanon, Mokenge Malafa

Collection and assembly of data: Paul B. Jacobsen, Ji-Hyun Lee, William J. Fulp, Fred Schreiber, Richard Brown, Richard M. Levine, Thomas H. Cartwright, Guillermo Abesada-Terk Jr, George P. Kim, Carlos Alemany, Douglas Faig, Philip Sharp, Merry-Jennifer Markham

Data analysis and interpretation: Paul B. Jacobsen, Ji-Hyun Lee, William J. Fulp, Erin M. Siegel, Jhanelle E. Gray

Manuscript writing: All authors

Final approval of manuscript: All authors