The Use of a Brief 5-Item Measure of Family Satisfaction as a Critical Quality Indicator in Advanced Cancer Care: A Multisite Comparison

Abstract

Background: Although family satisfaction is recognized as a critical indicator of quality for patients with advanced cancer, it is rarely assessed as part of routine clinical care. Measurement burden may be one barrier to widespread use of family satisfaction measures.

Objective: The goal of this study was to test the ability of a new, brief 5-item measure of family satisfaction with care to accurately capture differences across hospital settings.

Design: Using data from the Palliative Care for Cancer Patients study, a prospective study of 1979 patients and caregivers, we used multivariate regression analysis to detect significant differences across five sites.

Settings: Hospitalized patients with advanced cancer and their caregivers

Methods: We used both the shortened 5-item version of the FAMCARE scale (previously developed using Item Response Theory) and the original 20-item FAMCARE to measure family satisfaction.

Results: On the 5-item FAMCARE, sites ranged from mean scores of 5.5–6.9 out of a possible high score of 10. Family members at one care site (n = 783) were significantly (p < 0.05) less satisfied with their care than family members at four other care sites. The original 20-item measure failed to differentiate satisfaction levels between all hospital sites.

Discussion: Variability in family satisfaction with advanced cancer care across hospital settings can be more sensitively detected using a brief 5-item questionnaire versus longer measures. The development of less lengthy and burdensome measures for monitoring family satisfaction, which are still valid, can facilitate routine assessments to maintain and promote high-quality care across care settings.

Background

As reform efforts shift the U.S. healthcare system's focus toward accountability and value, an increasing emphasis has been placed on the use of quality indicators. Consequently, programs providing medical care are increasingly required to collect, report, and respond to data on quality. Coordinated quality measurement, a priority focus for cancer care,¹ will require the use of valid, reliable, feasible, and actionable quality indicators.

Satisfaction with care has emerged as a key outcome measure for evaluating quality of care for patients with advanced illnesses such as cancer.^2–4 Directly measuring satisfaction in patients with advanced illness can be difficult because of their clinical status; that is, patients are often too ill, too medicated, or too disoriented to report on their experience. The measurement of family satisfaction, therefore, is critical both as a proxy measure for patients and as in independent measure of quality itself. Indeed, according to the 2014 Institute of Medicine report on end-of-life care,⁵ care should ideally be family oriented and consider not only the needs of patients but also those of family and/or caregivers.

In the research setting, at least 14 different instruments have been used to measure family satisfaction in the context of cancer and other serious illnesses.⁶ These existing measures of family satisfaction are lengthy and may be burdensome—particularly in the setting of advanced illness where families may be heavily involved in the stress of caregiving and end-of-life decision making and planning.⁷ Because of the need for a reliable, valid, and brief measure of family satisfaction, we previously developed a brief 5-item version of The Family Satisfaction with End-of-Life Care (FAMCARE), from the original 20-item version,^8–10 using Item Response Theory (IRT). In this shortened version, we excluded items such as “availability of nurses to the family” and “availability of hospital bed,” which were least informative according to the magnitude of IRT-generated discrimination parameters.⁷ These items were also identified in other studies to be poor items.¹¹ Furthermore, we used differential item functioning (DIF) to determine whether individuals in groups with the same underlying level of family satisfaction had different probabilities of endorsing an individual item. Using this methodology, we included items that were relatively invariant across groups differing in patient gender, age, education, and race (black or white).¹² We excluded one item, “information given about patients' tests” that showed the most salient DIF. Finally, IRT analyses demonstrated that the number of response categories could be reduced, further reducing respondent burden.

Although the shortened measure demonstrated good reliability and ability to discriminate between patients with different levels of satisfaction, what remained unknown is how the shortened measure would perform in the “real world” at discriminating among hospital sites. Multiple patient-, family-, and facility-level factors have been associated with variation in family satisfaction.^13–15 We therefore hypothesized that we would detect differences across treatment sites in satisfaction with care, given variation between hospital sites in region, size, overall structure, and patients served. The goal of this study was to test the ability of this 5-item short form scale (the FAMCARE-5) to detect differences in family satisfaction with care across hospital sites. In addition to maintaining the provision of high-quality care for patients and families, measuring variation in satisfaction with care is important to larger healthcare systems whose payments will be increasingly linked to quality measures (e.g., Hospital Value-Based Purchasing).

Methods

Sample

The Palliative Care for Cancer Patients (PC4C) study is a multisite observational study of the effect of inpatient palliative care on patient health outcomes and health services use among patients with advanced cancer.¹⁶ Hospitalized patients with advanced cancer were recruited from five high-volume tertiary-care medical and cancer centers in New York, Virginia, Pennsylvania, Ohio, and Wisconsin.

Patients were required to meet the following criteria: age 18 and older; diagnosis of metastatic solid tumor or central nervous system malignancies; locally advanced head and neck or pancreas cancers; metastatic melanoma; or transplant-ineligible lymphoma and myeloma. Participants were excluded if they did not speak English; had a diagnosis of dementia; were admitted for routine chemotherapy; had previously received palliative care consultation; or if their attending physician did not give permission to recruit their patients. During hospitalization, patients identified the individual who was their primary contact person. Patients were followed until hospital discharge or death, and postdischarge surveys were conducted via telephone with patients and/or family members. Caregivers were identified by the patient in the hospital as the “person that best knows about your health or who helps you the most with your routine daily activities.” Caregivers were interviewed via telephone one week after patient discharge. If a patient died during the hospitalization, the caregiver was contacted by telephone two months after the patient's death.

All patients signed written informed consent and caregivers provided either written or telephone consent either at the time of hospitalization or at follow-up. The study was approved by the institutional review board of each participating facility.

Measures

Family satisfaction with end-of-life care (FAMCARE) for cancer patients

The FAMCARE is a 20-item scale designed to measure family satisfaction with advanced cancer care.^8,9 An overall score for the original scale is calculated by summing the 20 five-point Likert-scale items, with a higher score indicating greater satisfaction with care.

FAMCARE-5

We previously used IRT analysis to create a unidimensional 5-item short form of FAMCARE called the “FAMCARE-5.” The shortened measure demonstrated high reliability and little evidence of DIF across items.^7,12,17 The FAMCARE-5 items were coded as follows: “Very satisfied” as 2, “satisfied” as 1, and indecision or “dissatisfaction” as 0. Total scores range from 0 to 10, with 10 indicating highest satisfaction.

Demographic and clinical factors

We also examined the following demographic factors that have previously been associated with family satisfaction with care: patient age, gender, race, marital status, education, Medicaid status, and family member relationship to patient. Clinical data come from medical record review completed by trained project staff and from patient baseline interviews and daily symptom inventories. We categorized cancer diagnosis as gastrointestinal, gynecological, breast, brain, head/neck, lung, lymphoma/myeloma, and other. We used continuous variables for comorbidities (Elixhauser index¹⁸) and the Condensed Memorial Symptom Assessment Scale (CMSAS).¹⁹ We controlled for patient functional status based on complete need for assistance with at least one activities of daily living (ADL).²⁰ We also adjusted for length of hospital stay and receipt of palliative care consultation during hospitalization.

Analysis

We first used ANOVA, with Bonferroni correction for multiple comparisons, to examine differences between each hospital site's FAMCARE-5 score. We then used multiple ordinary least squares (OLS) regression analysis to examine whether there were site differences in family satisfaction with care after controlling for key clinical and demographic characteristics. First, we examined the association of each site relative to the lowest scoring site. Next, we controlled for patient age, gender, race, education, Medicaid status, marital status, and caregiver's relationship to patient. In our final model, we also included cancer type, number of patient's comorbidities, symptom score, ADL status, hospital length of stay, and receipt of palliative care consultation. These findings were compared to site differences detected using the original 20-item FAMCARE. Because of the data distribution of the 20-item FAMCARE, we used generalized linear models with Poisson distribution with power-2 links and calculated average marginal effects. Individuals who did not respond to 50% of the items (1.1% of sample) were omitted from analyses. For those with complete data, the scale was scored by taking the mean of all items. For those missing individual items, we imputed the missing values based on the mean score across completed items, and then took the sum across all items. As a sensitivity analysis, we also ran this model limiting our sample to those with complete data on all 20 items.

All analyses were completed using Stata version 13.1.

Results

Three thousand ninety-six patients were enrolled in the study and 89% identified a caregiver. Two thousand three hundred ninety of identified caregivers consented to participate (87%). We collected FAMCARE survey data from 1979 caregivers across five hospital sites ranging from 118 to 682 per site (Table 1), representing 83% of consented caregivers. Seventy-seven percent of caregivers completing the survey were family members, the majority of whom were living with the patient. More than half of the patients included were female (55%) with a mean age of 59.7. More than one-quarter of the patients had gastrointestinal cancer; 13% had lung cancer; and 12% had gynecological cancer. Eighteen percent received palliative care during their hospital stay. The vast majority of patients were discharged from the hospital alive (97%).

Table 1.

Demographic and Clinical Characteristics of Patients with Advanced Cancer by Healthcare Treatment Site

		Site
	Overall (n = 1979)	1 (n = 192)	2 (n = 118)	3 (n = 274)	4 (n = 713)	5 (n = 682)	p
Female, % (n)	55.0 (1088)	46.9 (90)	55.1 (65)	51.8 (142)	57.6 (411)	55.7 (380)	0.0782
Age, mean (SD)	59.7 (11.8)	59.9 (11.4)	63.5 (12.1)	58.5 (12.3)	60.0 (11.7)	59.1 (11.6)	0.0020
Race, % (n)
White	75.3 (1490)	85.9 (165)	85.6 (101)	70.1 (192)	92.6 (660)	54.6 (372)	0.0001
Black	20.8 (411)	10.4 (20)	11.9 (14)	18.6 (51)	6.0 (43)	41.5 (283)	0.0001
Other	3.9 (78)	3.6 (7)	2.5 (3)	11.3 (31)	1.4 (10)	4.0 (27)	0.0001
Married/cohabiting, % (n)	58.6 (1159)	61.5 (118)	67.0 (79)	56.6 (155)	69.2 (489)	46.6 (318)	0.0001
Insurance-Medicaid, % (n)	11.7 (232)	8.9 (17)	4.2 (5)	12.4 (34)	3.1 (22)	22.6 (154)	0.0001
Education, % (n)
College or higher	30.2 (594)	32.8 (63)	20.3 (24)	49.8 (136)	30.6 (216)	22.9 (155)	0.0001
HS/some college	53.3 (1048)	60.4 (116)	61.9 (73)	38.8 (106)	62.2 (439)	46.3 (314)	0.0001
<HS	16.5 (325)	6.8 (13)	17.8 (21)	11.4 (31)	7.2 (51)	30.8 (209)	0.0001
Caregiver relationship, % (n)
Family member living with patient	41.8 (828)	20.3 (39)	44.9 (53)	59.1 (162)	74.9 (534)	5.9 (40)	0.0001
Family member not living with patient	35.1 (694)	35.4 (68)	27.1 (32)	22.3 (61)	13.5 (96)	64.1 (437)	0.0001
Other	11.9 (236)	16.2 (31)	11.0 (13)	7.7 (21)	2.5 (18)	22.4 (153)	0.0001
Unknown	11.2 (221)	28.1 (54)	17.0 (20)	11.0 (30)	9.1 (65)	7.6 (52)	0.0001
Palliative care, % (n)	18.2 (360)	9.4 (18)	22.0 (26)	7.3 (20)	10.7 (76)	32.3 (220)	0.0001
Cancer type, % (n)
GI	26.7 (528)	18.8 (36)	21.2 (25)	29.2 (80)	32.4 (231)	22.9 (156)	0.0001
GYN	12.2 (242)	6.8 (13)	8.5 (10)	13.5 (37)	14.9 (106)	11.1 (76)	0.0130
Brain	2.5 (50)	3.6 (7)	3.4 (4)	1.1 (3)	2.9 (21)	2.2 (15)	0.3523
Breast	12.2 (241)	8.3 (16)	9.3 (11)	9.1 (25)	13.6 (97)	13.5 (92)	0.0781
Head and neck	6.0 (118)	5.2 (10)	0.0 (0)	11.7 (32)	3.5 (25)	7.5 (51)	0.0001
Lung	12.9 (255)	12.0 (23)	26.3 (31)	9.9 (27)	7.2 (51)	18.0 (123)	0.0001
Lymphoma/myeloma	7.0 (139)	9.9 (19)	9.3 (11)	2.9 (8)	6.9 (49)	7.6 (52)	0.0283
Othera	20.5 (406)	35.4 (68)	22.0 (26)	22.6 (62)	18.7 (133)	17.2 (117)	0.0001
Number of Elixhauser comorbidities, mean (SD), range: 0–10	2.8 (1.5)	2.8 (1.2)	4.0 (1.7)	2.5 (1.4)	2.4 (1.4)	3.1 (1.4)	0.0001
Needs complete assistance with 1+ ADL, % (n)	6.5 (129)	8.3 (16)	16.1 (19)	4.4 (12)	2.7 (19)	9.3 (63)	0.0001
Number of symptoms, mean (SD), range 0–14	6.9 (3.6)	7.2 (3.6)	8.9 (3.1)	6.4 (3.6)	5.8 (3.6)	7.9 (3.3)	0.0001
Hospital length of stay, mean (SD), range 0–46	5.8 (4.8)	4.7 (3.6)	6.7 (5.0)	5.9 (4.9)	5.7 (4.7)	5.9 (5.0)	0.0067

^aOther cancer types include kidney/renal cell cancer, melanoma, osteosarcoma/sarcoma cancer, penis and prostate cancer, carcinoid, thyroid cancer, and other.

ADL, activities of daily living; GI, gastrointestinal; GYN, gynecological; HS, high school.

Mean FAMCARE-5 family satisfaction scores across sites ranged from 5.5 (SD = 1.9) to 6.9 (SD = 2.6) out of a possible score of 10 indicating the highest level of satisfaction (Fig. 1). Pairwise comparisons with Bonferroni corrections indicated that one site differed significantly from three others.

FIG. 1.

Mean scores and 95% confidence intervals across five healthcare treatment sites (n = 1979). Note: FAMCARE-5 is a summary score of 5 items (range = 0–10).

In our fully adjusted models, we continued to see that four sites clustered together and were significantly different in terms of FAMCARE-5 score (ranging from 0.4 points to 1.2 points) than one site (p < 0.05) (Table 2). Because of high missingness (11%) for relationship of caregiver to patient, we modeled unknown relationship status and ran a sensitivity analysis excluding cases with missing status with no differences in findings. Similarly, to ensure that results were not sensitive to model specification, in sensitivity analyses we also fit our model using generalized linear model (GLM) with Poisson distribution and power-2 link. The results were similar across model specification (Supplementary Table S1; Supplementary Data are available online at www.liebertpub.com/jpm).

Table 2.

Multivariate Regression Analysis Examining Association between Healthcare Treatment Site and Family Satisfaction with Care Using the FAMCARE-5 (n = 1979)

	Model 1				Model 2a				Model 3b
Variable	Beta	SE	t	p	Beta	SE	t	p	Beta	SE	t	p
Intercept	5.54	0.09	60.13	<0.0001	5.12	0.35	14.57	<0.0001	5.91	0.40	14.87	<0.0001
Site 1	1.36	0.20	6.89	<0.0001	1.26	0.21	6.01	<0.0001	1.18	0.21	5.56	<0.0001
Site 2	0.62	0.24	2.58	0.0098	0.43	0.26	1.69	0.092	0.60	0.26	2.27	0.024
Site 3	1.17	0.17	6.80	<0.0001	1.02	0.20	5.15	<0.0001	0.88	0.20	4.37	<0.0001
Site 4	0.84	0.13	6.50	<0.0001	0.57	0.18	3.15	0.002	0.43	0.18	2.33	0.020
Adjusted R2	0.039			0.048				0.063

Referent group is site 5.

^aControlling for patient age, gender, race, education, marital status, Medicaid status, relationship to patient.

^bControlling for patient age, gender, race, education, marital status, Medicaid status, relationship to patient, cancer type, number of comorbidities, symptom score, ADL status, hospital length of stay, and receipt of palliative care consultation.

ADL, activities of daily living.

We then sought to determine whether the original 20-item FAMCARE would detect similar between-site differences. The average score on the 20-item scale (possible range 20–100) was 82.3 with a range of 80.6–84.9 across sites. Although the 20-item scale was highly correlated with the 5-item scale (r = 0.89), the 20-item scale only detected statistically significant differences between two hospital sites (sites 1 and 5) using multivariate modeling of the mean score (see Supplementary Table S1 for comparison of marginal effects using GLM models for the 5-item and 20-item FAMCARE measures). In addition to testing the 20-item scale using all cases (Supplementary Table S1), we also tested it using the summary score of participants who had complete data on all 20 FAMCARE items (n = 62%). Findings remained the same (data not shown).

Discussion

Family satisfaction is a critical indicator of quality of care for advanced cancer patients. Our study indicates that variability in family satisfaction with advanced cancer care across sites can be detected using a brief 5-item questionnaire. Having a brief scale may facilitate the more routine assessment of family satisfaction for internal quality improvement and may fill a much needed measurement gap in accountability programs.²¹

Our results demonstrate that not only was the FAMCARE-5 effective in detecting between-site differences it also improved on the original 20-item scale in discriminating among treatment sites. The better performance of FAMCARE-5 is likely due to the fact that it excludes items from the original scale that performed poorly psychometrically (e.g., availability of nurses), and thus were not appropriately capturing the construct of family satisfaction with care, and in turn, diluting the scale's ability to detect differences. Furthermore, the 5 items we included were free of DIF, a critical step in scale validation. The original 20-item scale contains at least 1 item with salient DIF. Finally, better ability to discriminate among sites is consistent with “known groups” validity analyses. Based on previous research,¹³ we posited that family members of older persons, whites and those with higher education, would be more satisfied with care. These hypotheses were confirmed using the 5-item scale. The p-values for differences in scores between younger age groups and those 65 and older trended toward significance (p = 0.072); whites were more satisfied than blacks (p < 0.001) and those with a high school education and above were more satisfied than those with lower educations (p < 0.001). The effect sizes (mean differences) for the 20-item scale were proportionately smaller and were not significant for age.

The ability to easily detect differences in quality across sites with a brief non-burdensome instrument will facilitate the ability of healthcare systems to improve cancer care. The development and dissemination of an instrument that can detect differences across sites will lay the foundation for subsequent research that can improve clinical practice. The next critical step is to determine why these differences exist (lack of support staff, poor training, etc.), through widespread use of this tool.

There are several study limitations worth noting. We had limited demographic data on caregivers who completed the satisfaction measure, which may explain site differences in satisfaction with care. We recognize that numerous family, cultural, and socioeconomic factors may influence caregivers' responses to family satisfaction measures. Although such data were not available in this secondary data analysis, we strongly encourage future researchers to examine a wider variety of caregiver variables potentially related to family satisfaction with care, including type of relationship, relationship quality, caregiver health, and caregiver resources. Missing covariate data were dropped using listwise deletion in multivariate models. Our sample was limited to patients with a specific group of malignancies and while we capture receipt of palliative care, we did not include all treatment information. In addition, the sample was limited to caregivers who spoke English, limiting study generalizability. Although we collected data across five hospitals, all were academic inpatient facilities. We will continue to test the FAMCARE-5 in more diverse healthcare treatment settings to provide the data needed to support widespread dissemination of this tool.

The results of this study, and the body of earlier developmental work, support the usefulness of FAMCARE-5. The development and dissemination of short and well-studied measurement tools such as the FAMCARE-5 are critical to providing high-quality care to patients with advanced cancer. The absence of robust measures for psychological, social, and spiritual distress in cancer care has been identified as a key gap area in quality measurement.²² Although not routine, care experience surveys for bereaved family members are utilized. For example, the U.S. Department of Veterans Affairs administers a bereaved family survey when a veteran dies after using end-of-life care,²³ and the Hospice Quality Reporting Program (HQRP)²⁴ now includes the Hospice Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey for bereaved family members.²⁵ Yet, the assessment of satisfaction with care during end-of-life care is rarely practiced. For example, although the National Quality Forum (NQF) has endorsed nearly 50 measures pertaining to cancer care, there is no endorsed measure of family satisfaction,²¹ a notable gap in CMS accountability programs. Within palliative care, leaders have prioritized the development of valid and implementable quality indicators to measure quality of care as evidenced by the Measuring What Matters (MWM)^26,27 consensus project. MWM currently includes 10 specific quality indicators, including pain treatment, discussion of emotional or psychological needs, and care consistency with documented care preferences. While their recommendations include a global indicator of patient/family perceptions of care, no specific survey instrument is identified.

Conclusion

Assessing satisfaction with care is critical to ensuring high-quality care in the setting of advanced cancer. Our results indicate that family satisfaction can be variable both within and between hospitals that provide advanced cancer care, highlighting the critical importance of routinely assessing family satisfaction to maintain and promote patient care. The use of FAMCARE-5 can reduce the burden of assessing satisfaction for family members in both clinical and research settings and can decrease barriers to measuring and ensuring high-quality advanced cancer care.

Acknowledgments

This project was funded by the National Cancer Institute, Grant 5R01CA116227-05 and the Claude Pepper Older Americans Independence Center: National Institute on Aging, P30, AG028741. Dr. Ornstein was supported by the National Institute on Aging, Grant K01AG047923, and a career development award from and the National Palliative Care Research Center. Dr. Schnur was supported by the National Cancer Institute, Grant R21CA173163. Dr. Garrido was supported by VA HSR&D 11-201/12-255. The views expressed in this article do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government. Earlier versions of this work were presented at the 2015 European Association for Palliative Care meeting and the 2015 American Academy of Hospice and Palliative Medicine meeting.

Author Disclosure Statement

No competing financial interests exist.