Reliability and Validity Testing of the Caregiver Centered Communication Questionnaire

Abstract

Objective:

The purpose of this study is to test the reliability and construct validity of a new instrument, the Caregiver Centered Communication Questionnaire (CCCQ), designed to assess the extent to which family caregivers feel their perspectives and needs are appropriately acknowledged and addressed by the health care team.

Methods:

We administered the CCCQ to adult family caregivers of hospice patients. We calculated Cronbach’s alpha and performed structural equation modeling. Following the estimation of the congeneric model, modification indices (MI) were examined to establish cross-loading. We used RMSEA, CFI and TLI, and SRMR to determine the quality of model fit.

Results:

525 caregivers completed the CCCQ instrument. The Cronbach’s alpha was α = 0.98, indicating good reliability. Our final model demonstrated good fit to our data, χ²(354)=711.653, RMSEA=0.044(0.039, 0.049), CFI=0.99, TLI=0.99, SRMR= 0.029.

Conclusion:

Caregivers are asked to play an ever-increasing role in managing aspects of care. This role is not fully reflected in available toolkits to assess processes of care, and the CCCQ, designed to address this gap, was found to be reliable and valid. Assessing caregiver centered communication should and can be integrated into the evaluation of health care services delivered to patients and families.

1. Introduction

Family caregivers play a critical role in the delivery of health services to patients with chronic and serious illness and our health care system heavily relies on their mostly unpaid work. In 2020, approximately 53 million adults in the United States were family caregivers who provided unpaid care to a relative or other individual who was ill, disabled or aged; 24% of them provided care for more than one person and 26% of them reported having difficulties coordinating care (National Alliance for Caregiving, 2021). In many instances when patients face cognitive and/or functional limitations or their communication is severely impacted by disease, caregivers are called to function as proxy decision makers for critical treatment and disease management decisions. In these cases, caregivers often feel overwhelmed by these responsibilities, consider themselves unprepared for this role and find significant challenges in their communication with health care providers (Oliver et al, 2017).

Family centered care that recognizes and addresses the needs of family caregivers facilitates better adherence to treatment plans, and has the potential to lead to increased patient and family satisfaction, and ultimately better health outcomes (Powazki, 2011). While the active engagement of caregivers in the process of care and the value of meaningful and inclusive communication are recognized as important goals by health care delivery systems, there is a lack of reliable and valid tools that measure caregiver centered communication. In recent years, instruments have been developed to assess the family caregiver experience such as the Caregiving Health Engagement Scale, which assesses the caregiver’s engagement in health care in terms of readiness to be an active, skilled and motivated player in the care process of their loved ones (Barello et al., 2019), and the CaTCoN (Cancer Caregiving Tasks, Consequences and Needs Questionnaire), which measures caregivers’ experiences pertaining to their tasks and responsibilities (Lund et el, 2014). In the context of end of life care, the Family Resilience Questionnaire assesses family dynamics and resources for caregivers of a loved one with cancer (Hjoerleifsdottir et al., 2019), and the Canadian Health Care Evaluation Project introduced a scale to assess family’s perception and satisfaction with end of life care (Heyland et al., 2010). These scales examine aspects of the caregiver experience but focus solely on one setting of care or condition (eg, cancer, end of life) or investigate caregiver attitudes, but do not address the extent to which communication with the caregiver addresses caregiver needs and expectations.

In order to address this gap, we developed an instrument that measures caregiver centered communication meant to be used in a broad range of settings and for a variety of clinical conditions. This tool is designed to assess the extent to which caregivers feel their perspectives and needs are appropriately acknowledged and addressed by the health care team, and to identify any communication barriers. The instrument labeled the Caregiver-Centered Communication Questionnaire (CCCQ) was informed by the National Center Institute (NCI) framework for patient-centered communication by Epstein and Street (2007) expanded to reflect the caregiver perspective. The questionnaire includes 5 subscales informed by the Epstein and Street (2007) framework: exchange of information, relationship with team, emotions, managing care, and decision-making. It has 30 items and answer choices for all items use a 5-point Likert scale, with 1 meaning “strongly disagree” and 5 meaning “strongly agree.” Preliminary testing included face validity by domain experts and reliability testing that indicated a Cronbach α for the entire scale at 0.97 (ranging from 0.82 to 0.93 for each of the 5 subscales) (Demiris et al., 2020). This previous work did not focus on validity testing and had a small sample size. The purpose of this study is to further test the CCCQ and more specifically, to confirm the previous reliability test with a larger sample size and also to assess the instrument’s construct validity.

2. Materials and Methods

Participants and Data Collection

We administered the CCCQ to family caregivers of hospice patients who participated in two large randomized clinical trials (one on the East Coast approved by the University of Pennsylvania Institutional Review Board and one in the Midwest approved by Washington University Institutional Review Board). Data included in the present analysis were obtained from the first data collection point at which the CCCQ was administered to minimize any potential effects of study procedures or interventions. Individuals were eligible to participate in the study if they were 18 years or older and identified as the family caregiver of an adult patient in hospice. Caregivers were excluded if they were not able to speak and read English. Informed consent was obtained by all study participants.

Data Analysis

To confirm findings of the previous reliability analysis [6] with a larger sample, Cronbach’s alpha was examined for all subscales and the full scale. As in the previous tests, we considered a Cronbach’s alpha of .7 or greater evidence of good reliability of the instrument (Silverstein, 1980).

The factor structure of the CCCQ is based on theory as well as a priori hypotheses about how items load on factors, therefore, to validate the measure, we conducted a confirmatory factor analysis. Due to the observed variables being ordinal Likert scale-type items, we used diagonal weighted least squares to estimate the models, which is the most appropriate estimator for these types of items and deals well with data that are not normally distributed (Li, 2016). Elsewhere, reliability for each item within their respective constructs has been established, therefore we began with a congeneric structure, i.e., no cross-loadings of manifest variables between latent variables, of the measure (Demiris et al., 2020). Following the estimation of the congeneric model, modification indices (MI) were examined to establish cross-loading and correlated measurement errors that were either theoretically or practically applicable based on content or item wording. Due to no previous factor analysis being performed on this measure, we used a MI threshold of 5 to determine potential changes to the model. Selected modifications were pooled into groups of 5 MI’s and added to the model one at a time. Throughout this process, for each subsequent model specification, non-significant item loadings were removed, and model fit was reassessed. Chi-square difference testing was used to confirm that the model improved after adding each group of changes. In general, models are considered good fit when chi-square values are not statistically significant, RMSEA <0.05, CFI and TLI >0.95, and SRMR <0.08 (Hu & Bentler, 1999). Due to the sensitivity of chi-square to sample size, we primarily used RMSEA, CFI and TLI, and SRMR to determine the quality of model fit.

3. Results

A total of 525 caregivers completed the entire CCCQ instrument (and another 18 completed only parts of the scale). The majority of caregivers (79%) were female and average age was 57.1 years. The CCCQ total score can range from 30 to 150 and in our sample the average score was 119 (SD 23.3). A description of the participant sample as well as descriptive statistics for the total CCCQ score in the completed sample can be found in Table 1.

Table 1.

Summary of Participant Demographics and CCCQ Score.

	Overall (N=525)
Gender
Female	355 (78%)
Male	98 (22%)
Age (years)
Mean (SD)	57.9 (12.1)
Median [Min, Max]	59.0 [22.0, 94.0]
Race
White	339 (76%)
Black/AA	90 (20%)
Other	18 (4%)
Marital Status
Divorced/separated	50 (10%)
Other	3 (1%)
Partnered	343 (66%)
Single, never partnered	83 (16%)
Widowed	43 (8%)
Education
Graduate/professional degree	113 (22%)
High school/GED	100 (19%)
Less than high school	12 (2%)
Other	1 (0%)
Some college/trade school	165 (32%)
Undergraduate degree	132 (25%)
Relationship to Patient
Adult child	245 (47%)
Spouse/partner	167 (32%)
Other relative	53 (10%)
Other	59 (11%)
Lives with Patient
Co-reside	291 (56%)
Less than 1 hour away	199 (38%)
More than 1 hour away	28 (5%)
Days in Hospice (days)
Mean (SD)	21.6 (12.0)
Median [Min, Max]	18.0 [0, 79.0]
Patient Residence
Private residence	426 (81%)
Nursing home	68 (13%)
Other	30 (6%)
CCCQ Total Score
Mean (SD)	119 (23.3)
Median [Min, Max]	119 [30.0, 150]

In our sample, participants rated the quality of caregiver centered communication on average as high. Table 2 lists the mean scores for all items grouped by constructs. In this analysis all 543 responses (even for the 18 partially completed surveys) were used.

Table 2.

Descriptive Summary of Individual CCCQ Items (grouped by underlying contstructs)

	Overall (N=543)
	Mean (SD)
Exchange of Information (EOI)
1. My health-care team tries to explore my beliefs, needs and preferences as a caregiver.	4.0 (0.98)
2. My health-care team shares sources of medical information.	4.0 (0.93)
3. My health-care team helps me use and understand medical information.	4.1 (0.94)
4. I feel comfortable sharing information with my health-care team.	4.4 (0.76)
11. My health-care team helps me understand my loved one’s illness.	4.0 (0.97)
Fostering health relationships with healthcare team (FHR)
5. My health-care team knows how to deliver news whether good or bad.	4.1 (0.89)
6. My health-care team is honest and open in disclosing information.	4.2 (0.87)
7. I trust my health-care team’s skills and knowledge.	4.3 (0.82)
8. My health-care team shows that he/she is committed to my loved one’s care.	4.3 (0.90)
9. My health-care team cares about me.	4.1 (0.93)
10. I feel that I have a connection with my health-care team.	4.0 (0.99)
26. My health-care team discusses with me when and where we will talk.	3.9 (1.0)
27. My health-care team discusses with me the roles of the team.	4.0 (0.92)
28. I feel like I am building a partnership with my health-care team.	3.9 (1.0)
29. My health-care team respects my background and culture.	4.1 (0.87)
Recognizing and responding to emotions (RRE)
13. My health-care team helps me deal emotionally with the uncertainty of my loved one’s illness.	3.9 (1.0)
15. My health-care team tries to determine whether I am anxious, depressed or stressed.	3.4 (1.1)
16. My health-care team makes me feel that it is ok to feel the way I do.	3.9 (0.95)
17. My health-care team reassures me and sympathizes with me.	4.0 (0.96)
18. My health-care team helps me deal with my emotions.	3.6 (1.0)
Managing care (MC)
12. My health-care team lets me be actively involved in my loved one’s care if 1 want to.	4.4 (0.81)
14. My health-care team helps me manage and solve problems that 1 face as a caregiver.	3.9 (1.0)
24. My health-care team helps me navigate the complex health-care system.	3.7 (1.0)
25. My health-care team helps me arrange the next step in my loved one’s care.	3.9 (0.96)
Decision making (DM)
19. My health-care team supports me when making decisions about my loved one’s care.	4.1 (0.93)
20. My health-care team helps me make choices and carry out a plan.	4.0 (0.96)
21. My health-care team helps me evaluate my decisions.	3.7 (0.97)
22. My health-care team discusses my choices with me.	3.9 (0.96)
23. My health-care team helps me set my priorities.	3.6 (1.0)
30. My health-care team discusses different possible options for care.	3.9 (0.99)

The results of the reliability analysis indicated confirmation of the original results [6] with a larger sample. The Cronbach’s alpha for the overall measure was α = 0.98, indicating good reliability. The exchange of information, fostering health relationships, recognizing and responding to emotions, managing care, and decision-making subscales also indicated good reliability, with Cronbach’s alphas of α = .91, α = .96, α = .94, α = .87, and α = .95, respectively.

After fitting the congeneric measurement model represented by Figure 1, examining MI’s produced 175 possible cross-loadings or correlated measurement errors, of which 41 were determined to be either theoretically or practically appropriate (16 cross-loadings and 25 correlated measurement errors). Each chi-square difference test while adding selected MI’s to the model was significant, indicating that the model did in fact improve with each modification. Our final model, represented by Figure 2, demonstrated good fit to our data, χ²(354)=711.653, RMSEA=0.044(0.039, 0.049), CFI=0.99, TLI=0.99, SRMR= 0.029. Item loadings for each item are presented in table 3.

Figure 1.

Congeneric Model of the CCCQ Measure.

Figure 2.

Final Model of the CCCQ Measure.

Table 3.

Factor Loadings for CCCQ Subscales

Scale Item	Estimate	SE	z	Significance Level	CI Lower Bound	CI Upper Bound
Exchange of Information
CCCQ 1	0.471	0.090	5.249	<0.001	0.295	0.647
CCCQ 2	0.338	0.087	3.889	<0.001	0.168	0.509
CCCQ 3	0.365	0.094	3.895	<0.001	0.181	0.548
CCCQ 4	0.346	0.177	1.958	0.050	0.000	0.693
CCCQ 11	0.127	0.046	2.761	0.006	0.037	0.218
CCCQ 12	0.332	0.085	3.886	<0.001	0.164	0.499
CCCQ 26	−0.396	0.197	−2.016	0.044	−0.781	−0.011
CCCQ 30	0.199	0.054	3.682	<0.001	0.093	0.305
Fostering Healthcare Team Relationships
CCCQ 4	0.575	0.197	2.922	0.003	0.189	0.961
CCCQ 5	0.858	0.014	60.398	<0.001	0.830	0.885
CCCQ 6	0.903	0.011	85.972	<0.001	0.883	0.924
CCCQ 7	0.909	0.010	90.021	<0.001	0.889	0.929
CCCQ 8	0.919	0.010	90.877	<0.001	0.899	0.939
CCCQ 9	0.690	0.051	13.473	<0.001	0.589	0.790
CCCQ 10	0.908	0.010	91.725	<0.001	0.888	0.927
CCCQ 13	0.133	0.067	1.978	0.048	0.001	0.265
CCCQ 17	0.250	0.056	4.497	<0.001	0.141	0.359
CCCQ 18	−0.162	0.077	−2.116	0.034	−0.312	−0.012
CCCQ 26	1.130	0.222	5.085	<0.001	0.694	1.565
CCCQ 27	0.841	0.016	53.126	<0.001	0.810	0.872
CCCQ 28	0.939	0.009	110.058	<0.001	0.922	0.956
CCCQ 29	0.513	0.066	7.738	<0.001	0.383	0.643
Recognizing and Responding to Emotions
CCCQ 9	0.261	0.054	4.869	<0.001	0.156	0.365
CCCQ 13	0.809	0.064	12.732	<0.001	0.685	0.934
CCCQ 15	0.829	0.017	50.228	<0.001	0.797	0.861
CCCQ 16	0.900	0.013	70.541	<0.001	0.875	0.925
CCCQ 17	0.674	0.054	12.451	<0.001	0.568	0.781
CCCQ 18	1.091	0.071	15.269	<0.001	0.951	1.231
CCCQ 29	0.354	0.067	5.244	<0.001	0.222	0.486
Managing Care
CCCQ 02	0.579	0.126	4.603	<0.001	0.332	0.825
CCCQ 03	0.582	0.138	4.231	<0.001	0.312	0.852
CCCQ 11	0.802	0.053	15.226	<0.001	0.699	0.905
CCCQ 12	0.570	0.125	4.569	<0.001	0.325	0.814
CCCQ 14	0.909	0.010	94.278	<0.001	0.890	0.928
CCCQ 21	−0.870	0.260	−3.354	0.001	−1.379	−0.362
CCCQ 24	0.420	0.188	2.235	0.025	0.052	0.789
CCCQ 25	0.337	0.178	1.891	0.059	−0.012	0.686
Decision-making
CCCQ 1	0.486	0.157	3.095	0.002	0.178	0.794
CCCQ 19	0.917	0.010	89.950	<0.001	0.897	0.937
CCCQ 20	0.941	0.007	136.857	<0.001	0.927	0.954
CCCQ 21	1.802	0.260	6.941	<0.001	1.293	2.311
CCCQ 22	0.916	0.009	105.770	<0.001	0.899	0.933
CCCQ 23	0.905	0.010	90.170	<0.001	0.885	0.925
CCCQ 24	0.414	0.188	2.201	0.028	0.045	0.783
CCCQ 25	0.497	0.178	2.794	0.005	0.148	0.845
CCCQ 30	0.721	0.076	9.523	<0.001	0.572	0.869

4. Discussion

The CCCQ instrument is conceptualized as a tool to assess caregivers’ perceptions of the extent to which communication with health care providers/ the health care team is caregiver centered. Caregivers are asked to play an ever-increasing role in providing supportive services, managing aspects of care and even participating in shared decision making shaping the plan of care. This crucial role is not fully reflected in the available toolkits to assess processes of care, and this instrument is designed to address this gap and provide a platform for a comprehensive assessment of caregiver needs and preferences when it comes to communication with health care teams. The CCCQ instrument can be applied to a broad spectrum of health care settings and contexts where family caregivers play a critical role and not only in hospice care which was the setting for this study.

Reliability testing confirmed the high reliability of the instrument. Additionally, the confirmatory factor analysis indicated that each set of subscale items represented a unique construct. While there were some cross-loading of items, in general, each set of items meant to represent an independent factor did so. The CCCQ survey was developed with five subscales based on an expansion of the conceptual framework of patient centered communication. While each subscale is meant to be conceptually distinct, we found in our preliminary testing moderate to high item-correlations among 5 subscales confirming that each subscale was not an entirely discernable component of the CCCQ. The cross-loading described above confirms this.

There are limitations to this study. Our population was moderately diverse and included only caregivers in the Midwest and East Coast regions of the United States. While participants received the survey early in each of the studies, the fact that these were caregivers who had already consented to participate in a research study may introduce bias in the findings in that they potentially may differ when compared to caregivers who did not consent to participate.

The CCCQ instrument facilitates an assessment of the quality of the communication between health care providers or teams and family caregivers, and informs ways to promote meaningful and caregiver-centered information exchange and decision making. This study confirmed the reliability and construct validity of the CCCQ, supporting its use in future research and practice. The instrument can be used in a variety of settings and the term “health care team” can be replaced by “health care provider” or a specific provider role. Clinical settings such as hospice or dementia care where families play an essential role in the delivery of care, lack instruments that focus on the quality of communication with family caregivers. The CCCQ addresses this gap. Nurses can utilize this scale to better understand how family caregivers are experiencing their communication with the health care team and whether caregivers’ needs and preferences are being met. This would be applicable to various settings of care for patients with serious illness or complex care needs and in situations where the caregiver is sometimes called to act as a proxy decision maker. This is of great importance especially when shared decision making is an objective of the processes of care. The instrument can be used over time to assess potential changes in communication patterns and also evaluate behavioral or educational interventions that are meant to improve communication, trust and transparency. Nurses can identify potential deficiencies in team communication and both work with the rest of the health care team to improve communication skills as well as with family members to empower them to raise their concerns and preferences and navigate the clinical landscape. Additionally, the scale can be of use in nursing education as it highlights the essential constructs to caregiver centered communication and increases awareness of empathic and effective communication with family members that can ultimately increase family satisfaction with care, and potentially improve patient and caregiver outcomes.

Nurses need to recognize and address communication needs and preferences of both patients and their family caregivers in clinical encounters. The use of reliable and valid tools to assess caregiver centered communication can inform strategies and interventions to meaningfully engage family caregivers as care partners and promote shared decision making.