Development and Validation of WeCares, a Survey Instrument to Assess Hospitalized Patients’ and Family Members’ “Willingness to Engage in Your Care and Safety”

Abstract

Background:

Patient engagement is recognized as a method to improve care quality and safety. A research team developed WeCares (Willingness to Engage in Your Care and Safety), a survey instrument assessing patients’ and families’ engagement in the safety of their care during their hospital stay. The objective of this study is to establish the preliminary construct validity and internal consistency of WeCares.

Methods:

WeCares was distributed to patients and families. With the survey responses, exploratory factor analysis (EFA) was performed to identify the factorial structure of WeCares. The internal consistency was assessed using Cronbach’s alpha. Descriptive and comparative analysis was also performed to summarize patients’ and families’ responses.

Results:

A total of 247 patients and families responded to the WeCare survey, of which 224 were used for EFA. EFA resulted in a 13-item, four-factor structure, including (1) comfortable sharing concerns, (2) responsibility for preventing errors, (3) perception of care team members’ attitude, and (4) patients’/families’ role in preventing errors. The Cronbach alphas were 0.716—0.866, indicating acceptable internal consistency. Overall, patients and families were comfortable sharing concerns with clinicians but preferred to remain anonymous. They believed that the care team members hold most responsibility for error prevention, however, and agreed on their ability to help prevent errors.

Conclusion:

WeCares was developed to assess patients’ and families’ willingness to engage. WeCares can also be used to facilitate conversation about safety concerns and shared responsibility. The study team believes this would lead to patient activation in guarding their own care and ultimately improve patient outcomes and safety.

Patient engagement in their care and safety is increasingly encouraged and recognized as a method to improve safety. As experts in their own care, patients and families are valuable sources of information. A growing body of literature examines the impact of patient engagement on preventing errors and improving patient safety. Advocates propound that “if the focus on patient safety doesn’t begin with, and include the patient, a valuable piece of the health care process is lost.”^{1 (p. 2)} One strong endorsement for the involvement of patients and families in reducing medical errors comes from the World Health Organization (WHO). In 2005 WHO launched the Patients for Patient Safety program with a vision to engage, empower, encourage, and facilitate patients and their families to partner with health care professionals and policy makers to make health care safer, more integrated, and more people-centered.² Waterman and colleagues, in a study exploring patients’ perspectives on error prevention, interviewed 2,078 discharged patients from 11 different Midwest hospitals. Their findings revealed 91% agreement that patient involvement could help prevent errors.³ In addition, a systematic review reported that patient and family engagement strategies significantly improve medication safety outcomes.⁴

Although patients can identify medical errors not otherwise reported,^5–7 their comfort levels for disclosing safety concerns vary. Some studies found that patients are hesitant to voice their concerns,^8,9 while others show that they are generally comfortable with the idea.^3,10,11 Many factors may affect patients’ comfort in reporting safety concerns, such as lack of knowledge or confidence in their knowledge about health care, illness severity, demographic factors,¹² or fears that speaking up may result in negative repercussions.¹³

The idea of shared responsibility was first introduced in 1992, when a study reported that most patients and members of the public want their physicians and families to share the responsibility for decision making and ensuring that high-quality care is provided.¹⁴ A number of studies have determined that patients, their families and/or caregivers, hospital administrators, and health care providers share responsibility for error prevention and patient safety.^15,16 A recent study interviewed medical-surgical patients and concluded that engagement is a dual responsibility for patients and nurses.¹⁷

To promote high-quality and safe patient care, it is important to understand the barriers to and facilitators of patient engagement in the safety and shared responsibility of their care. Several tools are available for assessing patient engagement.^18–21 However, none of them are designed to assess patients’ willingness to act as advocates for the safety and shared responsibility of their care. As a result, we developed a survey instrument, WeCares (the Willingness to Engage in Your Care and Safety survey), to assess patient and family members’ willingness to engage during their hospital stay. In this study, we refined and established the factorial structure of WeCares.

METHODS

Survey Instrument

Based on a review of the literature, we drafted 27 questions to assess adult inpatients’ and family members’ willingness to engage in their care. Draft questions were piloted with patients to assess face and content validity.^3,11,22–25 To shorten the survey as well as establish its face validity, we conducted a Delphi study²⁶ in which we invited patient safety subject matter experts (SMEs) to rate and comment on the survey questions for their significance and relevancy to patients’ and family member’s willingness to engage in their care. The 10 patient safety SMEs included the study’s steering committee members (both internal and external patient safety experts) and some clinical and administrative leaders from Brigham and Women’s Hospital, Boston. The Delphi study survey was sent out to those patient safety SMEs via REDCap electronic survey software.²⁷ After revisions based on the Delphi study, the WeCares survey tested in this study consisted of 19 items with three sections: (1) attitudes (4 items), (2) responsibility (6 items), and (3) comfort (9 items). The attitudes and comfort items were rated on a 4-point Likert scale, in which patients were asked for their agreement or disagreement about their engagement, to provide direction for improvement (Attitude: 1 = strongly disagree; 2 = disagree; 3 = agree; 4 = strongly agree; Comfort: 1 = very uncomfortable, 2 = slightly uncomfortable, 3 = slightly comfortable, 4 = very comfortable); the Responsibility items were rated on a 5-point Likert scale, which allows for the idea of equally shared responsibility between patient and family and the care team (1 = Patients/family are completely responsible; 2 = Patients/family are mostly responsible; 3 = Patients/family and the care team are equally responsible; 4 = The care team is mostly responsible; 5 = The care team is completely responsible). In addition, we asked their preference in identifying themselves versus remaining anonymous when sharing concerns in the areas of medication, pain, plan of care, waiting time, room, hygiene, communication, and privacy. Demographics questions asked included age, gender, race, education, planned or urgent admission, preferred language, literacy, and past inpatient experience. We also included one general health condition question from the modified 12-item Short-Form Health Survey (SF-12) survey²⁸—“In general, how would you say the patient’s health was one month ago?”—using a 5-point Likert scale (1 = excellent; 5 = poor). Health literacy was measured with a single validated question—“How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?”—using a 5-point Likert scale (1 = never; 5 = always).²⁹ All questions were asked in English only.

Data Collection

We conducted a cross-sectional study at a large academic medical center in the northeastern United States from May 2016 to August 2017. The WeCares survey was hosted in REDCap and was administered to patients and families on four units: a medical ICU, a general medicine unit, an intermediate cardiovascular unit, and a surgical ICU. Research staff approached patients on these units during patient engagement rounds on weekdays to participate in the survey and distributed the survey in either paper or electronic format, based on the patient’s preference. The patient was not approached if the nurse believed that it was not clinically appropriate or the patient was not well enough to engage or respond to a survey. We approached only family members in the ICU. Friends who are the primary caregiver could also answer the survey. On the medical ICU, paper copies of the survey, instructions, and a lockbox were provided in the waiting room for families to complete the survey. With the recommended 10:1 subject-to-item ratio,³⁰ we anticipated needing at least 190 responses to have a sufficient sample size for analysis. All activities were approved by the Institutional Review Board.

Data Analysis

We used exploratory factor analysis (EFA) to identify common themes (factors) in survey items and the underlying factorial structure of WeCares. We performed EFA with principal axis factoring extraction method and non-orthogonal Promax rotation to allow factors correlation. We also examined factor loadings as well as number of factors suitable to explain and support the assessment of patient and family member engagement. Before the EFA analysis, we transformed the responses from the 4-point scale based on the rescaling approach by Dawes.³¹ Dawes’s rescaling method is a simple arithmetic approach in which the scale endpoints are anchored. For example, in this study, we rescaled the 4-point scale to 5 points: 1 remains as 1, but 4 is rescaled to 5. The midpoints are adjusted for equal numerical intervals. As a result, the rescaled value of the 4-point scale becomes 1, 2.33, 3.67, and 5. Studies have used rescaling approaches to produce comparable data and demonstrated that the rescaling approaches do not alter or bias the results.^31,32

The EFA process was conducted iteratively to examine factor loadings and eliminate items cross-loading—for example, items with factor loadings of 0.32 or higher on two or more factors, or when an item’s primary loading was not twice its secondary loading.^33,34 The item reduction process eliminates unsuitable items with weak contribution to factors, retaining only “good” items representing the common themes (factors). For internal consistency assessment, we calculate Cronbach’s alpha of each subscale (factor) generated by the EFA.

In addition, we conducted descriptive analysis to summarize patients’ and family members’ willingness to engage in their care and safety. We also performed one-way analysis of variance (ANOVA) with post hoc tests using Bonferroni correction to compare group differences, including demographics, self-reported health condition, and whether they preferred to be identified or remain anonymous, or had no preference, when sharing their concerns.

RESULTS

The original survey development included 27 items tested for face validity with experts in the Delphi survey. After the Delphi survey, WeCares was revised to include 19 items as described above. We received a total of 247 responses. Among those, 242 responses (161 patients, 78 family, 3 friends) had less than 20% missing data, and 224 were complete responses with no missing data. EFA was performed using the 224 completed responses. Table 1 describes the demographics of the participants.

Table 1.

Participant Demographics

Age (in years)	n (%)
21–30	16 (6.6)
31–40	20 (8.3)
41–50	35 (14.5)
51–60	61 (25.2)
61–70	53 (21.9)
71–80	39 (16.1)
> 81	15 (6.2)
Total	239 (98.8%)
Gender	n (%)
Male	105 (43.4)
Female	134 (55.4)
Total	239 (98.8%)
Race	n (%)
White	193 (79.8)
Black	30 (12.4)
Asian	4 (1.7%
Native	0 (0)
Other	12 (5.0)
Total	239 (98.8%)
Reading assistance needed	n (%)
Never	161 (66.5)
Rarely	39 (16.1)
Sometimes	23 (9.5)
Often	12 (5.0)
Always	5 (2.1)
Total	240 (99.2%)
Unit	n (%)
Surgical ICU	24 (9.9)
Intermediate cardiovascular	184 (76.0)
Medical ICU	19 (7.9)
General medicine	15 (6.2)
Total	242 (100%)
Respondent	n (%)
Patient	159 (65.7)
Family	80 (33.1)
Friend	3 (1.2)
Total	242 (100%)
Education	n (%)
Some high school	10 (4.1)
High school	53 (21.9)
Some college	50 (20.7)
College	70 (28.9)
Graduate school	54 (22.3)
Total	237 (97.9%)
Experienced error in care	n (%)
Yes	89 (36.8)
No	114 (47.1)
Not sure	37 (15.3)
Total	240 (99.2%)

Before we proceeded to EFA, the Kaiser-Meyer-Olkin and Bartlett’s Test was used to assess the appropriateness of the sample and correlation matrices for an EFA analysis. We investigated the number of factors suitable for the factor structure using parallel analysis and the rule of eigenvalue > 1. Parallel analysis suggested nine factors but did not exhibit justifiable constructs; the rule of eigenvalue > 1 suggested six factors with justifiable constructs. Based on our baseline assumption of three constructs in WeCares, we explored quantities of three to six factors to examine several potential factorial structures. After iterative analysis with careful elimination of cross-loadings items and discussion, a four-factor solution was determined to provide the most meaningful and justifiable factorial structure with a total of 13 survey items and 53.26% variance explained. Table 2 presents the factor loadings (coefficients) of the final four-factor solution of the WeCares survey. A high coefficient represents that the item has a high contribution to the factor. The four factors are comfortable sharing concerns, responsibility for preventing errors, perception of care team members’ attitude, and patients’/family’s role in preventing errors. The four factors exhibit Cronbach alphas from 0.716 to 0.866, indicating acceptable internal consistency. Table 3 shows that the factors have little correlation. We then performed EFA with orthogonal Varimax rotation, and confirmed the same factorial structure with consistent strong factor loadings ( > 0.39) and no cross-loadings.

Table 2.

Factor Pattern Matrix

Reliability	Item	Factor
		1	2	3	4
F1: Comfortable Sharing Concerns Cronbach α = .766 Variance Explained = 18.603%	1. How comfortable are you sharing concerns about your care with the physician?	.982	.019	−.044	−.018
	2. How comfortable are you sharing concerns about your care with the nurse?	.798	.079	−.103	.097
	3. How comfortable are you sharing concerns about your care with hospital management?	.550	−.054	.087	.033
	4. How comfortable are you asking whether the care team member washed their hands?	.411	−.114	.098	−.067
	5. If you believed an error had occurred in your/the patient’s care, how comfortable would you be notifying the care team member involved?	.404	−.053	.146	−.092
F2: Responsibility for Preventing Errors Cronbach α = .716 Variance Explained = 13.790%	6. Who is responsible for making sure a new treatment or medication is safe for the patient?	−.024	.790	−.020	−.002
	7. Who is responsible for making sure the patient knows why they need a new treatment or medication?	.012	.775	−.032	−.044
	8. Who is responsible for making sure patients get information about their care, such as their test results or schedule for the day?	−.008	.490	.040	−.052
	9. Who is responsible for making sure concerning events or situations in patient care are fixed?	−.121	.478	.115	.101
F3: Perception of Care Team Members’ Attitude Cronbach α = . 866 Variance Explained = 12.249%	10. Care team members do not like it when patients/family share concerns to help prevent medical errors in patient care, such as concerns about receiving the wrong medication or that a care team member did not wash their hands.	.077	.072	.971	−.005
	11. Care team members do not like it when patients/family ask questions about their care, such as questions about new medications or telling the team they have not received a test result.	.047	−.012	.780	.018
F4: Patients’/ Family’s Role in Preventing Errors Cronbach α = .770 Variance Explained = 8.616%	12. Patients/family can help the care team prevent medical errors in patient care.	−.015	−.054	.078	.867
	13. Patients/family can prevent medical errors in patient care.	−.020	.039	−.068	.746

Extraction method: principal axis factoring

Rotation method: Promax with Kaiser normalization

Rotation converged in 5 iterations

Table 3.

Factor Correlation Matrix

Factor	F1	F2	F3	F4
F1. Comfortable Sharing Concerns	1.000	.055	−.024	.254
F2. Responsibility for Preventing Errors	.055	1.000	−.065	.153
F3. Perception of Care Team Members’ Attitude	−.024	−.065	1.000	−.030
F4. Patients’/family’s Role in Preventing Errors	.254	.153	−.030	1.000

Extraction Method: principal axis factoring

Rotation Method: Promax with Kaiser normalization

Descriptive Analysis and Group Comparisons

With the four-factorial structure found in EFA, we further performed descriptive analysis of the responses (Table 4). In general, patients and family members felt comfortable sharing their concerns with health care providers (mean = 4.47; standard deviation [SD] = 0.70). They also considered that the care team members should be primarily responsible regarding prevention of medical errors (mean = 4.00; SD = 0.67). However, they believed that the care team members would not mind if the patient or family member asked about their medications or shared concerns related to medical errors (mean = 3.85; SD = 0.67). Finally, patients and family members agreed that they can help in preventing medical errors (mean = 3.89; SD = 0.71).

Table 4.

Descriptive Analysis

		n	Mean	SD
F1: Comfortable Sharing Concerns Mean = 4.47 SD = 0.70	1. How comfortable are you sharing concerns about your care with the physician?	242	4.80	.69
	2. How comfortable are you sharing concerns about your care with the nurse?	242	4.82	.69
	3. How comfortable are you sharing concerns about your care with hospital management?	236	4.37	1.16
	4. How comfortable are you asking whether the care team member washed their hands?	238	3.70	1.52
	5. If you believed an error had occurred in your/the patient’s care, how comfortable would you be notifying the care team member involved? (After rescaled: 1 = Very uncomfortable; 5 = Very comfortable)	240	4.66	.85
F2: Responsibility for Preventing Errors Mean = 4.00 SD = 0.67	6. Who is responsible for making sure a new treatment or medication is safe for the patient?	240	4.31	.85
	7. Who is responsible for making sure the patient knows why they need a new treatment or medication?	241	4.25	.90
	8. Who is responsible for making sure concerning events or situations in patient care are fixed?	239	3.56	.97
	9. Who is responsible for making sure patients get information about their care, such as their test results or schedule for the day? (1 = Patients/family are completely responsible; 5 = The care team is completely responsible)	241	3.87	.97
F3: Perception of Care Team Members’ Attitude Mean = 3.85 SD = 0.89	10. Care team members do not like it when patients/family share concerns to help prevent medical errors in patient care, such as concerns about receiving the wrong medication or that a care team member did not wash their hands.	240	3.82	.97
	11. Care team members do not like it when patients/family ask questions about their care, such as questions about new medications or telling the team they have not received a test result. (5 = Strongly disagree; 1 = Strongly agree) (After rescaled: 1 = Strongly agree; 5 = Strongly disagree)	241	3.88	.93
F4: Patients’/ Family’s Role in Preventing Errors Mean = 3.89 SD = 0.71	12. Patients/family can help the care team prevent medical errors in patient care.	241	3.93	.78
	13. Patients/family can prevent medical errors in patient care. (After rescaled: 1 = Strongly disagree; 5 = Strongly agree)	241	3.84	.78

SD, standard deviation.

We also asked patients and family members whether they preferred being identified or remaining anonymous, or had no preference, when sharing their concerns about medication, pain, plan of care, waiting time, room, hygiene, communication, and privacy. We found that those who reported they were less comfortable sharing concerns preferred to remain anonymous when sharing concerns about plan of care (p < 0.05), waiting time (p < 0.001), room (p < 0.005), hygiene (p < 0.001), and communication (p < 0.02), compared to patients and family members who preferred to be identified or had no preference. When examining the association between a self-reported health condition (one general health condition question from the modified SF-12) and willingness to engage, we did not find patients’ health condition correlated to any of the four factors in WeCares.

We did not find differences in responses (by patient, family, or friend) when we examined age, gender, education, and unit. However, we found that white respondents reported that the care team has more responsibility for preventing errors than did black respondents (mean = 4.07 vs. 3.68, p = 0.013) and respondents of other races (mean = 4.0 vs. 3.50, p = 0.018). Patients or family members who reported “sometimes” needing help when reading instructions, pamphlets, or other written material from the physician or pharmacy also reported that the care team has less responsibility for preventing errors compared to patients and family members who reported “never” (mean = 3.41 vs. 4.08, p < 0.001) or “often” needing help (mean = 3.41 vs. 4.39, p < 0.001). In addition, 33.3% (n = 10) of black respondents reported “sometimes,” “often,” or “always” needing reading assistances compared to 14.0% (n = 27) of white respondents. Not surprisingly, patients or family members who had experienced errors in care reported being less comfortable sharing concerns (mean = 4.31 vs. 4.57, p = 0.028) and having a more negative perception of care team members’ attitude (mean = 3.64 vs. 4.04, p = 0.004).

DISCUSSION

In this study we established the face validity and content validity of WeCares. We also demonstrated its preliminary construct validity through EFA, resulting in 13 items and the following four factors: (1) comfortable sharing concerns, (2) responsibility for preventing errors, (3) perception of care team members’ attitude, and (4) patients’/family’s role in preventing errors. This structure explained just over half the variance in the sample and had acceptable internal consistency. Our findings indicate that patients and family members overall are willing to engage in the safety of their care in collaboration with health care providers, who have primary responsibility for their safety.

We identified that a portion of the participants preferred to remain anonymous in sharing concerns about plan of care, waiting time, room, hygiene, and communication, confirming results from a prior study by our team that captured safety concerns in real time from hospitalized patients using an electronic app, MySafeCare, that supported anonymous reporting.³⁵ In the earlier study we determined that 55.6% of patients opted to report concerns anonymously and that concerns related to plan of care and hygiene had the highest rates of anonymous reporting.³⁵ In related work, Zimmerman and colleagues assessed patient satisfaction with a self-administered questionnaire and switched between anonymous to non-anonymous completion. They discovered that anonymity was not associated with patient satisfaction ratings—whether the patient’s identity was known versus hidden did not affect how they responded.³⁶ In contrast, a systematic review conducted in 2002 revealed that higher satisfaction scores are likely to be recorded when evaluations are conducted via interviews or on-site surveys—a result the authors attributed to patients wishing to give socially acceptable answers in circumstances in which their anonymity is compromised.³⁷ Lack of anonymity could also lead to patients being less willing to address sensitive or embarrassing issues.³⁷ Another study discussed the paradox of anonymity, explaining that patients generally prefer anonymity for effective feedback, whereas clinicians see anonymity as a barrier to feedback and improvement.³⁸ There is limited research on the topic of anonymous reporting. Our data, however, suggest that there is a need to preserve anonymity in the context of patient safety reporting. More studies should be done to characterize patient anonymity preferences when expressing concern, providing feedback, or asking questions.

Our data also indicate that those with limited health literacy (self-reporting “sometimes” needing help with medical information)³⁹ may believe that the patient and family have more responsibility for preventing errors, compared to patients and family members with adequate or very limited health literacy. Health literacy refers to the set of skills (including reading, writing, and numeracy) necessary to make appropriate and informed health decisions⁴⁰ and plays a vital role in preventing errors and maintaining patient safety. Limited health literacy is associated with medication non-adherence and error,^41–44 higher medical expenses, and increased hospitalization.^45,46 The findings from our study require further investigation to fully explore the relationship between safety engagement and health literacy, particularly in light of other evidence in this area described above.

Furthermore, patients or family members who have experienced errors in care reported feeling less comfortable sharing concerns and having a more negative perception of care team members’ attitude, WeCares can serve as an assessment tool to identify patients’ and family members’ barriers to sharing safety concerns, such as prior experience, health literacy, and anonymity. WeCares can also be used to facilitate conversations about engaging in the safety of and sharing responsibility for one’s own care, following tailored patient/family education based on the WeCares assessment. We foresee that these uses would lead to patient activation in guarding their own care and ultimately improve patient outcomes and safety.

Limitations

The survey participants were patients and family members. Because family members often serve as advocates for hospitalized patients, they were an intended target of this survey. However, due to the small sample size we were not able to stratify the analysis between patients and family members. Future work should evaluate for differences between these two groups. The sample was from a single tertiary care hospital, and patients had to be well enough to respond to the survey or have a care partner respond to the survey, which may have resulted in selection bias. Our participants were mainly white (79.8%) and “never” or “rarely” required reading assistance (82.6%). This work should be replicated in other hospitals and could be expanded to additional patient populations. Other forms of validity could be assessed in the future, such as confirmatory factor analysis to verify the factorial structure and predictive and convergent validity to examine the association between WeCares and other patient engagement and safety measures.

CONCLUSION

We validated the four-factor, 13-item WeCares survey for use with patients and families in the hospital setting. This survey can be used to better understand patients’ and families’ perceptions and attitudes related to safety prevention with their care team. Patient-generated data capture is increasing, including direct capture of safety concerns. These perceptions and attitudes may directly affect safety reporting and disclosure of safety-related concerns by patients and families, leading to variable rates of reporting. A greater understanding of the relationship between perceptions and attitudes related to engagement in care and safety and patient-generated safety data sets will be important to patient safety organizations and data-driven efforts to reduce harm in the inpatient setting.