Learning from Patients: The Impact of Using Patients’ Narratives on Patient Experience Scores

Abstract

Background:

Enthusiasm has grown about using patients’ narratives — stories about care experiences in patients’ own words — to advance organizations’ learning about the care that they deliver and how to improve it but studies confirming association have not been published.

Purpose:

We assessed whether primary care clinics that frequently share patients’ narratives with their staff have higher patient experience survey scores.

Approach:

We conducted a one-year study of 5545 adult patients and 276 staff affiliated with nine clinics in one health system. We used multi-level models to analyze survey data from patients about their experiences and from staff about exposure to useful narratives. We examined staff confidence in own knowledge as a moderator because confidence can influence use of new information sources.

Results:

Frequency of sharing useful narratives with staff was associated with patient experience scores for all measures, conditional on staff confidence in own knowledge (p-values<0.01). For operational measures (e.g., care coordination), increased sharing correlated with subsequently higher performance for more confident staff and lower performance or no difference for less confident, depending on measure. For relational measures (e.g., patient-provider communication), increased sharing correlated with higher scores for less confident staff and lower scores for more confident.

Conclusion:

Sharing narratives with staff frequently is associated with better patient experience survey scores, conditional on confidence in knowledge.

Practice Implications:

Frequently sharing useful patient narratives should be encouraged as an organizational improvement strategy. However, organizations need to address how narrative feedback interacts with their staff’s confidence to realize higher experience scores across domains.

Introduction

Positive patient care experiences are a core feature of health systems that deliver value (Berwick, Nolan, & Whittington, 2008), a tenet reflected in international growth of care reimbursement contingent not only on cost-effective care but also on patient experience survey scores (Fujisawa & Klazinga, 2017). High scores indicate that care is patient-centered and that patients experience what they should during their interactions with care providers and the system e.g., timely access to care, respectful communication, and coordinated care (Anhang Price et al., 2014). Unfortunately, experience scores persistently indicate that a significant proportion of adults in the United States (U.S.) who visited a doctor or hospital had undesirable experiences (Agency for Healthcare Research and Quality (AHRQ), 2023). Even when satisfaction ratings “top out” near 100 percent (Bland et al., 2022), patients often simultaneously report undesirable experiences (Ranard et al., 2016). These reports have increased since the COVID-19 pandemic, even as patient express gratitude for healthcare workers (Leapfrog, 2022).

In response to the troubling data, healthcare organizations (e.g., hospitals and medical groups) have increasingly turned to their patients’ narratives as an information source that might enable them to better understand the care experiences that they currently deliver and then improve them, and their scores in turn (Emmert, Meszmer, & Sander, 2016; Schlesinger, Grob, & Shaller, 2015a). Patients’ narratives are accounts of healthcare experiences reported in patients’ own words that are written or shared verbally (Schlesinger et al., 2015a). Organizations elicit them through open-ended questions following the score-generating, closed-ended questions in their patient experience surveys. The open-ended questions typically ask patients to describe their positive and negative experiences during their last healthcare visit and share anything else that they wish. Some elicitation sets such as the AHRQ-endorsed Consumer Assessment of Healthcare Providers and System (CAHPS) Narrative Item Set (NIS) also ask patients to share their expectations, whether visit interactions met their expectations, and how well they got along with their provider (AHRQ, 2021).

Qualitative analyses of patients’ narratives suggest that they can be a useful tool for organizations because of the depth and novelty of information that they provide (Grob et al., 2019). Experts thus argue that narratives are a valuable complement to survey scores as an informational and motivating source for improvement (AHRQ, 2021; Grob et al., 2019; World Health Organization, 2018). However, there is no published evidence, to our knowledge, that organizations’ use of patients’ narratives is associated with better patient experience scores, leaving ambiguity about the utility of investing effort and resources in using narratives for quality improvement. The nearest study analyzed hospital patients’ narratives on Yelp, a website for consumer experience-sharing across industries. That study identified ten narrative themes (e.g., caring clinicians and staff) correlated with Yelp patient ratings, which in turn correlated with CAHPS Hospital Survey (HCAHPS) scores, the U.S. standard for measuring inpatient experience (Ranard et al., 2016). Notably, this study focused on narratives that were not systematically collected by organizations or researchers but rather shared online to inform other patients and did not assess hospital use of narratives nor evaluate correlation with HCAHPS scores directly. Thus, while the results are suggestive, they are insufficient evidence that the systematic use of patients’ narratives is associated with better patient experience scores.

To address this gap, we examined the relationship between organizations’ use of patients’ narratives and their subsequent performance on patient experience survey scores for primary care, the cornerstone of health and healthcare in many countries (Starfield, Shi, & Macinko, 2005).

Theory

Qualitative analyses suggests that narratives enrich information in five ways; they offer elaborating, distinctive, representative, actionable, and motivating information. In their narratives, patients often elaborate on why they chose their closed-ended responses, allowing for greater depth of understanding about measured dimensions of care experiences (Schlesinger et al., 2015b). Narratives also provide distinctive information as they often discuss aspects of experiences not assessed by closed-ended questions (Grob et al., 2019; Kilaru et al., 2016; Ranard et al., 2016). They yield more representative information across sub-populations because, although structurally disadvantaged patients (e.g., less educated, racialized minorities, older, etc.) may have lower survey response rates than advantaged counterparts, they share narratives with similar frequency and richness, thereby offering more insight about their experiences and the survey scores they assign than is available otherwise (Grob et al., 2016; Schlesinger et al., 2020). Furthermore, narratives provide actionable insight because patients often include their creative solutions for issues and interaction details (e.g., the who, what, when, etc.) that facilitate intervention (Grob et al., 2019; Kilaru et al., 2016; Lee, Grob, Nembhard, Shaller, & Schlesinger, 2022). Finally, narratives – with their image-rich and emotional content – can be memorable and motivating (Greenhalgh, 2016), prompting leaders and professionals to enact changes that improve care experiences (Valentine, Asch, & Ahn, 2022).

The various ways in which patients’ narratives support organizational learning and improvement suggest that primary care clinics that systematically collect and use patient’s narratives will deliver better patient care experiences provided those narratives are seen by key actors (administrative or clinical) as being useful i.e., capable of being used for desirable purpose. Care delivery is likely to be better because those actors have added insight to guide choice and implementation of patient-centered practices. Much research shows that successful implementation depends on staff engagement (e.g., Azevedo et al., 2021). Thus, clinics that facilitate active use of patients’ narratives by sharing them with their staff frequently may be particularly poised to achieve performance benefits, reflected in higher scores. They gain the additional advantage of their staff regularly engaging with the narratives themselves – emotionally and cognitively, which allows them to integrate their patients’ stories with their frontline knowledge to design more patient-centered practices (Valentine et al., 2022). These information and engagement pathways to better performance mirror those identified in research on mechanisms underlying the effect of customer feedback and storytelling in other industries (Gorry & Westbrook, 2011; Kipfelsberger, Herhausen, & Bruch, 2016; Vaerenbergh & Orsingher, 2016). We therefore hypothesized that:

Hypothesis 1: Primary care clinics that share systematically collected, useful patient narratives with their staff more frequently have higher patient experience scores than those that share them less frequently.

This hypothesis is premised on the value of access to new information. Having access, however, does not mean that such information will be used. Psychological research on information processing and learning (e.g., Bandura, 1997) suggests that characteristics of information recipients, particularly their self-efficacy or confidence in their pre-existing knowledge, often influences response to new information sources and their content. Confidence refers to the firmness or strength of belief in one’s judgment and ability; it is a cognitive mechanism that mediates motivation, thoughts, emotional reactions, and behavior, and is therefore one of the most influential moderators of response to novelty and feedback at work (Bandura, 1991, 1997). We thus also hypothesized that:

Hypothesis 2: The strength of the positive relationship between frequently sharing useful narratives and performance depends on clinic staff’s confidence in their knowledge about patients and practice.

We refrained from hypothesizing whether higher or lower confidence strengthens the relationship because results related to information use in other industries is mixed (Silver, Mitchell, & Gist, 1995). Some research suggests that greater confidence increases receptivity and response because the more confident believe that their competence enables them to learn and change practice effectively as needed (Ajzen & Fishbein, 1980; Bandura, 1991, 1997; Yoon & Kayes, 2016). That increases their uptake and effort to apply feedback to improve. Other research implies that the less confident are more receptive and apply feedback because they desire information that helps them to improve, whereas the more confident are disposed to question and resist feedback that does not accord with their beliefs (Dweck, 1986; Krumrei-Mancuso, Haggard, Labouff, & Rowatt, 2020; Sieck & Arkes, 2005). That limits their shifting to performance-improving practices. We aimed to address ambiguity about the main effect of patient narrative use on patient experience score performance and possible interaction with organizational-level confidence via empirical study in what is still an exploratory field.

Methods

Study Design

We conducted a one-year study using survey data from nine adult primary care clinics affiliated with a U.S. health system. Specifically, we linked clinic staff’s survey data about their past six-month exposure to patients’ narratives and their confidence in own knowledge of their patients and practice to their subsequent patients’ experience survey responses, which were collected for the entire year after the staff survey. We then assessed the correlation between staff’s narrative exposure and their clinics’ score performance using patient-level models. We compared clinics high versus low in staff confidence. All clinics served diverse patients, reflecting their location in a diverse, Eastern U.S. city.

Patient Experience Assessment and Response in the Study Setting

The health system had had a defined strategy to improve patients’ experiences for over a decade when our study began. Its strategy included analyzing patient survey data, choosing indicators to improve, setting target scores for the chosen priority indicators, and deploying known best practices (e.g., informing patients of test results and using provider communication tools) to improve chosen metrics. All study clinics followed this strategy. We were not involved in these efforts.

As part of the health system’s strategy, it used a standardized process for assessing patients’ experiences across all clinics. All patients were invited to complete a patient experience survey in English or Spanish immediately after their office visit via email or mail. Another invitation was sent five days later if the patient had not responded. The survey included validated measures of patient experience derived from the CAHPS Clinician & Group Survey (CG-CAHPS^®; items described in Measures below), questions about patient characteristics (listed in Measures) and a series of open-ended questions to elicit a narrative about their visit experience. The latter series was the only variance in survey. In six clinics, patients received three questions authored by the system’s survey vendor: What do you like best about our office? What did you like least about our office? Is there anything else that you would like to share about your experience? Patients in the remaining three clinics received the five-question CAHPS NIS (AHRQ, 2021), which the system was trialing to capture richer narratives. NIS questions, available here https://www.ahrq.gov/cahps/surveys-guidance/item-sets/elicitation/index.html, are validated for effectiveness in prompting patients to tell a clear story about their care experiences (Grob et al., 2016; Schlesinger et al., 2020).

For each clinic, the system’s Patient Experience (PX) Office aggregated survey scores for closed-ended questions and the composites that they formed, then shared them with respective leadership weekly via email. Clinic leaders also received all verbatim patient responses to the open-ended questions in a separate document sent weekly, with select positive and negative comments related to system targets highlighted, a practice begun eight years before our study. Supplement 1 shows examples of narrative comments received by clinics during our study. Score targets were the same for all clinics and someone from the PX Office met monthly with clinic leadership and staff charged with advocating for patient-centered care to review the comments and survey scores and discuss improvement strategies. Each clinic’s leadership decided how to use the patient feedback, specifically with whom to share and frequency, creating variance in experiences with narratives across clinics. Sharing via meetings and policy of sharing positive and negative feedback were stable across clinics during our study. We leveraged variance in “implementation” of narrative use (proportion of staff that received narratives and how often) to assess the link between staff’s narrative exposure and later patients’ experience scores, using health system data to account for clinic differences that might impact patient experiences. Baseline learning-improvement activity was similar across clinics based on staff reports and no initiatives not derived from clinics’ patient-linked feedback were deployed during our study, which was exempted by Yale University’s Institutional Review Board.

Data Sources and Sample

Patient experience surveys.

A total of 5,751 of the 80,891 surveys sent to patients in the nine clinics during our 12-month study period were returned a response rate of 7.11% (clinic response rate mean=7.21%; standard deviation (S.D.)=5.66%; range=3.56%−21.07%). Of the returned, 5,545 surveys contained responses to patient experience items, with 258 to 1,817 respondents per clinic. Table 1 shows characteristics of respondents. Most were female, White, non-Hispanic, with at least some college experience, and in at least “good” health. Average age was 59 years old.

TABLE 1:

Description of patients (N=5,545) and clinics (N=9) in analytic sample

A. Patient Characteristics	N	%, unless otherwise indicated
Age, mean (S.D.)	5,543	58.89 (23.01)
Missing	2	Not applicable
Sex
Male	1,682	69.67
Female	3,863	30.33
Race
White	2,507	45.21
Black	566	15.62
Other	1,351	24.36
Missing	821	14.81
Ethnicity
Hispanic	2,195	39.59
Not Hispanic	2,817	50.80
Missing	533	9.61
Education
Less than high school	866	15.62
High school	792	14.28
Some College	1,048	18.90
College+	2,424	43.72
Missing	415	7.48
Self-reported health status
Poor	197	3.55
Fair	1,035	18.67
Good	1,696	30.59
Very Good	1,492	26.91
Excellent	901	16.25
Missing	224	4.04
Self-reported mental health status
Poor	122	2.20
Fair	725	13.07
Good	1,310	23.62
Very Good	1,502	27.09
Excellent	1,658	29.90
Missing	228	4.11
B. Clinic Characteristics		Mean (S.D.)
Annual patient volume		36,237 (17,187)
Proportion of patients with Medicaid insurance		0.62 (0.32)
Proportion of patients with Medicare insurance		0.26 (0.29)

Staff surveys.

In the month prior to the year-long collection of the patient experience data, a paper survey was administered to all clinical and administrative staff to collect information about their experiences with patients’ narratives in the prior six months and their characteristics (e.g., role and confidence in knowledge). The survey was administered during a standing monthly staff meeting wherein staff were informed of its focus on written comments in patients’ own words. All attendees (100%) returned their survey. Of the 276 returned, 248 (86%) contained responses to all questions needed for our analysis. Completion rate ranged from 81%−100% at the clinic-level. Respondents (9 to 48 per clinic) were direct care team members (physicians, nurses, medical assistants; N=154; 58%), administrative support staff (N=89; 33%), and supervisors/administrators (N=25; N=9%).

Administrative Data.

We obtained information about clinics (Table 1) from the health system’s administrative records to account for clinic differences that might affect patient experiences.

Measures

Patient experience scores.

We examined scores for five validated measures of patient experience. The first two – timely access to care and care coordination – capture the operational side of care because they depend largely on the functioning of organizational systems under the auspices of administrative leaders. The last three – office staff courtesy and helpfulness, provider-patient communication, and patient rating of provider – capture the relational side of care because they depend on the quality of interpersonal interactions. The items used to measure these aspects of care are presented in Table 2. Patients indicated whether they experienced the action in each item using response scales ranging from “Never” to “Always” or “No” to “Yes, Definitely” for all but provider rating. Consistent with the “top-box” approach for reporting CAHPS responses (AHRQ, 2023), we created a binary variable for each survey item to indicate whether a patient answered “Always” or “Yes, Definitely” because we were interested in whether patients had a high-quality experience. For provider rating, patients gave a score from 0 (lowest rating) to 10 (highest rating) and we again used the top-box approach by creating an indicator variable for the most positive ratings (9 or 10). For all measures except the single-item provider rating, we averaged the related binary variables to create a composite score that ranged from 0 to 1. For use in robustness tests, we also calculated the alternative approach to top-box: the raw mean for composites and actual provider rating.

TABLE 2:

Study measures

Measure	Items	Response Scale
A. Patient Experience Measures
Timely Access to Care	When you made this appointment for care you needed right away, did you get this appointment as soon as you thought you needed?* When you made this appointment for a check-up or routine care, did you get this appointment as soon as you thought you needed?* In the last three months, when you phoned this provider’s office during regular office hours, how often did you get an answer to your medical question that same day? In the last three months, when you phoned this provider’s office after regular office hours, how often did you get an answer to your medical question as soon as you needed? Wait time includes time spent in the waiting room and exam room. During this visit, did you see this provider within 15 minutes of your appointment time?*	Never Sometimes Usually Always OR * No or Yes
Care Coordination	In the last three months, when this provider ordered a blood test, x-ray, or other test for you, how often did someone from this provider’s office follow-up to give you the result? During this visit, did this provider have your medical records?* In the last 3 months, how often did you and anyone on your health care team talk about all the prescription medicines you were taking?	Never Sometimes Usually Always OR * No or Yes
Office Staff Courtesy and Helpfulness	During your most recent visit… Were clerks and receptionists at this provider’s office as helpful as you thought they should be? Did clerks and receptionists at this provider’s office treat you with courtesy and respect?	No Yes, somewhat Yes, definitely
Patient-Provider Communication	During your most recent visit… Did this provider explain things in a way that was easy to understand? Did this provider listen carefully to you? Did you talk with this provider about any health problems or concerns?* Did this provider seem to know the important information about your medical history? Did this provider give you easy to understand instructions about taking care of these health problems or concerns? Did this provider show respect for what you had to say? Did this provider spend enough time with you?	No Yes, somewhat Yes, definitely OR *No or Yes
Provider Rating	Using any number from 0 to 10, where 0 is the worst provider possible and 10 is the best provider possible, what number would you use to rate this provider?	0 to 10 with 0=worst provider to 10=best provider
B. Staff Survey Measures
Organization use of patients’ narratives	Measured by frequency of sharing of useful narratives with clinic staff, which was calculated by multiplying frequency of narrative exposure and usefulness of patient narratives to which exposed, then dividing by 5	5-point scale
Frequency of narrative exposure	In the last six months, how often have you seen comments from patients (in their own words) about the care they received in this practice? * This question is consistent with the formal definition of narratives stated in the Introduction. It is broad because it includes written comments from any patient in the system (not just own) and narrow because limited to seen comments in patients’ own words.	5-point scale: 1=never, 2=rarely, 3=occasionally, 4=frequently, 5=very frequently
Usefulness of patient narratives to which exposed	Patients comments about their care… Provide useful information about how well our practice runs Provide useful information about what our patients want Are helpful for improving target indicators	5-point scale: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree
Confidence in knowledge of patients and practice^	I have a good sense of… what our patients want from their medical care how patients’ lives affect their ability to stick with their care plans what patients want from me how well I am able to communicate with patients our patients think of our practice	5-point scale: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree

Note: All patient experience measures were computed using the “top-box” approach for reporting CAHPS responses: a binary variable for each survey item was created that indicated whether a patient reported having had the highest quality experience (top box category, e.g., 9 or 10 for provider rating) (AHRQ, 2023). To create the composite score for each measure, we averaged across items within composite. Staff survey measures were averaged at the clinic-level. In robustness analyses, we used the average of responses to items in each experience composite and actual provider rating.

^{^}These items capture confidence via the stem (i.e., “I have a good sense of”). This wording mirrors scales that intend to capture strength of belief about own expertise without explicitly asking “how confident are you” or using a confidence rating scale (e.g., 1–100), mindful that admitting limited confidence about work-related knowledge can be particularly challenging for healthcare staff. Our measure emulates validated surveys like the US Health and Retirement Survey that use self-belief items (e.g., Q33d I don’t have a good sense of what it is I’m trying to accomplish in life.”).

Organization use of patients’ narratives: frequent sharing of useful narratives with clinic staff.

This measure was created by combining two constructs in the staff survey: frequency of narrative exposure and usefulness of patients’ narratives to which exposed. The newly created items used to assess these are presented in Table 2, Part B. We deemed it important to weight frequency by usefulness because more useful feedback should enable more positive change, whereas more frequent receipt of feedback with limited usefulness should not. It may frustrate instead. Past research demonstrates that perceived usefulness is highly correlated with objective usefulness (e.g., Venkatesh & Davis, 2000), supporting the validity of our survey measure. For each respondent, we multiplied the average of their responses to the narrative usefulness items (Cronbach’s alpha=0.84) by their frequency of narrative exposure to create a single measure of the frequency with which the clinic exposed them to useful patient narratives. We then averaged responses from all staff within each clinic to arrive at clinic-level measures of frequency of sharing useful narratives with staff (r_wg=0.86). While nine clinic-level estimates is few, it is sufficient to test our hypotheses, similar to biopharmaceutical studies that assess varying but few doses of a drug using patient-level analyses.

Confidence in knowledge of patients and practice.

We assessed this attribute using staff responses to five survey items developed for this study (Table 2, Part B). We did not ask these items in relation to narratives. Like our measure of narrative use however, we created these items because suitable measures did not exist, although we relied on literature review to craft them. We averaged responses across our items (Cronbach’s alpha = 0.86) and then across all respondents in each clinic to create a clinic-level measure of staff confidence in knowledge of patients and practice (r_wg=0.89). We verified that this confidence measure differentiated from our measure of organization’s use of patients’ narratives via exploratory factor analyses using principal axis factoring with promax rotation. Separate factors were found with moderate correlation (r=0.64), each with eigenvalue above one meaning that each factor (composite measure) explained more variance than a single item in it. Correlation between confidence and tenure in the clinic was insignificant (r=0.04), ameliorating concern about conflating these experience-related variables. One might have conjectured that confidence was proxying for tenure, and thus years of experience providing care is what interacts with narrative use in relation to experience scores. The insignificant correlation rejects that theory.

Covariates.

We had three sets of covariates. The first set included patient characteristics related to reports about their care experiences: age, gender, education, race/ethnicity, overall health status, and mental health status (Zaslavsky et al., 2001). These data were collected via the patient survey, measured as indicated in Table 1, and treated as time-varying covariates because some could change for respondents who appeared more than once in the sample. The second set of covariates included four clinic characteristics – proportion of patients with Medicaid and Medicare insurance, patient volume, and campus location – used to adjust for workload and population differences that could influence the experience that clinics could offer and their use of patient narratives. The last covariate category was an indicator variable for whether the patient received a survey with three open-ended questions authored by its survey vendor at the end or the five-question CAHPS NIS (AHRQ, 2021).

Analyses

We used multilevel panel regression analyses with random clinic intercepts to test our hypotheses and account for unobservable clinic differences such as staff turnover rates, change implementation approach, and/or unrevealed improvement initiatives though such initiatives were unlikely given system centralization. To test Hypothesis 1 (frequent sharing of useful narratives with clinic staff is associated with higher patient experience scores), we regressed scores for each of our five measures of patient experience on organizations’ sharing of patients’ narratives and covariates, using separate models for each measure. Coefficients in these models indicate that a one-unit increase in narrative sharing, for example, changes the proportion of items in which patients reported having a top-box experience in a domain by the coefficient’s amount. To test Hypothesis 2 (confidence in knowledge moderates the relationship in Hypothesis 1), we first added our moderator variable, confidence in knowledge, to each Hypothesis 1 model, and then added an interaction term created by multiplying organization’s sharing of patients’ narratives with staff and staff confidence in knowledge. To improve interpretation and reduce multicollinearity without disturbing model fit, we mean-centered these variables prior to entering them in models. We evaluated the significance of the interaction terms to assess support for Hypothesis 2. When significant, we calculated the effect of frequent sharing of useful patients’ narratives on experience scores for clinics that were high versus low on confidence (i.e., one standard deviation above versus below mean), illustrated the effects for each group in figures, and assessed the significance of each effect using the simple slopes tests.

All models adjusted for month of office visit to account for trends in patient experience scores and time-invariant characteristics that may correlate with clinics’ sharing of narratives. We clustered standard errors at the clinic level to account for patients nested within clinics and to minimize intraclass correlation that would otherwise bias estimates downwards. To assess robustness of our results with respect to using top-box measures, we replicated analyses using the alternative measures.

Results

Table 3 reports summary statistics for patient experience measures in our sample alongside national statistics. Sample means ranged from 0.63 to 0.90 with standard deviations (S.D.) ranging from 0.21 to 0.48, indicating that 63% to 90% of patients, on average, reported receiving the highest quality experience for our measures with substantial variation around these means. The reported frequency of sharing useful narratives ranged from 2.07 to 3.89 on a 5-point scale (mean=2.82, S.D.=1.14) across clinics, a wide range that supports use of few clinic-level values or doses for patient-level analyses. Staff confidence was high with a mean of 4.05 and range of 3.87 to 4.24 (S.D.=0.14). Bivariate correlations of focal variables with experience scores are presented in Supplement 2.

TABLE 3:

Summary statistics for patient experience measures

	Study Sample		CG-CAHPS National Database Sample^
Measure	N	Mean (S.D.)	Mean US Census Regional Practices/All Primary Care Practices/All Medical Specialty Practices
Timely Access to Care	5,397	0.73 (0.32)	0.69 0.63 0.66
Care Coordination	5,386	0.70 (0.30)	0.74 0.72 0.73
Office Staff Courtesy and Helpfulness	5,323	0.85 (0.33)	0.78 0.78 0.79
Patient-Provider Communication	5,393	0.90 (0.21)	0.87 0.86 0.85
Provider Rating	5,323	0.79 (0.40)	0.81 0.79 0.80

S.D. = Standard Deviation. Sample sizes reflect the number of patients who provided answers to the items for each measure. Thus, sample sizes for measures are less than the total patient sample size of 5,545 due to patient choice not to answer items in some sections of the survey.

^{^}Statistics drawn from the CG-CAHPS Database for 2018, which included 313,706 patients from 2,024 medical practices (AHRQ, 2023).

Table 4 reports key estimates from the regression analyses. Full models are in Supplement 3. We found some support for Hypothesis 1. The first model for each patient experience measure shows that frequent sharing of useful patients’ narratives with clinic staff was not associated with higher experience scores, without accounting for staff confidence. However, when we accounted for confidence in the second model for each measure, sharing of useful patients’ narratives with staff was positively and significantly associated with an increase in two of five measures (p<0.01): timely access to care (B=0.107, S.E.=0.02) and office staff courtesy and helpfulness (B=0.179, S.E.=0.02).

TABLE 4:

Results of regression analyses of sharing patients’ narratives with staff, staff confidence and patient experience measures

Panel A: Operational Patient Experience Measures
	Timely Access to Care	Timely Access to Care	Timely Access to Care	Care Coordination	Care Coordination	Care Coordination
	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)
IV: Frequency of sharing useful patient narratives with staff	−0.051	0.107***	0.102***	−0.058*	−0.020	−0.039***
	(0.039)	(0.017)	(0.011)	(0.031)	(0.049)	(0.011)
MV: Staff confidence in knowledge		−0.248***	−0.109**		−0.060	0.506***
		(0.021)	(0.048)		(0.048)	(0.049)
IV x MV			0.376***			1.535***
			(0.087)			(0.100)
Indicator for NIS elicitation protocol	−0.035	0.010*	−0.012	−0.009	0.002	−0.090***
	(0.027)	(0.006)	(0.009)	(0.015)	(0.018)	(0.010)
Observations	5395	5395	5395	5384	5384	5384

Panel B: Relational Patient Experience Measures
	Office Staff Courtesy/Helpfulness	Office Staff Courtesy/Helpfulness	Office Staff Courtesy/Helpfulness	Provider Communicate	Provider Communicate	Provider Communicate	Provider Rating	Provider Rating	Provider Rating
	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)	B (SE)
IV: Frequency of sharing useful patient narratives with staff	−0.087	0.179***	0.189***	0.001	0.004	0.010	−0.036	−0.038	−0.011
	(0.066)	(0.018)	(0.022)	(0.007)	(0.013)	(0.010)	(0.041)	(0.059)	(0.016)
MV: Staff confidence in knowledge		−0.416***	−0.716***		−0.005	−0.180***		0.003	−0.8.26***
		(0.031)	(0.082)		(0.017)	(0.034)		(0.058)	(0.084)
IV x MV			−0.814***			−0.474***			−2.250***
			(0.152)			(0.058)			(0.180)
Indicator for NIS elicitation protocol	0.010	0.085***	0.133***	0.008**	0.009	0.037***	0.010	0.010	0.144 ***
	(0.045)	(0.009)	(0.016)	(0.004)	(0.005)	(0.007)	(0.004)	(0.023)	(0.017)
Observations	5321	5321	5321	5391	5391	5391	5321	5321	5321

^***p<0.01,

^**p<0.05,

^*p<0.1. B = coefficient; SE = standard error; IV = independent variable; MV = moderating variable; NIS = the CAHPS Narrative Item Set. Sample sizes are two less that than reported in Table 3 because two people who did not provide their age were excluded. Models control for sex, level of education, race/ethnicity, self-reported health, self-reported mental health at the individual level and patient volume, proportion of patients on Medicaid, proportion of patients on Medicare at the clinic level, and campus. Models also control for month and standard errors are clustered at the clinic level. See Supplement 1 for full models.

The third model for each measure in the table shows strong support for Hypothesis 2. Not only does staff confidence in their own knowledge of their patients and practice have a significant association with patient experience scores for all measures, the interaction term in each model is significant indicating that frequently sharing useful patients’ narratives with clinic staff was associated with patient experience scores, conditional on staff confidence (all p-values< 0.01). Figure 1 shows the nature of the interaction between these factors, which differed for operational versus relational measures. Simple slopes tests confirmed that all lines, except for the one depicting the interaction of low confidence and narrative sharing on timely access to care, have a slope that is significantly different from zero using a threshold of p<0.05.

FIGURE 1.

The interaction of sharing patients’ narratives with staff and staff confidence on patient experience scores

Notes: Low and high confidence are one standard deviation below and above the mean for confidence in knowledge of patients and practice, respectively. The linear prediction is the incremental effect for being in the top box for each measure. All slopes are significant using the simple slopes test (P<0.05), except for the one depicting the effect of sharing patients’ narratives on timely access to care for clinics with low confidence staff.

For the operational patient experience measures shown in Panel A, increased frequency of sharing useful comments with staff was positively associated with scores for clinics with highly confident staff. For clinics with less confident staff, such sharing was not associated with any change in scores for timely access to care but was associated with a decrease in scores for care coordination. For the relational patient experience measures shown in Panel B, increasing the frequency of sharing useful narratives with clinic staff was positively associated with scores when staff were less confident. When they were highly confident, there was a negative association. Notably, the less confident outscored the more confident on relational measures. Also, the third model in Table 4 shows that use of the NIS elicitation protocol was associated with higher scores for the relational measures (i.e., office staff helpfulness, provider-patient communication, and patient rating of provider), but lower scores for care coordination, relative to the three-item elicitation protocol. All results replicated using our alternative experience measures except timely access to care. The interaction was no longer significant but a positive, main effect of narrative use was (p<0.01) (Supplement 4).

Discussion

Our results indicate a significant relationship between organizations’ use of patients’ narratives and their patient experience survey scores and show that the nature of this relationship depends greatly on context. Organizations that frequently share useful patients’ narratives with their staff have higher patient experience scores contingent on two factors: domain of patient experience measured and staff’s pre-existing confidence in knowledge about patients and practice. For organizations with more confident staff, higher narrative use is associated with higher scores for operational measures of patient experience (a positive interaction) and is associated with lower scores for relational measures (a negative interaction). However, for organizations with less confident staff, higher narrative use has no or negative association for operational measures and a positive association for relational measures, which also benefit from organizations’ use of the NIS five-question elicitation protocol. Care coordination scores benefit from sharing feedback from the three-item protocol instead.

These results affirm prior qualitative studies that suggest that patients’ narratives can impact organizational learning and improvement. They also extend that work by indicating when positive impact occurs more readily versus not. Prior work (highlighted in our Theory section) had indicated multiple informational benefits of patients’ narratives for organizations but not demonstrated that the insights from narratives can be applied to yield better patient experiences and (in turn) scores, with impact varying by group attributes. Our study offers supporting evidence for that possibility in primary care, extending research on the beneficial performance effects of customer feedback and storytelling in other industries for organizations that respond conscientiously to the feedback (Gorry & Westbrook, 2011; Kipfelsberger et al., 2016; Vaerenbergh & Orsingher, 2016). Our results are also notable because the effect of narrative use is comparable to recent policy and physician-focused patient experience interventions (e.g., Caregiver Advise, Record, Enable (CARE) Act (Lee, Taggert, Coe, & Chatterjee, 2023); shadow coaching (Quigley, Elliott, Slaughter, Talamantes, & Hays, 2023)). All of these provide an opportunity for those who serve patients to see, hear, and/or read more of patients’ stories. Using narratives has the added asset of constantly bringing insight for ongoing learning.

A quandary in the patient experience improvement community has been that systematic and scoping reviews of research on the use of patient experience data for quality improvement have documented a wide range of improvement initiatives sparked particularly by poor survey scores, but little evidence that collecting these data and score-motivated efforts are associated with improved patient experience (Gleeson et al., 2016). Our findings suggest two explanations for this quandary. One is that scores alone lack specific actionable insight to appropriately direct effort; narratives provide needed complementary insights. A second is that score-motivated efforts may be responded to differently in settings where staff have (among other differences) more versus less knowledge confidence, which prior studies have not addressed. While our data do not permit us to determine a definitive explanation, they suggest that accounting for contingencies identified here is necessary.

Research on information use has highlighted the role of confidence (Ajzen & Fishbein, 1980; Bandura, 1991, 1997) but organizational and individual learning research, with few exceptions (e.g., Edmondson, 1999; Yoon & Kayes, 2016), have not. Our results, however, indicate that knowledge confidence alters learning possibilities and interacts with domain in significant ways, making it a deserving of greater study for more complete understanding. Differences by experience domain may exist because operational domains may be perceived as system issues to be tackled creatively, enticing the more confident to solve. In contrast, relational domains may feel more personal to providers and thus more threatening, resulting in defensive response in highly confident clinics. The less confident already suspect that they can improve so may be more willing to digest and use the information. Future research should test this logic and explore other possible, non-confidence contingencies (e.g., staff morale) so that organizations can address them as well and optimize the benefits of using narratives. It should also examine the potential for a cycle. Experience scores may alter narrative use and confidence, which may alter performance in the next period and restart the cycle. This study assessed just part of this potential cycle i.e., score as output (not input). Multi-wave study might reveal a cycle to be managed and/or leveraged.

Future research should also examine which mechanisms most drive the associations we found. For example, if the mechanism is more elaborating and/or distinctive information, the field might be served by greater use of elicitation protocols that better generate this information. If it is actionable information and creative improvement ideas, a question might be added to protocols to invite patients’ solutions to identified problems. To the extent that narratives enable identification of contributors to inequitable experiences and ideas for how to address these, organizations might further encourage those structurally disadvantaged to complete these surveys through greater outreach and via the mode most attractive them.

Of course, the richer content inherent in narratives becomes more useful when presented in ways that allow staff to digest it with ease. One evidence-based option to achieve this goal is implementing electronic reporting interfaces on organizations’ intranet sites to share with staff narrative patterns and themes coded using validated taxonomies (Shaller et al., 2023). Even with the aid of such technology, moving from narrative insights to better patient experiences may not be easy. Using patient survey data for improvement is notoriously challenging due to barriers such as competing priorities and limited resources to support change efforts (Davies & Cleary, 2005; Gleeson et al., 2016). Our study, however, suggests, that the investment is worthwhile.

Our study has limitations. First, although we analyzed patient scores received after our narrative use survey to assess directionality of effect, and although our sample size of patient-generated survey scores was robust, we cannot claim causality. Second, we studied a single health system (and just nine clinics) that had an experience-improvement strategy for a decade before we conducted our survey. Consequently, our results may not generalize, although our sample’s mean experience scores were similar to national samples (Table 3), our staff survey completion rates were high, and the variation in measures sufficient to reveal significant results. Our variation may be narrower though, which might mean that our results underestimate effects. As more health systems incorporate narratives in their patient experience assessment, evaluation of generalizability across settings and types of care organizations (hospitals, long-term care facilities, etc.) will be possible. Third, for this first assessment of the relationship between narrative use and scores, we focused on confidence in knowledge as our sole moderator because of its centrality in organizational theory on information processing. However, other moderators may also be influential and should be examined, alongside whether implementation strategy differences for ideas derived from narratives influenced results. Fourth, we used newly developed and staff-reported measures. It is possible, for example, that staff misjudge the usefulness of some feedback, even though narratives often add richness that makes judging their usefulness easier, or that staff rate some feedback less useful because circumstances prevent them using it, although insightful. Future work should validate our measures and continue to develop measures for narrative studies. Last, we used aggregated staff data on narrative use and confidence because we were unable to link staff to their patients’ data due to guaranteed anonymity for patient respondents. Research in the future may be able to link staff attributes and interactions with their own patients’ narratives, responses, and scores for even greater understanding of the links between these. It may be able to push our analyses further by examining effects by role to assess whether our found interactions are stronger or differ at the role level, which we could not do due to our sample size at that level. The sharing with and confidence of those in roles most influential for a particular measure (e.g., clinical providers for patient-provider communication), and the ratio of positive to negative feedback shared might be especially critical. Concurrent with that query, a key next step is to identify patient-inspired initiatives that yielded improved experience scores.

Practice Implications

Our results imply that efforts to learn from patients’ narratives should be encouraged as an organizational improvement strategy but will not yield score benefits across domains unless organizational leaders attend to staff mindset toward such learning. Leaders need to complement narrative use with confidence-building or affirming communications and actions for initiatives related to operational domains of performance (e.g., “we can fix this” or “we should do more of this”), and to complement narrative use with humble and/or threat-reducing messages for relational domains (e.g., “this feedback does not mean we’re not capable; it means we’re missing something”). Complementary messaging in both cases may foster an inquiry orientation toward narratives characterized by curiosity (asking how and why) along with openness to listening and individual learning, which enables organizational learning and innovation (Singer & Edmondson, 2008). Organizations may also need to support specific knowledge acquisition in the areas in our survey questions because lower confidence may reflect true knowledge gaps that organizations can address.

Notably, we examined one strategy for organizations’ use of patients’ narratives: frequently sharing useful narratives with staff. Our results thus imply that organizations should share useful narratives frequently. Even in our study setting, which systematically collected narratives, such sharing was meager, indicating that there is likely room to increase sharing in many organizations. Identifying the interval most helpful to staff and their improvement efforts is a next step for research. Future studies should also evaluate options for improving the usefulness of narrative feedback to increase the value of sharing. Already for practice however, our results suggest that the series of open-ended questions used to elicit narratives matters for usefulness. Clinics that were assigned to use the CAHPS NIS elicitation questions had higher scores on all three relational measures in the following year, and lower scores for care coordination. Thus, organizations may benefit from using the NIS to enhance the feedback they receive about relational aspects of care. They may need to investigate operations more to understand feedback related to care coordination.

Conclusion

This study provides evidence that organizations’ use of patients’ narratives by frequently sharing useful narratives with their staff is associated with their subsequent patient experience scores. It also shows that the nature of association varies significantly with the domain of patient experience (operational versus relational) and staff confidence in own knowledge about patients and practice. Based on these findings, organizations should be encouraged to collect useful patients’ narratives and share them with their staff with the understanding that achieving the score benefits of such sharing across experience domains requires organizational leaders to attend to their staff’s mindset toward this feedback and manage narrative use accordingly. Staff are responsive to the information in their patients’ narratives. Thus, using that information for improvement is an opportunity.