Leveraging Patients’ Creative Ideas for Innovation in Health Care

Abstract

Policy Points.

• Patients’ creative ideas may inform learning and innovation that improve patient‐centered care.

• Routinely collected patient experience surveys provide an opportunity to invite patients to share their creative ideas for improvement. We develop and assess a methodological strategy that validates question wording designed to elicit creative ideas from patients.

• Health care organizations should consider how to report and use these data in health care delivery and quality improvement, and policymakers should consider promoting the use of narrative feedback to better understand and respond to patients’ experiences.

Context

Learning health systems (LHSs) have been promoted for a decade to achieve high‐quality, patient‐centered health care. Innovation driven by knowledge generated through day‐to‐day health care delivery, including patient insights, is critical to LHSs. However, the pace of translating patient insights into innovation is slow and effectiveness inadequate. This study aims to evaluate a method for systematically eliciting patients’ creative ideas, examine the value of such ideas as a source of insight, and examine patients’ creative ideas regarding how their experiences could be improved within the context of their own health systems.

Methods

The first stage of the study developed a survey and tested strategies for elicitation of patients’ creative ideas with 600 patients from New York State. The second stage deployed the survey with the most generative open‐ended question sequence within a health care system and involved analysis of 1,892 patients’ responses, including 2,948 creative ideas.

Findings

Actionable, creative feedback was fostered by incorporating a request for transformative feedback into a sequence of narrative elicitation questions. Patients generate more actionable and creative ideas when explicitly invited to share such ideas, especially patients with negative health care experiences, those from minority racial/ethnic backgrounds, and those with chronic illness. The most frequently elicited creative ideas focused on solving challenges, proposing interventions, amplifying exceptional practices, and conveying hopes for the future.

Conclusions

A valid and reliable method for eliciting creative ideas from patients can be deployed as part of routine patient experience surveys that include closed‐ended survey items and open‐ended narrative items in which patients share their experiences in their own words. The elicited creative ideas are promising for patient engagement and innovation efforts. This study highlights the benefits of engaging patients for quality improvement, offers a rigorously tested method for cultivating innovation using patient‐generated knowledge, and outlines how creative ideas can enable organizational learning and innovation.

For the past decade, the national academy of medicine has envisioned learning health systems (LHSs) as a way for American health care to achieve the elusive goal of high‐quality, patient‐centered care while containing costs. ¹ , ² , ³ An important capability of the LHSs is fostering innovation (i.e., new policies, practices, and technologies ⁴ ) fueled by continuously generated knowledge, especially from day‐to‐day health care delivery. ¹ , ³ , ⁵ , ⁶ Patients are recognized as an important source of such knowledge. ⁷ , ⁸ Although LHSs routinely extract insights from patient data, the pace of translating these insights into innovations is slow and effectiveness inadequate. ⁸ , ⁹ , ¹⁰ As a result, many health care delivery organizations fall short of their potential as LHSs capable of evolving and innovating using knowledge obtained directly from their patients. ³ , ⁸

We hypothesize that these shortfalls could be at least partly rectified through methods for more effectively eliciting and learning from patients’ creative ideas, defined as ideas that are novel to the setting and useful for the task at hand. ¹¹ Creative ideas become innovations once implemented. ¹¹ , ¹² , ¹³ Although little has been established about patients as a source of creative ideas, studies of specific patient populations reveal both the potential and constraints of this approach. ¹⁴ For example, patients with multiple chronic conditions, who may consult up to 16 clinicians annually, ¹⁵ acquire a unique grasp of the system and concomitant capacity to generate creative ideas but rarely have time to participate in learning initiatives. ⁸ Likewise, patients from minority racial backgrounds hold underrepresented perspectives necessary to tailor treatment to their needs and values, yet systemic inequities have constrained the spaces where their voices are invited. ¹⁶ , ¹⁷ Research has not yet fully assessed the potential value of patients’ creative ideas or how to elicit them effectively across diverse populations for health system learning. ⁷ , ¹⁸

This study's objective is to fill that gap by examining new approaches to effectively obtain patients’ creative ideas and how they may be utilized for quality improvement. Its first aim is to develop and test an entirely new method for systematically eliciting patients’ creative ideas that combines strengths from existing approaches, namely patients’ creative idea generation from codesign and utilization of representative knowledge from large‐scale patient surveys. ⁷ , ⁸ , ¹⁹ The second aim is to examine the value of patients’ creative ideas by testing hypotheses about the characteristics of such ideas within representative patient samples and specific patient subgroups who may hold unique vantage points. The final aim of this study is to outline how creative ideas may enable organizational learning for innovation by categorizing and describing patients’ creative ideas generated from experiences in a real‐world health care system, providing organizations specific and actionable pathways to translate patients’ creative ideas into innovation.

This study makes three primary contributions. First, it enriches understanding of an overlooked form of patient knowledge—patients’ creative ideas—as a complement to patient‐reported outcomes (PROMs) and reports of patient experience through standardized surveys and other extensively gathered patient data. ²⁰ Our findings demonstrate the benefits of soliciting patients’ creative ideas as part of comprehensive, deliberate learning practices in health care systems. In doing so, our findings also introduce a novel dimension to patient engagement and centeredness. Second, we contribute to health services research by offering a rigorously tested methodological strategy for eliciting patients’ creative ideas to assist LHSs in harnessing innovations from continuously generated patient knowledge. This method may address challenges patients encounter in voicing their opinions in health care systems, such as power imbalance (compared with health professionals and administrators) and limited platforms to share their ideas. ⁸ , ¹⁵ , ²⁰ Finally, this work advances understanding of how patients stimulate organizational innovation, enriching theory and practice on creativity, learning, and innovation.

Patients’ Actionable and Creative Ideas: Theory and Hypotheses

The three study aims—to develop and test a new method for systematically eliciting patients’ creative ideas, to examine the value of patients’ creative ideas, and to outline how creative ideas enable organizational learning for innovation—will be addressed with hypotheses that were generated by integrating literatures on creativity, health services research, service management, and learning. The study's hypotheses are summarized in Figure 1. Our first series of hypotheses focus on important characteristics of ideas for quality improvement and innovation, including creativity and actionability of ideas, which are defined in Table 1 along with other key constructs.

Figure 1.

Conceptual Framework [Colour figure can be viewed at wileyonlinelibrary.com]

Note: H1, hypothesis 1.

Table 1.

Definition of Key Constructs

Construct	Definition
Patient centeredness	An approach to health care that prioritizes the individual needs, values, and preferences of patients and seeks to place the patient at the center of all decision making. 36 This approach aims to ensure that care is respectful, responsive, and tailored to the unique circumstances and goals of each patient. 75
Creativity	The generation of novel and useful ideas. 60 In health care (and other settings), creativity involves thinking beyond established practices to develop new strategies, interventions, or solutions that address health care challenges, improve patient outcomes, or enhance the quality of care.
Novelty	The quality of being new, original, or unprecedented—the distance of the idea from the referent practice. 60 , 76 In health care, novelty pertains to the introduction of new ideas, interventions, or practices that have not been previously encountered or widely adopted within the field. Novelty often implies a departure from existing practice. 77
Usefulness	The ability to achieve desired goals. 60 , 78 In health care, it assesses the extent to which an idea effectively responds to a specific challenge, improves outcomes, or improves health care delivery.
Idea	An alternative for a possible course of action to approach the task at hand. 59 A single patient narrative response in a survey may contain multiple ideas.
Creative idea	An idea that is novel to the setting and useful to the task at hand. 60 Every idea has its own level of creativity. 60
Actionability	The extent to which an item can inform improvement efforts by supplying specific and concrete details. 61 In health care, it denotes the degree to which an idea can be translated into practice within a real health system by providing concrete details that can be translated into practice. For example, actionable items included in an idea might refer to the what, where, when, and why of an idea, 61 emphasizing the translation potential of the idea into practice.

Develop and Test a New Method for Systematically Eliciting Patients’ Creative Ideas

Explicit Invitation

Hypothesis 1 (H1) is that patients generate more actionable and creative ideas when invited explicitly to do so using narrative elicitation methods compared with when not invited explicitly. Narrative elicitation methods emphasize the value of patients’ words and experiences, promoting a learning‐oriented, curious, and reflective approach that encourages the sharing of rich and valuable insights. ¹⁸ , ²¹ , ²² We propose that narrative elicitation that explicitly encourages patients to share improvement suggestions will lead to highly actionable and creative ideas from patients. The actionability and creativity of these ideas are likely to be high because patients may feel that the organization is receptive to their input, and therefore, it is worthwhile to be specific (enhancing actionability) and novel (enhancing creativity) in their responses. ¹⁸ , ²³

Invitation Emphasizing Creativity or Past Experiences

H2 is that invitations that emphasize creativity or past experiences can elicit highly actionable and creative ideas. This hypothesis rests on the observation that patients need to be primed to feel comfortable conveying creative ideas. Because of human bias against novelty and the social risk of deviating from the status quo, it is essential to address patients’ concerns about idea feasibility and acceptance when inviting creativity. Two approaches fostering comfort have been identified in the literature. First, explicit invitations to think “outside the box” have been suggested as crucial elicitation techniques to ensure respondents feel comfortable sharing their ideas. ²³ , ²⁴ Second, studies of user‐based innovation suggest that asking respondents to remember and interpret their past experiences can effectively generate detailed and creative ideas tailored to address past experiences. ²⁵ , ²⁶ We therefore hypothesize that both approaches hold promise for eliciting highly actionable and creative ideas (H2) and empirical testing is required to determine which method is superior.

Invitation Emphasizing System Impact

For H3, we posit that actionable and creative feedback depends on patients’ perceptions of an organization's responsiveness to their feedback. This is consistent with past research on customers’ decisions to share written feedback in service industries. ²³ , ²⁴ , ²⁷ , ²⁸ When customers believe their feedback will not be taken seriously or utilized by management, they are less likely to provide feedback. ²⁴ , ²⁵ , ²⁹ , ³⁰ We thus anticipate that these perceptions will be particularly relevant to contexts such as health care in which there is distrust of institutions, which is likely to deter patients from sharing ideas unless deliberate effort is made to signal enthusiasm for learning from patients in order to improve care. ¹⁷ , ³¹ , ³²

The Value of Patients’ Creative Ideas: Experience‐Based Knowledge

Patients as Coproducers: Firsthand Knowledge

In recent years, the concept of coproduction in health care, in which patients and providers work together to create the service and its value, has gained acceptance. ³³ This idea has been bolstered by patient advocacy movements, consumer empowerment trends, and the rise of personalized medicine. ³⁰ , ³⁴ , ³⁵ Although patients have demonstrated their willingness to share diverse forms of knowledge, the extent to which they can contribute helpful ideas for future care has not been explored. We hypothesize that patients can generate highly actionable and creative ideas for improved care experiences (H4) because their first‐hand experiences with care delivery yield an enhanced ability to identify patterns and possibilities. Acknowledging inherent differences in patients’ experiences, ¹⁹ , ²⁹ , ³⁶ certain subgroups may possess a unique vantage point for generating especially actionable and/or especially creative ideas. We highlight those subgroups next.

Patients With Negative Care Experiences: Activated Knowledge

H5 posits a positive association between patients with negative care experiences and the actionability of ideas, suggesting that patients with negative care experiences are likely to generate ideas with more actionable content. Health system leaders and experts have long emphasized that patients with negative care experiences, much like health care professionals with negative work experiences, are motivated to instigate change, which is supported by research. ³⁷ , ³⁸ Patient grievance studies demonstrate the constructive aspirations of patients with negative experiences, with a high proportion of their feedback being detailed and focused on prompting organizational changes that can benefit future patients and families. ³¹ , ³⁹

Patients With Minority Racial/Ethnic Backgrounds: Distinctive Knowledge

H6 posits that patients from minority racial/ethnic backgrounds are more likely to generate and share high‐creativity ideas if motivated to overcome persisting levels of distrust in health care providers and systems. ⁴⁰ Patients with minority racial and ethnic backgrounds are likely to generate highly creative ideas because historical underrepresentation of their voices and ideas in health care has induced practices that are inadequately responsive to their distinctive preferences or social circumstances. ¹⁷ , ⁴⁰ , ⁴¹ , ⁴² We expect that experiencing a minority racial/ethnic background yields distinctive knowledge about effects of current practices that produces higher creativity ideas for change; that is, ideas that differ from established practices and are considered new to the setting.

Patients With Extensive Exposure to System: Amassed Knowledge

H7 is that patients managing chronic illnesses, and thus having more exposure to health care systems, are likely to generate a higher volume of ideas compared with patients without chronic illnesses. Scholars of user‐based innovation have observed that consumers with extensive experience are most prone to generate innovation‐ready ideas to serve their own needs because of their breadth of exposure. ²⁶ , ⁴³ , ⁴⁴ Applying this rationale to health care, we identify patients managing chronic illness as sophisticated users, as they engage with as many as 16 clinicians annually across various care settings. ¹⁵ , ⁴⁵ Their varied experiences, such as navigating insurance challenges, advocating for themselves, and coproducing care, contribute to a substantial reservoir of amassed knowledge from which to generate more ideas for organizational change. ⁴⁶

How Creative Ideas May Enable Organizational Learning for Innovation

LHSs gather knowledge generated from patients that can fuel innovation, yet little is understood about how patients’ creative ideas can be used to guide organizational learning for innovation. ¹ , ⁷ , ¹⁰ We contend that patients’ creative ideas can spur the receiving health care organization to innovate by highlighting problems and potential solutions. This underscores the dynamic between patients as creators and organizations as assessors, developers, and implementers of these ideas. ³⁰ We offer three pathways for how patients’ creative ideas can inspire organizational learning for innovation. These ideas can 1) solve extant challenges, 2) specify new interventions, and/or 3) amplify exceptional practices and people whose processes should be better understood, formalized, and implemented broadly.

Solving Extant Challenges

H8 posits that patients’ creative ideas may prompt organizations to recognize existing challenges that may otherwise go undetected and inspire strategies for resolution. Studies reveal that organizational challenges drive creative idea generation, ⁴⁷ , ⁴⁸ , ⁴⁹ which in turn can solve those extant challenges. Additionally, organizational learning frameworks suggests that organizations can intuit areas for patient concern from creative ideas or learn through their efforts to interpret and make sense of ideas shared with them and thus use their new knowledge to address problems. ¹² , ⁴⁹ , ⁵⁰ In health care, challenges in need of problem‐solving may reveal overlooked yet salient issues visible to patients, ⁸ , ³⁰ , ³⁴ supporting patient‐centered care principles and recognizing patients’ unique expertise in aligning care with their preferences and values. ¹ , ⁷ , ⁸ , ³⁶

Specifying New Interventions

H9 contends that patients’ creative ideas hold potential to delineate novel strategies or interventions, offering avenues for organizational development. User‐based innovation studies emphasize how such ideas position consumers at the beginning of the innovation process. ²⁶ , ³² , ⁴³ , ⁴⁹ Patient‐derived interventions are less susceptible to the dilution of colleague influences, which often affects creativity. ⁵¹ , ⁵² Although not all patient recommendations may be immediately executable, they can infuse fresh perspectives into existing practices and proposed innovations, helping to identify those meriting resource or time investment. ⁵¹ Creative ideas that specify new interventions may catalyze interpretive processes within organizations, facilitating integration, adaptation, and eventual implementation of these ideas to become routine practice. ¹³ , ⁵⁰

Amplifying the Impact of Exceptional Practices or People

H10 is that patients’ creative ideas have the potential to amplify exemplary practices or individuals whose significant contributions may go unnoticed. Insights from customer service research reveal that customers exhibit a strong inclination to identify past practices that led to their satisfaction and individuals responsible for outstanding experiences. ²³ , ²⁷ , ⁵³ , ⁵⁴ , ⁵⁵ Through the lens of organizational learning, creative ideas that amplify exceptional practices or people provide evidence regarding effective practices or human resources from a customer perspective, thereby offering invaluable insights for the organization to replicate and accentuate these positive assets. ⁵⁵ , ⁵⁶ Having insights that illuminate exceptional practices or individuals provides organizations with rich data to support the spread of their most efficacious practices and honor their most effective people. ⁵⁴ , ⁵⁷

Methods

To test the three aims and related hypotheses, we conducted multistage mixed‐method studies of patients’ creative ideas. The first stage field study included survey development and testing with a representative sample of 600 patients from New York State and their creative ideas. This study supported the first two aims: to develop and test an entirely new method for systematically eliciting patients’ creative ideas and to examine the value of patients’ creative ideas. Aim 1 was assessed by the survey development and testing of a new narrative elicitation approach to collecting patients’ creative ideas. Aim 2 was assessed with a quantitative analysis of drivers and characteristics of ideas generated by patients from the survey data. The second‐stage field study was a real‐world health systems test with a sample of 2,948 creative ideas from 1,892 patient surveys from a large health system in New York City. This study supported the aim to identify how patients’ creative ideas may enable organizational learning for innovation. Aim 3 was assessed with a qualitative content analysis of creative ideas and inductive identification of thematic categories. We will now outline each study and related aims and activities.

Stage 1 Study: A Survey Development and Testing Study With a Representative Sample of Patients and Their Creative Ideas

Setting and Sample

Survey participants were a representative sample of patients from a web‐enabled Ipsos KnowledgePanel, a probability‐based panel representative of the US adult population. Participants were New York State adults who were English‐language survey takers who saw or talked to a doctor or other health care professional about their health in the last 12 months, including those who also met criteria for a serious, life‐threatening, or other long‐term condition. These patients were representative of the US adult population on demographic measures such as age, education, race/ethnicity, gender, household size, household type, income, marital status, Census region, ownership status, and employment status. These participants were drawn from a random selection of telephone numbers and residential addresses. Table 2 presents the characteristics of the study sample compared with the US adult population. Although the study population was generally representative of the general US adult population, it was also on average a slightly older, more White (versus racial minority group members), and higher income population. Ipsos provided a laptop and internet connection to those who agreed to participate but who did not have access to a laptop or a computer. Participants were contacted by email, and the online survey was accessed through a link in the email notification. The survey field period was 21 days, with up to two email reminders. The survey, which had a 49% response rate, was fielded to 1,224 Ipsos panel members in order to reach a target of 600 completed surveys. This target was to allow for the four‐arm experiment (described later in the Methods), with N = 150 per arm, with sufficient power to detect significant differences among arms. As intended, 250 of the 600 respondents met the criteria for a serious, life‐threatening, or other long‐term condition. The median completion time for the entire survey was 8 minutes, which includes 3 minutes for those who completed all of the open‐ended questions. Completing the creativity‐focused question took a median of 30 seconds.

Table 2.

Representative Patient Study Sample Compared With US Adult Population

Individual Characteristics	Respondents (n = 600), %	US Population, %
Age, years 79
<30	11	21
30‐44	17	26
45‐60	28	27
>60	44	26
Race/ethnicity79
White	78	66
Black	10	12
Hispanic or Latino	7	15
Other	5	8
Educationa
High school or less	39	42
Some college	30	29
College graduate	31	29
Chronic health problems 80
Yes	42	50
No	58	50
Physician visits in the previous year 80
1	21	31
2‐3	43	43
4‐0	29	22
>9	7	4
Time with current physician, years 80
≤1	24	37
2‐3	20	19
3‐4	17	12
≥5	39	32

Sources for the US population are as follows: Sociodemographic (from Census Current Population Survey 2019) and Health Status and Utilization (from 2019 National Health Interview Survey).^79,80

Survey

The survey included validated closed‐ended measures of patient experience from the ambulatory Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG‐CAHPS) survey, open‐ended measures from the CG‐CAHPS Narrative Item Set (NIS; which was designed to supplement the closed‐ended CG‐CAHPS survey) and a new open‐ended item focused on eliciting patients’ creative ideas, which was the subject of development and testing. The closed‐ended measures from the CG‐CAHPS survey were patients’ rating of the provider (one item), how well providers communicate with patients (composite of four items), and helpful, courteous, and respectful office staff (composite of two items). The open‐ended measures from the CG‐CAHPS NIS asked patients to share in their own words what they look for in a provider, how their provider measures up, what has gone well, what hasn't gone well, and how they get along with their provider. ¹⁸ , ²¹ We tested a sixth open‐ended question focused on eliciting patients’ creative ideas by randomly assigning one‐quarter of the participants to each of four phrasing alternatives. Each alternative contained an evidence‐based approach to plausibly elicit patients’ creative ideas via survey wording.

Analytic Approach for Aim 1

To develop and test an entirely new method for systematically eliciting patients’ creative ideas, we developed and tested four narrative elicitation approaches for collecting such ideas. The lead author conducted an environmental scan and literature review to identify existing approaches to elicit customer and patient feedback on improvement in health and other industries. Thirty‐four peer‐reviewed publications on customer or patient feedback surveys from the fields of service quality, design thinking, user‐based innovation, and quality improvement were reviewed resulting from PubMed and Business Source Complete searches (example search terms included “survey,” “experience,” “user,” “customer,” “improvement,” and “feedback”). The authors also collectively reviewed seven existing health care surveys that included questions on patients’ suggestions for improvement.

From this environmental scan and literature review, the author team discussed and identified four principles essential to eliciting customers’ or patients’ suggestions for improvement. These four principles were as follows: 1) draw on past exemplars of excellent experiences, ²⁵ which is an approach used in service quality research that anchors respondents in salient details of past service experiences; 2) induce creativity explicitly, ²⁴ , ⁵⁸ which is an approach from design thinking research that primes and invites respondents to access their creative mindset; 3) remove psychological barriers or obstacles to sharing ideas, ²³ which is an approach used in user innovation research that recognizes that psychological or external barriers to voicing suggestions can dampen willingness to participate in improvement; and 4) emphasize health system impact of ideas, ³⁰ , ⁴⁴ which is an approach used in health care improvement that informs patients that the health system receiving their feedback is eager to listen and hear their perspectives. The author team developed 12 draft survey questions, with three questions for each design principle. A patient focus group was convened to provide feedback on these 12 draft questions, and cognitive testing of the questions was conducted to inform the selection of plausible question options. Participants rank ordered the alternatives, and the research team reviewed rank order results. The final four‐question alternatives (and the design principle contained therein) are presented in Table 3. Our online Appendix describes details of the patient focus group and cognitive testing procedures.

Table 3.

Four‐Question Alternatives for Eliciting Patients’ Creative Ideas

Literature	Item
Alternative 1: draw on past exemplars of excellent experiences	Now think back to some of the best health care experiences you have had; what might your current provider and their staff do to be more like these experiences?
Alternative 2: draw on past exemplars of excellent experiences and emphasize system impact	Now think back to some of the best health care experiences you have had; what might your current provider and their staff do to be more like these experiences? Hearing from you is important to improving the care we deliver.
Alternative 3: induce creativity	If you could, what would you change about your care or interactions with your provider and their staff?
Alternative 4: induce creativity and emphasize system impact	If you could, what would you change about your care or interactions with your provider and their staff? Hearing from you is important to improving the care we deliver.

To compare these four‐question alternatives, survey participants in the representative sample were randomized into four arms of 150 people each. Each arm received the same survey except for the last question, which was one of the four drafted questions shown in Table 3. Following the logic of H1 that patients will generate more actionable and creative ideas when invited explicitly to do so using narrative elicitation methods, we used the criteria of actionability and creativity to examine the most effective elicitation method among those being tested.

To test H2 and H3 that the approaches of drawing on past exemplars of excellent experiences or inducing creativity, with or without emphasizing system impact, were associated with more actionable items per idea and higher creativity responses than approaches that did not include the element, we quantitatively compared the level of creativity and number of actionable items generated by each of the four‐question alternatives using multivariate mixed models. In these models, the “past exemplars” alternative was used as the reference category, with “past exemplars with system impact,” “induce creativity,” and “induce creativity with system impact” as levels to be tested in the analyses.

Analytic Approach for Aim 2

To examine the value of patients’ creative ideas, we quantitatively assessed the relationship between patient characteristics and creative idea characteristics from the survey data. There are different analytic plans for H4 vs. H5‐8. To test H4 that patients can generate creative ideas for improved care experiences, we quantitatively compared the level of creativity and number of actionable items generated by the additional survey question focused explicitly on inviting patients’ creative ideas, compared with what was generated by the NIS questions (no explicit invitation for creative ideas). We used multivariate mixed models and created the following idea‐focused variables for the analysis.

Number of Actionable Items per Idea. Three coders (the first author and two Masters‐of‐Public‐Health–trained researchers) independently identified 626 ideas from the 600 patient responses using the working definition of an idea as “an alternative for a possible course of action to approach the task at hand.” ⁵⁹ Of the 626 ideas analyzed, 100 were generated in response to the standard NIS survey questions (no explicit invitation for creative ideas) compared with the 526 generated in response to the creativity‐focused question. A related concept in this study is a creative idea, which is an idea that is novel to the setting and useful to the task at hand. ⁶⁰ Every idea had its own level of creativity (measurement described in next section).

One patient response could contain more than one idea. For example, this response contains two ideas:

There are a couple of ways health care could be better. It would be helpful if my doctor could create a set of picture cards that explain common health conditions and treatments, instead of reading a long pamphlet. For example, a card with a picture of a pill and a clock to say “take this medicine at 8am”. This would make it easier to follow the doctor's advice! Also, it would be great if we could use Siri to help us book appointments. I could call a number and say “I need to see my doctor on Tuesday” and the system would handle the rest.

Ideas were then coded for actionability, which is the extent to which an item can inform improvement efforts by supplying specific and concrete details. ⁶¹ Using the coding schema for actionability from Grob and colleagues, ⁶¹ specific coding categories included the “who” (e.g., “doctor” or “staff”), “what” (action described), “when” (e.g., “before”, “during” or “after” clinical interaction), “where” (setting), and “how” (e.g., response elicited) of the narrative description. The coders collaboratively identified the number of actionable items per creative idea for all 626 ideas, spending 20 hours in total coding. Using the first creative idea in the hypothetical response above a coder could identify five distinct actionable items: “who” (doctor), “what” (create picture cards), “when” (during clinical interaction), “where” (in doctor's office), and “how” (make it easier to follow doctor's advice).

Idea Creativity. The level of creativity for each idea was measured using a gold‐standard method for creativity assessment, known as the “Consensual Assessment Technique.” ⁶⁰ Three experts with backgrounds in quality improvement and ambulatory care (three family medicine clinicians working in different New York area outpatient practices) rated all ideas for novelty (1 to 5 scale, with 1 being least novel and 5 being most novel) and usefulness (1 out of 5 scale, with 1 being least useful and 5 being most useful). Instructions included contextual information about how these ideas were generated by patients in New York who recently attended an outpatient practice and who may be managing a chronic disease, the goals of the research (to understand more about the ideas generated by patients to improve their care experience), and instructions on how to approach rating. This included rating each idea in relation to each other (not compared with ideas seen in your institution) and instructions to guide evaluation of the two idea characteristics. For example, “If an idea strikes you as potentially effective in achieving better care outcomes, this idea has high usefulness” and “If an idea is new or very different from existing practices or ideas, this idea has high novelty.” Each expert's novelty and usefulness scores for each idea were multiplied together, then divided by 5 to give one creativity score per idea. To generate an average creativity score across the raters for each idea, the raters’ scores were added together and divided by 3. There was 84% (first and second coder) and 87% (first and third coder) intercoder reliability for their ratings of creativity (formed from novelty and usefulness scores), considered high and acceptable for this technique. ⁶⁰

Covariates

Covariates were how recently the respondent had seen a clinician; if the respondent had been treated for a severe or life‐threatening health condition in the last 12 months; if the respondent had a long‐term medical condition that required regular medical monitoring or treatment in the last 12 months; and the respondent's age, education, race/ethnicity, gender, housing type, household income, marital status, region within state, ownership status of living quarters, presence of household members, and employment status.

To test H 5–7, we statistically assessed relationships between characteristics of the patients in the sample and their ideas’ characteristics.

Patients with Negative Care Experiences

We assessed patient care experiences using patients’ responses to three constructs in CG‐CAHPS: patients’ rating of the provider (one item), how well providers communicate with patients (composite of four items), and helpful, courteous, and respectful office staff (composite of two items). The “patients’ rating of the provider” question asked patients to rate their provider on a scale of 0 to 10, with 0 being the worst provider possible and 10 being the best provider possible. The “how well providers communicate with patients” composite measure asked patients how often their providers explained things clearly, listened carefully, showed respect, and spent enough time with the patient, with a four‐item frequency response scale (never, sometimes, usually, or always). The “helpful, courteous, and respectful office staff” composite measure asked patients how often office staff were helpful and treated them with courtesy and respect, with a four‐item frequency scale response (never, sometimes, usually, or always). Patients with negative experiences have lower ratings or frequency.

Patient With Minority Racial/Ethnicity Backgrounds

We assessed the race and ethnicity of patients using self‐identified responses in the survey. There were five categories available in the survey: White, Black/African American, Hispanic/Latino, Asian, or other. We treated these as categorical variables, with “White” being the reference category.

Patient With Extensive Exposure to System (Because of Managing Chronic Illness)

We assessed if a patient had extensive exposure to health care systems because of managing a chronic illness by their self‐reported response to the survey item: “Are you actively managing a serious, life‐threatening, or other long‐term condition?” We created a categorical variable for this measure, with “no” being the reference category.

Analyses for H4‐7. Multivariate, mixed models were conducted to examine H4‐7. For H4, we used separate models in which the level of creativity or number of actionable items (dependent variables) generated by the question focused on eliciting patients’ creative ideas was compared with the NIS items that do not explicitly ask for creative ideas (independent variable). For H5, we used models in which the relationship between patients’ care experiences (independent variable) and the number of actionable items per creative idea (dependent variable) was analyzed. H6 assessed the relationship between patients’ race/ethnicity (independent variable) and the level of creativity of their ideas (dependent variable). Finally, H7 assessed the relationship between patients’ managing a chronic illness (independent variable) and the number of creative ideas generated (dependent variable).

Stage 2 Study: A Real‐World Health Systems Test With a Sample of Creative Ideas From Patient Surveys From a Large Health System in New York City

Setting and Sample

We collected ideas from patients who completed patient care experience surveys over 18 months (July 1, 2020, to December 31, 2021). The survey was sent by mail and email to a random sample of patients who had recently had an in‐person ambulatory care visit in one New York City health system. The average age of these patients was 53 years old, and most patients were primarily English speakers (85%) and received services in the internal medicine specialty (65%). Fifty‐five percent of patients received Medicaid, and 26% of patients received Medicare. Twenty‐eight percent of patients were White.

Survey

The survey included the closed‐ended CG‐CAHPS survey (complete survey), followed by the five‐item NIS, followed by the newly developed and tested (in stage 1) item focused on eliciting patients’ creative ideas (invitation to emphasize creativity and emphasis on system impact).

Analytic Approach for Aim 3

To identify how patients’ creative ideas may enable organizational learning for innovation, we conducted a qualitative content analysis of the ideas shared by patients in their survey responses and inductive identification of thematic categories. A subteam of this paper's authors inductively created a new coding schema for the ideas based on capturing different ways that patients’ creative ideas may inform organizational learning by those receiving these data (e.g., health care providers, administrators, patient experience, or quality improvement professionals). The new codes included ideas focused on the following: solving extant challenges, which illuminated an extant challenge and provided a strategy for solving; specifying new interventions, which identified a new intervention that could be developed by the organization; amplifying exceptional practices that might be overlooked as initiatives worth spreading; and hopes for the future, articulating a vision for better future care. An additional code differentiated among the level of specificity of ideas. All ideas were subsequently coded by a coder with deep expertise in coding patient narrative data. The coder discussed progress and questions with a subteam of this paper's authors over the course of 12 meetings. The first author qualitatively analyzed and described the thematic categories and the creative ideas in each category.

Results

Stage 1 Study: A Survey Development and Testing Study to Examine the Value of Patients’ Creative Ideas

Aim 1 Results: Develop and Test a New Method for Systematically Eliciting Patients’ Creative Ideas

H1 that patients will generate more actionable and creative ideas when invited explicitly to do so using narrative elicitation methods was supported. Specifically, we found that a survey question that included both the invitation to induce creativity and emphasized system impact performed best on our criteria of high actionability and creativity. That is, inducing creativity, the approach tested in H2, and emphasizing system impact, the approach tested in H3, were supported. The question wording that combined these two approaches was as follows: “If you could, what would you change about your care of interactions with your provider and their staff? Hearing from you is important to improving the care we deliver” (shown as arm 4 in Table 3).

Table 4 shows the results of the quantitative analyses comparing the four arms. The item with this specific wording of arm 4, described above, generated creative ideas with an average of 1.01 actionable items and a 2.85 (out of 5) creativity score. Compared with arm 1 (reference question), arm 4 generated an additional 0.69 more actionable items per idea and a 3.34‐point increase in creativity score, which are statistically significant differences. Table 5 provides exemplars of highly creative ideas with varied levels of actionability (low, medium, and high).

Table 4.

H2 and H3: Association Between Question Alternatives and Elicitation of Actionable and Creative Ideas (N = 626 Ideas)

Q6 Alternatives	Unadjusted Mean	Actionable Items (Number per Idea)	Idea Creativity (out of 5)
1. Past exemplars	Actionability mean (SD): 0.39 (0.53) Creativity mean (SD): 2.14 (2.75)	Reference category	Reference category
2. Past exemplars and impact	Actionability mean (SD): 0.57 (0.62) Creativity mean (SD): 2.40 (3.76)	0.03 (0.09)	1.02 (0.88)
3. Induce creativity	Actionability mean (SD): 0.37 (0.59) Creativity mean (SD): 2.64 (3.42)	0.20 (0.09) **	1.67 (0.83) **
4. Induce creativity and impact	Actionability mean (SD): 1.01 (1.16) Creativity mean (SD): 2.80 (4.80)	0.69 (0.09) **	3.34 (0.77) **

H2, hypothesis 2; Q6, question 6; SD, standard deviation.

^** p < 0.001.

Covariates included how long it has been since the patient has seen doctor, if they have been treated for a severe or life‐threatening health condition in the last 12 months, and if they have some other long‐term medical condition that required regular medical monitoring or treatment in the past 12 months as well as the patient's age, education, race/ethnicity, gender, housing type, household income, marital status, region based on the state of residence, ownership status of Yliving quarters, presence of household members, and current employment status.

Table 5.

Examples of Patient Ideas by Characteristics

Idea Characteristics	Idea Description	Explanation
High creativity, low actionability	“I think the clinic should have a calming vibe room with music. That would make me feel better instantly.”	Although highly creative, this idea may be challenging to implement because of few actionable details that could guide organizations to feasibly translate the idea into practice.
High creativity, medium actionability	“People in the community help patients understand medical stuff. There could be friendly guides who explain things like how my body is functioning in simple words.”	This creative idea falls in the medium actionability range, potentially requiring volunteer organization and training to improve health literacy.
High creativity, high actionability	“I like how my clinic has fruit in the waiting area. How about a healthy cooking corner? We could learn how to cook healthy meals on a budget. The greeters could give out simple recipes or veggies sometimes.”	This idea combines creativity with high actionability, promoting health education and access to nutritious food through a simple clinic‐based program. The idea provides many concrete details on how the idea could be successfully implemented.

High‐creativity ideas have a creativity level of 3 or higher of 5, and many ideas in the sample met this threshold. We do not showcase low‐creativity ideas in this table because they are potentially less interesting to organizations seeking new ideas that diverge from existing practice.

Aim 2 Results: The Value of Patients’ Creative Ideas

Creative ideas (N = 626 ideas) were collected from all 600 responses. H4 that patients can generate creative ideas for improvement, was supported. As shown in Table 6, adding a survey question that explicitly invites patients to share their creative ideas generated ideas with double the actionability (1.96 more actionable items per idea) and higher levels of creativity (rated 4.03 out of 5) compared with ideas generated in response to the remaining NIS survey items.

Table 6.

H4: Idea Characteristics Elicited by a Creativity‐Focused Survey Question (N = 626 Ideas)

	Actionability		Creativity
	Mean (SD)	Actionable Items (Number per Idea)	Mean (SD)	Idea Creativity Score (out of 5)
Ideas generated in response to a standard NIS open‐ended survey a	0.59 (0.62)	Reference category	0.76 (2.56)	Reference category
Ideas generated in response to a creativity‐focused, open‐ended question only	1.01 (1.16)	1.96 (0.08) **	2.80 (4.80)	4.03 (0.78) **

H4, hypothesis 4; NIS, narrative item set; SD, standard deviation.

^* p < 0.05,

^** p < 0.001.

Covariates included how long it has been since the patient has seen doctor, if they have been treated for a severe or life‐threatening health condition in the past 12 months, and if they have had some other long‐term medical condition that required regular medical monitoring or treatment in the past 12 months as well as the patient's age, education, race/ethnicity, gender, housing type, household income, marital status, region based on the state of residence, ownership status of living quarters, presence of household members, and current employment status.

^a Validated five open‐ended question sequence asking patients to share in their own words what they look for in a provider, how their provider measures up, what has gone well, what has not gone well, and how they get along with their provider.

H5 that patients with negative health care experiences generate more actionable ideas was supported. As shown in Table 7, as scores on patient experience measures decreased, actionability of written creative ideas from patients increased significantly. When the provider rating score was reduced by 0.07 points, the communication score by 0.23 points, or the office staff score by 0.38, there was an additional actionable item included in each creative idea.

Table 7.

H5: Patient Care Experiences and Actionable Items per Idea (N = 626 Ideas Associated With 600 Respondents)

	Actionable Items per Idea (Number per Idea), β (SE)
Independent variables
CG‐CAHPS visit rating score (out of 10)	−0.071 (0.02) **
CG‐CAHPS provider communication score (out of 5)	−0.23 (0.06) **
CG‐CAHPS office staff score (out of 5)	−0.38 (0.09) **
Patient‐level covariates
Length of time since last saw provider	0.01 (0.02)
Treatment for severe or life‐threatening health condition in last 12 months	0.04 (0.09)
Long‐term medical condition that required regular medical monitoring or treatment in the last 12 months	−0.14 (0.08)
Age	−0.13 (0.03)
Education (ref: no less than high school)
High school	0.04 (0.51)
Some college	0.09 (0.04)
Bachelor's degree or higher	0.10 (0.05) *
Race/ethnicity (ref: White)
Black or African American	0.09 (0.06)
Hispanic or Latino	0.03 (0.04)
Other	0.02 (0.05)
Gender (ref: female)
Male	0.17 (0.12) **
Housing type	0.08 (0.02)
Household income (ref: low income <$40,000)
Medium income ($40,000‐$99,999)	0.01 (0.02)
High income (≥$100,000)	0.09 (0.03)
Marital status	0.02 (0.03)
Region based on state of residence	0.17 (0.12)
Ownership status of living quarters	0.02 (0.20)
Presence of household members	0.17 (0.07)
Current employment status	−0.05 (0.09)

CG‐CAHPS, Consumer Assessment of Healthcare Providers and Systems Clinician and Group; H5, hypothesis 5; ref, reference group; SE, standard error.

^* p < 0.05,

^** p < 0.001.

H6 that patients with minority racial/ethnic backgrounds will generate higher creativity ideas compared with White patients was supported. As shown in Table 8, patients who identified as Black/African American or Hispanic/Latino generated higher creativity ideas compared with White patients. Specifically, compared with White patients, patients who identified as Black/African American generated ideas that were more creative by 1.33 points (out of 5) and patients who identified as Hispanic/Latino generated ideas that were more creative by 1.22 points (out of 5).

Table 8.

H6: Patient Race/Ethnicity and Idea Creativity (N = 626 Ideas Associated With 600 Respondents)

	Dependent Variable Idea Creativity Score (out of 5)
Independent variable, race/ethnicity
White	ref
Black or African American	1.34 (0.22)**
Hispanic or Latino	1.22 (0.22)**
Other	1.04 (0.13)*
Patient‐level covariates
Length of time since last saw provider	0.10 (0.02)
Treatment for severe or life‐threatening health condition in last 12 months	0.04 (0.09)
Long‐term medical condition that required regular medical monitoring or treatment in the last 12 months	−0.14 (0.08)
Age	−0.33 (0.20) *
Education (ref: no less than high school)
High school	0.34 (0.83)
Some college	0.93 (0.48)
Bachelor's degree or higher	1.33 (0.39) **
Gender (ref: female)
Male	0.37 (0.59)
Housing type	0.08 (0.02)
Household Income (ref: low income <$40,000)
Medium income ($40,000‐$99,999)	0.09 (0.07)
High income (≥$100,000)	0.14 (0.02)
Marital status	0.02 (0.03)
Region based on state of residence	0.17 (0.12)
Ownership status of living quarters	0.02 (0.20)
Presence of household members	0.37 (0.60)
Current employment status	−0.09 (0.07)

H6, hypothesis 6; ref, reference group.

^* p < 0.05,

^** p < 0.001.

H7 that patients with extensive exposure to the care system generate a higher volume of creative ideas was supported. As shown in Table 9, patients who were managing a chronic illness generated a higher number of creative ideas than those who were not. Specifically, compared with patients who were not managing a chronic illness, patients who were managing a chronic illness had 2.3 times increased odds of creative idea generation.

Table 9.

H7: Patient Exposure to System (Management of Chronic Illness) and Idea Volume (N = 626 Ideas Associated With 600 Respondents)

	Dependent Variable Odds Ratio (Confidence Interval)
Independent variables
No long‐term medical condition that required medical monitoring or treatment in the last 12 months	Ref
Long‐term medical condition that required medical monitoring or treatment in the last 12 months	2.31 (1.93‐2.84) **
Patient‐level covariates
Age	1.52 (1.24‐4.78) *
Education (ref: no less than high school)
High school	4.49 (0.95‐21.12)
Some college	4.86 (1.04‐22.26) *
Bachelor's degree or higher	5.88 (1.28‐27.08) **
Race/ethnicity (ref: White)
Black/African American	1.88 (1.24‐4.15)
Hispanic	1.62 (0.92‐2.82)
Other	0.2 (0.38‐0.78)
Gender (ref: female)
Male	0.52 (0.38‐0.78) **
Housing type	0.08 (0.02)
Household Income (ref: low income <$40,000)
Medium income ($40,000‐$99,999)	0.93 (0.48‐1.80)
High income (≥$100,000)	0.34 (0.14‐0.83) **
Marital status	0.02 (0.03)
Region based on state of residence	0.17 (0.12)
Ownership status of living quarters	0.02 (0.20)
Presence of household members	0.17 (0.07)
Current employment status	−0.05 (0.09)

H7, hypothesis 7; ref, reference group.

^* p < 0.05,

^** p < 0.001.

Stage 2 Study: A Real‐World Health Systems Test to Identify How Patients’ Creative Ideas May Prompt Organizational Learning for Innovation

Aim 3 Results

We collected 2,948 ideas from 1,892 patients who completed patient care experience surveys (with a completed creativity‐focused question) over 18 months. A total of 2,035 surveys were received, but only 93% of these surveys had a completed creativity‐focused question and were therefore retained for analysis. Unfortunately, we are not able to provide data on the patients’ time to complete surveys in this setting because the organization's survey infrastructure does not capture this information. The expert coder spent 63 hours in total coding and categorizing the ideas contained in the patient responses. Table 10 shows characteristics and examples of the creative ideas generated by patients in the real‐world LHSs test. Most creative ideas were focused on problem‐solving extant challenges (43%) or specifying new interventions (42%), with a smaller proportion of ideas amplifying exceptional practices or people (5%) or forecasting hopes for the future (10%). Ideas amplifying the exceptional had the highest proportion of “very specific” content (70%), followed by those specifying new interventions (52%). This distribution was mostly consistent throughout the study period, with one exception. During the last 6 months of data collection (June 1, 2021, to December 31, 2021), the proportion of ideas with new interventions suggested exceeded the proportion of ideas solving extant challenges (41% vs. 37%).

Table 10.

Domains of Creative Ideas Elicited from Health System Patient Sample

Domain	Description	Example of Patient Response
Problem‐solving extant challenges	Creative idea shines a light on an extant challenge that might be hiding in plain sight and provides a strategy to solve the problem.	Can you assign me a physician who can oversee my overall health? I am finding my medical care so fragmented into specialties, and trying to get a hold of each doctor really bothers me.
Specifying new interventions	Creative idea identifies a new strategy or intervention that could be developed and implemented by the organization.	Would appreciate someone who can help me find a ride to my visit every month!
Amplifying exceptional practices and people	Creative idea identifies existing practices that may be overlooked or unrecognized as initiatives worth spreading throughout the organization.	My PCP is perfect. After blood tests, he telephones us at night to discuss the results and explains everything. Who does this? Only him, and he's done this for years.
Forecasting hopes for the future	Creative idea articulates a vision for better future care that matters to patients as a starting point for intervention development.	Here's to a future where my physicians’ well‐being and staff's well‐being can be safeguarded with better hours.

PCP, primary care physician.

There was a subset of ideas focused on issues during the COVID‐19 crisis, such as vaccine rollout and accessing non–COVID‐19 health care services, that are unlikely to endure in a postemergency phase of the health system. For example, “If I could, I would change the number of appointments [at] a given hour to decrease congestion [and] the risk of COVID transmitting while in closed space over an hour.” There was also a subset of ideas focused on issues prompted by the COVID‐19 crisis but is likely to endure in the years ahead, including highlighting workforce burnout, inadequate resources, and staffing shortages and need for cultural competency from providers and staff when working with patients from marginalized or underserved populations. For example, “I think to the degree you can safeguard your physicians’ well‐being and staff's well‐being, that's great, and maybe that involves allowing for slightly longer visits allotted per patient and good breaks, too.” The remainder of ideas were focused on issues that endured from pre–COVID‐19 times, such as improving relationships with clinicians and recognizing outstanding examples of model clinicians or practices. For example, “Dr. X. is a medical rock star. He uses analogies like plumbing or puzzles to explain my condition during my visits. He has saved my life on more than one occasion.”

Discussion

Patients’ creative ideas are derived from their health care experience, and they can provide a potentially powerful form of knowledge for health systems seeking to innovate for better patient‐centered care. ⁷ This research considered methods for systematically eliciting patients’ creative ideas, the value of patients’ creative ideas, and the nature of creative ideas that a real‐world health system elicits. We discuss our findings and learnings in more detail below.

How to Elicit Creative Ideas From Patients: Narrative Elicitation

This research developed and tested a valid and reliable strategy for eliciting creative ideas from patients using methods that seamlessly integrate open‐ended questions into routine survey operations. We worked with patients to codesign and test an open‐ended survey question that can be inserted at the end of NISs that follows conventional closed‐ended patient experience surveys. ²² Of the four options tested, the final open‐ended question generated roughly double the number of actionable items and highly creative ideas. This question was distinct in its approach to induce creativity explicitly and emphasize system impact in a second sentence. This validates the notion that patients will share their creative ideas when provided the opportunity and assurance that it will be valued. The survey elicitation method we tested also adds to evidence that rigorous survey methods can provide a platform for patients’ voices. ²¹ , ²² Many organizations already survey patients, so this additional question may be a relatively easy strategy to implement with sizable impact for idea generation, organizational learning, and innovation.

Creative Ideas Can Come From Various Subsets of Patients

This work evaluates the value of patients’ creative ideas, adding to the evidence on the value of other forms of feedback and input from patients, including perspectives on care experiences, care design, and care outcomes. ⁸ , ²⁰ , ⁶² Although most of these forms of patient voice focus their content on past care experiences, creative ideas are focused on actions to improve future care. The act of generating and sharing a creative idea requires a degree of analysis (reflecting, interpreting, and learning) ⁵⁰ as well as agency (sharing an alternative vision for health care) from the patient. ⁶³ As such, inviting patients to generate and share their creative ideas may support their agency and autonomy in health care systems that need more of both and that seek to create more opportunities for patient feedback and empowerment. ³³ , ⁴⁷

Our results suggest that all patients who experience health care, that is all patients who receive standard patient experience surveys postvisit (or their surrogates), are capable and willing to provide their creative ideas for improvement. To our knowledge, we know of no other studies that have investigated this widespread potential of systematically eliciting patients’ creative ideas. Studies utilizing crowdsourcing and qualitative interview methods show patients with chronic conditions or living with HIV illness generate many ideas when prompted with a question on quality improvement. ⁶⁴ , ⁶⁵ Other studies show the benefits of engaging small samples of highly activated patients for their input on specific care experiences. ⁹ , ¹⁴ Our study builds on these past streams of research and extends them by showing how patients’ creative ideas can be elicited systematically as part of routine quality improvement in health care organizations.

Moreover, our findings are significant because democratizing patient participation in health care improvement is important for health equity. ⁵ , ⁷ , ⁵⁷ It is well known that disparities in experiences of care differ by characteristics such as race/ethnicity, income, education status, and other characteristics. ¹⁶ , ¹⁷ , ³⁷ , ⁴⁰ , ⁴¹ , ⁴⁵ Providing an opportunity for patients to suggest how these inequitable experiences can be improved could be an important step to narrow this gap.

Patients With Negative Health Care Experiences Generate the Most Actionable Ideas

The more negative the patients’ care experiences are, the more actionability is embedded in their improvement ideas. Health care leaders are often concerned that wronged patients are focused on complaints. ³⁵ , ³⁹ In contrast, our findings suggest patients want to help make health care better and have specific suggestions on how to achieve positive change. We encourage health care leaders to embrace the participation of patients who may have had poor care experiences in the past, as they may be especially motivated to make care better in the future.

Patients With Minority Racial/Ethnic Backgrounds Generated the Highest Creativity Ideas

Patients who identified as Black/African American or Hispanic/Latino generated significantly higher creativity ideas than patients who identified as White. Their ideas were rated as most novel (new to the setting) and useful, which suggests that their ideas may offer a distinctive perspective that is helpful for learning and innovation. ¹¹ , ⁶⁵ , ⁶⁶ Studies show that patients from racial or ethnic minority backgrounds receive systematically suboptimal care. ¹⁶ , ¹⁷ , ⁴¹ , ⁴² Our findings suggest democratizing patient participation in sharing ideas is potentially important for health equity, ⁷ , ⁶⁷ providing an opportunity for underrepresented patients with poor experiences to suggest how these experiences can be improved. ¹⁷

Patients With Extensive System Exposure Because of Chronic Illness Are Most Likely to Generate Greater Numbers of Creative Ideas

Patients who were managing a chronic illness generated the greatest number of ideas in the project. These patients, who may visit as many as 16 clinicians in a year, have a unique grasp on navigating the system that appears to result in many creative ideas. ¹⁵ , ⁴⁵ We encourage health care leaders to support the subsets of patients highlighted in this study (those with negative care experiences, minority racial/ethnic backgrounds, and extensive system exposure because of chronic illness) and be especially attentive to the rich and numerous insights they have on how to make navigating health care systems easier and better. ¹ , ²⁹

How Patients’ Creative Ideas May Enable Organizational Learning for Innovation

The LHSs test of eliciting patients’ creative ideas showed four possibilities for patients’ creative ideas to enable learning for innovation. Creative ideas can solve extant challenges. LHSs could integrate these creative ideas into agenda setting and prioritization of patient experience “pain points” in settings in which problem identification is key, such as evidence‐based codesign or lean improvement methods such as Kaizen boards. ⁶⁸ , ⁶⁹ Creative ideas can specify new interventions. LHSs could use these creative ideas to help with intervention development during processes such as team huddles or plan–do–study–act cycles in which generating solutions for improvement is important. ⁶⁹ , ⁷⁰ Patients who generated creative ideas could also be part of solution development in settings such as patient and family advisory councils or innovation development teams. Creative ideas can amplify exceptional practices or people. These creative ideas may support staff development efforts, including efforts to retain, develop, and enhance morale of clinicians and staff who benefit from seeing the impact of their work on patients. ⁶⁷ Finally, patients’ creative ideas can forecast hopes for the future. This knowledge may provide important input for organizational strategic planning and forecasting, providing a sense of what patients value and hope for the future, even if the specifics may not be feasible in the short term.

Limitations

This research is not without limitations. Although the value of patients’ creative ideas have been explored in this work, creative idea generation does not guarantee implementation in organizations. ¹¹ , ⁶³ Patients’ creative ideas as a source of knowledge for LHSs, and the methods supporting their implementation, are the starting point in a long journey from idea generation to implementation. ¹¹ Eliciting patients’ creative ideas, furthermore, will require some level of organizational investment of resources, including time and expertise in coding the resulting ideas for use by the organization. These resources will likely compete with other demands during a time of historic resource and supply constraints in health care. ⁶⁷ Finally, our sampling strategy (oversampling patients with chronic conditions and ending data collection once 600 responses to the creativity‐focused question were complete) did not allow us to address nonresponse bias. Our Ipsos sample included disproportionately older and more highly educated respondents than the general population and, therefore, may not completely represent the general population. Nevertheless, this work shows the potential use of patients’ creative ideas as a complementary source of knowledge for organizations seeking to better integrate patient voice in innovation. Further, the methods studied could serve as a starting point for possible synchronous codesign efforts being led in organizations, such as by patient and family advisory councils.

Implications for Policy and Practice

This study contributes to policy on advancing patient‐centered care by specifying creative ideas as a potentially important but overlooked form of patient knowledge to complement PROMs, reports of patient experience, and other forms of patient data currently collected on a large scale. ²⁰ Our findings highlight the benefits of engaging patients beyond elevating their voice for the purpose of enhanced participation. ³⁶ The ideals of LHSs, patient coproduction, and codesign all encourage patients to be engaged as partners in improving and innovating health care. It seems possible that many patients have important suggestions for improvement, but this potential is overlooked, perhaps deprioritized in health care environments with competing demands, lack of time, and organizational capacity to focus on patient perspectives. We anticipate that by demonstrating the potential for patient‐reported creativity, we will galvanize greater exploration of this potential and evidence regarding its impact on prevailing health system practices, thereby shifting priorities in the patient experience community and among health care administrators. Patients’ creative ideas, which may have in the past gone unrecognized or unreported, contribute critical knowledge and distinctive inputs for innovation—a new frontier of patient engagement.

This paper documents a rigorously tested methodological strategy to assist LHSs in cultivating innovations using continuously generated knowledge from patients. The method has the potential to address certain challenges that patients face in sharing their voice in health systems, including a perceived lack of power (compared with medical professionals and administrators) and few forums to share their ideas. ⁸ , ¹⁵ , ²⁰ For health care professionals working on innovation, quality improvement, and operations, our findings suggest that patients’ creative ideas can aid learning and guide effective organizational responses. Furthermore, they can be assured that inserting the validated survey question at the end of NISs (which follow conventional closed‐ended patient experience surveys [e.g., CG‐CAHPS]) is an evidence‐based strategy for eliciting a higher volume of actionable and creative ideas from patients.

Innovations in reporting and using these data in routine health care delivery and operations are also crucial. Health care organizations should examine how patients’ creative ideas could be available with limited delay to administrators, clinicians, and staff responsible for quality improvement and service recovery. Ensuring access to patients’ creative ideas may democratize the process of quality improvement intervention in organizations by allowing many clinicians and staff the opportunity to be inspired and informed by patients’ ideas. Policymakers and regulators may also consider making such data available beyond the confines of single organizations, perhaps by crowdfunding creative ideas across geographic regions to gain a better understanding of the priorities and needs of specific patient populations. Patients’ creative ideas could also be made public in innovation tournaments and hackathons, in which the most promising ideas are developed for widespread dissemination across care delivery systems and sectors, inviting cross‐institutional and sectoral collaborations that benefit the population.

As unstructured forms of data from patients, especially narrative accounts in their own words, gain prominence and dissemination in health systems worldwide, health systems and individual health care organizations could consider linking these data with other forms of performance data. Just as our research showed the significant association of patients’ creative ideas with closed‐ended patient experience scores, so too should future research investigate the possible relationship of patients’ creative ideas with other forms of performance data, such as measures of clinical quality, equity, cost efficiency, and provider well‐being. All pillars of the US system's Quintuple Aim. Patients’ creative ideas should also be linked to data being generated in innovation centers hosted under the auspices of governments and consulting firms and internally within health care organizations, as patient‐generated inputs into innovation could provide a powerful complement to innovations generated from other sources, such as executives or consultants. Patients’ creative ideas collected via large‐scale survey methods like those tested in this study should also complement ongoing patient and family advisory council and experience‐based codesign efforts. Ideas stemming from these narrative methods can provide generalizable data from a broad sample of patients, ensuring the perspectives of underserved and marginalized patients are being represented and providing data on trends over time and by demographic and socioeconomic characteristics.

Finally, for leaders in health care organizations interested in implementing this approach, the following steps may aid implementation success. First, identify and invite colleagues who may be interested and focused on patient‐centered improvement efforts. This may include members of the patient experience, quality improvement, safety, and clinical and administrative teams. Second, assessing current capability and systems for gathering feedback from patients is important, too, including identifying current surveys (if any) that allow for the inclusion of open‐ended questions to complement closed‐ended survey questions. Adding the open‐ended survey question focused on inviting patients to share their creative ideas for improvement and innovation, described in this project, should occur next. As shown in this work, adding this question did not incur any additional costs to survey operations and posed minimal burden to respondents and implementors in terms of time and attention. Patients spent a median time of 30 seconds to respond to the creativity‐focused survey question, and coders spent a median time of 2 minutes on each patient narrative response that contained creative ideas, a limited time burden for patients and implementors with what we believe we have demonstrated to be high‐value yield. Based on our results, organizations may benefit from oversampling patients managing a chronic disease, belonging to minority racial/ethnic groups, or experiencing negative experiences to obtain distinctive creative ideas. Third, organizations must develop and execute plans to analyze and share the resulting data routinely with administrators, clinicians, and staff. Analyses of narrative data can be done efficiently and effectively using human coders or artificial intelligence/large language models, although large language models still require more training to achieve the fidelity of human coding. ⁷¹ Regardless of analytic approach, recent research indicates that when that narrative sharing occurs frequently, for example, through meetings and huddles ⁷² or through an online dashboard for narrative reporting, ⁷³ patient experience scores can improve significantly. Therefore, a sharing plan is essential. Sharing allows for responsive action to improve patient experience. Finally, organizations must have an enabling structure for review and implementation of creative ideas. Recent research indicates that using a structured approach to change (e.g., plan–do–study–act cycles) and staffing implementation teams with those central in the organization's social network are particularly helpful for implementation of creative ideas. ⁷⁴

Conclusion

Our results suggest that inviting patients’ creative ideas could inspire positive dynamics within health care organizations and provide access to untapped expertise on how to reimagine care experiences for the better. In an organizational and policy environment that desires, needs, and incentivizes patient engagement, systematically eliciting patients’ creative ideas from large‐scale surveys holds substantial promise. Patients can do more than just report their experiences: they can interpret those experiences, envision how to improve them, and articulate those ideas. This work indicates that they are poised to take up their well‐deserved position as a unique and distinct source of expertise on how to achieve patient‐centered care.

Funding/Support

This work was supported by grant NYSF‐18‐06107 from the New York Health Foundation and grant U18 HS016978 from the Consumer Assessment of Healthcare Providers and Systems V of the Agency for Healthcare Research and Quality.

Conflict of Interest Disclosures

The authors have no conflict of interest to declare.

Feedback on the 12 draft questions was collected via a patient focus group. The lead author convened a group of six participants who were patients with frequent access to ambulatory care services in New York State. Participants were recruited as a convenience sample with assistance from a community‐based organization focused on patient advocacy. The participants reflected diversity in backgrounds and care experiences (50% female, 67% Black, 33% managing a chronic condition for over 2 years). The group evaluated and critiqued the draft questions and provided input on language and cultural conventions that may result in unanticipated responses. For example, participants flagged that a question that described a “magic wand” that could change their care experience might be interpreted as making light of a severe situation or an example of cultural appropriation. Two questions were removed based on focus group recommendations.

Cognitive testing of the 12 draft survey questions was also conducted to inform the selection of plausible question options. Fifteen individuals were recruited for their expertise as a patient with frequent access to ambulatory care services (n = 8), patient experience professionals in a health care system (n = 2), or patient advocates (n = 5). Participants were emailed a document with the 12‐draft survey questions and asked to rank the item alternatives from 1 to 12. The lead author then conducted 30‐to‐60‐minute telephone conversations with each participant to obtain their feedback on questions. The author team reviewed rank order results and discussed cognitive interview themes, which centered on if questions prompted respondents to share an idea and how respondents interpreted the questions’ potential for actionability. A critical insight that emerged from the cognitive interviews was that respondents were generally encouraged by including a second sentence in the survey question that emphasized the impact of a response by the system. The top four ranked questions across the 15 interviewees were also the questions evaluated as the more comprehensive and solicitous (showing care and attentiveness to responses). The final four‐question alternatives (and the design principle contained therein) are described in Table 2.