Abstract
Background:
Person-centred coordinated care (P3C) is a priority for stakeholders (ie, patients, carers, professionals, policy makers). As a part of the development of an evaluation framework for P3C, we set out to identify patient-reported experience measures (PREMs) suitable for routine measurement and feedback during the development of services.
Methods:
A rapid review of the literature was undertaken to identity existing PREMs suitable for the probing person-centred and/or coordinated care. Of 74 measures identified, 7 met our inclusion criteria. We critically examined these against core domains and subdomains of P3C. Measures were then presented to stakeholders in codesign workshops to explore acceptability, utility, and their strengths/weaknesses.
Results:
The Long-Term Condition 6 questionnaire was preferred for its short length, utility, and tone. However, it lacked key questions in each core domain, and in response to requests from our codesign group, new questions were added to cover consideration as a whole person, coordination, care plans, carer involvement, and a single coordinator. Cognitive interviews, on-going codesign, and mapping to core P3C domains resulted in the refinement of the questionnaire to 11 items with 1 trigger question. The 11-item modified version was renamed the P3C Experiences Questionnaire.
Conclusions:
Due to a dearth of brief measures available to capture people’s experience of P3C for routine practice, an existing measure was modified using an iterative process of adaption and validation through codesign workshops. Next steps include psychometric validation and modification for people with dementia and learning difficulties.
Keywords: clinician–patient relationship, patient/relationship centered skills, patient expectations, patient feedback, quality improvement, relationships in health care, survey data
Background
Measuring patient experiences of health-care delivery is now recognized as a valuable mechanism to evaluate the quality of care from the perspective of the patient (1,2). Such measures are often included in the evaluation of new models of care aimed at improving both the outcomes and the quality of care for people with long-term conditions (LTCs) and multiple long-term conditions (MLTCs) (3 –5).
People with LTCs and MLTCs often describe their care as fragmented and lacking in continuity (6 –8) and call for better coordination of their care (9). Individuals also often feel that their wishes and preferences are not taken into consideration (10 –20). These experiences can be described as non–person-centred care (PCC), which is often exacerbated by poor coordination (10,21,16). A recent systematic review identified care coordination, clinical guidelines, (non-)PCC, and shared decision-making as critical areas that require improvement in people with LTCs (22).
Person-centred care can be described as “care where individuals’ values and preferences are elicited to guide all aspects of their health care, supporting their realistic health and life goals” (23, p. 16). To recognize the importance of care coordination for the delivery of PCC for people with LTCs and MLTCs (23,24), we brought these concepts together in a single operating framework; what we call person-centred coordinated care (P3C). Our framework (25) has been influenced by the work of Ekman et al who have identified 3 core practice routines for the delivery of PCC, which emphasize personhood, equality, and personalized outcomes (26): through the (a) elicitation of a personal narrative, (b) the establishment of a partnership and treatment goals, and (c) the documentation of this in a care plan (26,27). We emphasize a fourth routine “agreeing to act with the person and their family to coordinate the care plan” (25, p. 24). For P3C to be achieved, we argue that activities within the following 5 core domains need to take place: “person-centred information and communication,” “the identification of goals/outcomes and what is important to the person,” “person-centred care planning,” “the management of health states and service transitions,” and “collaborative/shared decision making” (9,13,28,29,30).
Our domain model was created to support services develop new models of person centred coordinated care (P3C). According to these core domains, an accompanying evaluation framework probes multiple perspectives (ie, that of patients, staff, and the organization) (31). In developing this framework, our initial aim was to identify a measure capable of probing if care received is both person-centred and coordinated, from the perspective of the patient. However, due to the lack of suitable measures available, we modified our aim and developed an existing measure suitable for this purpose using stakeholder codesign.
Methods
Our methods included a rapid review, stakeholder codesign workshops and measurement modification, and cognitive testing of the modified measure.
Rapid Review
A rapid review (32) was undertaken to identify candidate measures that capture experiences of P3C (Figure 1). We searched Pubmed, Medline, and Cochrane databases of systematic reviews for recently published literature on the topic using the terms “integrated care,” “coordinated care,” “person-centred care” and “continuity of care” and “systematic/review” as keywords. Seven relevant systematic reviews were identified of which 2 were on integrated care (33,34), 2 on continuity of care (35,36), 2 on person-centredness (37,38), and 1 on care coordination (39). Reports from key organizations such as the Health Foundation (40), Picker Institute Europe (41), Agency for Healthcare Research and Quality (42), Department of Health (DH) (43), and gray literature were considered next. Outcome measure databases such as the database by the Patient-Reported Measurement Group of Oxford University (http://phi.uhce.ox.ac.uk) and Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID) (http://www.proqolid.org/) were also searched. Finally, measures that were used or currently being used by new care models were taken into account (44).
Figure 1.
A schematic representation of the rapid review.
Measures were included if they covered the patient’s perspective (a) of PCC and/or (b) their experiences of coordinated/integrated care. Of the measures that were identified, 71 were considered for inclusion (Appendix A) of which only 7 met our inclusion criteria (see Table 1).
Table 1.
Brief Outline of the Selected Measures.
| Name of Measure | No. of Items | Country of Development | Aim of Measure |
|---|---|---|---|
| 1. Patients Perceptions of Integrated Care Survey (PPIC) (45) | 80 | Harvard School of Public Health, United States | To measure patient’s perception of integrated care |
| 2. Patient Perceived Continuity from Multiple Clinicians (CC-MC) (46) | 34 | Canada | To measure continuity of care from the perspective of patients who regularly see more than 1 clinician |
| 3. Patient Assessment of Chronic Illness Care (PACIC) (47) | 20 | United States | To measure the extent to which patients receive care consistent with the chronic care model (CCM) |
| 4. Client Perception of Coordination Questionnaire (CPCQ) (48) | 31 | National Centre for Epidemiology and Population Health, Australia | To measure coordination of health care |
| 5. Long-Term Condition 6 (LTC6) questionnaire (43) | 6 | DH Quality, Innovation, Productivity and Prevention (QIPP) LTC team, United Kingdom | To measure patient’s confidence in services/care given and own abilities to self care |
| 6. Patient-Service User Questionnaire—Integrated Care Evaluation Pilots (44) | 26 | United Kingdom | To evaluate patient experiences of integrated care |
| 7. North West London Integrated Care Pilot: Patient Survey (49) | 19 | Integrated Care Pilot team in North West London, United Kingdom | To measure patient’s perception of integrated care |
Reasons for excluding measures were (a) not aimed at LTCs or older people (eg, generic national surveys), (b) only 1 domain of integrated or PCC covered (eg, communication, empathy), (c) minimal questions of relevance to P3C, (d) covering only episodic care (eg, hospital visit), (e) condition specific (eg, chronic pain, cancer), or (f) measure still in development.
To critically examine included measures, these were mapped to the core domains of our P3C framework. The identified domains and questions were then compared with existing patient experience frameworks (Institute of Medicine framework (50), the Picker Institute framework (51), Warwick Patient Experiences framework (52), and NICE guidelines (53)) to identify missing constructs.
Codesign Workshops
The 7 included measures were presented to stakeholders in a series of workshops to explore their acceptability, utility, and strengths/weaknesses. The first workshop with 12 attendees comprised patient and carers (patients n = 3; female aged 70, male aged 67, female aged 57; carer n = 1, female aged 65), commissioners (n = 2), clinical staff (n = 2), social care representatives (n = 1), and the academic research team (n = 3) and lasted for 3 hours. First, each individual measure was discussed in detail in small groups. The top 3 measures were then selected by a final voting process and agreed by all.
The top 3 measures were then presented and discussed in further 3 workshops. The first of these had a mixed representation of stakeholders (n = 11) involving patient representatives (n = 3; 1 male aged 58, 2 females aged 57 and 68 years), commissioners (n = 1), clinical staff (n = 2), social care representatives (n = 2), and academic research team (n = 3). The final 2 were organized solely with patient representatives (n = 5 and n = 7; age ranged: 58-80, 7 females and 5 males). All workshops lasted for an hour and a half. Stakeholders had an opportunity to comment on each of the 3 questionnaires. Discussions focused on the content and language used in the measures but also relevance to lived experience, length, and layout. All patient representatives had LTCs or MLTCs and were identified through general practice patient participant groups. Their participation was a patient representatives and codesign experts and not as research participants.
Measurement Selection and Modification
The preferred measure was subject to the modifications that focused on adding extra questions as per the feedback obtained from codesign workshops. The Modified Long-Term Condition 6 (LTC6; renamed Person-Centred Coordinated Care Experiences Questionnaire [P3CEQ]) underwent several iterations based on feedback from the team, other researchers working in P3C, and our codesign workshops where questions were redesigned by patients.
Cognitive Testing of the Modified Version
The next step was refine the modified measure through cognitive interviews with patient representatives. This is a method of collecting additional information on a questionnaire from end users which is then used to determine whether each question is generating the response/information that is intended by the developers (54). It explicitly focusses on the cognitive process that a person uses to answer questions which may include comprehension of question, retrieval from memory of relevant information, judgment, and estimation process and response process (ie, mapping answers to response options) (55). It also seeks to ensure that the questions are meaningful to those the measure is aimed at.
Patient representatives were approached through a local public engagement coordinator and 5 participants (4 males, aged 55-69, and 1 female, aged 67) volunteered to take part in these interviews. The participants either had an LTC such as diabetes and renal problems or cared for someone with health (physical and mental) difficulties. Verbal consent was obtained and the interviews were audio recorded. T.S., a trained interviewer conducted these interviews, and each interview lasted around 1 hour. Monetary compensation was provided for their time. Both the “think aloud” and “verbal probing” techniques were used throughout to comprehend how participants were responding to each questions (54,55).
Results
Rapid Review
Our review work identified 13 important subdomains of patient experience which fall within our 5 core domains (28): (a) goal setting, (b) empowerment/activation, (c) self-management, (d) carer involvement, (e) care plan, (f) case manager/key person, (g) single point of contact, (h) care coordination (i within teams; ii across teams), (i) continuity of care (eg, regular appointment, follow-up), (j) involvement in decision-making, (k) relational continuity, (l) information gathering/sharing, and (m) knowledge of patient/familiarity.
Each item from the 7 included measures (see Table 1) was mapped to these domains and subdomains by 2 reviewers (T.S. and H.L.) independently to ascertain that if the measure probed that subdomain, any disagreements were resolved on discussion.
This process examined the extent a measure probed P3C, according to the definition and core domains we identified. We also sought to identify the extent to which a measure had been psychometrically validated. The mapping process for the selected 7 measures is presented in Table 2. This table depicts if the measure probes that domain and subdomain and the number of questions in each. The top row details the domains with the subdomains presented underneath.
Table 2.
Item Mapping of the Selected Measures to Core Coordinated Care Domains.
| No. | “I” Statements | My Goals/Outcomes | Care Planning | Transitions | Decision-Making | Information and Communication | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Person-Centred Coordinated Care Domains | Goal setting | Empowerment/activation | Self- Management | Carer Involvement | Care Plan | Case Manager/Key Person | Single Point of Contact | Care Coordination | Continuity of Care | Involvement in Decision- Making | Relational Continuity | Information Gathering/Sharing | Knowledge of Individual/Familiarity | ||
| Within Teams | Across Teams | ||||||||||||||
| 1 | Patient Perceptions of Integrated Care (PPIC) | ✓ (2) | ✓ (11) | ✓ (13) | ✓ (1) | ✓ (1) | ✓ (5) | ✓ (4) | ✓ (3) | ✓ (8) | ✓ (12) | ✓ (2) | ✓ (12) | ✓ (8) | ✓ (12) |
| 2 | Patient Perceived Continuity from Multiple Clinicians (CC-MC) | ✓ (3) | ✓ (14) | ✓ (1) | – | ✓ (6) | ✓ (3) | ✓ (4) | ✓ (7) | ✓ (12) | ✓ (12) | ✓ (12) | ✓ (1) | ✓ (11) | ✓ (6) |
| 3 | Patient Assessment of Chronic Illness Care (PACIC) | ✓ (3) | ✓ (7) | ✓ (3) | – | ✓ (3) | – | – | ✓ (1) | ✓ (1) | ✓ (5) | ✓ (2) | ✓ (2) | ✓ (7) | ✓ (2) |
| 4 | Client Perception of Coordination Questionnaire (CPCQ) | – | ✓ (7) | ✓ (2) | ✓ (3) | – | ✓ (1) | – | ✓ (2) | ✓ (6) | ✓ (5) | ✓ (5) | ✓ (2) | ✓ (1) | ✓ (1) |
| 5 | Long-Term Condition 6 (LTC6) questionnaire | ✓ (1) | ✓ (2) | ✓ (3) | – | – | – | – | – | ✓ (1) | – | ✓ (1) | – | ✓ (2) | – |
| 6 | Patient-Service User Questionnaire—Integrated Care Evaluation Pilots | – | ✓ (5) | ✓ (5) | – | ✓ (3) | – | ✓ (1) | ✓ (4) | ✓ (3) | ✓ (4) | ✓ (3) | ✓ (1) | ✓ (8) | ✓ (5) |
| 7 | North West London Integrated Care Pilot: Patient Survey | ✓ (1) | ✓ (5) | – | – | ✓ (5) | – | – | ✓ (3) | ✓ (3) | – | ✓ (5) | ✓ (2) | ✓ (1) | ✓ (1) |
Codesign Workshops and Measurement Modification
Our codesign workshops selected the LTC6 questionnaire as the most preferred for routine measurement due to its content, short length, utility, and tone. The other measures were rejected due to length and/or because they lacked a person-centred focus.
In response to our codesign workshops, we modified the LTC6 (43). As presented in Table 2, the focus of the questionnaire was on self-management (including goal setting and empowerment/activation). There were 2 free-text questions on how care could be improved and what support is needed to make them feel more confident. Initial development data showed that the LTC6 has been used with 1592 individuals across 29 sites participating in the Quality, Innovation, Productivity and Prevention LTC Work stream in September 2012, with no apparent floor and ceiling effects (unpublished DH report).
The Modified LTC6 (renamed P3CEQ) underwent several iterations based on feedback from the team, other researchers working in P3C, and our wider codesign group. The modifications are described below:
The term “well-being” was added to “health” to questions 1, 3, 4, and 6 to ensure the questionnaire was applicable to both health and social care settings.
As per (i), the terms “treatment” in question 2 and “support” in question 5 were replaced with the term “care.”
The original question 3 on the amount of information received by the person was modified to reflect not only the quantity of the information but also its perceived usefulness.
A comments section was added to each question to allow individuals to expand their views or provide an example in their own words.
Eight new questions were included that probed: “being considered as a ‘whole’ person rather than a disease/condition,” “repeating information that should have been in care records,” “carer involvement”; “a single named person to coordinate the care,” and 4 questions on care planning (covering availability, accessibility, utility, and applicability).
Cognitive Interviewing
We used cognitive interviewing to understand if the modified LTC6 measure was understandable to people with LTCs and MLTCs. We also wanted to ensure that specific terms such as “well-being,” “whole person,” “plan of care,” “joined up care,” “coordinated care,” and “information” were meaningful to people. Positive feedback from the interviews related to the relevance of the questionnaire to P3C, overall language, response codes, length, and the time taken for completion. Minor suggestions were made in relation to layout, formatting, ordering of questions, and language. Major changes included in the reduction in the number of main questions to 11. This was achieved by including a trigger question on care plans as part of the main questions, and 3 further care plan questions that would be applicable only to people who respond positively to the initial care plan question.
The positive feedback from the cognitive interviews and from the ongoing stakeholder engagement in terms of relevance to P3C and understanding of the questions ensured face validity (ie, the measure was investigating a person’s perspective on whether care was experienced as person centred and coordinated). Throughout the 12 iterations of the P3CEQ, items within the revised version were continuously being mapped to the core domains of our model to maintain the relevance of the measure for capturing peoples experiences of P3C and thus ensuring content validity. How the original and modified LTC6 questionnaire compared to the core P3C domains is presented in Table 3.
Table 3.
Comparison of the Original and Modified LTC6 against Core P3C Domains.
| No. | “I” Statements | My Goals/Outcomes | Care Planning | Transitions | Decision-Making | Information and Communication | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Person-Centred Coordinated Care Domains | Goal Setting | Empowerment/Activation | Self- Management | Carer Involvement | Care Plan | Case Manager/Key Person | Single Point of Contact | Care Coordination | Continuity of Care | Involvement in Decision Making | Relational Continuity | Information Gathering/Sharing | Knowledge of Individual/Familiarity | ||
| Within Teams | Across Teams | ||||||||||||||
| 1 | Long-Term Condition 6 (LTC6) questionnaire | ✓ (1) | ✓ (2) | ✓ (3) | – | – | – | – | – | ✓ (1) | – | ✓ (1) | – | ✓ (2) | – |
| 2 | Person-Centred Coordinated Care Experiences Questionnaire (P3CEQ) | ✓ (2) | ✓ (3) | ✓ (4) | ✓ (1) | ✓ (4) | ✓ (1) | – | ✓ (3) | ✓ (3) | ✓ (1) | ✓ (2) | ✓ (3) | ✓ (2) | ✓ (1) |
The P3CEQ
The P3CEQ has 11 questions and is free to use on completing a user agreement form. The P3CEQ has been translated into 5 European languages and is now being used across a range of national and international evaluations of new models of integrated care and self-management (http://www.selfie2020.eu/ and http://www.sustain-eu.org/).
Although the current FKG level of 7.3 is appropriate for most individuals, we recognize that it may not be suitable for people with learning difficulties. Feedback that the measure may not be suitable for completion by people with dementia and cognitive difficulties has also been taken into consideration. To adjust for these difficulties, we are exploring the use of Talking Mats, a pictorial framework that has been validated for this subset of this population (45).
Discussion
The importance of measuring the individual patient’s perspective of P3C is widely acknowledged in practice, research, and policy. In response to repeated calls for the need of patient-reported experience measures, there has been a rapid surge in the development of new measures. Hence, this study was initiated as a review with the aim of identifying a measure that could be used in routine practice to evaluate P3C experience from the perspective of the patient. Despite a large number of measures identified, many failed to cover the core domains of P3C or were too long for routine practice improvement projects. This finding was in line with the findings of the recent relevant systematic reviews (33 –35,37,39). Due to the lack of a suitable measure, we adapted the LTC6 to cover the core domains of P3C in a timely and cost-effective manner using codesign to produce the P3CEQ.
A particular strength of our work is in the ongoing involvement and validation by key stakeholders throughout the process of measure selection and modification. This has ensured that the focus of the work is person centred and is relevant to the end user. Recent reports/articles have confirmed the domains of interest probed by the P3CEQ as key constructs of P3C (23,24). The mapping of the items from the modified questionnaire to these core P3C domains has ensured that the measure is strongly aligned with new models of care. Finally, the use of cognitive interviewing (50) has helped refine the questionnaire, making it more user-friendly. The P3CEQ has completed wide-scale psychometric testing, the results of which will be published imminently.
Conclusion
The P3CEQ is a brief, generic measure that covers core domains of person-centred coordinated care from the perspective of the patient. This measure therefore offers an opportunity for services to develop P3C by acting on the evaluative statements of those receiving care and support. This will only be achieved, however, if the data from it are fed back to practice to drive change, a short easy to administer measure such as the P3CEQ advances progress toward this aim.
Supplemental Material
Supplemental Material, Appendix_1_Px for Codesigning a Measure of Person-Centred Coordinated Care to Capture the Experience of the Patient: The Development of the P3CEQ by Thavapriya Sugavanam, Ben Fosh, James Close, Richard Byng, Jane Horrell, and Helen Lloyd in Journal of Patient Experience
Acknowledgments
The authors would like to thank all the patient and carer advisors and professionals who helped us develop the P3CEQ. The help and support was invaluable.
Author Biographies
Thavapriya Sugavanam is a post-doctoral researcher in Rehabilitation at RRIO, NDORMS, University of Oxford and previously worked as a Research Assistant for PenCLAHRC at Plymouth University where this work was undertaken. She has a PhD and MSc in Physiotherapy from Queen Margaret University, Edinburgh. Her research interests include person-centred care, goal setting, patient experience measures, exercise/physical activity, and rehabilitation with a special interest in systematic reviews.
Ben Fosh is a PenCLAHRC research assistant in the Community and Primary Care Research Group. Ben interests are around quantitative methodology, psychometrics, Bayesian statistics, and bioinformatics. Ben's primary role is to develop feedback software for practitioners and organizations, and conduct quantitative analysis of evaluative work examining the impact of person-centred coordinated care on patients with multiple long-term conditions. Ben has a master’s degree from the University of York in Cognitive Neuroscience.
James Close is a SW CLAHRC research fellow in the Community and Primary Care Research Group at the University of Plymouth. With a background spanning genome biology, hematology, and psychiatry. His current research interests are focused on “system levers” for person-centred care, and how organizations can efficiently reorganize for the challenges of 21st entry medicine. He is supporting the person-centred coordinated care (P3C) program of work via evaluation of initiatives such as “SPQS” (a prototype for de-incentivization of QOF), the development of our portal for patient-reported measures at p3c.org.uk, and supporting the development of our own suite of measures for P3C.
Jane Horrell is a research assistant in the Community and Primary Care Research Group, Plymouth University, Peninsula Schools of Medicine and Dentistry. She has worked on a number of projects and evaluations of person-centred coordinated care (P3C), including the development of both the P3CEQ and Organisational Change Tool (P3C-OCT).
Helen Lloyd is a senior researcher for PenCLAHRC and an experienced mixed methodologist with a training in medical anthropology. Helen uses participatory action and qualitative and quantitative methodologies to create health and social improvements. Her research has explored how identity influences health behavior and the experiences of illness and the outcomes of care. A central theme of her work with PenCLAHRC focuses on complex interventions and patient experience where she is leading a program of research on person-centred coordinated care (P3C). This work aims to advance theory, measurement, and service improvement of P3C for older people with complex health and social needs.
Appendix A
List of measures considered for inclusion in stage 1
Adapted Picker Institute Cancer Survey
Ambulatory Care Experiences Survey (ACES)
Brief 5 A’s Patient Survey
Canadian Survey of Experiences With Primary Health Care Questionnaire
Care Evaluation Scale for End-of-Life Care (CES)
Care Transitions Measure (CTM)
CCAENA Questionnaire—Continuity of Care Between Care Levels
Client-Centred Care Questionnaire
Client Perception of Coordination Questionnaire (CPCQ)
Client Satisfaction Questionnaire (CSQ)
Coleman Measures of Care Coordination
Components of Primary Care Index (CPCI)
Consultation and Relational Empathy (CARE) Measure
Consultation Care Measure
Consumer Assessment of Healthcare Providers and Systems (CAHPS)
Family Satisfaction in the Intensive Care Unit (FS-ICU 24)
Family-Centred Care Self-Assessment Tool
Follow-Up Care Delivery
General Practice Assessment Questionnaire (GPAQ-R2)
Giving Youth a Voice (GYV)
Health Care Empowerment Questionnaire
Health Tracking Household Survey
Icelandic Perceived Involvement in Care Scale (I-PICS)
Improving Chronic Illness Care Evaluation (ICICE)
Interpersonal Processes of Care Survey
Korean Primary Care Assessment Tool (KPCAT)
Long-Term Condition (LTC6) Questionnaire
Measure in development—Nuffield Trust
Measure of Processes of Care (MPOC-28)
Medical Home Index (MHI)
National Survey for Children’s Health (NSCH)
National Survey of Children With Special Health Care Needs (CSHCN)
National Voices I Statements
NHS in Patient Adult Survey
North West London Integrated Care Pilot: Patient Survey
Oncology Patients’ Perceptions of the Quality of Nursing Care Scale (OPPQNCS)
Outcomes Assessment for People With Long-Term Neurological Conditions
OxPie
Parents’ Perceptions of Primary Care (P3C)
Patient Assessment of Chronic Illness Care (PACIC)
Patient Perceived Continuity of Care From Multiple Providers (CC-MC)
Patient Perception of Continuity Instrument (PC)
Patient Perception of Patient-Centeredness (PPPC)
Patient Perceptions of Care (PPOC)
Patient Assessment of Care for Chronic Conditions (PACIC)
Patient Assessment of Care for Chronic Conditions (PACIC) +
Patient Perceptions of Integrated Care Survey (PPIC)
Patient-Service User Questionnaire—Integrated Care Evaluation Pilots
Personal Identity Threat
Person-Centred Climate Questionnaire
Person-Centred Inpatient Scale
Person-Directed Care (PDC)
Physician Office Quality of Care Monitor (QCM)
Picker Patient Experience (PPE-15)
PREPARED Survey
Primary Care Assessment Survey (PCAS)
Primary Care Assessment Tool (PCAT)
Primary Care Multimorbidity Hassles for Veterans With Chronic Illnesses
Primary Care Questionnaire for Complex Pediatric Patients
Primary Care Satisfaction Survey for Women (PCSSW)
Promoting Healthy Development Survey PLUS—(PHDS-PLUS)
Quality From Patient’s Perspective (QPP) Questionnaire—Long
Quality From Patient’s Perspective (QPP) Questionnaire—Short
Relational and Management Continuity Survey in Patients With Multiple Long-Term Conditions
Resources and Support for Self-Management (RSSM)
Responsiveness of Primary Care Services
R Outcomes
Schizophrenia Quality Indicators for Integrated Care
Symphony Patient Experience Questionnaire
The Satisfaction Profile (SAT-P)
The Outcomes and Experiences Questionnaire
Authors’ Note: The P3CEQ is free to use, however, as we would like to monitor its use, we are requesting users to complete a user agreement form before use. Therefore, we plan on not attaching the measure as an appendix with the article if selected for publication. However, for purposes of peer review, we have attached the P3CEQ as a supplementary file. Patient and professional representatives were involved in the development of P3CEQ as part of stakeholder engagement workshops and cognitive interviews. They were involved as “experts” in this area and not as research participants. Our expert patients were part of an established group who work as part of the NIHR CLAHRC South West Peninsula (PenCLAHRC). Verbal consent was obtained for audio recording the cognitive interviews and this is mentioned in the manuscript under the cognitive interviews section. All authors consent for this manuscript to be submitted to this journal for publication.
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula at the Royal Devon and Exeter NHS Foundation Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health. Thavapriya Sugavanam’s current role is funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care Oxford at Oxford Health NHS Foundation Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
ORCID iD: Helen Lloyd 
http://orcid.org/0000-0002-2916-1874
Supplemental Material: Supplemental material for this article is available online.
References
- 1. Lloyd H, Jenkinson C, Hadi M, Gibbons E, Fitzpatrick R. Patient reports of the outcomes of treatment: a structured review of approaches. Health Qual Life Outcomes. 2014;12:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Wolf JA, Niederhauser V, Marshburn D, LaVela SL. Defining patient experience. Patient Exp J. 2014;1:7–19. [Google Scholar]
- 3. World Health Organisation. WHO Global Strategy on Integrated People-Centred Health Services 2016-2026: Executive Summary. Geneva, Switzerland: WHO; 2015. http://www.who.int/servicedeliverysafety/areas/people-centred-care/global-strategy/en/. [Google Scholar]
- 4. National Collaboration on Integrated Care and Support. Integrated Care and Support: Our Shared Commitment. England, UK: NHS England; 2013. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/198748/DEFINITIVE_FINAL_VERSION_Integrated_Care_and_Support_-_Our_Shared_Commitment_2013-05-13.pdf. [Google Scholar]
- 5. The King’s Fund. Implementing the NHS Five Year Forward View: Aligning Policies with the Plan. London, UK: NHS England; 2015. http://www.kingsfund.org.uk/sites/files/kf/field/field_publication_file/implementing-the-nhs-five-year-forward-view-kingsfund-feb15.pdf. [Google Scholar]
- 6. Kodner DL. All together now: a conceptual exploration of integrated care. Healthc Q. 2009;13 Spec No:6–15. [DOI] [PubMed] [Google Scholar]
- 7. Bodenheimer T. Coordinating care--a perilous journey through the health care system. N Engl J Med. 2008;358:1064–71. [DOI] [PubMed] [Google Scholar]
- 8. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297:831–41. [DOI] [PubMed] [Google Scholar]
- 9. National Voices. Integrated Care: What do Patients, Service Users and Carers Want? London, UK: National Voices; 2013. http://www.nationalvoices.org.uk/sites/www.nationalvoices.org.uk/files/what_patients_want_from_integration_national_voices_paper.pdf. [Google Scholar]
- 10. Ryan J, Doty MM, Shah T, Schneider EC. How high-need patients experience health care in the United States: findings from the 2016 commonwealth fund survey of high-need patients. Issue Brief (Commonw Fund). 2016;43:1–20. [PubMed] [Google Scholar]
- 11. Curry N, Goodwin N, Naylor C. Practice-Based Commissioning. Reinvigorate, Replace or Abandon? London, UK: Kings Fund; 2008. [Google Scholar]
- 12. Sheaff R, Halliday J, Øvretveit J, Byng R, Exworthy M, Peckham S, et al. Integration and continuity of primary care: polyclinics and alternatives, an organisational analysis. Health Serv Deliv Res. 2015;3: 1–105. [PubMed] [Google Scholar]
- 13. National Voices. A Narrative for Person Centred Coordinated Care. London, UK: National Voices; 2013. [Google Scholar]
- 14. Physicians RCo. Putting the Pieces Together: Removing the Barriers to Excellent Patient Care. London, UK: Royal College of Physicians; 2015. [Google Scholar]
- 15. Humphries R. Health and social care for older people: progress, problems and priorities. Qual Ageing Older Adults. 2015;16:27–31. [Google Scholar]
- 16. Schoen C, Osborn R, Squires D, Doty M, Pierson R, Applebaum S. New 2011 survey of patients with complex care needs in eleven countries finds that care is often poorly coordinated. Health Aff (Millwood). 2011;30:2437–48. [DOI] [PubMed] [Google Scholar]
- 17. Oliver D, Foot C, Humphries R. Making Our Health and Care Systems Fit for an Ageing Population. London, UK: The Kings Fund; 2014. [Google Scholar]
- 18. Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380:37–43. [DOI] [PubMed] [Google Scholar]
- 19. Goodwin N, Smith J, Anna D, Claire P, Rebecca R, Alisha D, et al. Integrated Care for Patients and Populations: Improving Outcomes by Working Together. London, UK: The King’s Fund and the Nuffield Trust; 2012. [Google Scholar]
- 20. Kupeli N, Leavey G, Harrington J, Lord K, King M, Nazareth I, et al. What are the barriers to care integration for those at the advanced stages of dementia living in care homes in the UK? health care professional perspective. Dementia (London). 2016:pii: 1471301216636302 [DOI] [PMC free article] [PubMed]
- 21. Schoen C, Osborn R, How SK, Doty MM, Peugh J. In chronic condition: experiences of patients with complex health care needs, in eight countries, 2008. Health Aff (Millwood). 2009;28: w1–w16. [DOI] [PubMed] [Google Scholar]
- 22. Sinnott C, Mc Hugh S, Browne J, Bradley C. GPs’ perspectives on the management of patients with multimorbidity: systematic review and synthesis of qualitative research. BMJ Open. 2013;3:e003610. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. American Geriatrics Society Expert Panel on Person-Centered Care. Person-centered care: a definition and essential elements. J Am Geriatr Soc. 2016;64:15–8. [DOI] [PubMed] [Google Scholar]
- 24. Harding E, Wait S, Scrutton J. The State of Play in Person-Centred Care. London, UK: The Health Policy Partnership; 2015. http://www.healthpolicypartnership.com/wp-content/uploads/International-Environment-Scan-on-Person-Centred-Care-18th-Dec-FINAL.pdf. [Google Scholar]
- 25. Lloyd H, Wheat H, Horrell J, Close J. A Guide for Commissioners on the Use Metrics for Person Centred Coordinated Care for People With Long Term Conditions, Multiple Long Term Conditions and Those at the End of their Life. London, UK: National Health Service England; 2017. [Google Scholar]
- 26. Ekman I, Swedberg K, Taft C, Lindseth A, Norberg A, Brink E, et al. Person-centered care--ready for prime time. Eur J Cardiovasc Nurs. 2011;10:248–51. [DOI] [PubMed] [Google Scholar]
- 27. McCormack B, Dewing J. The implementation of a model of person-centred practice in older person settings. Ireland, Dublin: final report, office of the nursing services director, health services executive; 2010. http://www.science.ulster.ac.uk/inhr/public/pdf/HSE_IrelandPCPProgrammeFINAL.pdf.
- 28. Horrell JL, Sugavanam T, Close J, Byng R. Creating and facilitating change for person centred coordinated care (P3C): the development of the organisational change tool (P3C-OCT). Health Expect. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Coulter A, Roberts S, Dixon A. Delivering Better Services for People with Long-Term Conditions. London, UK: The King’s Fund; 2013. [Google Scholar]
- 30. Kathryn M. Care Coordination Measures Atlas Update. Rockville, MD: Quality. AFHRA; 2014. [Google Scholar]
- 31. Lloyd Helen M, Pearson M, Wheat H, Asthana S, Horrell J, Valderas J, et al. Collaborative action for person-centred coordinated care (P3C). Health Res Policy Syst. 2017;15:98. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an emerging approach to evidence synthesis in health technology assessment. Int J Technol Assess Health Care. 2014;30:20–7. [DOI] [PubMed] [Google Scholar]
- 33. Strandberg-Larsen M, Krasnik A. Measurement of integrated healthcare delivery: a systematic review of methods and future research directions. Int J Integr Care. 2009;9:e01. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Lyngsø AM, Godtfredsen NS, Høst D, Frølich A. Instruments to assess integrated care: a systematic review. Int J Integr Care. 2014;14:e027. [PMC free article] [PubMed] [Google Scholar]
- 35. Uijen AA, Heinst CW, Schellevis FG, van den Bosch WJ, van de Laar FA, Terwee CB, et al. Measurement properties of questionnaires measuring continuity of care: a systematic review. PLoS One. 2012;7:e42256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Saultz JW. Defining and measuring interpersonal continuity of care. Ann Fam Med. 2003;1:134–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Hudon C, Fortin M, Haggerty JL, Lambert M, Poitras ME. Measuring patients’ perceptions of patient-centered care: a systematic review of tools for family medicine. Ann Fam Med. 2011;9:155–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Edvardsson D, Innes A. Measuring person-centered care: a critical comparative review of published tools. Gerontologist. 2010;50:834–46. [DOI] [PubMed] [Google Scholar]
- 39. Schultz EM, Pineda N, Lonhart J, Davies SM, McDonald KM. A systematic review of the care coordination measurement landscape. BMC Health Serv Res. 2013;13:119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Helping Measure Person-Centred Care. London, UK: The Health Foundation; 2014. http://www.health.org.uk/sites/default/files/HelpingMeasurePersonCentredCare.pdf. [Google Scholar]
- 41. King J, Gibbons E, Graham C, Walsh J. Developing Measures of People’s Self-Reported Experiences of Integrated Care. West Way, Oxford: Picker Institute Europe and University of Oxford; 2013. http://www.pickereurope.org/wp-content/uploads/2014/10/Developing-measures-of-IC-report_final_SMALL.pdf. [Google Scholar]
- 42. Care Coordination Measures Atlas Update. Rockville, MD: Agency for Healthcare Research and Quality; 2014. http://www.ahrq.gov/professionals/prevention-chronic-care/improve/coordination/atlas2014/index.html. [Google Scholar]
- 43. QIPP LTC Year of Care Funding Model Project Team. QIPP Long Term Conditions—Supporting the Local Implementation of the Year of Care Funding Model for People With Long-Term Conditions. London, UK: Department of Health; 2012. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215060/dh_133652.pdf. [Google Scholar]
- 44. Ernst & Young LLP. National Evaluation of the Department of Health’s Integrated Care Pilots. Cambridge, UK: RAND Europe; 2012. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/215103/dh_133127.pdf. [PMC free article] [PubMed] [Google Scholar]
- 45. Talking Mats. 2015. http://www.talkingmats.com/.
- 46. Singer SJ, Friedberg MW, Kiang MV, Dunn T, Kuhn DM. Development and preliminary validation of the Patient Perceptions of Integrated Care survey. Med Care Res And Rev: MCRR. 2013;70:143–64. PubMed PMID: 23161612. [DOI] [PubMed] [Google Scholar]
- 47. Haggerty JL, Roberge D, Freeman GK, Beaulieu C, Bréton M. Validation of a generic measure of continuity of care: when patients encounter several clinicians. Ann Of Fam Med. 2012;10:443–51. PubMed PMID: 22966108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005;43:436–44. PubMed PMID: 15838407. [DOI] [PubMed] [Google Scholar]
- 49. Mastellos N, Gunn L, Harris M, Majeed A, Car J, Pappas Y. Assessing patients’ experience of integrated care: a survey of patient views in the North West London Integrated Care Pilot. Int J Of Integr Care. 2014;14:e015–e. PubMed PMID: 24987321. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. http://iom.nationalacademies.org/Reports/2001/Crossing-the-Quality-Chasm-A-New-Health-System-for-the-21st-Century.aspx. [PubMed] [Google Scholar]
- 51. Picker Institute. Patient Experience Surveys: The Rationale. West Way, Oxford: Picker Institute Europe; 2008. [Google Scholar]
- 52. Staniszewska S, Boardman F, Gunn L, Roberts J, Clay D, Seers K, et al. The Warwick patient experiences framework: patient-based evidence in clinical guidelines. Int J Qual Health Care. 2014;26:151–7. [DOI] [PubMed] [Google Scholar]
- 53. National Institute for Health and Care Excellence. Patient Experience in Adult NHS Services: Improving the Experience of Care for People using Adult NHS Services. London, UK: Royal College of Physicians; 2012. nice.org.uk/guidance/qs15. [PubMed] [Google Scholar]
- 54. Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA: SAGE Publications; 2005. [Google Scholar]
- 55. Caspar RA, Lessler JT, Willis GB. Cognitive interviewing: a “How To” guide: Research Triangle Institute, Short course. Paper presented at: Meeting of the American Statistical Association, 1999. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Material, Appendix_1_Px for Codesigning a Measure of Person-Centred Coordinated Care to Capture the Experience of the Patient: The Development of the P3CEQ by Thavapriya Sugavanam, Ben Fosh, James Close, Richard Byng, Jane Horrell, and Helen Lloyd in Journal of Patient Experience