Using Electronic Health Record Data to Measure Care Quality for Individuals with Multiple Chronic Medical Conditions

Abstract

OBJECTIVES:

To inform the development of a data-driven measure of quality care for individuals with multiple chronic conditions (MCCs) derived from an electronic health record (EHR).

DESIGN:

Qualitative study using focus groups, interactive webinars, and a modified Delphi process.

SETTING:

Research department within an integrated delivery system.

PARTICIPANTS:

The webinars and Delphi process included 17 experts in clinical geriatrics and primary care, health policy, quality assessment, health technology, and health system operations. The focus group included 10 individuals aged 70–87 with three to six chronic conditions selected from a random sample of individuals aged 65 and older with three or more chronic medical conditions.

MEASUREMENTS:

Through webinars and the focus group, input was solicited on constructs representing high-quality care for individuals with MCCs. A working list was created of potential measures representing these constructs. Using a modified Delphi process, experts rated the importance of each possible measure and the feasibility of implementing each measure using EHR data.

RESULTS:

High-priority constructs reflected processes rather than outcomes of care. High-priority constructs that were potentially feasible to measure included assessing physical function, depression screening, medication reconciliation, annual influenza vaccination, outreach after hospital admission, and documented advance directives. High-priority constructs that were less feasible to measure included goal setting and shared decision-making, identifying drug–drug interactions, assessing social support, timely communication with patients, and other aspects of good customer service. Lower-priority domains included pain assessment, continuity of care, and overuse of screening or laboratory testing.

CONCLUSION:

High-quality MCC care should be measured using meaningful process measures rather than outcomes. Although some care processes are currently extractable from electronic data, capturing others will require adapting and applying technology to encourage holistic, person-centered care.

Characteristics of high-quality care of individuals with multiple chronic conditions (MCCs) are easy to understand and difficult to measure. Care should be safe, person-centered, individualized, goal-directed, affordable, and facilitated through effective communication and care coordination.^1–7 In qualitative studies, individuals with MCCs report a desire for timely access to care, high-quality patient–clinician relationships, and minimal treatment and financial burdens.^8,9

Despite agreement on these principles, there are no measures of care quality designed specifically for individuals or populations with MCCs.^4,10 There have been multiple calls to develop and implement such metrics, and one of the main priorities of the U.S. Department of Health and Human Services initiative on MCC is to “identify, develop, endorse, and use key quality metrics, in the form of performance measures, to promote best practices in the general care of individuals with MCC.”¹¹

Creating valid MCC quality measures is difficult because optimal MCC care is described in terms of care processes rather than easily quantifiable outcomes.^1,4,12–14 Current quality assessments have focused on disease-specific quality measures that are easily quantifiable—and increasingly use electronic data—but for individuals with MCCs, disease-specific measures can overemphasize one disease over another and result in inadequate attention to some conditions or harm from duplicative or conflicting treatments. In a world in which quality assessment increasingly depends on available electronic data, it is urgent to identify measures of MCC care quality that can be extracted from electronic health records (EHRs) and used to encourage person-centered, appropriate MCC care delivery.

To establish a foundation for developing a data-driven measure of MCC care quality, content experts and individuals with MCCs were engaged to identify constructs reflecting high-quality MCC care that could be measured using available electronic data. The goal was to identify and refine a list of variables that could be extracted from the EHR and ultimately combined into a preliminary composite measure of MCC care quality. Once developed, such a measure will permit evaluation of interventions to improve MCC care, guide systematic approaches to MCC care delivery, and focus attention on MCC care as a distinct, valuable entity.

METHODS

Study Design and Setting

This qualitative investigation was conducted in Kaiser Permanente Colorado (KPCO), an integrated not-for-profit healthcare delivery system with a well-developed EHR and associated data warehouse. To identify constructs reflecting high-quality MCC care, interactive webinars were conducted with a panel of content experts in July 2014 and a separate focus group with individuals with MCCs in September 2014. The content experts were then engaged in a modified Delphi process to prioritize endorsed MCC care constructs and potential associated measures. Finally, in conjunction with KPCO data experts, the feasibility of extracting data from the EHR on each of the recommended potential measures was assessed.

Participants

Content experts participating in webinars represented a range of expertise in MCC care, research, and quality measurement and were recruited through snowball sampling within the Agency for Healthcare Research and Quality MCC Research Network and the International Research Community on Multimorbidity.^15,16 Areas of expertise included clinical geriatrics and primary care, health policy, MCC research, quality assessment, health technology, and health system operations (see Acknowledgments). The focus group included 10 individuals aged 70–87 with three to six chronic conditions. They were selected from a random sample of individuals aged 65 and older with three or more of 10 common chronic medical conditions (hypertension, congestive heart failure, hyperlipidemia, diabetes mellitus, coronary artery disease, chronic obstructive pulmonary disease, osteoarthritis, osteoporosis, depression, obesity.) Individuals were purposively sampled to represent men and women along a spectrum of age and morbidity burden. Although multimorbidity is not limited to individuals aged 65 and older, this population was focused on because of its high prevalence of MCC.

Data Collection

Each of four interactive webinars included two to six experts. Webinars started with an outline of the project, including the ultimate goal of developing an electronic measure of MCC care quality. Open-ended input on constructs reflecting high-quality MCC care was then solicited. To prompt discussion, possible constructs were suggested, including advance care planning, care transitions, and medication management. Participants were then asked to recommend variables reflecting constructs that could potentially be extracted from electronic data. Prompts included examples such as patient-reported functional status and pharmacy data.

To complement expert opinion with patient perspectives, one focus group was conducted with 10 individuals with MCCs. Because there is already substantial literature on patient perceptions of high-quality care, exploration was limited to one focus group to corroborate existing evidence in the context of the experts’ recommendations.¹⁷ Participants were provided background information on how and why quality of care is measured and the goal for the current project. Using open-ended questions, participants were asked to describe what they considered to be high-quality medical care for individuals with MCCs. Examples included, “What do you think of when you think of high-quality care for persons with MCC?” Prompts included, “For example, do you think it is important to receive an annual flu vaccine? What about having a doctor ask about mood or pain?” Care processes that might be particularly relevant for individuals with MCC, such as shared decision-making, were also asked about. Finally participant opinions on the importance of constructs that the expert panels raised during the webinars were asked about.

Data Analysis

Two authors (EB, DM) separately reviewed and coded the focus group and webinar transcripts. An inductive approach was used to identify constructs reflecting high-quality MCC care that experts or patients mentioned, and constructs were grouped into overarching domains.¹⁸ Any measures associated with specific constructs were also identified from the webinar transcripts. Finally, if no metrics were suggested for constructs, the research team proposed example measures. For example, the expert panel might suggest a construct of mood assessment and a metric of depression screening using the Patient Health Questionnaire (PHQ)-2 measure, or focus group participants might suggest a construct of care continuity for which a known measure of continuity of care was proposed.

The expert panel members then prioritized and revised these constructs and possible metrics using a modified Delphi process.¹⁹ The expert panel members who participated in the webinars completed a survey that listed constructs and possible metrics to measure each construct. Suggestions from expert panel members and patients were incorporated equally into the survey. For each construct and possible metric, the experts were asked to assign a high, medium, or low priority to the construct and metric; to explain the rationale for their decision; to comment on the feasibility of extracting such information from an EHR; and, if relevant, to recommend a measurement interval (e.g., annual assessment). Panel members were asked to emphasize measures relevant to the broad MCC population rather than specific subpopulations.

A prioritized list of constructs reflecting high-quality MCC care and possible metrics was generated from this modified Delphi process. The list was then reviewed with KPCO data experts on the research team who had a good understanding of KPCO’s EHR to confirm the feasibility of measuring selected constructs, and a matrix of potential quality metrics was generated based on prioritization and measurement feasibility.

The KPCO institutional review board reviewed and approved the study.

RESULTS

Overarching domains from discussions with content experts reflected the importance of patient-centered decision-making, understanding a patient’s healthcare context, ensuring safety, attending to relevant condition-specific care needs, and respecting the patient experience of care. Patients echoed these general domains and also focused on logistics of system access, communication, and efficient care delivery.

Table 1 illustrates overarching domains and specific constructs that webinar participants and the focus group proposed. The focus group and experts endorsed each overarching domain, reflecting a shared recognition of patient-focused care and the multidimensional nature of MCC care delivery, but within each domain, experts focused more on specific constructs, potential measures, and discrete processes of care (e.g., medication reconciliation), whereas patients focused on constructs that are less easily measurable using available tools but reflected their definition of high-quality MCC care (e.g., customized communication, easy access to specific providers).

Table 1.

Domains, Constructs, and Measures of High-Quality Multiple Chronic Conditions Care from Webinars and Focus Group

Domain	Construct	Example Measure or Description of Documentation	Stakeholdersa
Deliver contextually relevant and compassionate care	Assess caregiver support at home	Needs assessment such as Medicare health assessment	Expert panel
	Shared decision-making	Documentation of shared decision-making	Both
	Advance care planning	Documentation of advance directive on record	Expert panel
	Address social determinants	Needs assessment such as Medicare health assessment	Expert panel
	Understand patients’ perceptions of themselves with MCC	Development needed	Both
Respect patients	Care guided by patient goals and priorities	Documentation of patient goals and outcomes	Both
		Attainment of patient goals
	Address patient questions	Visit length	Patients
	Consider patient finances	Redundant care, return visits	Patients
	Patient experience of care	Patient satisfaction survey	Both
Avoid harm	Care coordination	Phone contact or visit after care transition	Both
	Readmissions	Length of stay, Acuity of admission, Charlson comorbidity index, and Emergency department use score (van Walraven, CAMJ, 2010)	Both
	Functional assessment	Activities of daily living	Both
	Medication reconciliation	Periodic medication reconciliation	Expert panel
	Use of potentially inappropriate medications or doses	Beers or Screening Tool of Older Persons’ potentially inappropriate Prescriptions criteria (Gallagher, Age and Aging 2008)	Expert panel
		Drug-drug interactions
		Diagnoses suggestive of inappropriate medications or doses e.g., hypoglycemia	Expert panel
Provide high-quality condition-specific care	Selected disease-specific metrics	Physician Quality Reporting System composite (https://www.cms.gov)	Expert panel
	Depression assessment	Patient Health Questionnaire-2 or 9 (Kroenke, Medical Care, 2003)	Both
	Anticipatory management of clinical needs	Reminders for needed laboratory studies	Patients
		Annual flu vaccine, pneumonia vaccine	Both
	Pain assessment	0—10 pain scale	Expert panel
	BMI assessment	Systematic documentation of body mass index	Expert panel
Minimize logistical barriers to care	Timeliness of care	Time to return call, time to appointment	Patients
	Access to care	Ability to reach or see desired provider	Patients
Optimize patient–clinician communication	Informational continuity between providers	Quantify EHR use by care team	Patients
	Customizing communication to patient preference	Written communication of treatment plan	Patients
	Interpersonal continuity of care	Continuity of Care Index (Bice and Boxerman, Medical Care, 1977)	Both
	Effective team-based care	Measure team continuity	Expert panel
	Accuracy of EHR documentation	Patient review of EHR	Patients
Optimize efficiency	Wasted resources	Diagnoses or procedures suggestive of overuse or redundant care: e.g., inappropriate cancer screening or duplicate lab tests	Both
	Cost of care	Total cost of care	Patients
	Billing inefficiencies	Patient satisfaction survey	Patients

^aExpert panel, patients in focus group, or both.

EHR = electronic health record.

The matrix in Figure 1 summarizes the modified Delphi process of prioritizing constructs and potential metrics and estimating the feasibility of extracting potential measures from the EHR. High-priority domains with higher estimated measurement feasibility included assessment of physical function, depression screening, medication reconciliation, annual influenza vaccination, systematic outreach after hospital admission, and assessment of the presence of advance directives. Delphi panelists noted that medication reconciliation (especially after care transitions) was critical in preventing adverse outcomes.

Figure 1.

Modified Delphi process findings regarding salience and feasibility of measuring specific multiple chronic conditions quality domains.

^aRequires specific denominator population

The focus group and the Delphi panel highly valued collaborative goal setting and shared decision-making. One panelist described these constructs as the “holy grail” of MCC care. However, these constructs along with those of assessing social support, common drug–drug interactions, and timely and tailored communication with patients had lower estimated measurement feasibility. The construct of assessing social support was also considered very important; however, the panel did not feel that the metric available in some EHRs of “whether someone lives alone” was an adequate proxy for support.

Several constructs were assigned relatively lower clinical or measurement priority by the panel. Although technically feasible to measure, the panel concluded that telephone and e-mail response times would vary based on content and target respondent and therefore were a lower priority. Assessing written communication of treatment plans (such as printing after-visit instructions) was considered feasible, but measuring adherence to patient communication preferences was less so. Although quantifiable in the EHR, measures of pneumonia vaccination, pain assessment, BMI measurement, and anxiety screening were ranked by the Delphi panel as having lesser importance as potential measures of care quality. The panel noted insufficient evidence in the literature for anxiety screening.

The Delphi panel and the focus group both felt that inappropriate and wasteful care were potentially important indicators of care quality, but it was thought that identifying measurement intervals for wasteful or duplicative care would be difficult because there is a lack of consensus regarding specific services that represent low value care. Likewise measures of inappropriate care (such as inappropriate cancer screening) may not be broadly applicable to individuals with MCCs and often reflect individualized decision-making. One participant commented that it was not clear if there is enough screening overuse in this population to rise to the level of a quality measure. Finally, panel members noted that while some constructs represented important aspects of MCC care (such as systematic pain assessment and BMI measurement), assessing these metrics as measures of care quality may only be relevant in certain clinical situations.

DISCUSSION

Potential MCC quality metrics identified in this study included a range of care processes reflecting holistic, person-centered care in the context of multiple competing demands for patients and clinicians. The purpose of developing and implementing quality metrics for persons with MCCs is not to attain high scores but to promote optimal care delivery.

Many of the overarching domains and associated constructs elicited from participants reflect established recommendations for MCC care. National Quality Forum (NQF) recommendations for individuals with MCCs include process measures with specific attention to shared decision-making and the National Quality Strategy, which targets the broader population and focuses on safe, contextual, coordinated care.^4,20 Selected broadly applicable measures under the Physician Quality Reporting System include influenza vaccination, depression screening, and pain assessment.²¹ The current investigation advances these and other recommendations by identifying constructs and associated metrics that may be extractable from electronic medical data and applicable across the MCC population.

Metrics that are easily quantifiable and have evidence-based criterion standards of care and unambiguous outcomes are usually disease specific.^22,23 Exclusively targeting such metrics risks inefficient care and unintended consequences for individuals with MCCs because they are more vulnerable to interacting effects of symptoms and treatments that reaching these goals causes.²⁴ To advance beyond disease-specific measures, the capacity must be developed to document, extract, and quantify processes of care as well as outcomes of care. EHR documentation will need to capture these processes in extractable fields without imposing additional burden on clinicians at the point of care.

Several care processes recommended as possible quality metrics reflect the difficulty of measuring the nuance of person-centered care delivery. Although some important constructs (e.g., depression screening, presence of advance directives) were considered feasible to measure, others (e.g., goal setting, shared decision-making) were deemed more difficult. Other recommended metrics (e.g., visit length, call response time) are technically feasible to extract, but experts questioned their application across a range of clinical scenarios.²⁵ A multifaceted approach highlighting important care constructs coupled with national incentives such as value-based Medicare payment structures may help harness data-driven quality measurement to guide and improve MCC care. Several of the measures considered feasible and salient are already included as quality measures under the Medicare EHR incentive programs.

There is substantial interest in incorporating patient-reported outcomes (PROs) as part of quality assessment.^4,26–29 The expert panel distinguished between the content of PRO data and the process of collecting such information. They felt that, for the broad MCC population, high-quality care should reflect the process of collecting and acting on important PROs rather than the absolute level of the PRO measure. For example, high-quality care entails systematic assessment of functional status but does not require that all individuals achieve a certain level of functioning. Current NQF recommendations for PRO performance measures include collection with minimal patient burden and real-time measurement and reporting in the EHR.²⁹

There are several limitations to this investigation. First, the results reflect a consensus-based method (expert opinion) combined with available evidence to develop a foundational quality measure—a method appropriate for a field with an emerging evidence base.^22,30 One goal for expanding on this foundational work is to develop a composite measure or a strategic menu of individual measures that paint a picture of comprehensive high-quality MCC care. Such a composite or list would need to be validated against outcomes important to patients, such as satisfaction with care and care received that is concordant with patient wishes or priorities. Second, only expert panel members participated in the modified Delphi process. Patients may have prioritized potential measures differently but would have been less able to comment on measurement feasibility. Third, some recommended constructs of high-quality MCC care (e.g., medication reconciliation, advance care plans) are already considered standard care, but it is not clear to what extent these standards are being met for individuals with MCCs, and if so, for whom and in which settings. Fourth, input from a 10-person focus group in an integrated delivery setting could limit generalizability. Finally, constructs that require denominated sub-populations to measure were not focused on. For example, disease-specific measures are necessary for many individuals with MCCs but not sufficient to encourage optimal care for the broad MCC population.

CONCLUSION

Although some processes reflecting high-quality MCC care are currently quantifiable using electronic data, capturing others will require adapting clinical workflows and EHR technology to assess person-centered goal setting and other “holy grails” of MCC care. Once developed, MCC quality measures can promote person-centered care delivery and serve as outcomes for interventions improve MCC care.