Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2022 May 9;29(7):1233–1243. doi: 10.1093/jamia/ocac059

Challenges and opportunities for advancing patient-centered clinical decision support: findings from a horizon scan

Prashila Dullabh 1,, Shana F Sandberg 2,1, Krysta Heaney-Huls 3, Lauren S Hovey 4, David F Lobach 5, Aziz Boxwala 6, Priyanka J Desai 7, Elise Berliner 8,2, Chris Dymek 9, Michael I Harrison 10, James Swiger 11, Dean F Sittig 12
PMCID: PMC9196686  PMID: 35534996

Abstract

Objective

We conducted a horizon scan to (1) identify challenges in patient-centered clinical decision support (PC CDS) and (2) identify future directions for PC CDS.

Materials and Methods

We engaged a technical expert panel, conducted a scoping literature review, and interviewed key informants. We qualitatively analyzed literature and interview transcripts, mapping findings to the 4 phases for translating evidence into PC CDS interventions (Prioritizing, Authoring, Implementing, and Measuring) and to external factors.

Results

We identified 12 challenges for PC CDS development. Lack of patient input was identified as a critical challenge. The key informants noted that patient input is critical to prioritizing topics for PC CDS and to ensuring that CDS aligns with patients’ routine behaviors. Lack of patient-centered terminology standards was viewed as a challenge in authoring PC CDS. We found a dearth of CDS studies that measured clinical outcomes, creating significant gaps in our understanding of PC CDS’ impact. Across all phases of CDS development, there is a lack of patient and provider trust and limited attention to patients’ and providers’ concerns.

Discussion

These challenges suggest opportunities for advancing PC CDS. There are opportunities to develop industry-wide practices and standards to increase transparency, standardize terminologies, and incorporate patient input. There is also opportunity to engage patients throughout the PC CDS research process to ensure that outcome measures are relevant to their needs.

Conclusion

Addressing these challenges and embracing these opportunities will help realize the promise of PC CDS—placing patients at the center of the healthcare system.

INTRODUCTION

A goal of the Affordable Care Act (ACA) was to transition to a healthcare system that is focused on value-based, whole-person care that prioritizes patient outcomes and the rapid dissemination of learnings from patient-centered outcomes research (PCOR).1 The 2019 PCOR Trust Fund reauthorization provides additional opportunities to support patient-centered care.2 The transition to patient-centered care will likely require improved accessibility to high-quality evidence in the form of clinical guidelines and the dissemination of PCOR findings to guide patient and clinician decision-making.

Traditionally, clinical decision support (CDS) focused on assisting clinicians at the point-of-care by delivering diagnostic and treatment guidance based on clinical guidelines.3 However, the shift towards patient-centered care has increased interest in and a need for evidence-based CDS that directly engages patients and incorporates patient-specific data. At a minimum, CDS that is patient-centered incorporates outcomes and measures that are meaningful to patients. Beyond this standard, patient-centered CDS (PC CDS) exists on a continuum that reflects the degree to which its knowledge base, data, delivery, and use focus on patient needs and experiences (Figure 1).4

Figure 1.

Figure 1.

Patient-centered factors for PC CDS. PC CDS exists on a continuum that reflects the degree to which its knowledge base, data, delivery, and use focus on patient needs and experiences. Abbreviations: PC CDS: patient-centered clinical decision support.

PC CDS is based on comparative effectiveness research or PCOR that incorporates outcomes and measures that are meaningful to patients. PC CDS may be further informed by data that are generated directly from patients (ie, patient-generated health data or PGHD), patient-reported outcomes (PROs), patient preferences, and/or nonclinical patient-centric data such as social determinants of health (SDOH). Patient engagement in CDS can take many forms, including alerts that prompt a patient to seek preventive care based on their health data; apps that transmit patients’ PGHD to their providers from Bluetooth-enabled devices; PROs submitted through an app or patient portal that inform healthcare decision-making; and patient preferences collected electronically during a visit (eg, questionnaires, electronic health record [EHR]-based clinical notes) or outside of a clinical visit (eg, symptoms reported via the patient portal). This engagement may continue into the clinical encounter, where PC CDS can support shared decision-making (SDM), a process where patients, caregivers, and care teams share and discuss health information and patients’ values and preferences to reach mutually acceptable health-related decisions.5,6

A range of use cases exists among PC CDS interventions within the categories of prevention, diagnosis, management (eg, addressing patient symptoms or providing advice for patient self-care), and treatment. Some interventions have shown positive outcomes, but gaps in measurement make it difficult to assess their success. What is apparent is that we have not fully harnessed the potential of PC CDS in delivering evidence-based, patient-centered care.7 To advance PC CDS, we must understand the challenges that remain and the opportunities to address them.

OBJECTIVES

We conducted a horizon scan to (1) identify and assess the current state of PC CDS challenges and (2) identify future directions for PC CDS in the near (3–5 years) and long (10–20 years) term.

MATERIALS AND METHODS

We employed 3 methods to conduct the horizon scan: (1) soliciting comments from a technical expert panel (TEP), (2) conducting a scoping literature review, and (3) interviewing key informants. The study was submitted for Institutional Review Board (IRB) review and classified as exempt.

Technical expert panel

We convened a 22-member TEP to refine the scoping review, guide key informant selection, and assist with synthesizing findings. We selected panel members through a convenience sample of subject matter experts and thought leaders who represented a range of PC CDS stakeholders with expertise in PC CDS design, implementation, knowledge representation, and measurement: federal agencies (n = 4), academic medical centers (n = 3), health information technology (IT) app vendors (n = 4), patient advocacy organizations (n = 2), researchers/research organizations (n = 4), health systems clinical staff and providers (n = 2), health plans and value-based purchasers (n = 2), and quality standards and measures developers (n = 1). The TEP met in person in February 2020 to consult on the project scope and virtually in September 2020 to provide feedback on the landscape assessment and key informant interviews.

Scoping review

We searched PubMed, Google, and Google Scholar for peer-reviewed and gray literature focused on: (1) the current state of PC CDS and emerging trends, (2) PC CDS gaps and challenges, and (3) future directions for PC CDS.8 Given that PC CDS is a relatively new term, we used search terms encompassing key areas of CDS, such as mobile health (mHealth) and digital health. To capture the full breadth and depth of relevant literature, we conducted supplemental searches to target specific topics after initial review of search results: decision science, behavioral economics, patient preferences, and PC CDS standards. We screened titles and abstracts using prespecified inclusion/exclusion criteria. We then reviewed the full-text of the remaining literature, ultimately including 215 resources (Figure 2). This article cites 67 resources, 58 of which are from the scoping review. We developed a tool to systematically extract key data from these resources including use case, clinical condition, population, setting, users, technology platform, and findings/outcomes. The Supplementary Appendix provides additional information on our review methods.

Figure 2.

Figure 2.

Published and gray literature search flow chart. We screened peer-reviewed and gray literature using prespecified inclusion/exclusion criteria. Following screening, full-text review, and supplemental searches, we included 215 resources in our scoping review.

Key informant discussions

We identified a convenience sample of potential informants to ensure that interviews provided a diverse understanding of opportunities for PC CDS. There was limited overlap between this sample and the TEP (1 TEP member also served as a key informant). We identified both primary and secondary choices to ensure perspectives from across the CDS field were well represented. Ultimately, we recruited 18 experts in a range of disciplines related to PC CDS: health IT vendors/consultants (n = 5), healthcare clinical staff and providers (n = 4), clinical content vendors (n = 3), researchers/research organizations (n = 3), patient representatives/patient advocacy organizations (n = 1), federal agencies (n = 1), and payers (n = 1) to serve as key informants. We used semi-structured interview guides during each 60-minute interview to gather perspectives on facilitators, challenges, and areas for future PC CDS research. Transcript-style notes were created after each interview.

Analysis and synthesis

We based our analytic framework on the 4 phases of the Analytic Framework for Action (AFA), which provides a comprehensive view of the PC CDS landscape and outlines the phases involved in translating evidence into PC CDS interventions: Prioritizing, Authoring, Implementing, and Measuring.9 The AFA comes from the Agency for Healthcare Research and Quality’s (AHRQ’s) PC CDS Learning Network. It was developed through a multi-stakeholder process that included patient advocates, provider organizations, payers, CDS content, and EHR vendors. It provided a tested, pragmatic framework for analyzing PC CDS. We also considered external factors that impact PC CDS development, such as policy and governance issues. Table 1 describes each framework component.

Table 1.

Analytic framework for action (AFA)

Phases of AFA lifecycle Application to PC CDS
Prioritizing Applying objective measures of evidence for identifying and prioritizing evidence-based findings that are to be transformed and disseminated via PC CDS, assessing, or defining their capacity for implementation, and defining stewardship and governance requirements.
Authoring Applying accepted data and knowledge standards for translating findings into one or more PC CDS intervention types that support key decisions, actions, and communications that are essential to ensuring that the finding improves care and outcomes.
Implementing Applying standardized, best practice methods and architectures for operationalizing PC CDS interventions into clinical workflows that deliver the right information to the right users in the right formats through the right channels at the right times (“CDS Five Rights”).10
Measuring Ensuring that PC CDS interventions measurably improve clinician and patient decision-making, care processes, and outcomes.
External Factors External factors including the marketplace, policy, legal, and governance issues that impact development, dissemination, and implementation processes for PC CDS.

We analyzed the literature and interview transcripts using qualitative thematic synthesis.11 Three researchers reviewed abstracted data from the literature to identify descriptive themes and map them to the AFA.8,11 The full research team reviewed and refined these themes, drawing findings for each AFA phase. For interview analysis, we used an inductive approach with simultaneous phases of data collection and analysis.11 Two senior staff reviewed the transcript-style notes to refine the themes identified in each interview. Over the course of multiple discussions, they further refined the salient themes and findings both within and across interviews. The team then mapped the themes and findings back to the AFA.

RESULTS

We identified 12 salient challenges that must be addressed to further the development, implementation, and impact of PC CDS (Table 2). These challenges spanned all phases of PC CDS development and highlighted issues that could benefit from a more patient-centric approach. Below, we discuss these patient-centered challenges and opportunities for development in each phase.

Table 2.

Twelve salient PC CDS challenges

Analytic framework for action phase and challenges
Prioritization  
  • Absence of patient voice in prioritizing topics for PC CDS

  • Lack of guidelines for multiple chronic conditions

Authoring  
  • Variations in guideline translation

  • Delays in keeping PC CDS up-to-date

  • Lack of patient-centered terminology standards

Implementation  
  • Limited application of research on patient decision-making

  • Lack of alignment of PC CDS modalities with patients’ daily lives

  • Lack of PC CDS integration into clinical workflows

Measuring  
  • Lack of studies that measure clinical outcomes

  • Limited studies that measure patient and provider engagement

External factors  
  • Lack of patient and provider trust in app safety and efficacy

  • Concerns regarding the privacy and security of patient data

Prioritizing

We identified 2 major challenges in the prioritizing phase focused on the evidence that informs PC CDS development.

Absence of patient voice in prioritizing topics for PC CDS

We found that studies of PC CDS did not discuss how researchers selected evidence or guidelines to disseminate via PC CDS. In addition, the studies did not discuss whether the prioritization process involved patient representatives, families, and/or caregivers. According to the key informants, much of the research used to develop clinical guidelines and PC CDS focuses on encouraging clinically “desirable” behaviors that providers like to see (eg, weight loss or adherence to a diuretic without side effects), which may or may not be important outcomes for patients, families, and caregivers. Furthermore, we identified limited PC CDS research focused on conditions (eg, behavioral health conditions, pain management) where guidelines emphasized selecting treatments that meet patient’s individual needs and preferences.12,13 Notably, these conditions lend themselves to SDM, a process where a patient and provider discuss a patient’s health, preferences and/or values, and treatment options to facilitate decision-making,14 due to the tradeoffs associated with each treatment.

Lack of guidelines for multiple chronic conditions

Another challenge is the lack of focus on comprehensive care that addresses co-existing conditions. The majority of studies we identified targeted a single chronic condition. This limited focus represents a critical gap given that multiple chronic conditions affect 42% of American adults.15,16 Consequently, most guidelines fail to address the complexity and potential conflicts of associated treatment plans. They also ignore the potential benefits of PC CDS in managing chronic conditions by eliciting patient preferences and goals and fostering patient–provider communication to enable priority setting and SDM.

Authoring

The development and use of standards for data representation and transmission is a key factor needed for scaling PC CDS. The use of standards enables PC CDS knowledge to more widely interoperate, both syntactically and semantically, with data from EHRs and patient engagement applications, and to integrate more easily into healthcare workflows, we identified 3 major challenges in this phase.

Variations in guideline translation

A central challenge to PC CDS knowledge translation is heterogeneity in CDS interventions based on different interpretations and implementations of a clinical guideline. Typically, the guidelines do not provide clarity on patient-centeredness or on different patient contexts, meaning that there is heterogeneity in how healthcare organizations translate clinical guidelines into computable knowledge (ie, the CDS algorithm) and in how they develop and implement site or product-specific CDS interventions. The result is that CDS guidance may differ by stakeholder or location. Some adaptation is necessary to support specific populations or context; however, local customization without corresponding specification of logic used to guide these distinctions can reduce guideline clarity.17 This issue has implications for the use of PC CDS, given that lack of guideline clarity can impede patient and provider SDM discussions. In response to variation in guideline translation, the Centers for Disease Control and Prevention initiated “Adapting Clinical Guidelines for the Digital Age,” in order to standardize and improve the timeliness, accuracy, and consistency of guideline translation and implementation.17–19 Initiatives like Option Grid and “Care that Fits” deliver clinical guidelines to providers and patients through the development of evidence-based decision tools.20,21 These tools can be used to facilitate SDM.

Delays in keeping PC CDS up-to-date

PC CDS must be updated as evidence changes. Delays in updating guidelines limit the usefulness of PC CDS and require healthcare organizations to make their own updates. Rather than using off-the-shelf products for the content and timing of updates, the key informants indicated that healthcare organizations may favor home-grown solutions. This impedes scalability of PC CDS across organizations. The key informants also noted that developers of direct-to-consumer apps may revisit the underlying logic used in their products less frequently than health systems that control their own content, further limiting the timeliness of evidence. These delays have potentially serious consequences for the quality or safety of patient care, widening disparities in care, and inferior patient outcomes.

Lack of patient-centered terminology standards

Terminology standards that are important for patient-centered care are either in development or do not exist. Data elements need to be defined to capture patients’ perspectives on their conditions, progress, acceptance of various interventions, and level of adherence to therapies. To date, much of the progress in standardizing PGHD capture has focused on electronic PROs.22 Terminology standards are needed to describe patient preferences across multiple conditions.23 To support patient-centered care and SDM, existing clinical coding systems should be enhanced to incorporate terms that are understandable by patients in order to support patient-centered care and SDM.24,25

The diversity of data types and devices used to collect PGHD presents a challenge when managing data across platforms.26 Standardized data elements are lacking for PGHD, hindering the seamless incorporation of PGHD into care, the pooling of data from multiple participants and data sets, and the consistent analysis and interpretation of PGHD.26,27 While some PRO instruments (eg, PROMIS measures) are codified in Logical Observation Identifiers Names and Codes (LOINC), a standardized ontology is needed that will constrain the syntactic interoperability—so patient-centered outcomes can be collected across institutions to enable population-scale research.22

Controlled terminologies to support SDOH are another gap area in patient-centric data standards. Recent policy and research emphasis on collection and use of SDOH reflect new understanding of how significantly demographics, neighborhood, and social risk factors can affect health outcomes. While the Gravity Project has developed a minimum data set required to exchange information within multiple SDOH domains,28 providers and researchers need standardized representations of SDOH terms that appear in widely used tools (eg, Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experiences, PRAPARE). SDOH terminology standards would result in better data on SDOH and the addition of SDOH data into PC CDS algorithms. In addition, the incorporation of the SDOH data into PC CDS tools may help address the potential for bias in the underlying evidence that informs PC CDS algorithms. These algorithms are often based on clinical studies that do not capture outcomes or risk factors for all populations (eg, a range of races and ethnicities, all genders, geographic areas). The incorporation of SDOH data may help mitigate the potential bias from these gaps.

Implementing

We identified 3 major challenges centered on implementing patient decision-making and PC CDS into patients’ lives and providers’ workflows.

Limited application of research on patient decision-making

There is a large amount of literature in cognitive psychology and behavioral economics examining the heuristics and biases that influence human decision-making.29,30 This literature has spurred further research to understand the factors that influence the way that providers and patients make healthcare decisions.31–34 Findings indicate that patient decision-making processes differ from provider decision-making, and are informed by different information and, at times, divergent priorities. Research on using decision aids to facilitate SDM shows that these tools can enhance patient participation in decision-making.35 Yet, there was limited use of decision aids in the PC CDS studies in our literature review. A recurrent theme in the key informants’ discussion of SDM was the need to identify and respond to efficacious uses of PC CDS in the real world, in terms of both patient and provider needs. In addition, a better understanding of the literature examining individual and SDM processes would allow PC CDS developers to (1) target interventions to foster SDM and (2) support patients from an informed perspective about the way they make decisions that reflect their goals and preferences. The key informants noted gaps in the application of research on patient decision-making processes to PC CDS as a limitation of current PC CDS design.

Lack of alignment of PC CDS modalities with patients’ daily lives

To augment in-person health and SDM support, many opportunities for PC CDS exist outside the clinical setting. While minimally discussed in the literature, the key informants noted the importance of designing PC CDS that fits into patients’ daily lives, so that they are likely to use it. Design considerations include the broader context of patients’ lives (eg, setting, timing, clinical condition, health literacy, information preferences), the most effective forms of communication (eg, video, voice, SMS messaging) for individual patients, and how PC CDS integrates into patients’ routines.36 For patients who require ongoing care, such as those with multiple chronic conditions, PC CDS must be designed to support SDM about how a potential treatment or management routine fits into a patient’s daily life in the context of existing treatment protocols. The key informants emphasized “meet[ing] patients where they are with the technology” by providing multiple modalities for communicating information because different patients will have different levels of interest in, and capability of, engaging with technology.

Lack of PC CDS integration into clinical workflows, which can increase provider burden

Several informants stated that PC CDS tools are challenging to implement because of the need to integrate them into clinical workflows, a potentially resource-intensive process. This theme was echoed in the literature. To be most effective, PC CDS must enhance communication and fit into the clinical workflow when patients and providers interact.37–39 However, leveraging existing infrastructure requires a high degree of integration between new health apps and legacy EHRs to make the apps useful in clinical practice.40 To date, there is no agreement across EHR vendors on where to place CDS trigger points (eg, provide dose checking as soon as each order is entered, or wait until a full session of multiple orders is ready to be signed). Additional work is needed to standardize CDS insertion points and ensure optional alignment with clinical workflows.

Other major challenges to integrating PGHD is that most EHRs lack an infrastructure for receiving, storing, displaying, and using PGHD in a manner that (1) maximizes accessibility by considering user-centered design for provider-facing data and (2) reduces provider’s burden for reviewing and leveraging these data.41–44 Clinical integration of PGHD into EHRs has lagged behind other types of data for reasons that include insufficient standards to support data linkage and scaling across healthcare systems, lack of interfaces for healthcare providers, and lack of incorporation of PGHD into workflows and accessibility at the point of care. In addition, there are variable levels of provider engagement.45–47 While a developer may cite data showing a PC CDS intervention can be effective, the provider will likely not use the tool if it cannot be efficiently integrated into their EHR-based workflow before, during, or after the patient visit.

Measuring

Use of data to inform continuous improvement is critical to a learning health system. Ideally, a continuous feedback loop from PC CDS research to real-world quality improvement efforts should exist. This feedback can help ensure that data from PC CDS interventions are used to improve clinician and patient decision-making and health outcomes. We found 2 challenges in this area.

Lack of studies that measure clinical outcomes

The published literature focuses on measuring aspects of PC CDS implementation such as the acceptability and feasibility of study interventions. While several studies did include some measure of clinical effectiveness, these studies largely focused on processes related to chronic disease management (eg, adherence to measuring blood glucose levels). Few studies measured impact on clinical outcomes (eg, improvement in blood glucose levels); and among those that did, the results were mixed.

Limited studies and metrics to measure patient and provider engagement

PC CDS tools require different levels of patient–provider engagement depending on the use case and population. While studies suggest design features that support more user-friendly interactions with PC CDS tools (eg, easier to read navigation buttons,48 less text,49 or links to educational resources),38 less is understood about what is most effective for creating and maintaining patient engagement (eg, longitudinal tracking capabilities,50 frequency of notifications,51 or ability to directly engage providers).52 A recent review found that while a limited number of health app studies for managing chronic conditions used simple consumer engagement measures (eg, number of log-ins recorded, frequency of interactions with the app), gaps remain in understanding what types of engagement lead to improved clinical and other outcomes.53 Without the evidence about which PC CDS tools and strategies are effective, for which clinical conditions and patient populations, and when they are most appropriate, the adoption and sustained use of PC CDS interventions will remain limited. To effectively scale PC CDS that supports the patient–provider relationship and SDM, there is a need to move toward studies that measure and compare outcomes related to provider and patient engagement and that refine engagement assessment metrics.

External factors

These are challenges related to trust and privacy permeated the Prioritizing, Authoring, Implementation, and Measuring phases.

Lack of patient and provider trust in app safety and efficacy

Putting patients at the center of PC CDS requires that they trust both the tools and the underlying evidence.54 Patients and providers need to have confidence in the validity and effectiveness of health apps and other PC CDS tools. The potential lack of trust in these tools has long been a concern in the field of CDS.55–57 The key informants felt the research that undergirds these tools should be explicit so that patients can trust the tools and providers can raise concerns about unvetted or unreliable PC CDS. Likewise, they must have confidence in the underlying research and clinical guidelines that support the PC CDS.

Concerns regarding the privacy and security of patient data

Patients and providers must trust the data they share through PC CDS tools are secure and remain private. Lack of clarity around data ownership and related legal considerations remains a barrier to PGHD use.58 For example, a patient may opt-in to data sharing for one purpose (eg, a treatment decision) but may not be given opportunity to opt-out of having it used or sold for other purposes (eg, marketing).27 Mobile app developers and companies, health systems, EHR vendors, and academic institutions must be transparent about their data use and ownership policies so that patients can make informed choices about when and how to share PGHD.58 Operational strategies to ensure patient privacy include developing standards, policies, and guidelines regarding the collection, transmission, use, and ownership of PGHD.58

Illustrative quotes

Our conversations with the key informants provided a robust discussion of these challenges. Table 3 lists these challenges along with illustrative quotes from key informants.

Table 3.

Select salient challenges and illustrative quotes

Current challenges Key informant quotes
Prioritization  
  • Absence of patient input in prioritizing topics for PC CDS

“We often don’t generate the evidence with the patient engaged in the conversation. Whatever you do downstream in terms of the interaction about the compliance or behavior change … or whatever, it pushes it more towards doing what the clinicians want, or believe, is correct…” (Federal Stakeholder)
Authoring  
  • Variations in guideline translation

“Everyone uses a different methodology to create recommendations and then they write them up in ways that are not understandable to clinicians, let alone people who haven't read and reviewed the literature. So, it's really starting at the top and making all of the guidance clear to follow…I think once we get that done … the wobbliness of the interpretations of the differences in implementation will slowly phase out; there will be less wiggle room about what people should be doing and the specific situations.” (Implementing Partner)
“…the more we can have centralized sources that keep [knowledge] up [to date] and translate clinical guidelines into structured logic that EHRs can look to, [the easier it will be for] the rest of us to do the implementation and work with people in clinical settings to use them.” (Healthcare Provider)
Implementation  
  • Limited understanding of patient decision-making

  • Need for translation of decision-making best practices into PC CDS development

  • Lack of alignment of PC CDS modalities with patients’ daily lives

“Some people are early adopters and they'll take a new device or they'll try new things. But for a lot of people … if it's not really easy for them to incorporate what they already do, they're probably going to forget about it or not use it.”(Researcher)
“You can’t expect someone to be sitting on a bus and thinking about cancer treatment. You expect them to be in a place that is somewhat comfortable, maybe looking at a computer screen and looking at a pamphlet and writing things down and doing a cost benefit structure. I think that envisioning the environments where people are going to use these tools will help. (Federal Stakeholder)”
Measuring  
  • Limited research that measures or compares methods for increasing patient and provider engagement

“I think one of the areas [for future investment] clearly is in consumer engagement and what are the methods that work and don't work, and what are the key criteria that have to be met to actually successfully engage consumers in their care. That is … a huge remaining issue, because you can create all this cool technology, you can push all this stuff out, but if you don't really get people's attention, you've done nothing.” (Health IT Vendor)
External factor  
  • Lack of patient and provider trust in app safety and efficacy

“We want to be providing them the ability to make decisions for themselves and to make sure that what goes out is safe. We need research on the effect of apps and safety of apps for patient care and patient-centered decision support.” (Implementing Partner)
“There probably should be some industry-wide principles or style guides saying here is the safe and effective thing to do for a consumer facing app…on the consumer side of the apps today is that there is no downside to making a mistake. When we talk about managing diabetes or your sleep apnea or whatever at home the intended or unintended outcomes will become more important and these principles will need to be established.” (Health IT and App Vendor)

DISCUSSION

The 12 challenges identify key gaps that can inform research and development opportunities to advance PC CDS. Broadly, increased patient and provider engagement in PC CDS development have enormous potential to improve patient health and well-being. Based on the challenges, below we discuss future opportunities for PC CDS specific to each AFA phase. Discussions with the key informants and the TEP informed the development of these opportunities summarized in Table 4.

Table 4.

Future directions for PC CDS

Current challenges Areas for future activities
Prioritization  
  • Absence of patient input in prioritizing topics for PC CDS

  • Lack of guidelines for multiple chronic conditions

Establish methods to incorporate the patient voice in evidence prioritization.
Expand research and development of PC CDS that incorporate guidelines for multiple conditions in a patient-centric way. Specifically, understand patient needs and priorities by engaging with patients and caregivers through qualitative research. Solicit their responses to use cases to clarify what PC CDS can and should do.
Authoring  
  • Variations in guideline translation

  • Keeping PC CDS up to date

  • Lack of patient-centered terminology standards

Accelerate clinical knowledge translation into PC CDS by leveraging ongoing initiatives on open standards—eg, Mobilizing Computable Biomedical Knowledge (MCBK), which is supporting an ecosystem built on open standards for accessing biomedical knowledge–(make knowledge Findable, universally Accessible, highly Interoperable, and readily Reusable; ie, FAIR principles).59
Standardize publication of real-world evidence in computable forms, so that this evidence can be automatically converted to PC CDS, thereby limited variability in interpretation.
Standardize and expand the terminologies for PC CDS data elements.
Implementation  
  • Limited understanding of patient decision-making

  • Aligning PC CDS modalities with patients’ daily lives

  • Lack of PC CDS integration into clinical workflows

Engage in PC CDS codesign activities with patients to understand their needs, desires, expectations, strengths, and limitations.
Establish industry-wide principles on effective ways to deliver patient-facing PC CDS and principles for implementing PC CDS to support patients.
Explore how PC CDS can be delivered through multiple modalities to support patient–provider SDM about risk and options.
Enhance standards-based application programming interfaces (APIs) for sharing information between apps and with EHRs to facilitate communication between provider and patient devices. This includes the development of new FHIR resources and implementation guides and improving the performance and availability of existing FHIR resources in current implementations.
Focus on prototype software that creates an acceptable workflow between patients and providers for requesting, capturing, and sharing PGHD.
Measuring  
  • Lack of studies that measure clinical outcomes

  • Limited studies that measure patient and provider engagement

Measure more substantive targeted outcomes for PC CDS effectiveness, impacts of PC CDS on clinical outcomes, and research on effective ways to engage patients through PC CDS.
Engage patients and caregivers across the measurement lifecycle—from selecting outcomes to be tested, to designing the study and intervention, and ultimately contextualizing results.
External factors  
  • Lack of patient and provider trust

  • Concerns regarding the privacy and security of patient data

Define standards to support the metadata used to communicate trust eg, for data provenance standards. These would be part of consumable artifacts to enhance the ease of sharing and likelihood of trusting PC CDS artifacts.60
Increase transparency through development of rating systems, professional endorsements, and other safeguards to protect patient health, safety, and privacy.
Develop guidance on best practices for health app developers and/or a rating system for the health app industry to certify they have met certain safety and quality criteria for, eg, plain language information on source of recommendations, any modifications to recommendations that have been tailored to patient-specific preferences and conditions.

Future directions for PC CDS

Prioritization phase

The field of CDS should establish methods to elicit and incorporate information from patients, families, and caregivers during evidence prioritization to ensure that tools address outcomes important to these groups. For example, an international consortium, the Multi-Stakeholder Engagement (MuSE) project, is currently building consensus-based guidance for evidence-based strategies for “equitable and meaningful” engagement of stakeholders in guideline development.61 It should also expand research and development of PC CDS that leverages multiple guidelines in a patient-centric way. There is a need for PC CDS development for disease prevention, including preventive care tools (eg, condition-specific risk assessments) that could be used at home. PC CDS experts, industry stakeholders, and patients should be consulted to discuss under what circumstances PC CDS and SDM could most effectively advance whole-person care, and to collaboratively develop a research agenda. Shifts in the PC CDS landscape are likely as tools evolve from helping clinicians manage individual patients in individual encounters to a population health-based approach toward managing many patients with the same conditions at the same time.

Authoring phase

The field of CDS should establish a coordinating entity to standardize clinical knowledge translation from clinical guidelines into structured use in EHRs to accelerate translation into PC CDS and address process gaps. One pathway is leveraging ongoing work such as Mobilizing Computable Biomedical Knowledge (MCBK), which is establishing a community and approaches to build around open standards through which biomedical knowledge can meet the FAIR principles.59,62 AHRQ has developed tools that can be used at the time of guideline writing to ensure consistency between recommendations and PC CDS and to promote interoperability, available through the AHRQ CDS Authoring Tool and Repository.62

To address the lack of standardization of PGHD, the CDS field can focus on a prototype software framework such as SMART Markers,22 that leverages the SMART (Substitutable Medical Applications and Reusable Technologies) on Fast Healthcare Interoperability Resources (FHIR) application programming interface (API) to create a seamless workflow between patients and providers for capturing and sharing PGHD.22,63 There is an ongoing need to standardize and expand the terminologies for the data elements within the FHIR profiles, to make the data supported by FHIR information systems more comprehensive.64–66

Implementation phase

There are opportunities to standardize PC CDS implementation and better address patients’ needs. Implementation refers to the building process and architecture (eg, at a particular site) associated with supporting a CDS use case. Delivery refers to the final stage when PC CDS reaches users in a form with which they can interact. The key informants suggested establishing industry-wide principles or style guides about safe and effective ways to deliver patient-facing PC CDS (ie, reaching users in a form they can interact with). They also suggested establishing principles for building and configuring PC CDS within the local architecture to support patients (eg, addressing health and technological literacy, language, culture). In order to design patient-centered tools that incorporate patient preferences in multiple ways, CDS developers (ie, individuals who create and design patient-facing apps that incorporate CDS) must learn how to understand patients’ needs, expectations, strengths, and limitations. Accomplishing this objective will require patient-centered design research and meaningful codesign activities, with a focus on facilitating provider and patient CDS use and access to information.

To expand the range of PC CDS, future research should explore PC CDS delivery through multiple modalities to support patient–provider conversations, in particular SDM, about risk and options.37 Some experts point to a future where artificial intelligence may reduce the level of integration needed with EHRs and require less provider engagement for disease detection and management apps to be effective.40 Although use of conversational agents would still require design by someone with expertise in human factors, human–computer interaction, and or UI/UX, experts noted the potential for artificial intelligence (AI)-enabled chatbots to function outside of the EHR and reduce the need for provider engagement by providing patient SDM and/or support and assisting with symptom triage. For example, once a glucose management program is established between patient and provider, a human coach who provides guidance based on wearable glucose sensor results could be replaced by an AI-powered virtual coach.40,67 They also suggest that PC CDS may need to be designed to fit a changing healthcare ecosystem in which care is increasingly delivered outside the traditional patient encounter. This evolution relies in large part on advancing the availability of standards-based APIs to enable information sharing between apps and with EHRs to facilitate communication between provider and patient devices (eg, “write” access to EHRs, CDS Hooks that support appropriate workflow triggers for patients and providers). This sharing will require interoperability and accessibility of PGHD through consistent use of device identifiers and alignment with data exchange standards. The CDS field should establish FHIR interoperability resources to become a component of the Health IT Certification program as the standards mature.65

Measuring phase

The success of PC CDS requires that we develop outcome measures that are meaningful to patients. Researchers should engage patients and caregivers across the measurement lifecycle—from selecting study outcomes, to designing studies and interventions, and contextualizing results—to ensure that PC CDS research is measuring what matters to patients and their families. This may include the use of PRO measures that directly reflect patient experiences (eg, function, quality of life, pain). Research should measure substantive, targeted outcome measures for PC CDS effectiveness; assess the impact of PC CDS on clinical outcomes and PROs; and research effective ways to engage patients through PC CDS.

External factors

To foster trust, providers and patients need greater transparency regarding how PC CDS is developed from clinical guidelines. Likewise, the key informants noted that patients may not fully understand the potential privacy risks associated with using these apps—specifically, whether Health Insurance Portability and Accountability Act (HIPAA) protections apply. Patients may inadvertently consent to the app’s secondary use of health information, which allows the selling of their protected health information. Methods for enhancing trust and safety as PC CDS may include development of rating systems from trusted and vetted sources and safeguards that protect patient health, safety, and privacy. Such practices will help foster user confidence in the PC CDS systems, whether they are accessed in EHRs, via the web, or through an app. In addition, researchers and multi-stakeholder public–private partnerships should develop guidance on best practices for health app developers (eg, an ethical and safety framework), and/or a rating system to certify that developers have met certain privacy, safety, and quality criteria, particularly when the apps involve patient-specific data and life-altering clinical decisions.

Limitations

Our horizon scan has limitations. The terminology for PC CDS is continuously evolving. While we searched for literature on related terms, our scoping review was comprehensive, but not exhaustive. We interviewed a small group of experts. While these individuals had robust PC CDS experience, it is possible that they did not consider all challenges currently facing the field of CDS. In addition, there was limited patient involvement in the KIIs and the TEP; which may have limited the ability of our findings to fully represent the patient perspective on the landscape for PC CDS.

CONCLUSIONS

The vision of PC CDS will continue to evolve as advancements are made in patient-facing apps for CDS and the collection and use of PGHD. A future landscape for PC CDS and supportive systems should: (1) support patients in developing the skills, knowledge, and interest necessary to be active in SDM and (2) help providers implement safe and effective care plans. To support patient-centered care, there needs to be a greater focus on the development of PC CDS tools that address priority conditions, populations, and national health goals. Achievement of these objectives will depend in part on broader, longer-term efforts to move culture across the health system toward prioritizing patient-centeredness and incorporating SDM into clinical care and payment models. There are multiple opportunities to increase patient engagement, incorporate patient preferences into clinical evidence, and ensure that PC CDS fits into both patients’ daily lives and providers’ workflows.

The medical model of healthcare delivery emphasizes the role of the clinician. The shift to person-centered care places the patient at the center of healthcare and seeks to empower individuals to participate in maintaining or improving their own health. Thus far, movement in this direction has been gradual, and patients remain at the periphery of the process. Rapid change seems unlikely. Nonetheless, this article has uncovered important ways that PC CDS can contribute to gradual development of a genuinely patient-centered health system.

FUNDING

This work is based on research conducted by NORC at the University of Chicago under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. HHSP233201500023I).

AUTHOR CONTRIBUTIONS

All authors made substantial contributions to conception and design of this research. PD, SFS, KHH, LH, DFL, and DFS led data acquisition and the analysis and interpretation of data. All authors were involved in drafting the manuscript or revising it critically for important intellectual content and gave final approval of the version published.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Supplementary Material

ocac059_Supplementary_Data

ACKNOWLEDGMENTS

The authors thank Ed Lomotan at the Agency for Healthcare Research and Quality who provided helpful feedback on this paper. We also thank Nora Marino for her assistance in manuscript preparation.

CONFLICT OF INTEREST STATEMENT

AB is a stockholder in Elimu Informatics.

DATA AVAILABILITY

The data underlying this article will be shared on reasonable request to the corresponding author.

Contributor Information

Prashila Dullabh, NORC at the University of Chicago, Bethesda, Maryland, USA.

Shana F Sandberg, NORC at the University of Chicago, Bethesda, Maryland, USA.

Krysta Heaney-Huls, NORC at the University of Chicago, Bethesda, Maryland, USA.

Lauren S Hovey, NORC at the University of Chicago, Bethesda, Maryland, USA.

David F Lobach, Elimu Informatics, El Cerrito, California, USA.

Aziz Boxwala, Elimu Informatics, El Cerrito, California, USA.

Priyanka J Desai, NORC at the University of Chicago, Bethesda, Maryland, USA.

Elise Berliner, Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA.

Chris Dymek, Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA.

Michael I Harrison, Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA.

James Swiger, Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA.

Dean F Sittig, School of Biomedical Informatics, University of Texas Health Science Center, Houston, Texas, USA.

REFERENCES

  • 1. Smith M, Saunders R, Stuckhardt L, et al. , eds. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2013. [PubMed] [Google Scholar]
  • 2.Further Consolidated Appropriations Act, 2020. 2020. https://www.congress.gov/116/plaws/publ94/PLAW-116publ94.pdf. Accessed August 2, 2021.
  • 3. Sittig DF, Wright A, Osheroff JA, et al.  Grand challenges in clinical decision support. J Biomed Inform  2008; 41 (2): 387–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Patient-Centered and Patient-facing Clinical Decision Support | Patient-Centered CDS Learning Network. https://pccds-ln.org/node/276. Accessed August 2, 2021.
  • 5.Patient-Centered Clinical Decision Support (CDS) Learning Network. https://pccds-ln.org/index.php/. Accessed August 2, 2021.
  • 6. Charles C, Gafni A, Whelan T.  Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med  1997; 44 (5): 681–92. [DOI] [PubMed] [Google Scholar]
  • 7. Richardson JE, Middleton B, Osheroff JA, et al.  The PCOR CDS-LN Environmental Scan: Spurring Action by Identifying Barriers and Facilitators to the Dissemination of PCOR through PCOR-Based Clinical Decision Support. Research Triangle Park, NC: RTI International; 2016. https://www.evicore.com/-/media/files/evicore/insightdocuments/blogs/pcor-cds-ln_environmental-scan-report_2016-11-04.pdf?la=en. [Google Scholar]
  • 8. Arksey H, O’Malley L.  Scoping studies: towards a methodological framework. Int J Soc Res Methodol  2005; 8 (1): 19–32. [Google Scholar]
  • 9. Marcial L, Richardson JE, Blumenfeld BH. Development of a Proposed Analytical Framework for Patient-Centered CDS. 2017. https://pccds-ln.org/sites/default/files/2017-09/ihealth_poster_2017_final.pdf. Accessed August 2, 2021.
  • 10. Osheroff JA, ed. Healthcare Information and Management Systems Society. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. 2nd ed. Chicago, IL: HIMSS; 2005. [Google Scholar]
  • 11. Barnett-Page E, Thomas J.  Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol  2009; 9: 59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Skelly AC, Chou R, Dettori JR.  Noninvasive Nonpharmacological Treatment for Chronic Pain: A Systematic Review. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ); 2018. [PubMed] [Google Scholar]
  • 13. Sutton RT, Pincock D, Baumgart DC, et al.  An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med  2020; 3: 17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Office of the National Coordinator for Health Information Technology. Shared Decision Making Fact Sheet. 2013. https://www.healthit.gov/sites/default/files/nlc_shared_decision_making_fact_sheet.pdf. Accessed August 2, 2021.
  • 15. Buttorff C, Ruder T, Bauman M.  Multiple Chronic Conditions in the United States. Santa Monica, CA: RAND Corporation: 1–33. https://www.rand.org/pubs/tools/TL221.html. Accessed August 2 2021. [Google Scholar]
  • 16.The AHRQ Multiple Chronic Conditions Research Network. http://www.ahrq.gov/patient-safety/settings/long-term-care/resource/multichronic/index.html. Accessed August 2, 2021.
  • 17.Adapting Clinical Guidelines for the Digital Age. 2020. https://www.cdc.gov/ddphss/clinical-guidelines/index.html. Accessed August 3, 2021.
  • 18. Lomotan EA, Meadows G, Michaels M, et al.  To Share is Human! Advancing evidence into practice through a national repository of interoperable clinical decision support. Appl Clin Inform  2020; 11 (1): 112–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Mission | DynaMed. https://www.dynamed.com/about/mission/. Accessed November 16, 2021.
  • 20.Care That Fits. Care That Fits. https://carethatfits.org/. Accessed November 16, 2021.
  • 21.Option Grid. Option Grid. http://optiongrid.org/. Accessed November 16, 2021.
  • 22. Sayeed R, Gottlieb D, Mandl KD.  SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med  2020; 3: 9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Cameron B, Douthit B, Richesson R.  Data and knowledge standards for learning health: a population management example using chronic kidney disease. Learn Health Syst  2018; 2 (4): e10064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Keselman A, Logan R, Smith CA, et al.  Developing informatics tools and strategies for consumer-centered health communication. J Am Med Inform Assoc  2008; 15 (4): 473–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Hong Y, Ehlers K, Gillis R, et al.  A usability study of patient-friendly terminology in an EMR system. Stud Health Technol Inform  2010; 160: 136–40. [PubMed] [Google Scholar]
  • 26. Lavallee DC, Lee JR, Austin E, et al.  mHealth and patient generated health data: stakeholder perspectives on opportunities and barriers for transforming healthcare. mHealth  2020; 6: 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Tarver M.  Executive Summary for the Patient Engagement Advisory Committee Meeting. Washington, DC: Department of Health and Human Services Food and Drug Administration; 2018. https://www.fda.gov/media/122887/download. [Google Scholar]
  • 28.Gravity Project FHIR IG—Patient Care—Confluence. https://confluence.hl7.org/display/PC/Gravity+Project+FHIR+IG. Accessed August 3, 2021.
  • 29. Kahneman D, Slovic P, Tversky A.  Judgment under Uncertainty: Heuristics and Biases. Cambridge: Cambridge University Press; 1982. [Google Scholar]
  • 30. Thaler RH, Sunstein CR.  Nudge: Improving Decisions about Health, Wealth, and Happiness, Rev. and expanded ed. New York: Penguin Books; 2009. [Google Scholar]
  • 31. Benin AL, Wisler-Scher DJ, Colson E, Shapiro ED, Holmboe ES.  Qualitative analysis of mothers’ decision-making about vaccines for infants: the importance of trust. Pediatrics  2006; 117 (5): 1532–41. [DOI] [PubMed] [Google Scholar]
  • 32. Limaye RJ, Malik F, Frew PM, et al.  Patient decision making related to maternal and childhood vaccines: exploring the role of trust in providers through a relational theory of power approach. Health Educ Behav  2020; 47 (3): 449–56. [DOI] [PubMed] [Google Scholar]
  • 33. Saposnik G, Redelmeier D, Ruff CC, et al.  Cognitive biases associated with medical decisions: a systematic review. BMC Med Inform Decis Mak  2016; 16 (1): 138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Langford BJ, Daneman N, Leung V, et al.  Cognitive bias: how understanding its impact on antibiotic prescribing decisions can help advance antimicrobial stewardship. JAC Antimicrob Resist  2020; 2 (4): dlaa107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Stacey D, Légaré F, Col NF, et al.  Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2014; (1): CD001431. [DOI] [PubMed] [Google Scholar]
  • 36. Valdez RS, Brennan PF.  Exploring patients’ health information communication practices with social network members as a foundation for consumer health IT design. Int J Med Inform  2015; 84 (5): 363–74. [DOI] [PubMed] [Google Scholar]
  • 37. Melnick ER, Lopez K, Hess EP, et al.  Back to the bedside: developing a bedside aid for concussion and brain injury decisions in the emergency department. eGEMs  2015; 3 (2): 6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Fiks AG, Mayne S, Karavite DJ, et al.  A shared e-decision support portal for pediatric asthma. J Ambul Care Manage  2014; 37 (2): 120–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Montori VM, Hargraves I, McNellis RJ, et al.  The care and learn model: a practice and research model for improving healthcare quality and outcomes. J Gen Intern Med  2019; 34 (1): 154–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Cohen AB, Martin SS.  Innovation without integration. NPJ Digit Med  2020; 3: 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Abdolkhani R, Gray K, Borda A, et al.  Patient-generated health data management and quality challenges in remote patient monitoring. JAMIA Open  2019; 2 (4): 471–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.EHRIntelligence. Top Challenges to Leveraging Patient-Generated Health Data. EHRIntelligence. 2016. https://ehrintelligence.com/news/top-challenges-to-leveraging-patient-generated-health-data. Accessed August 23, 2020.
  • 43. Narus SP, Rahman N, Mann DK, et al.  Enhancing a commercial EMR with an open, standards-based publish-subscribe infrastructure. AMIA Annu Symp Proc  2018; 2018: 799–806. [PMC free article] [PubMed] [Google Scholar]
  • 44. Tcheng JE, Bakken S, Bates DW, et al. , eds. Optimizing Strategies for Clinical Decision Support: Summary of a Meeting Series. Washington, DC: National Academy of Medicine; 2017. [PubMed] [Google Scholar]
  • 45. Cohen DJ, Keller SR, Hayes GR, et al.  Integrating patient-generated health data into clinical care settings or clinical decision-making: lessons learned from project health design. JMIR Hum Factors  2016; 3 (2): e26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Patient Safety Network. Alert Fatigue. Patient Sefety 101: Primers. 2019. https://psnet.ahrq.gov/primer/alert-fatigue. Accessed 3 August 3, 2021.
  • 47. Rudin RS, Perez S, Rodriguez JA, et al.  User-centered design of a scalable, electronic health record-integrated remote symptom monitoring intervention for patients with asthma and providers in primary care. J Am Med Inform Assoc  2021; 28 (11): 2433–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Fleisher L, Ruggieri DG, Miller SM, et al.  Application of best practice approaches for designing decision support tools: the preparatory education about clinical trials (PRE-ACT) study. Patient Educ Couns  2014; 96 (1): 63–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Cooley ME, Abrahm JL, Berry DL, et al.  Algorithm-based decision support for symptom self-management among adults with Cancer: results of usability testing. BMC Med Inform Decis Mak  2018; 18 (1): 31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. de Jong JM, Ogink PA, van Bunningen CG, et al.  A cloud-based virtual outpatient clinic for patient-centered care: proof-of-concept study. J Med Internet Res  2018; 20 (9): e10135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Cushing A, Manice M, Ting A, et al.  Feasibility of a novel mHealth management system to capture and improve medication adherence among adolescents with asthma. Patient Prefer Adherence  2016; 10: 2271–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. O’Malley DM, Davis SN, Devine KA, et al.  Development and usability testing of the e-EXCELS tool to guide cancer survivorship follow-up care. Psychooncology  2020; 29 (1): 123–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53. Pham Q, Graham G, Carrion C, et al.  A Library of analytic indicators to evaluate effective engagement with consumer mhealth apps for chronic conditions: scoping review. JMIR Mhealth Uhealth  2019; 7 (1): e11941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Middleton B, Platt J, Richardson JE, et al.  Recommendations for Building and Maintaining Trust in Clinical Decision Support Knowledge Artifacts. Research Triangle Park, NC: Patient-Centered Clinical Decision Support Learning Network; 2018. https://pccds-ln.org/sites/default/files/2018-09/TFWG%20White%20Paper_final.pdf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Alexander GL.  Issues of trust and ethics in computerized clinical decision support systems. Nurs Adm Q  2006; 30 (1): 21–9. [DOI] [PubMed] [Google Scholar]
  • 56. Carter SM, Rogers W, Win KT, et al.  The ethical, legal and social implications of using artificial intelligence systems in breast cancer care. Breast  2020; 49: 25–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57. Gretton C.  Trust and transparency in machine learning-based clinical decision support. In: Zhou J, Chen F, eds. Human and Machine Learning. Cham: Springer; 2018: 279–92. https://link.springer.com/chapter/. [Google Scholar]
  • 58. Petersen C, DeMuro P.  Legal and regulatory considerations associated with use of patient-generated health data from social media and mobile health (mHealth) devices. Appl Clin Inform  2015; 6 (1): 16–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.ABOUT. Mobilizing Computable Biomedical Knowledge (MCBK). https://mobilizecbk.med.umich.edu/about. Accessed August 3, 2021.
  • 60. Richardson JE, Middleton B, Platt JE, et al.  Building and maintaining trust in clinical decision support: recommendations from the Patient‐Centered CDS Learning Network. Learn Health Syst  2020; 4 (2). e10208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Huguet N, Angier H, Marino M, et al.  Protocol for the analysis of a natural experiment on the impact of the Affordable Care Act on diabetes care in community health centers. Implement Sci  2017; 12 (1): 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.CDS Authoring Tool | CDS Connect. https://cds.ahrq.gov/cdsconnect/authoring. Accessed August 3, 2021.
  • 63.SMART Markers—A Framework for Patient Generated Data. SMART Health IT. 2019. https://smarthealthit.org/smart-markers-a-framework-for-patient-generated-data/. Accessed August 3, 2021.
  • 64. Kukhareva P, Warner P, Rodriguez S, et al.  Balancing functionality versus portability for SMART on FHIR applications: case study for a neonatal bilirubin management application. AMIA Annu Symp Proc  2019; 2019: 562–71. [PMC free article] [PubMed] [Google Scholar]
  • 65. Payne TH, Corley S, Cullen TA, et al.  Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc  2015; 22 (5): 1102–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66. Horvath K, Sengstack P, Opelka F, et al. ; Association of American Medical Colleges. The vision for a person-centered health information system. NAM Perspectives  2018; 8 (10): 13. [Google Scholar]
  • 67. Melnick ER, Holland WC, Ahmed OM, et al.  An integrated web application for decision support and automation of EHR workflow: a case study of current challenges to standards-based messaging and scalability from the EMBED trial. JAMIA Open  2019; 2 (4): 434–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ocac059_Supplementary_Data

Data Availability Statement

The data underlying this article will be shared on reasonable request to the corresponding author.


Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES