Consumer health informatics: results of a systematic evidence review and evidence based recommendations

Abstract

An increasing array of technology based tools are available for patient and consumer utilization which claim to facilitate health improvement. The efficacy of these Consumer Health Informatics tools has not previously been systematically reviewed. As such a systematic evidence review of the efficacy of consumer health informatics tools was conducted. This review also sought evidence of any barriers to future widespread utilization of these tools and evidence of economic impact of these tools on health care costs. The findings of this review indicate that while more work needs to be done, the available literature does suggest a positive impact of consumer health informatics tools on select health conditions and outcomes. Many barriers remain that must be overcome prior to widespread utilization of these tools. There was insufficient data regarding economic impact of consumer health informatics tools on healthcare costs.

INTRODUCTION

We are living in a time, in the evolution of the US healthcare system, when significant forces, including rising costs, a growing proportion of the population is over the age of 65, and the increasing prevalence of chronic disease, are suggesting a need for significant changes in the traditional healthcare system [1] Effective chronic medical care, unlike acute disease treatment, is often a much more collaborative process between patients and providers. It involves a much larger reliance on patient and caregiver engagement and shared decision making [1]. Recent reports on patient safety [2] and healthcare disparities [3], among others, document the inability of the current healthcare system to provide consistent high-quality care to every patient. In addition, a review of 31 national surveys found that two thirds of Americans do not actively and consistently perform the actions directly linked to benefiting from available healthcare [4]. At the same time, approximately 10 million people on any given day or by 2009, a total of approximately 175 million Americans have used the Internet to search for health information [5]. Taken together, these data suggest that there is both great patient need and consumer appetite for resources, especially electronic resources, to enable and improve their engagement in healthcare.

Although one previous review did evaluate barriers and drivers of utilization (6), prior to the present study, a comprehensive, systematic evaluation of the efficacy of these types of electronic tools has not previously been undertaken. In 2001, the term Consumer Health Informatics was introduced in an attempt to describe a focus on patient oriented information needs [6]. For the purposes of this review, consumer health informatics applications or tools were defined as any electronic tool, technology, or system that is: (1) primarily designed to interact with health information users or consumers (anyone who seeks or uses healthcare information for nonprofessional work), (2) interacts directly with the consumer who provides personal health information to the CHI system and receives personalized health information from the tool application or system, and (3) is one in which the data, information, recommendations, or other benefits provided to the consumer, may be used in coordination with a healthcare professional but is not dependent on a healthcare professional [7]. Here, we distinguish patients (individuals who are already ill) from consumers (individuals who are not ill).

To understand the potential impact of these consumer health informatics tools, we conducted a Systematic Evidence Review of the available scientific literature. For the purposes of this review, we have excluded point of care devices (e.g., glucometer, remote monitoring devices), prescribed clinical devices that are part of the provision of clinical care, general information websites, message boards, and applications that are designed for use in a healthcare delivery environment. This definition has the following advantages: first, it keeps the focus of the review on how CHI applications meet the needs of consumers rather than the needs of clinicians. Second, it helps avoid a categorical disease-oriented evaluation of every clinical technological development for every disease which is not necessarily focused on the needs of consumers. Third, it helps to keep the focus of the review on studies that demonstrate impact, value or efficacy from the perspective of consumers. Finally, it facilitates categorization of CHI applications in ways that may be more meaningful for patients.

Potential categories of CHI tools or applications included in the review, but not be limited to, are (a) applications that facilitate knowing, tracking, or understanding clinical parameters (disease management); (b) applications that facilitate knowing/tracking/understanding observations of daily living; (c) applications and technologies that facilitate calendaring (lifestyle management assistance); (d) applications and tools that facilitate prevention and health promotion; (e) applications that facilitate self-care; and (f) applications that facilitate assisted care and care giving.

Primary objectives of this review were to review and synthesize the available evidence regarding the impact of currently developed CHI applications on health and healthcare processes and outcomes, identify barriers to the use of CHI applications, identify the gaps in published information on costs, benefits, and net value of these applications, and finally, to identify what critical information is needed for consumers, their families, clinicians, and developers to clearly understand the value of CHI applications.

METHODS

A core team of experts, who have strong expertise in clinical and health sciences informatics, clinical trials, systematic literature reviews, epidemiological studies, and general medicine was assembled along with two advisors who have done extensive research in the areas of open access, health policy, eHealth, and CHI. Additionally seven external technical experts from diverse professional backgrounds including consumer advocates, research methodologies ethics, decision aids, CHI, to CHI user acceptance were identified. Finally, two additional peer reviewers, who were not otherwise involved in the project, were identified to provide comments and feedback on the review.

The core team worked with the external advisors, technical experts, and representatives of the Agency for Healthcare Research and Quality (AHRQ), which commissioned the review, to refine a set of guiding questions The final key questions to be answered by this study included are shown in Table 1.

Table 1.

Key Questions

1) What evidence exists that CHI applications impact

a) Health care process outcomes (e.g., receiving appropriate treatment) among users,

b) Intermediate health outcomes (e.g., self-management, health knowledge, and health behaviors) among users,

c) Relationship-centered outcomes (e.g., shared decision making or clinician-patient communication) among users,

d) Clinical outcomes (including quality of life) among users, e) Economic outcomes (e.g., cost and access to care) among users?

2) What are the barriers that clinicians, developers, consumers, and their families or caregivers encounter that limit utilization or implementation of CHI applications?

3) What knowledge or evidence exists to support estimates of cost, benefit, and net value with regard to CHI applications?

4) What critical information regarding the impact of CHI applications is needed to give consumers, their families, clinicians, and developers a clear understanding of the value proposition particular to them?

To answer Key Question 1, we reviewed research employing randomized controlled trial (RCT) study designs. To answer Key Question 2, we reviewed research that was designed to look at barriers to use of CHI employing any study design. All research identified in the process of evaluating Key Questions 1 or 2 was also used to evaluate Key Questions 3 and 4.

The study team developed the conceptual model, shown in Fig. 1, to address the key questions (above). The goals of the model were to direct our review of the relevant literature and to assist reviewers in understanding which articles applied to our strict criteria for inclusion.

Fig. 1.

CHI conceptual model

Searching the literature involved identifying reference sources, formulating a search strategy for each source, and executing and documenting each search. For the searching of electronic databases, we used medical subject heading (MeSH) terms. To identify articles that that were potentially relevant to Key Question 1, we searched for terms relevant to our definition of CHI applications, combined with terms relevant to our definition of “consumer,” combined with terms identifying RCTs as the study design of interest. To identify articles that that were potentially relevant to Key Question 2, we searched for terms relevant to our definition of CHI applications, combined with terms relevant to barriers; the search was not limited by study design. Eligible studies were also identified by reviewing the references in pertinent reviews and by querying our experts.

Our comprehensive search included electronic searching of peer-reviewed literature and gray literature databases as well as hand searching. On December 22, 2008, we conducted searches of the MEDLINE®, EMBASE®, The Cochrane Library, Scopus, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases. This search was while review findings were reviewed by AHRQ and external experts to ensure we included the most current relevant articles; this search was extended to June 1, 2009. A supplemental search targeting gray literature sources was conducted on January 7, 2009; it was also extended to June 1, 2009. Sources searched were: Health Services Research Projects in Progress, Institute of Electrical and Electronics Engineers (IEEE) Conference Proceedings, Institution of Engineering and Technology (IET) Conference Proceeding, Proceedings of the American Society for Information Science and Technology (Wiley InterScience), World Health Organization (WHO)-International Clinical Trials Registry Platform, American Public Health Association 2000–2008, OpenSIGLE-System for Information on Gray Literature in Europe, and The New York Academy of Medicine—Gray Literature (see Tables 2 and 3).

Table 2.

Detailed scientific literature search strategy

Database	Terms
PubMed	((“Medical Informatics Applications”[Mesh] OR “Informatics”[Mesh] OR “medical informatics”[mh] OR telemedicine[mh] OR informatics[tiab] OR internet[tiab] OR internet[mh] OR “Consumer Health Information”[Mesh] OR “Support systems”[tiab]) AND (consumer[tiab] OR “Patients”[Mesh] OR patients[tiab] OR patient[tiab] OR parents[mh] OR parents[tiab] OR parent[tiab] OR “age groups”[mh] OR Caregivers[mh] OR caregiver[tiab] OR “care giver”[tiab] OR “persons”[mh] OR persons[tiab] OR person[tiab] OR people[tiab] OR individual[tiab] OR individuals[tiab]) AND English[] AND (“randomized controlled trial”[pt] OR “randomized controlled trials as topic”[mh] OR “randomized controlled trial”[tiab] OR “randomized controlled trial”[tiab] OR “controlled trial”[tiab] OR “clinical trial”[tiab]) NOT (editorial[pt] OR letter[pt] OR comment[pt]) NOT (animal[mh] NOT human[mh]) AND ((“1900/01/01”[PDat] : “2009/06/01”[PDat])))
	OR
	((“Medical Informatics Applications”[Mesh] OR “Informatics”[Mesh] OR “medical informatics”[mh] OR telemedicine[mh] OR informatics[tiab] OR internet[tiab] OR “internet”[MeSH Terms] OR “Consumer Health Information”[Mesh] OR “Support systems”[tiab]) AND (consumer[tiab] OR “Patients”[Mesh] OR patients[tiab] OR patient[tiab] OR “parents”[MeSH Terms] OR parents[tiab] OR parent[tiab] OR “age groups”[mh] OR “caregivers”[MeSH Terms] OR caregiver[tiab] OR “care giver”[tiab] OR “persons”[mh] OR persons[tiab] OR person[tiab] OR people[tiab] OR individual[tiab] OR individuals[tiab]) AND (Access[tiab] OR barrier[tiab] OR facilitator[tiab] OR compatibility[tiab] OR incompatibility[tiab] OR “user-centered”[tiab] OR “user centered”[tiab] OR “work flow”[tiab] OR workflow[tiab] OR “reimbursement mechanisms”[mh] OR reimbursement[tiab] OR “attitude to computers”[mh] OR attitude[tiab] OR “health knowledge, attitudes, practice”[mh] OR “computer literacy”[mh] OR (computer[tiab] AND literacy[tiab])) AND English[lang] NOT (editorial[pt] OR letter[pt] OR comment[pt]) NOT (“animals”[MeSH Terms] NOT “humans”[MeSH Terms]) AND ((“1900/01/01”[PDat] : “2009/06/01”[PDat]))) AND ((“1900/01/01”[PDat]: “2009/06/01”[PDat]))
EMBASE	((‘informatics’:ti,ab OR telemedicine:ti,ab OR internet:ti,ab OR ‘consumer health information’:ti,ab) AND (consumer:ti,ab OR ‘patients’:ti,ab OR parents:ti,ab OR ‘age groups’:ti,ab OR caregivers:ti,ab) AND (‘randomized controlled trial’:ti,ab OR (controlled:ti,ab AND trial:ti,ab) OR (clinical:ti,ab AND trial:ti,ab))) OR ((‘informatics’:ti,ab OR telemedicine:ti,ab OR internet:ti,ab OR ‘consumer health information’:ti,ab) AND (consumer:ti,ab OR ‘patients’:ti,ab OR parents:ti,ab OR ‘age groups’:ti,ab OR caregivers:ti,ab) AND (access:ti,ab OR barrier:ti,ab OR facilitator:ti,ab OR compatibility:ti,ab OR incompatibility:ti,ab OR ‘user centered’:ti,ab OR ‘work flow’:ti,ab OR reimbursement:ti,ab OR attitude:ti,ab OR (computer:ti,ab AND literacy:ti,ab))) AND ([article]/lim OR [editorial]/lim OR [review]/lim) AND [english]/lim AND [humans]/lim
Cochrane library	((“Medical Informatics applications”:ti,ab,kw or “Informatics”:ti,ab,kw or (telemedicine):ti,ab,kw or (internet):ti,ab,kw or “Consumer Health Information”:ti,ab,kw or “Support systems”:ti,ab,kw) AND ((consumer):ti,ab,kw or “Patients”:ti,ab,kw or (parents):ti,ab,kw or “age groups”:ti,ab,kw or (Caregivers):ti,ab,kw) AND ((randomized controlled trial):ti,ab,kw or (controlled trial):ti,ab,kw or (clinical trial):ti,ab,kw))
	OR
	((“Medical Informatics applications”:ti,ab,kw or “Informatics”:ti,ab,kw or (telemedicine):ti,ab,kw or (internet):ti,ab,kw or “Consumer Health Information”:ti,ab,kw or “Support systems”:ti,ab,kw) AND ((consumer):ti,ab,kw or “Patients”:ti,ab,kw or (parents):ti,ab,kw or “age groups”:ti,ab,kw or (Caregivers):ti,ab,kw) AND ((Access):ti,ab,kw or (barrier):ti,ab,kw or (facilitator):ti,ab,kw or (compatibility):ti,ab,kw or (incompatibility):ti,ab,kw or “user centered”:ti,ab,kw or “work flow”:ti,ab,kw or Reimbursement:ti,ab,kw or “attitude to computers”:ti,ab,kw or “computer literacy”:ti,ab,kw))
SCOPUS	((TITLE-ABS-KEY(“Medical Informatics applications”) OR TITLE-ABS-KEY(telemedicine) OR TITLE-ABS-KEY(internet) OR TITLE-ABS-KEY(“Consumer Health Information”)) AND (TITLE-ABS-KEY(consumer) OR TITLE-ABS-KEY(“Patients”) OR TITLE-ABS-KEY(caregivers)) AND (TITLE-ABS-KEY(“randomized controlled trial”) OR TITLE-ABS-KEY(“clinical trial”))) OR ((TITLE-ABS-KEY(“Medical Informatics applications”) OR TITLE-ABS-KEY(telemedicine) OR TITLE-ABS-KEY(internet) OR TITLE-ABS-KEY(“Consumer Health Information”)) AND (TITLE-ABS-KEY(consumer) OR TITLE-ABS-KEY(“Patients”) OR TITLE-ABS-KEY(caregivers)) AND (TITLE-ABS-KEY(access) OR TITLE-ABS-KEY(barrier) OR TITLE-ABS-KEY(facilitator) OR TITLE-ABS-KEY(“user centered”) OR TITLE-ABS-KEY(“attitude to computers”) OR TITLE-ABS-KEY(“computer literacy”) OR TITLE-ABS-KEY(“health knowledge, attitudes, practice”))) AND (LIMIT-TO(DOCTYPE, “ar”) OR LIMIT-TO(DOCTYPE, “re”) OR LIMIT-TO(DOCTYPE, “rp”)) AND (LIMIT-TO (LANGUAGE, “English”))
CINAHL	((TX “Informatics” or TX telemedicine or TX internet or TX “Consumer Health Information” or TX “Support systems”) AND (TX consumer or TX “Patients” or TX parents or TX “age groups” or TX Caregivers) AND (TX “randomized controlled trial” or TX “controlled trial” or TX “clinical trial”)) OR ((TX “Informatics” or TX telemedicine or TX internet or TX “Consumer Health Information” or TX “Support systems”) AND (TX consumer or TX “Patients” or TX parents or TX “age groups” or TX Caregivers) AND (TX Access or TX barrier or TX facilitator or TX compatibility or TX incompatibility or TX “user centered” or TX “work flow” or TX Reimbursement or TX Attitude or TX “computer literacy”)) NOT ((PT editorial)or (PT letter) or (PT comment))

Table 3.

Detailed gray literature search strategy

Database	Terms
Health services research projects in progress	(((informatics OR internet OR consumer health information) AND (consumer OR patients OR parents OR caregivers) AND (randomized controlled trial OR clinical trial)) OR ((informatics OR internet OR consumer health information) AND (consumer OR patients OR parents OR caregivers) AND (access OR barrier OR facilitator OR compatibility OR user centered)))
IEEE CNF IEEE conference proceeding	((((((informatics or internet or consumer health information) and (consumer or patients or parents or caregivers) and (randomized controlled trial or clinical trial)) or ((informatics or internet or consumer health information) and (consumer or patients or parents or caregivers) and (access or barrier or facilitator or compatibility or user centered))))<in>metadata)) <and> (pyr>= 1990 <and> pyr <= 2009)
IET CNF IET conference proceeding
Proceedings of the American society for information science and technology (Wiley InterScience)	Informatics OR “health information” OR “consumer health information” OR internet
WHO-international clinical trials registry platform	Informatics applications OR consumer health information OR internet
American public health association (APHA)	Consumer health information OR health information OR consumer
2000–2008
OpenSIGLE—system for information on gray literature in Europe	(((informatics OR internet OR consumer health information) AND (consumer OR patients OR parents OR caregivers) AND (randomized controlled trial OR clinical trial)) OR ((informatics OR internet OR consumer health information) AND (consumer OR patients OR parents OR caregivers) AND (access OR barrier OR facilitator OR compatibility OR user centered)))
The New York academy of medicine—gray literature	Informatics OR “consumer health information” OR “health information application”

Search strategies specific to each database were designed to enable the team to focus the available resources on articles that were most likely to be relevant to the Key Questions (see Table 1). We developed a core strategy for MEDLINE®, accessed via PubMed, on the basis of an analysis of the MeSH terms and text words of key articles identified a priori. The PubMed strategy formed the basis for the strategies developed for the other electronic databases.

Studies were eligible for inclusion in the review if they applied to Key Questions 1 or 2 and did not have one of the following reasons for exclusion: no health informatics application, health informatics application does not apply to the consumer, health informatics applications is for general information only (e.g., general Web site) and is not tailored to individual consumers, study of a “point of care” device (defined as requiring a clinician to use or obtain and is part of the regular provision of care), or no original data.

We assessed the eligible studies on the basis of the quality of their reporting of relevant data. For the RCTs, we used the study quality scoring system developed by Jadad et al. [8]. For the other studies, we used a form to identify key elements that should be reported when reporting results. The quality assessments were done independently by paired reviewers.

We then created a set of detailed evidence tables containing information extracted from the eligible studies. We stratified the tables according to the applicable Key Question and subquestion (for Key Question 1). We did not quantitatively pool the data for any of the outcomes because of the marked heterogeneity of target conditions of interest and the wide variety of outcomes studied. Data were abstracted by one investigator and entered into online data abstraction forms using SRS (Mobius Analytics, Inc., Ottawa, Ontario, CA). Second reviewers were generally more experienced members of the research team, and one of their main priorities was to check the quality and consistency of the first reviewers’ answers.

At the completion of our review, we graded the quantity, quality, and consistency of the best available evidence for each type of outcome in each clinical area, using an evidence grading scheme recommended by the GRADE Working Group and modified for use by the Evidence-Based Practice Centers Program [9]. For each outcome of interest, two investigators independently assigned a grade, and then the entire team discussed their recommendations and reached a consensus.

RESULTS

As shown in Fig. 2, the literature search process identified 24,794 citations that were deemed potentially relevant to Key Questions 1 and/or 2 and 6,673 additional articles were identified through hand searching. We identified no additional eligible articles in the gray literature. We excluded 8,943 duplicate citations from the electronic search results. Most duplicates came from concurrently searching MEDLINE®, The Cochrane Library, EMBASE®, CINAHL, and SCOPUS. The search strategy used in all search engines was modeled after that which we used in MEDLINE®, with similar search terms. Additionally, the EMBASE® search engine allows the user to search the MEDLINE® database as well as EMBASE®, a strategy that often yields many duplicates between the two search sites but improves the sensitivity of the search.

Fig. 2.

Flow diagram

In the title review process, manuscript titles were reviewed by two investigators to preliminarily assess manuscript relevance. We excluded 19,377 citations that clearly did not apply to the Key Questions. In the abstract review process, we excluded 2,642 citations that did not meet one or more of the eligibility criteria. At the article review phase, we excluded an additional 357 articles that did not meet one or more of the eligibility criteria. Two more articles were removed from the pool of articles because the articles could not be located through any of the cooperating libraries or the manuscript authors. Ultimately, we were left with 146 articles that were eligible for inclusion in this report: One hundred and twenty-one for Key Question 1 and 31 for Key Question 2; six articles were eligible for both Key Questions 1 and 2. The complete list of publications has been published in the full report [7].

In terms of types of applications studied, 55% of studies evaluated interactive web-based applications or tailored educational websites. Another 15% of studies evaluated computer-generated tailored feedback applications. Interactive computer programs and personal monitoring devices were evaluated in approximately 8% of studies each. Finally, health risk assessments, decision aids, discussion or chat groups, and computer-assisted imagery were evaluated in less than 5% of studies each.

Ninety-nine studies reported user age, 77% (76/99) of those studies reporting age of participants targeted adults, approximately 12% of studies targeted adolescents, 3% of studies targeted seniors, and another 3% of studies targeted children. Five percent of studies targeted participants from overlapping age groups. Among studies reporting the race of the participants (n = 53), 92% (49/53) of the studies employed populations that were greater than 50% Caucasian. There was only one study with greater than 50% African-American participants and no studies with a majority of participants who were Hispanic, American Indian/Alaska Native, or Asian/Pacific Islander. Fifty-eight percent of studies reporting delivery location evaluated CHI applications that were used in the home or residence. A minority of evaluations were completed in schools (15%), clinical settings (17%), communities (3%), online (5%), or kiosks (2%).

The impact of CHI applications on health outcomes

The impact of CHI applications on healthcare process outcomes

There were only six studies that met the inclusion–exclusion criteria and thus were available to shed light on this question. Five of these studies focused on asthma and one additional study focused on contraceptive medication utilization. All of the asthma studies showed a significant positive effect of the CHI application on at least one healthcare process measure. The oral contraceptive medication use application failed to reduce contraceptive discontinuation. No study found any evidence of harm.

The impact of CHI applications on intermediate health outcomes (Key Question 1b)

This review identified 108 studies that addressed the influence of CHI applications on intermediate health outcomes in the context of nine categories of diseases or health conditions. These were breast cancer in three studies, diet, exercise, and physical activity (not obesity) in 32 studies, alcohol abuse in seven studies, smoking cessation in 19 studies, obesity in 11 studies, diabetes mellitus (or diabetes with associated conditions) in seven studies, mental health in eight studies, asthma/chronic obstructive pulmonary disease (COPD) in four studies, and miscellaneous health conditions in another 15 studies.

With regard to breast cancer, evaluated intermediate outcomes included social support, information competence, level of conflict, and satisfaction. All three studies reported significant positive effect on at least one intermediate health outcome. No study found any evidence of harm.

In terms of diet, exercise, and physical activity, not obesity, evaluated intermediate outcomes included self-management, knowledge, program adherence, and change in health behaviors. Eighty-nine percent of these studies demonstrated significant positive effect on at least one intermediate health outcome related to diet, exercise, and physical activity. No study found any evidence of harm.

Evaluated intermediate outcomes related to alcohol abuse included self-management, knowledge attainment, and change in health behaviors. All seven studies found significant positive effect on at least one intermediate outcome related to alcohol abuse. No study found any evidence of harm.

With regard to smoking cessation, intermediate outcomes assessed in these smoking cessation CHI trials included self-management, knowledge attainment, and change in health behaviors. Fifty-seven percent of these studies demonstrated a positive effect on at least one intermediate outcome related to smoking cessation. No study found any evidence of harm.

Evaluated intermediate outcomes of interest related to obesity included weight loss behaviors and body composition. Only 36% of studies demonstrated positive effect on intermediate outcomes related to obesity. No study found any evidence of harm.

Seven studies were identified to evaluate the influence of CHI on intermediate outcomes related to diabetes mellitus. Intermediate outcomes of interest included perceived self-efficacy, satisfaction, and readiness to change, perceived competence, exercise minutes per day, and self-reported global health. All seven studies found evidence of effect of CHI applications on one or more intermediate outcomes related to diabetes mellitus. No study found any evidence of harm.

Eight studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to mental health issues. Intermediate outcomes of interest included work and social adjustment, perceived stress, self-rated self-management, sleep quality, mental energy, and concentration. Seven of the eight studies found evidence of positive effect of CHI applications on at one or more intermediate outcomes related to mental health. No study found any evidence of harm.

Four studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to asthma/COPD. Intermediate outcomes of interest included adherence, knowledge, change in behavior, dyspnea knowledge, and self-efficacy. Only one of the four studies demonstrated a significant effect on any intermediate outcome related to asthma/COPD. No study found any evidence of harm.

Two studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to menopause or hormone replacement therapy (HRT). Only one study found evidence of significant effect on an intermediate outcome related to menopause/HRT utilization.

Finally, an additional 15 studies were identified to evaluate the influence of intermediate health outcomes in other clinical areas. These intermediate outcomes were in health areas related to arthritis, back pain, behavioral risk factor control, contraception, cardiovascular disease, cancer, caregiver decision making, fall prevention, health behavior change, headache, HIV/AIDS, and adolescent risk behaviors. Each of these studies found evidence of significant effect of the CHI application on intermediate outcomes related to the health condition under study. No study found evidence of harm.

The effect of CHI applications on relationship-centered outcomes (Key Question 1c)

Eight studies were identified that met the inclusion–exclusion criteria. Relationship centered outcomes of interest included social support, quality of life, decision making skill, social support, positive interaction with the provider, and satisfaction with care. These relationship-centered outcomes were evaluated in the context of care for HIV/AIDS, cancer, osteoarthritis, and pregnancy. Five of eight studies demonstrated significant effect of CHI on at least one aspect of relationship-centered care. No study found any evidence of harm.

The impact of CHI applications on clinical outcomes (Key Question 1d)

Twenty-eight studies addressed this question in the context of care for cancer (three studies), diabetes mellitus (three studies), mental health (seven studies), diet, exercise, or physical activity (five studies), and Alzheimer’s disease, arthritis, asthma, back pain, aphasia, COPD, HIV/AIDS, headache, obesity, and pain (one study each). Over 80% of the studies found significant influence of CHI applications on at least one clinical outcome.

Three studies evaluated the effect of CHI applications on breast cancer clinical outcomes, but only one found any evidence of significant CHI impact. Of the five studies that evaluated the effect of CHI applications on clinical outcomes related to diet, exercise, or physical activity, four studies found a significant positive effect on one or more clinical outcomes. Among the seven studies that evaluated the effect of CHI applications on mental health clinical outcomes, all seven found evidence of significant effect of CHI on one or more clinical outcomes. Three studies evaluated the effect of CHI applications on diabetes mellitus clinical outcomes. All three studies found evidence of significant effect of CHI on at least one clinical outcome. The remaining nine studies evaluated a CHI application in different health areas including Alzheimer’s disease, arthritis, asthma, back pain, aphasia, COPD, headache, HIV/AIDS, and general pain. With the exception of the general pain study, the eight remaining studies all found evidence of significant effect of CHI on one or more clinical outcomes. None of these 27 studies found any evidence of harm attributable to a CHI application.

The impact of CHI applications on economic outcomes (Key Question 1e)

Three studies addressed this question. Economic outcomes evaluated in these studies included cost of program delivery, cost of computer information system with manual data extraction versus cost of the computer system with use of an electronic patient record, materials costs, total costs, and incremental cost effectiveness. These outcomes were evaluated in the context of care for asthma, cancer, and obesity. Each of these studies used different economic metrics and methodologies. One study failed to provide any cost estimates for the control group. One study was done in an adult population, another in a pediatric population, and the third study did not provide any details regarding the age of study participants.

Barriers that limit utilization or implementation of CHI applications for clinicians, developers, consumers, and their families or caregivers encounter

Thirty-one studies addressed the question of barriers to CHI application use. Studies focused on a wide variety of clinical conditions including cancer, HIV/AIDS (and sexually transmitted disease), mental health, physical activity/diet/obesity, smoking cessation, prostate cancer, and hypertension. Because CHI applications involve the participation of consumers, their caregivers, may also include clinicians, and developers, this analysis included barriers that impede participation of any of the above groups. Identified barriers were grouped into healthcare system-level barriers and individual-level barriers. Six studies addressed systems-level barriers including Internet access at home or in the community and all six found this to be a barrier. One study identified hardware requirements and another study identified mobile device shape, design, or configuration as a systems-level barrier to use. Another five studies cited CHI tool incompatibility with current healthcare as a barrier.

Identified individual-level barriers included clinic staff who feared increased workloads, lack of built-in social support, forgotten passwords, automated data entry inability to allow for back entry of old data, lack of adequate user customization, and substantial financial investment. Nineteen studies queried application usability or user-friendliness and all 19 found evidence of lack of usability as a barrier to use. Eleven studies explored how patient knowledge, literacy, and computer skills could impact the use the CHI application. Ten found deficits in knowledge, literacy, or computer skills to be barriers. Six studies considered the possibility that users would find the application too time-consuming. Five of these studies simply cited this possibility in the “RESULTS” section of the manuscripts, while the one of these studies actually reported that “too many emails to participants” was found to be a barrier.

Utilization fees were also identified as a barrier. Five studies sought information about privacy concerns and four reported concerns over privacy as a barrier. These studies also found concerns over the control of information or lack of trust in the technology to be barriers. Only two studies queried for potential cultural barriers to use and one study found evidence of this. The expectations of consumers including acceptability, usefulness, credibility, expectations, and goals were found to be barriers in eight studies. Cost was mentioned as a barrier in only one study, and only one study found evidence that physical or cognitive impairment resulted in barriers to the use of CHI applications. Finally, anxiety over the use of computers, complaints about lack of personal contact with clinicians, and the belief that health IT would not be an improvement to current care were mentioned in two studies as barriers.

Knowledge or evidence needs to support estimates of cost, benefit, and net value with regard to consumer health informatics applications

The identified studies indicate that the available literature is at a very early stage of development. Many questions have only been evaluated by one study. Thus, confirmatory studies have generally not been done. In addition, no high-quality studies have been conducted regarding several important questions. Broadly, these questions can be grouped into at least one of four categories: patient-related questions; CHI utilization factors; technology-related issues (i.e., hardware, software, and platform related issues and health-related questions); and health-related questions.

Patient-related questions

The literature is relatively silent on the question of whether or not significant differences in patient preferences, knowledge, attitudes, beliefs, needs, utilization, and potential benefits exists across gender, age, and race/ethnicity. Beyond these demographic differences, the field of CHI is developing within the context of a global emergence of technology-based realities including the emergence of Web 2.0/Web 3.0 and ubiquitous computing, which are enabling an unprecedented level of user-determined content, interactivity, and functionality. The degree to which this functionality could be harnessed for the health benefit of consumers is unknown. The targeted uses of CHI applications must increasingly be focused on more than just the index patient. The role of sociocultural and community factors will likely exert significant effect on access, usability, desirability, and benefit of CHI applications. Issues related to trust, security, and confidentiality need to be further explored. Because the bulk of the currently available research has been conducted on the 18- to 65-year-old adult population, more work needs to be done among the populations that may have the most potential for using CHI applications. Seniors may stand to benefit from those applications that reduce social isolation to independence.

Adolescents are some of the most intense technology users. Their natural affinity for technology may prove advantageous to CHI applications that could be developed in the future. Finally, most of the current CHI research is being conducted among predominately white/Caucasian populations. Early evidence suggests that differential utilization patterns and preferences exist by race [10, 11]. Such differences could potentially lead to differential efficacy of emerging CHI applications. This could have the unintended consequence of enhancing rather than reducing some racial and ethnic disparities in healthcare. Age, gender, and race/ethnicity subgroup differences need to be netter understood and those differences incorporated into the development of emerging applications to ensure efficacy among all population subgroups.

CHI utilization factors

Despite a rapid increase in access to broadband services among all population groups, age groups, and geographic regions of the country, differential access to broadband Internet access may have significant implications in terms of health benefits that may be derived from these tools to applications. While many in the younger generations become very technically savvy at an early age, many Americans still have limited computer literacy. These CHI utilization factors suggest the need for a more robust evaluation of the epidemiology of broadband access and technology literacy in the USA [11, 12].

Technology-related issues

The majority of CHI applications are designed for use on personal computers as web-based applications. Many more potential types of platforms exist that have not been evaluated. In addition, emerging evidence is suggesting that the CHI applications and functionality that consumers want and need are not always what healthcare practitioners think they need. As a result, important human factors considerations (graphics vs. text based interfaces, mobile vs. desktop devices) may not get incorporated into emerging CHI applications and, therefore, lead to CHI applications with limited efficacy.

Health-related questions

Finally, most CHI applications that have been evaluated tend to focus on one or more domains of chronic disease management. Insufficient attention has been given to the role of CHI applications in addressing acute health problems. The role of CHI applications in primary, secondary, and tertiary prevention also needs to be more adequately explored. Sociocultural factors are increasingly important determinants of healthcare outcomes. The potential influence on social factors including social isolation and social support and perhaps even broader social determinants of health need to be evaluated and may prove useful in helping consumers address specific health concerns in the home and community-based setting.

Information needed to give consumers, their families, clinicians, and developers a clear understanding of the value proposition of CHI particular to them

Several critical information needs must be addressed to enable a clear understanding of the value proposition of CHI applications. It is likely that the knowledge gaps needed to establish a value proposition, while overlapping, are not identical across all potential stakeholders. Because providers are often most concerned about clinical outcomes and costs, it seems reasonable that questions of the impact of CHI applications on provider or healthcare processes, costs, and outcomes as addressed in this report will need to be more definitively characterized. In addition, the potential liability a provider might incur from a patient using a CHI application will also need to be addressed.

Patients often cite convenience and anonymity as the primary reasons the Internet has become such a major source of health information [10]. It is likely that the more these elements can be incorporated into emerging CHI applications, the more likely they will be considered of value by consumers. Other related factors such as usability, portability, and patient-centered functionality are likely important characteristics of CHI applications that may help drive utilization. Those technologies that exist and enable consumers to accomplish tasks (empower) without further complicating individuals’ lives may ultimately prove to be the most widely valued CHI applications. By expanding the number of and types of platforms available to consumers, CHI applications may become more appealing to a broader consumer base and thus prove valuable to those consumers who could most benefit but may not otherwise use a more traditional CHI application.

DISSCUSION

The results of this SER suggest several emerging themes. First, there may be a role for CHI applications in reaching consumers at a low cost and also in obviating the need for some activities currently performed by professionals. In addition, the data suggest that CHI applications may also be used to enhance the efficacy of interventions currently delivered by professionals. Several studies compared the use of a CHI application and traditional therapy against traditional therapy alone and found that the group receiving traditional therapy with a CHI application had more benefit than traditional therapy alone. Thirdly, the studies evaluated in this review tended to support the finding that at least three critical elements are most often found in effective CHI tools and applications including [7] individual tailoring, [1] personalization, and [2] behavioral feedback. Personalization involves designing the intervention to be delivered in a way that makes it specific for a given individual. Tailoring refers to building an intervention in part on specific knowledge of actual characteristics of the individual receiving the intervention. Finally, behavioral feedback refers to providing consumers with messages regarding their progression through the intervention. Interestingly, it is not clear from this literature that CHI-derived behavioral feedback is any better than feedback originating from human practitioners or others. Rather, it appears that the feedback must happen with an appropriate periodicity, in a format that is appealing and acceptable to the consumer.

Finally, despite the previously cited limitation of the available scientific literature, the body of the available scientific evidence suggests that CHI applications may hold significant future promise for improving outcomes across a variety of diseases and health issues. In terms of healthcare processes and relationship-centered outcomes, the literature is positive but very limited. Most of the currently available research has evaluated the impact of CHI applications on intermediate health outcomes. The literature appears strongest for CHI applications targeting intermediate outcomes related to smoking cessation. In terms of clinical outcomes, the weight of the evidence appears strongest for the use of CHI applications on mental health outcomes. Evidence-based conclusions regarding economic outcomes cannot be made at this time.

Study limitations

This review has several important limitations. First, our initial search for eligible studies proved to be challenging because of inconsistent use of terminology in the literature. We minimized this problem by searching multiple databases and supplementing our search with a review of selected journals and querying experts. The most important limitation was marked heterogeneity of interventions, populations, and outcomes, making synthesis across studies difficult and precluding meta-analysis. Inconsistent definitions and reporting of outcome measures further limited our ability to synthesize data, as many studies did not report enough data to support calculation of effect sizes. Methodologic (limited sample size, randomization scheme not specified, poor adjustment for potential confounders, etc.) limitations of many of the RCTs limit the strength of conclusions. Usually because of the relatively small number of available studies, but sometimes also due to variability in the quality of available studies, the strength of the body of evidence was often graded as low. Finally, there are several ongoing CHI studies that have not yet reported. This evidence report may need to be updated when the results of these studies are available.