Interventions for enhancing consumers' online health literacy

Abstract

Background

Access to health information is critical to enable consumers to participate in decisions on health. Increasingly, such information is accessed via the internet, but a number of barriers prevent consumers making effective use of it. These barriers include inadequate skills to search, evaluate and use the information. It has not yet been demonstrated whether training consumers to use the internet for health information can result in positive health outcomes.

Objectives

To assess the effects of interventions for enhancing consumers' online health literacy (skills to search, evaluate and use online health information).

Search methods

We searched: the Cochrane Consumers and Communication Review Group Specialised Register; Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library, Issue 1 2008); MEDLINE (Ovid); EMBASE (Ovid); CINAHL (Dialog); ERIC (CSA Illumina); LISA (CSA Illumina); PsycINFO (Ovid); Index to scientific and technical proceedings; SIGLE; ASLIB Index to Theses; ProQuest Dissertation Abstracts; National Research Register/UK CRN Portfolio database; Current Controlled Trials ‐ MetaRegister of Controlled Trials. We searched all databases for the period January 1990 to March 2008.

Selection criteria

Randomised controlled trials (RCTs), cluster RCTs and associated economic evaluations, quasi‐RCTs, interrupted time series analyses, and controlled before and after (CBA) studies assessing interventions to enhance consumers' online health literacy, in any language.

Data collection and analysis

Two review authors independently selected studies for inclusion, assessed their quality and extracted data. We contacted study authors for clarification and to seek missing data. We presented results as a narrative and tabular summary, and calculated mean differences where appropriate.

Main results

We included two studies: one randomised controlled trial (RCT) and one controlled before and after (CBA) study with a combined total of 470 participants. The RCT compared internet health information classes with patient education classes for participants with HIV infection. Only the RCT, which we rated as having a moderate risk of bias, reported statistically significant positive effects for primary outcomes related to online health literacy in the intervention group, for the following outcomes: 'Self‐efficacy for health information seeking', 'health information evaluation skills' and the 'number of times the patient discussed online information with a health provider'. The CBA, which we rated as having a high risk of bias, compared internet health information classes with a control group receiving no intervention among healthy adults aged 50+. It showed significant positive changes only in a secondary (behavioural) outcome in the intervention group, regarding the readiness to adopt the internet as a tool for preventive health information. No adverse effects were reported.There is low quality evidence that such interventions may improve some outcomes relevant to online health literacy in certain populations.

Authors' conclusions

Due to the small number of studies and their variable methodological quality, the evidence is too weak to draw any conclusions about implications for the design and delivery of interventions for online health literacy. There is a need for well‐designed RCTs to investigate the effects of such interventions. These should involve different participants (in terms of disease status, age, socio‐economic group and gender) to analyse the extent to which online health literacy reduces a barrier to using the internet for health information. Trials should be conducted in different settings and should examine interventions to enhance consumers' online health literacy (search, appraisal and use of online health information) like internet training courses, measuring outcomes up to at least one year after the intervention to estimate the sustainability of the intervention effects.

Interventions for enhancing consumers' online health literacy

Access to health information allows people to take an active part in their health care. The internet is an increasingly popular way for people to obtain health information, but there are a number of barriers that prevent people making full use of such information. In particular, people may not be motivated to use the internet, or they may not have the skills that they need. This review assessed whether teaching people to find, evaluate or use online health information (online health literacy) improves those skills and improves their health.

Only two studies met the inclusion criteria for this review. Both studies investigated the effect of adult education classes on the online health literacy of consumers. There is low quality evidence that these interventions improve aspects of online health literacy, specifically, regarding the outcomes 'self‐efficacy for health information seeking', 'health information evaluation skills', 'number of times the patient discussed online information with a health provider' and 'readiness to adopt the internet as a tool for preventive health information'. No adverse effects were reported. As the study participants were, respectively, HIV‐infected consumers, and people aged over 50, there is limited evidence on which to draw conclusions about the effect of these interventions on other population groups. More high quality studies on this topic are necessary.

Summary of findings

Summary of findings for the main comparison.

Interventions for enhancing the skills of consumers to find, evaluate and use online health information
Patient or population: Adult consumers Settings: United States of America: Atlanta, Georgia Intervention: Interventions for enhancing the skills of consumers to find, evaluate and use online health information
Outcomes	Assumed risk*	Corresponding risk	Relative effect (95% CI)	No of Participants (studies)	Quality of the evidence (GRADE)	Comments
	Control	Information and Computer Technology interventions for enhancing the skills of consumers to find, evaluate and use online health information
Self‐efficacy for health information seeking 4‐item Likert scale. Scale from: 0 to 10. (follow‐up: 9 months)	The mean self‐efficacy for health information seeking in the control groups was 5.2 points	The mean self‐efficacy for health information seeking in the intervention groups was 1.10 points higher (0.64 to 1.56 higher)	‐	448 (1)	⊕⊕⊝⊝ low1
Health information evaluation skills Difference in the quality ratings given to web pages. Scale from: ‐9 to 9. (follow‐up: 3 months)	The mean health information evaluation skill in the control groups was 1 point	The mean health information evaluation skill in the intervention groups was 0.6 points higher (0.11 to 1.09 higher)	‐	448 (1)	⊕⊕⊝⊝ low1
Number of times the patient discussed online information with a health provider Episodes the consumer had discussed online information with a health provider in the previous 3 months. Scale from: 0 to unlimited. (follow‐up: 9 months)	The mean number of times the patient discussed online information with a health provider in the control groups was 0.1 episodes	The mean number of times the patient discussed online information with a health provider in the intervention groups was 0.7 episodes higher (0.22 to 1.18 higher)	‐	448 (1)	⊕⊕⊝⊝ low1
Ability to search for online health information ‐ not measured	See comment column	See comment column	Not estimable	‐	See comment column	This criterion was not measured in the included studies
Knowledge and understanding of how to find, evaluate and use online health information ‐ not measured	See comment column	See comment column	Not estimable	‐	See comment column	This criterion was not measured in the included studies
Computer literacy ‐ not measured	See comment column	See comment column	Not estimable	‐	See comment column	This criterion was not measured in the included studies
Adverse events	See comment column	See comment column	Not estimable	‐	See comment column	No harms were reported
*The assumed risk is the final mean score in the control condition. The corresponding risk (and its 95% CI) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). CI: Confidence interval GRADE: GRADE Working Group grades of evidence

¹ Quality of evidence downgraded due to unclear reporting of allocation concealment, high loss to follow‐up and indirectness of evidence (due to the diseases status of the participants).

GRADE Working Group grades of evidence High quality (++++): Further research is very unlikely to change our confidence in the estimate of effect. Moderate quality (+++O): Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low quality (++OO): Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low quality (+OOO): We are very uncertain about the estimate.

Background

The importance of online consumer health information

Access to health information is an important pillar of consumer empowerment. Access to high quality information enables consumers to participate in their health decisions and increases patient satisfaction, health knowledge, recall of clinical consultations, and understanding of the risks and benefits of treatment (Coulter 2006a; Coulter 2006b). Whilst consumers consistently report that their preferred source of health information is a health professional, health information they receive from professionals seems to be inadequate or insufficient (Chen 2001; Coulter 1999; Saver 2007). Limited consultation times also increase the need for, and reliance on, information provided outside the clinical encounter. Online health information is available in overwhelming diversity, steadily increasing as internet use does. Internet use is broadly similar in all Western countries (Morahan‐Martin 2004), but is much less common in lower income countries. For example in 2010, 8.8 out of 100 people in Sub Saharan Africa and 5.5 out of 100 people in the 'South Asia' region used the Internet (World Bank 2011). In comparison, in the European Union 67.4 out of 100 people, and in the USA 75.8 out of 100 people used the internet (World Bank 2011). Up to 80% of internet users in the US have searched for health information (Baker 2003; Dutton 2007; Fox 2002; Powell 2006), and up to 66% of people using online health information claim it has had an impact on their healthcare decisions (Fox 2002). The widespread and sufficient use of online consumer health information may also reduce the inequities in health care between low, middle and high income countries.

Online information provision has a number of advantages. Once access is provided, use is convenient, private, and continuous. The information can be specific to the needs of the consumer, and may be frequently updated (Christensen 2000; Griffiths 2006). Internet use can be particularly effective where there are barriers to the use of traditional health information, for example for stigmatised conditions such as mental illness, incontinence, or sexually transmitted infections (Anderson‐Inman 1998; Berger 2005; Cline 2001).

Who is using the internet for health information?

Researchers have identified demographic characteristics of those who use the internet for health information and the types of information they seek. Four main categories of health internet users can be distinguished:

• those with an acute health problem,

• those with a long‐term health problem,

• those looking on behalf of someone else, and

• those who are currently well but looking for general information (Cain 2000).

The most common reason to seek health information is a recent personal diagnosis (Cline 2001), but 'indirect' use of the internet is very common; 50% of cancer patients report using internet‐based information, usually provided by a family member or friend (James 2007). Compared to people who do not use the internet for health information, those who do are more likely to have poor self‐reported health (Baker 2003; Rice 2006) or a stigmatised illness (Berger 2005), and are also more likely to be educated and literate, female, and have a higher income (Bylund 2007; Diaz 2002; Dutton 2007). Age is a particularly important barrier, with internet use dropping sharply after the age of 55 (Dutton 2007). Once access has been provided, however, there is little evidence of differences in online information‐seeking behaviour (Brodie 2000). People in 'deprived' groups may actually receive a relatively larger benefit from computer‐based health interventions (Gustafson 2002), and the relationship between internet use, information‐seeking, and 'deprivation' is not straightforward. There seems to be a subgroup of information‐seekers in these populations who turn to the internet to locate health information (Malone 2005).

Barriers to online health literacy

Barriers to consumer use of online health information include: lack of physical access to the internet, motivation, and a real or perceived lack of cognitive skills to find and use information. Of these, motivation and skills are critical. Simply increasing access to online information will not lead to increased use unless consumers are motivated and able to make effective use of the internet and the information they find. The expectation that online health information will be useful is one of the strongest predictors of internet use (Mead 2003), and a lack of skills is debated as one possible reason for not using the internet (Livingstone 2006). Specifically, potential users may lack the skills to effectively locate, evaluate and use online health information (e.g. Benotsch 2004; Gray 2005). For instance, internet users typically use sub‐optimal search and evaluation strategies (Eysenbach 2002a). Additionally, while consumers are generally satisfied with the information they find (Diaz 2002; James 2007), the quality of many web sites is poor (Eysenbach 2002b). Moreover, consumers report using online health information despite not trusting its quality (Powell 2006).The essential skills a consumer should have can be subsumed under the term 'health literacy', especially computer literacy and information literacy. Health literacy is the ability to obtain, understand, act on and communicate health information (Kickbusch 2006; Nielsen‐Bohlman 2004), and is a barrier that can affect every stage of finding, understanding, evaluating and using health information. Computer literacy refers to the ability to operate in computer‐supported environments, and includes both technical competence and an understanding of how information is presented electronically (Martin 2006). Information literacy is "knowing when and why you need information, where to find it, and how to evaluate, use and communicate it" (CILIP 2007).

Training people to find, evaluate and use online health information

Interventions to enhance people's online health literacy skills aim to address the barriers to consumer use of online health information. Interventions can take place in diverse settings and provide a variety of skills to different consumer groups. They may focus on any or all of the following areas:

• Internet knowledge and ability to use the internet, addressing motivational and computer literacy barriers;

• Search skills, addressing barriers related to information literacy skills;

• Evaluation skills, addressing barriers related to poor quality information and low information literacy;

• Ability to apply information in practice, addressing barriers related to low health literacy.

Some consumer skills interventions focus narrowly on one specific skill, such as training consumers to evaluate online information using a quality checklist. Interventions may also take a broader approach. For example, Kalichman 2006 included participants with no previous computer or internet experience and the intervention therefore included motivational introductions to computers and the internet, effective search skills, and personal use of online health information, as well as evaluation of information. Health‐focused consumer internet skills interventions have been used with patients with HIV (Kalichman 2006), oncology patients and their families (Edgar 2002), Spanish‐speaking communities in the US (Olney 2007), and older adults (Campbell 2005; Dauz 2004; Leaffer 2000). Interventions have been delivered in diverse settings including medical facilities (Edgar 2002), community centres (Olney 2007), a university (Kloda 2006), participants' homes (Dauz 2004) and public libraries (Oermann 2005). As this is a new area of research, many of these interventions are uncontrolled pilot or feasibility studies, or are evaluations of existing projects.

Potential benefits and harms of consumer internet skills training

Based on the Information, Motivation and Behavioural Skills (IMB) model (Fisher 2003), we hypothesise that enhancing consumers' online health literacy has a positive effect on health behavioural outcomes. Most studies suggest that information on its own has limited direct health benefits or ability to change behaviour (Jorm 2003; Vergouwen 2003), so we hypothesise that the health benefit is due to indirect effects of the intervention. The IMB model assumes that information and motivation influence behavioural skills, which subsequently influence health behaviours. Following Kalichman's hypothesised route of action (Kalichman 2002; Kalichman 2006), we suggest that interventions directly enhance internet skills, internet‐related self‐efficacy, and internet use, leading to increased health knowledge, a more active coping style, and improved information handling, which then may lead to improved health behaviours. Outcomes of the intervention can therefore be grouped into primary outcomes that measure consumer internet skills, internet use, internet self‐efficacy, or information‐handling, and secondary outcomes that measure more distal effects such as coping style, consumer satisfaction or health service use (see Figure 1). As these are complex interventions, outcomes are likely to be affected by aspects of the intervention (for example, mode of delivery or setting), and participant characteristics such as age or severity of health problem. This hypothesis is in line with research demonstrating that both participation in education and use of the internet for health information are associated with positive health and social outcomes (Aldridge 2000; Kalichman 2005).

Figure 1.

Hypothesised pathway of action (adapted from Kalichman 2002)

Trials of decision support systems suggest that access to evidence‐based information may reduce healthcare demand (Pencheon 1998). Access to unregulated and variable quality information on the internet may, however, work in the opposite direction. If consumers have difficulty appraising information they may make healthcare decisions uncritically, relying on non‐evidence‐based information, especially when most consumers do not discuss the information they find online with a health professional (Diaz 2002; Murray 2003). There is widespread concern, but limited anecdotal information, about the possible harms associated with the use of online health information (Crocco 2002). Crocco and colleagues note that this may be due to an actual low risk for harm associated with the use of information available on the Internet, to underreporting of cases, or to bias. Consumer skills interventions could potentially avert harm caused by inaccurate information or poor comprehension of the information. They could also cause harm if consumers who would not otherwise have used online health information are introduced to inaccurate or anxiety‐provoking information.

Why is this review necessary?

Poor online health literacy is a major barrier to locating, evaluating and using online health information. Interventions to enhance consumer internet skills that are also intended to improve health and social outcomes are in use in many settings and therefore it seems reasonable to assess the effects of these interventions in a systematic review. It is not yet clear whether training consumers in online health literacy has beneficial effects for their health. A 2002 review of consumer internet interventions found a lack of rigorous research into the health outcomes of such interventions (Bessell 2002). The included studies primarily compared the effect of having/not having access to the internet or to particular internet‐based patient education materials, but did not consider the skills of the participants. There is some evidence that interventions delivered over the internet can be effective at improving health outcomes (Griffiths 2006; Kaltenthaler 2006; Murray 2005). Other reviews excluded these interventions, such as Griffiths 2006, focused on defined, networked, interventions delivered over the internet (for instance, computerised cognitive behavioural therapy), and specifically excluded interventions that just aimed to improve consumer internet skills. Murray's review of interactive health communication applications (IHCAs) (Murray 2005) and Bailey's review of interactive computer based sexual health interventions (Bailey 2010) also excluded non‐interactive interventions.

Objectives

To assess the effects of interventions for enhancing consumers' online health literacy (skills to search, evaluate and use the online health information). In this review we use the term 'online health literacy' to subsume these abilities necessary for using online health information.

Methods

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCTs) or cluster RCTs and associated economic evaluations, plus quasi‐RCTs, controlled before and after (CBA) studies, and interrupted time series (ITS) studies, were eligible for inclusion in this review.

CBA and ITS studies were selected according to Cochrane Consumer and Communication Review Group guidelines (CC&CRG 2007; CC&CRG 2008; Ryan 2007) which are based on guidance from the Effective Practice and Organisation of Care (EPOC) Review Group. CBA studies were eligible for inclusion if they met the following two quality criteria:

• the timing of the periods for intervention and control groups were comparable; and

• the intervention and control groups were comparable on key characteristics.

ITS studies were eligible for inclusion if they met the following criteria:

• a clearly defined point in time at which the intervention occurred; and

• a minimum of three measurement points available before and after the intervention.

CBA and ITS studies that did not meet these criteria were excluded from the review.

Our rationale for including study types other than RCTs was that this is a relatively new area of study involving complex educational interventions. We therefore expected to find few RCTs answering the research question, but anticipated finding other comparative studies.

Types of participants

We included all patients/consumers, with no exclusions based on age, gender, ethnicity, or health status. We excluded studies of training healthcare professionals, as well as studies that trained non‐health professionals (e.g. community development workers or librarians) to use health resources. For studies in which both consumers and health professionals were trained, we would have sought data specifically about the consumers, although no such studies were identified.

We included participants in situations where the search for information was hypothetical as well as those with a true information need, participants who were searching for information for other people as well as for themselves, and people who trained at different ages. To reflect the rapid evolution of the internet we also included participants who were being assessed or re‐assessed a significant time after their training, as well as those who were trained recently. We intended to treat these groups separately during data analysis, but the small number of included studies prevented us from doing so.

Types of interventions

We included any type of intervention that trained consumers to find, evaluate, or use health information from the internet. Training can be defined as teaching a skill or type of behaviour, is task‐directed and focused on the practical application of skills. This is in contrast to 'patient education' which is a broader concept and often refers to simple provision of consumer health information or health knowledge (Campbell 1990).

Interventions may therefore have included training on one or more of the following subjects:

• Technical internet skills (e.g. how to use an internet browser);

• Effective search skills (e.g. using a search engine);

• The use of particular web sites (e.g. generic high quality sites such as MedlinePlus, condition‐specific sites, or sites requiring less advanced literacy skills);

• Evaluation skills (e.g. assessing the quality of a web site);

• Using or applying online health information (e.g. using information to make decisions about health service use).

We included studies in all settings, including non‐health settings (e.g. adult education), and from all types of intervention provider. There was no exclusion with respect to the teaching or learning method. The intervention could have been part of a complex intervention or larger study, but it had to be possible to separately identify the effects of the skill enhancement intervention. Complex interventions that involved enhancing online health literacy as well as other aspects of internet use were included; for example, interventions that included modules on online safety or use of web‐based email as well as search and appraisal skills. These interventions were included because internet use is a complex behaviour and use of publicly‐available online information is not a clearly delimited activity, for instance, consumers may wish to email the web site to ask for further information.

We included studies that made the following intervention comparisons:

• Enhancing the ability of consumers to use online health information versus no intervention or usual care (e.g. access to the internet without training);

• Enhancing the ability of consumers to use online health information versus other types of consumer education or training (e.g. patient education seminars or generic internet skills or information literacy interventions with no health component);

• Enhancing the ability of consumers to use online health information versus another type of intervention to train consumers to find and use online health information (e.g. studies that compare different training content or delivery method);

• Enhancing the ability of consumers to use online health information in participant groups different in certain characteristics (e.g. age).

Health information was defined broadly, to include conventional medical information as well as information about health conditions, treatments, complementary and alternative medicines, and physical or emotional well‐being. This broad definition was used because in practice consumers do not make distinctions between these different types of health information (Hardey 1999), and because the skills included in this review are applicable to all types of health information. Only internet‐based health information was included.

We excluded the following types of interventions:

• Unstructured or unplanned training sessions (e.g. before use of an internet‐based intervention) unless the effects of training could be separated from the effects of the intervention;

• Interventions with the sole aim of enhancing consumer skills in other aspects of internet use (e.g. use of specific interactive health communication applications, or to email professionals or to use chat rooms);

• Generic computer training or access to computer equipment without a health information component;

• Interventions with the sole aim of enhancing the skills of consumers to find, evaluate or use health information which is provided in other formats (e.g. printed materials, telephone help lines, CD‐ROMs);

• Interventions aimed at mediators/facilitators such as health professionals, non‐health professionals (e.g. community development workers or librarians) and 'train‐the‐trainer' studies which train consumers to help/train other people. This is because we considered their aim and likely pathway of action to be different to interventions that aim to directly enhance the skills of individual consumers.

• Interventions that just provide access to online health information (e.g. by giving participants a list of recommended health information web sites or an information prescription, providing internet kiosks in a health clinic, or comparing online information to printed information) without consumer training. These interventions primarily provide health information and do not aim to enhance the skills of consumers.

• Interactive health communication applications and computerised decision aids. These interventions provide health information and at least one other service, such as peer support, expert advice, or decision‐making tools (Murray 2005; O'Connor 2009), and do not primarily aim to enhance consumer skills at using online health information.

Types of outcome measures

A variety of outcomes may be affected by consumer skills interventions. We sought to extract and assess all the outcomes measured in the included studies. We a priori grouped outcomes into primary and secondary outcomes, based on the studies mentioned in the Background and our hypothesised mode of action. Primary outcomes were those affected directly by the interventions, and secondary outcomes are those that may be indirectly affected by the interventions. Outcomes reported by those studies described in the Background and the Consumer and Communication Review Group taxonomy of outcomes (CC&CRG 2007) were used to identify and categorise relevant primary and secondary outcomes.

Primary outcomes

We looked for the following primary outcomes for both consumers and their families/carers:

• Technical internet/computer skills (that is, level of computer literacy, specific to the use of the internet) e.g. self‐reported ability to perform internet‐related tasks (Leaffer 2000);

• Ability to search for online health information e.g. locating the correct answer to specific health questions (Eysenbach 2002a);

• Ability to evaluate/appraise online health information e.g. ability to discriminate between different quality web sites ;

• Ability to apply (use) health information to address health problems;

• Self‐efficacy for internet use and health information seeking (i.e. a belief in one's ability to use the internet and seek health information) e.g. the Eastin and LaRose Internet Self Efficacy Scale (Eastin 2000).

We included self‐reported measures of ability and self‐efficacy as well as objective or validated measures. This was because perceived ability may predict behaviour more accurately than objective competence, and because self‐reported measures are commonly used in this developing area of research. Potential bias introduced by the use of non‐validated measures was assessed during data extraction and analysis.

Secondary outcomes

We looked for the following secondary outcomes:

• Knowledge and understanding;

• Consumer health status;

• Behavioural outcomes including health behaviours and self‐care;

• Behavioural and cognitive outcomes including well‐being, confidence, coping, and health anxiety;

• Economic outcomes;

• Consumer and family/carer involvement in the care process;

• Evaluation of health care;

• Outcomes relating to health professionals including attitudes to the intervention or to consumer use of online health information, practice style, and provision of interventions.

We also looked for any data relating to adverse events or harms (such as increased anxiety) associated with the interventions.

Search methods for identification of studies

Electronic searches

We searched the following databases in March 2008:

• Cochrane Consumers and Communication Review Group Specialised Register (March 2008)

• Cochrane Central Register of Controlled Trials (CENTRAL), The Cochrane Library, (Issue 1, 2008)

• MEDLINE (Ovid) (January 1990 to March 2008)

• CINAHL (Dialog) (January 1990 to March 2008)

• ERIC (CSA Illumina) (January 1990 to March 2008)

• LISA (CSA Illumina) (January 1990 to March 2008)

• PsycINFO (Ovid) (January 1990 to March 2008)

• EMBASE (Ovid) (January 1990 to March 2008)

Conference proceedings

• Index to scientific and technical proceedings

Grey literature

• SIGLE.

• ASLIB Index to Theses.

• ProQuest Dissertation Abstracts.

Clinical trials

• National Research Register/UK CRN Portfolio database.

• Current Controlled Trials ‐ MetaRegister of Controlled Trials.

We searched in English but included papers in any languages. We searched from January 1990 because the internet was not used by the public before this time period.

The search strategy for use in MEDLINE (Ovid) is presented in Appendix 1. The strategies for other databases searched are in Appendix 2 to 8.

Searching other resources

We examined the reference lists and, where possible, the 'cited by' lists, of relevant articles to identify additional studies. We also contacted authors of included studies to ask if they were aware of additional studies meeting the inclusion criteria.

Data collection and analysis

Selection of studies

Two review authors independently assessed the eligibility of studies identified by the search strategy. We applied the selection criteria to titles and abstracts, with decisions erring on the side of caution, that is, to provisionally include all studies potentially reporting an intervention for enhancing consumers' online health literacy unless they could be definitely excluded. We then retrieved the full text of papers assessed as potentially relevant, and two review authors (AC and CU) assessed them independently against the selection criteria for inclusion. Disagreements about the inclusion or exclusion of particular studies were resolved by discussion between the two review authors and with one other review author (EK). One study author was contacted for further details when it was unclear whether the study met the inclusion criteria. Studies that appeared to fulfil the inclusion criteria but were later excluded from the review are detailed in the Characteristics of excluded studies table, along with the reasons for their exclusion (Higgins 2008a).

Data extraction and management

We extracted data from included studies using an adaptation of the Cochrane Consumers and Communication Review Group data extraction template (CC&CRG 2008). Items relating to CBA and ITS designs were adapted from the Cochrane EPOC Review Group data collection checklist and extraction form (EPOC 2002; EPOC 2007). Two review authors (AC and CU) independently extracted the data from each included study. Disagreements were resolved by discussion with each other and with a third review author (EK).

Extracted data were entered into RevMan by one review author, and the entries checked for accuracy against the original data form by another author.

Assessment of risk of bias in included studies

Following the guidelines of the Cochrane Consumers and Communication Review Group (Ryan 2007) and The Cochrane Handbook (Higgins 2008b), we assessed the methodological quality of included studies by explicitly reporting the following individual quality components for each included study. Each criterion was rated as yes (low risk of bias), unclear (uncertain risk of bias), or no (high risk of bias) (see Risk of bias in included studies).

We assessed RCTs for risk of bias according to the criteria outlined in The Cochrane Handbook (Higgins 2008b). We also reported on other quality indicators using the following criteria:

• Were outcome measurement tools validated?

• Were outcome measures reliable?

• Was ethical approval obtained?

We assessed the quality of the CBA study using the following criteria: EPOC 2002; EPOC 2007.

We also reported on other quality indicators using the following criteria:

• Were outcome measurement tools validated?

• Was ethical approval obtained?

In all cases, two review authors independently assessed the quality of included studies, with disagreements resolved by discussion and consensus. We contacted study authors for additional information about the included studies, or for clarification of the study methods as required.

Assessment of reporting biases

Investigation of publication bias

We included grey literature in our search strategy to minimise the effects of publication bias. Due to the limited number of studies it was not possible to investigate possible publication bias further.

Data synthesis

Following data extraction, we tabulated data to create a descriptive synthesis of the included studies. Due to the small number of included studies it was not possible to make all of the comparisons detailed in the protocol. Instead, we have described each included study in a narrative summary. The included studies were substantially heterogenous with respect to study design, population and outcomes, and so it was not appropriate to pool data in a meta‐analysis.

Consumer participation

We circulated the draft protocol and review to a consumer participation group for comments, which were incorporated into the review. At least one consumer referee commented on the protocol and review during the Cochrane Consumers and Communication Review Group's standard editorial process.

Results

Description of studies

See: Characteristics of included studies; Characteristics of excluded studies.

Results of the search

Our search strategy returned 41,225 citations. Due to the size of the data set and differences in the way in which citations were retrieved, it was not possible to easily remove duplicate citations at this stage, so we do not know how many of these were unique citations. Of these, 39 were potentially relevant. The large number of irrelevant citations can perhaps be attributed to the inclusion of non‐RCT study designs in our search strategy (Reeves 2008).

Of the 39 studies which we assessed in full text, we excluded 35 because they did not meet the inclusion criteria in terms of study design (26 studies), or type of intervention (9 studies). A further two studies were excluded; one study because it was later found not to match the criterion regarding population, and one study because it was discontinued due to recruitment problems. These two studies that appeared to fulfil the inclusion criteria but were later excluded from the review are detailed in the Characteristics of excluded studies table.

Included studies

Overview

We included one randomised controlled trial (RCT) (Kalichman 2006) and one small exploratory controlled before and after (CBA) study (Cortner 2006), the key features of which are described in the Characteristics of included studies table.These two studies included 470 adult participants in total. Four hundred and forty eight of these were participants in the Kalichman 2006 RCT. The Kalichman 2006 participants were people living with HIV/AIDS who had been recruited via health and social services, while the 22 CBA participants (Cortner 2006) were healthy adults aged 50+ who were recruited via a local newsletter for older adults. Both studies took place in specialist computer centres placed in community venues in the United States of America. Participants in Kalichman 2006 were only included if they had used the internet less than three times in the preceding month, whereas the inclusion of participants in Cortner 2006 was not based on their previous experience of the internet. Neither study reported consumer involvement in the research process.

Interventions and comparisons

Both studies compared one intervention group with one control group. The interventions were broadly similar in both studies, consisting of 'adult education style' discussion, instruction and practice in groups of six to ten participants. Kalichman's RCT provided a higher intensity intervention, with eight 120 minute sessions delivered over four weeks. Cortner's CBA provided four 90 minute sessions over four weeks. The CBA intervention primarily focused on locating trustworthy online health information and evaluating the quality of web sites, while the RCT intervention had a broader focus, and included using online health information (e.g. by taking it to a health professional), gaining social support online, and strategies for getting personal access to the internet in the future, as well as locating and evaluating online health information.

In both studies participants were searching for information for themselves, either related to HIV/AIDS and healthcare, or preventive health information for ageing. The information was therefore relevant and not entirely hypothetical, but it was not possible to tell from the published data whether the intervention met the real information needs of the participants.

Control group

The control condition differed between the studies. Cortner's CBA study used the participants on the waiting list as control. For the RCT a time‐and‐contact‐matched HIV information support group was the control. Sessions for this group included information about HIV, management of health problems, nutrition, medication access and adherence strategies, but were not skills based. Control participants met in a room in the same building as the intervention group, and had a tour of, and access to, the technology centre. They were therefore aware that internet access was available to them, but were not given training on how to use the facility.

Outcomes

Outcomes also differed between studies (see Effects of interventions). The CBA study focused on a single outcome, being the participants' readiness to adopt the internet as a tool for preventive health. The RCT measured as primary outcomes 'Self‐efficacy for health information seeking', 'health information evaluation skills' and the 'number of times the patient discussed online information with a health provider'.

Risk of bias in included studies

One study (Cortner 2006) has a significant risk of bias due to methodological weaknesses of the study design(Figure 2). The study is especially at risk of selection and performance bias and used unvalidated outcome measures. The study by Kalichman (Kalichman 2006) is less prone to bias although some sources of bias could not be analysed sufficiently. Details are reported for the different study designs in the Risk of Bias Tables in the Characteristics of included studies section.

Figure 2.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

The randomised controlled trial: Kalichman 2006

We applied eight criteria assessing risk of bias to Kalichman 2006. We judged the study as having a low risk of bias on five of the criteria (allocation sequence generation, blinding, baseline comparability, contamination and other risks) and an unclear risk of bias on three of the criteria (allocation concealment, incomplete data and selective reporting). We also assessed as additional quality indicators the use of validated and reliable outcome measures. Half the measures had been used in other studies or were validated instruments. The other measures had been developed and tested prior to use in this study. Ethical approval had been obtained. Overall, we consider this study to be at moderate risk of bias.

The controlled before and after study: Cortner 2006

We applied six criteria to assess risk of bias in Cortner 2006. We judged the study to be at a low risk of bias on two criteria (baseline and follow‐up), unclear risk on two criteria (contamination and outcome measures), and high risk on one criterion (blinding). One criterion was not applicable (second site characteristics). We also recorded high risk of bias regarding the validity of outcome measures. The study authors had obtained ethics approval. Overall, we consider this study to be at high risk of bias.

Effects of interventions

See: Table 1

Outcomes mainly consisted of ordinal data obtained from scales, but were reported as continuous data by study authors. We therefore calculated mean differences (MDs) for the outcomes. The primary outcomes of the RCT showed strong evidence of being positively skewed (Deeks 2008). Study authors reported that they transformed data when appropriate, to obtain a more normal data distribution. The higher effects in the intervention group must therefore be treated with caution. All individual outcomes and MDs are reported in Table 3. Data on primary outcomes are reported in the Table 1. Both studies reported outcomes that favoured the intervention group. The authors of the Kalichman 2006 RCT also reported outcomes that did not show a difference between the groups. Neither study reported outcomes that favoured the control group. Neither study reported any harms.

Table 1.

Outcomes and effect sizes

Study	Outcome (range of possible values)	Absolute effect size (unit)	Relative effect size (%)	Mean difference (95% CI)	Favours intervention or control
Kalichman 2006	Self‐efficacy for health information seeking (0 to 10)	1.1 (points)	‐	1.10 (0.64 to 1.56)	Intervention
Kalichman 2006	Health information evaluation skills (‐9 to 9)	0.6 (points)	‐	0.60 (0.11 to 1.09)	Intervention
Kalichman 2006	Number of times the patient discussed online information with health provider	0.7 (episodes)	‐	0.70 (0.22 to 1.18)	Intervention
Kalichman 2006	Affective depression (0 to 18)	‐0.1 (points)	2.0	‐0.10 (‐0.87 to 0.67)	No difference
Kalichman 2006	Total internet use	3.5 (episodes)	57	3.50 (‐1.56 to 8.56)	No difference
Kalichman 2006	Used email	2.6 (episodes)	96	2.60 (‐0.12 to 5.32)	No difference
Kalichman 2006	Searched for online health information	1.5 (episodes)	250	1.50 (0.78 to 2.22)	Intervention
Kalichman 2006	Searched for clinical trials	0.6 (episodes)	150	0.60 (‐0.01 to 1.21)	No difference
Kalichman 2006	Information seeking coping (likelihood of using strategy) (0 to 9)	0.1 (points)	1.4	0.10 (‐0.21 to 0.41)	No difference
Kalichman 2006	Ways of coping ‐ Problem focused (0 to 3)	0.1 (points)	5.6	0.10 (‐0.04 to 0.24)	No difference
Kalichman 2006	Ways of coping ‐ Information coping	0.2 (points)	12.5	0.20 (0.07 to 0.33)	Intervention
Kalichman 2006	Ways of coping ‐ social coping behaviours (0 to 3)	0.1 (points)	5.6	0.10 (‐0.04 to 0.24)	No difference
Kalichman 2006	Social support (15 to 60)	2.1 (points)	5.0	2.10 (0.56 to 3.64)	Intervention
Cortner 2006	Readiness to adopt the internet for preventive health (0 to 100)	34.19 (points)	62	34.19 (23.24 to 45.14)	Intervention

Follow‐up data were available for Kalichman 2006. We have used data from the longest time point available for each outcome (Deeks 2008). This was the 9 month follow‐up for all outcomes except 'health information evaluation skills', which was only measured at baseline and the 3 month follow‐up.

Below we present a narrative summary of the effects for each study design separately, organised by our specified primary and secondary outcome categories. To give an indication of the size of effect, we have reported the absolute difference and relative percentage difference between the intervention and control group at the latest time‐point, and the direction of effect for each outcome. We have not addressed some related outcomes from the RCT (Kalichman 2006) described 'group experience' and, 'use of the computer centre' as they were not relevant for our primary or secondary outcomes.

Primary outcomes: Kalichman 2006 (randomised controlled trial)

Self‐efficacy for health information seeking

Self‐efficacy for health information seeking was measured on a scale of participants' self‐reported beliefs about their ability to perform tasks related to online information such as finding information about HIV treatments. At 9 months post‐intervention, people in the intervention group had greater self‐efficacy for health information seeking. The absolute difference between the groups was 1.1 points, which is a relative difference of 21%. The mean difference between the groups was 1.10 (95% CI 0.64 to 1.56) (Analysis 1.1).

Analysis 1.1.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 1 Self‐efficacy for health information seeking.

Health information evaluation skills

Health information evaluation skills were measured at three months by participants giving quality scores on one evidence‐based and one non‐evidence based web page. The difference between the scores was used as a measure of participant ability to discriminate between the web pages. People in the intervention group were more able to discriminate between the good and poor quality information. The absolute difference was 0.6, which was a relative increase of 60%. The mean difference between the groups was 0.6 (95 % CI (0.11 to 1.09) (Analysis 1.2).

Analysis 1.2.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 2 Health information evaluation skills.

Number of times the patients discussed online information with a health provider

Kalichman 2006 measured whether participants had used online information by taking it to a health provider in the preceding three months. People in the intervention group were more likely to have talked to a health provider about online health information. The absolute increase was 0.7 episodes, which was a relative increase of 70%. The mean difference between the groups was 0.7 (95 % CI 0.22 to 1.18) (Analysis 1.3).

Analysis 1.3.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 3 Number of times the patients discussed online information.

The Kalichman 2006 study did not measure other primary outcomes for this review, namely internet/computer skills, search skills, and knowledge or understanding of how to find, evaluate or use health information.

Secondary outcomes: Kalichman 2006 (randomised controlled trial)

Consumer health status

Kalichman 2006 measured affective depression using six items from the Centers for Epidemiological Studies‐Depression Scale (CES‐D). At 9 months there was no statistically significant difference between the intervention and the control group (MD ‐0.10 (95% CI ‐0.87 to 0.67) (Analysis 1.4).

Analysis 1.4.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 4 Affective depression (showing symptoms the previous 7 days).

Behavioural outcomes

At the nine month follow‐up Kalichman 2006 measured total internet use and use of email in the preceding three months. The intervention group used the internet and email more often than the control group. The absolute differences between the groups were 3.5 episodes (a relative increase of 57%; MD 3.50 (95% CI ‐1.56 to 8.56); Analysis 1.5) for total internet use, and 2.6 episodes (a relative increase of 96%; MD 2.60 (95% CI ‐0.12 to 5.32); Analysis 1.6) for email use. Results for both outcomes seem to favour the intervention, but the results are not statistically significant and should be used with caution.

Analysis 1.5.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 5 Total Internet use.

Analysis 1.6.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 6 Used email.

At the nine month follow‐up the study also measured participants' use of the internet to search for health information and clinical trials in the preceding three months. The intervention group searched more often for general health information and clinical trials than the control group. Absolute differences between the groups were 1.5 episodes for general searching (MD 1.50 (95% CI 0.78 to 2.22) (Analysis 1.7)) and 0.6 episodes for clinical trial searches (MD 0.60 (95% CI ‐0.01 to 1.21) (Analysis 1.8)). Relative percentage increases were 250% and 150% respectively. Results for clinical trial searches are not statistically significant.

Analysis 1.7.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 7 Searched for online health information.

Analysis 1.8.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 8 Searched for clinical trials.

The study authors also assessed 'information seeking' as coping strategy for health (Miller 1988). The intervention group had a score of 0.1 points higher than the control group, a relative difference of 1.4%. The mean difference was not statistically significant, at 0.10 (95% CI ‐0.21 to 0.41) (Analysis 1.9).

Analysis 1.9.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 9 Information seeking coping (likelihood of using strategy).

For assessing participants' coping strategies in terms of problem‐focused coping, information coping, and social coping, the study used the Ways of Coping Inventory (Lazarus 1984). There was little difference between the groups, with the intervention group scoring very slightly higher on all measures (indicating a greater likelihood of using the strategies). The differences were as follows:

Outcome	Absolute difference	Relative difference	Mean difference (95% CI)	Analysis
Problem‐focused coping	0.1 points	5.6%	0.10 (95% CI ‐0.04 to 0.24)	Analysis 1.10
Information coping	0.2 points	12.5%	0.20 (95% CI 0.07 to 0.33)	Analysis 1.11
Social coping	0.1 points	5.6%	0.10 (95% CI ‐0.04 to 0.24)	Analysis 1.12

Only the results for information coping were statistically significant.

The perception of being socially supported by the intervention was measured using the Social Support Questionnaire (Brock 1996). The intervention group scored higher by 2.1 points at 9 months, indicating greater perception of social support. The relative difference between the groups was 5%, with a mean difference of 2.10 (95% CI 0.56 to 3.64) (Analysis 1.13).

Analysis 1.13.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 13 Social support (intensity of social support).

Secondary outcomes: Cortner 2006 (controlled before and after study)

The CBA by Cortner 2006 measured a single (secondary) outcome: Readiness to adopt the internet as a means of obtaining preventive health information. The scale consisted of a number of items corresponding to stages in the Diffusion of Innovation and Stages of Change theories. The study author supplied us with individual patient data for each item, but we decided the clearest measurement was the mean score. The intervention group had a mean score of 34.19 points higher than the control group, indicating a greater readiness to adopt the internet as a tool for health care. This was a relative increase of 62%. The mean difference between the groups was 34.19 (95% CI 23.24 to 45.14) (Analysis 1.14). According to the author's interpretation of the scale, this meant that the control group had minimal readiness with help definitely needed for adoption, whereas the intervention group were ready to adopt the innovation, with minimal help needed to do so (Cortner 2006).

Analysis 1.14.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 14 Readiness to adopt the internet for preventive health.

Discussion

Summary of main results

This review included only two studies: a randomised controlled trial (RCT) Kalichman 2006 and a controlled before and after study (CBA) Cortner 2006. The studies, both conducted in the USA, were heterogenous with respect to study design, participants, health problems, and outcomes, but examined quite similar educational interventions. The RCT authors reported significant beneficial effects of the intervention on five outcomes (Table 1), and no difference between the groups on other outcomes. No outcome favoured the control group, suggesting that the interventions had a low risk of harm, but neither study specifically reported adverse events. Due to the limitations of the study design, the asymmetrical distribution (skewness) of data and the outcomes for which the effect was not statistically significant (Table 1) it is not possible to make further generalisations about benefits or harms attributable to the interventions.

Overall completeness and applicability of evidence

It is clear that the included studies do not answer the review question sufficiently. This is mainly due to the small number of studies, but also the heterogenous character of the participants (HIV‐positive consumers, and healthy adults aged 50+) from one country, and to the type of outcomes reported. Outcomes mainly focused on consumer attitudes or behaviour rather than skills. It is unclear whether behavioural measures (such as number of times the internet is searched) reflect consumer skills. It is useful to know that the interventions increase likely or actual use of the internet to search for health information, but it would be helpful to also have a measure of the effectiveness of the searches. Many of the outcomes were measured using semantic differential or Likert‐type scales. While this is a feature of much research into complex educational interventions, it is also difficult to know what magnitude of change on these scales would be a significant change.

It is disappointing that the search conducted for this review returned a large number of relevant study publications reporting adult educational interventions similar to those of the included studies, that had to be excluded due to their methodologically weak study design. Obviously many of these interventions are applied without strong evidence of their effectiveness.

Quality of the evidence

This review includes two studies with a total of 470 participants, 448 of whom were involved in the RCT by Kalichman 2006. The included studies tested interventions that were comparable regarding content and format. Results of both the studies were consistent, in that they favoured the intervention. Nonetheless, the quality of evidence must be downgraded by the fact that only two studies could be included and one of them (Cortner 2006) was small and had methodological limits due to the non‐randomised exploratory CBA design. Apparent risks of bias in the RCT (Kalichman 2006) seem to be related to poor reporting rather than poor design, but a single study cannot be considered to sufficiently answer the review question. It is notable that Kalichman 2006 is the only trial that specifically incorporates material to enhance the ability of consumers to use or apply the information they find.

Potential biases in the review process

Our search strategy was comprehensive and erred on the side of sensitivity rather than specificity. We also included non‐randomised study designs in our search strategy. These two factors increased the likelihood of identifying relevant studies, but also greatly increased the number of results and the time spent selecting papers. Two review authors independently carried out the study selection and data extraction with a third review author to adjudicate disagreements as necessary, but none of the review authors who performed selection of studies were methodological experts, which may have introduced a degree of bias.

Agreements and disagreements with other studies or reviews

The included studies indicate that consumer information and communication technology skills interventions may have a positive effect on attitudes and to a limited extent, behaviour. This is consistent with the conclusion of other studies that show internet‐based interventions can be associated with improvement in certain outcomes related to health: For example, systematic reviews by Murray 2005 and Kaltenthaler 2006 found that interactive healthcare applications and computerised cognitive‐behavioural therapy had beneficial effects on users' knowledge, social support, health behaviours and clinical outcomes. It is also consistent with studies of consumer educational interventions, especially self‐management courses which aim to enhance consumer skills to manage their illness and healthcare. Although such courses do not usually include an online information module, there are a number of similarities between the courses and consumer internet skills interventions. Similar to the interventions covered in this review, self‐management programs aim to teach problem solving skills and not just impart information, and have self‐efficacy as a key concept (Bodenheimer 2002). A Cochrane systematic review of lay‐led self‐management education programs (Foster 2007) concluded that the programs could improve self‐efficacy, subjective health ratings (e.g. pain or fatigue), cognitive symptom management, and frequency of aerobic exercise but that there was no evidence of an improvement in psychological health, symptoms, or health‐related quality of life. The limited evidence from our review points also in this direction.

Authors' conclusions

Due to the small number and the variable methodological quality of included studies we cannot draw any conclusions about the implications of this review for the content or delivery of consumer internet skills interventions. The RCT (Kalichman 2006) provides some support for the premise that providing access to the internet without training is not sufficient to increase consumer use of online health information, since the control group participants were given internet access without training. This has implications for wider policy, and suggests that investment in infrastructure needs to be accompanied by investment in training.

As mentioned in the Background to this review, a large number of pilot and exploratory interventions are reported in the literature but rigorously conducted controlled trials are few. So the body of evidence to evaluate these interventions is weak despite the growing use of online health information by consumers. Future research should promote the development of effective interventions to enhance the online health literacy of consumers.Thus there is a need for well‐designed and rigorously conducted randomised controlled trials (RCTs). These RCTs should involve diverse participants (regarding disease status, age, socio‐economic group and gender) to analyse to what extent online health literacy reduces a barrier to using the internet for health information, or if socio‐economic group, gender and age are more important in influencing internet use (Livingstone 2006).Trials should be conducted in different settings (including low, middle and high income countries) and should examine interventions to enhance consumers online health literacy (search, appraisal and use of online health information) like internet training courses. For measuring the outcome types (e.g. disease prevention habits, disease status, cognitive skills and behaviour) it is important to use validated scales and to have measurements up to at least one year, to estimate the sustainability of the intervention effects.

Appendices

Appendix 1. MEDLINE (Ovid) search strategy

1.Caregivers/ 2.exp disabled persons/ 3.exp patients/ 4.(Patient$ or consumer$ or carer$ or caregiver$ or lay$).mp 5.or/1‐4 6.(Information adj4 (need$ or seek$ or us$ or util$ or literac$)).mp. 7.(Communicat$).mp. 8.Access to information/ 9.Information dissemination/ 10.Exp "Information storage and retrieval"/ 11.exp Information services/ [Remove databases] 12.(educat$ or skill$ or teach$ or train$ or coach$ or instruct$).mp. 13.exp Health education/ 14."Early Intervention (Education)"/ 15.exp Teaching materials/ 16.Teaching/ 17.Education, distance/ 18.Exp Education, nonprofessional/ 19.or/6‐18 20.exp Computers/ 21.Computer communication networks/ or Internet/ 22.(Computer$ or Internet or online).mp. 23.(World adj wide adj web) or (worldwide adj web).mp. 24.(Web adj site$) or (web site$).mp. 25.(Information adj technology).mp. 26.Medical informatics/ 27.(consumer$ adj health adj informatic$).mp. 28.exp Educational Technology/ 29.Computer user training/ 30.Computer assisted instruction/ 31.("E‐health" or "electronic health" or ehealth).mp 32.User‐Computer interface/ 33.Hypermedia/ 34.(blog$ or weblog$ or web‐log$).mp 35.(bulletin board$ or bulletinboard$ or messageboard$ or message board$ or forum$).mp 36."Search engine$".mp 37.Computer literacy/ 38.Attitude to computers/ 39.or/20‐38 40.Randomized Controlled Trial.pt. 41.Controlled Clinical Trial.pt. 42.Randomized Controlled Trials.sh. 43.Random Allocation.sh. 44.Double Blind Method.sh. 45.Single Blind Method.sh. 46.or/40‐45 47.(Animals not Humans).sh. 48.46 not 47 49.Clinical Trial.pt. 50.exp Clinical Trial/ 51.(Clin$ adj25 Trial$).ti,ab. 52.((Singl$ or Doubl$ or Trebl$ or Tripl$) adj25 (Blind$ or Mask$)).ti,ab. 53.Placebos.sh. 54.Placebo$.ti,ab. 55.Random$.ti,ab. 56.Research Design.sh. 57.or/49‐56 58.57 not 47 59.58 not 48 60.Comparative Study.sh. 61.exp Evaluation Studies/ 62.Follow Up Studies.sh. 63.Prospective Studies.sh. 64.(Control$ or Prospectiv$ or Volunteer$).ti,ab. 65.or/60‐64 66.65 not 47 67.66 not (48 or 59) 68.5 and 19 and 39 69.48 or 59 or 67 70.68 and 69

Appendix 2. EMBASE (via Ovid) search strategy

1.caregiver/

2.exp disabled person/

3.(patient$ or consumer$ or carer$ or caregiver$ or lay$).mp.

4.exp patient/

5.or/1‐4

6.(Information adj4 (need$ or seek$ or us$ or util$ or literac$)).mp.

7.communicat$.mp.

8.access to information/

9.information dissemination/

10.exp "Information storage and retrieval"/

11.exp Information services/ [remove databases]

12.(educat$ or skill$ or teach$ or train$ or coach$ or instruct$).mp.

13.exp health education/

14."early intervention (education)"/

15.exp teaching materials/

16.teaching/

17.Education, distance/

18.exp education, nonprofessional/

19.or/6‐18

20.exp computers/

21.computer communication networks/ or Internet/

22.(computer$ or Internet or online).mp.

23.((world adj wide adj web) or (worldwide adj web)).mp. [mp=title, abstract, subject headings, heading word, drug trade name, original title, device manufacturer, drug manufacturer]

24.((web adj site$) or web site$).mp.

25.(Information adj technology).mp.

6.medical informatics/

27.(consumer$ adj health adj informatic$).mp.

28.exp educational technology/

29.computer user training/

30.computer assisted instruction.mp.

31.("E‐health" or "electronic health" or "ehealth").mp.

32.user‐computer interface/

33.hypermedia/

34.(blog$ or weblog$ or web‐log$).mp.

35.(bulletin board$ or bulletinboard$ or messageboard$ or message board$ or forum$).mp.

36."search engine$".mp.

37.computer literacy/

38.Attitude to computers/

39.or/20‐38

40.randomized controlled trial/

41.controlled clinical trial/

42.randomized controlled trial/

43.exp randomization/

44.double blind procedure/

45.single blind procedure/

46.or/40‐45

47.(animals not humans).mp.

48.46 not 47

49.clinical trial/

50.exp clinical trial/

51.clin$ ad25 trial$.m_titl.

52.limit 51 to abstracts

53.((singl$ or doubl$ or Trebl$ or Tripl$) adj25 (blind$ or mask$)).m_titl.

54.limit 53 to abstracts

55.PLACEBO EFFECT/ or PLACEBO/ or placebo.mp.

56.placebo.m_titl.

57.limit 56 to abstracts

58.random.m_titl.

59.limit 58 to abstracts

60.research design.mp.

61.or/49‐60

62.57 not 47

63.58 not 48

64.comparative study.mp. or exp comparative study/

65.exp evaluation studies/

66.follow up studies.mp. or follow up/

67.prospective studies.mp. or prospective study/

68.(control$ or prospective$ or volunteer$).m_titl.

69.limit 68 to abstracts

70.or/64‐68

71.70 not 47

72.71 not (48 or 63)

73.5 and 19 and 39

74.48 or 63 or 72

75.73 and 74

Appendix 3. PsycINFO (Ovid) search strategy

1. caregivers/ 2. exp patients/ 3. (Patient$ or consumer$ or carer$ or caregiver$ or lay$).mp. 4. or/1‐3 5. (Information adj4 (need$ or seek$ or us$ or util$ or literac$)).mp. 6. Information seeking/ 7. Information literacy/ 8. Communicat$.mp. 9. exp information/ 10. Information dissemination/ 11. Automated information storage/ 12. exp automated information retrieval/ 13. Information services/ 14. Databases/ 15. (educat$ or skill$ or teach$ or train$ or coach$ or instruct$).mp. 16. exp health education/ 17. exp teaching/ 18. exp education/ 19. exp training/ 20. or/5‐19 21. exp computers/ 22. Internet/ 23. (Computer$ or Internet or online).mp. 24. ((World adj wide adj web) or (worldwide adj web)).mp. 25. web sites/ 26. ((Web adj site$) or web sites$).mp. 27. exp information technology/ 28. (consumer$ adj health adj informatic$).mp. 29. Computer training/ 30. exp computer assisted instruction/ 31. ("E‐health" or "electronic health" or ehealth).mp. 32. exp human computer interaction/ 33. Hypermedia/ 34. (blog$ or weblog$ or web‐log$).mp. 35. (bulletin board$ or bulletinboard$ or messageboard$ or message board$ or forum$).mp. 36. Digital divide/ 37. Computer literacy/ 38. Computer anxiety/ 39. Computer attitudes/ 40. exp electronic communication/ 41. or/21‐40 42. exp health/ 43. health$.mp. 44. ill$.mp. 45. exp medical sciences/ 46. medic$.mp. 47. exp health care services/ 48. exp health education/ 49. or/42‐48 50. (Clin$ adj25 Trial$).ti,ab. 51. ((Singl$ or Doubl$ or Trebl$ or Tripl$) adj25 (Blind$ or Mask$)).ti,ab. 52. (random$ or control$ or intervention or compar$ or evaluat$ or experiment$ or placebo$ or prospectiv$ or volunteer$).ti,ab. 53. (research adj3 design).ti,ab. 54. Clinical trials.sh. 55. Experimental design/ 56. Empirical methods.sh. 57. Placebo.sh. 58. Time series.sh. 59. "time series".ti,ab. 60. (pre test or pretest or posttest or post test).ti,ab. 61. "0400".md. 62. "0430".md. 63. "0451".md. 64. "2000".md. 65. "2260".cc. 66. or/50‐65 67. 4 and 20 and 41 and 49 and 66

Appendix 4. CINAHL (Dialog) search strategy

1. Exp caregivers/ 2. Exp patients/ 3. ((patient* or consumer* or carer* or caregiver* or lay*)).ti,ab 4. 1 or 2 or 3 5. (Information adj4 need*).ti,ab 6. (Information adj4 seek*).ti,ab 7. (Information adj4 us*).ti,ab 8. (Information adj4 util*).ti,ab 9. (Information adj4 literac*).ti,ab 10. 5 or 6 or 7 or 8 or 9 11. Communicat*.ti,ab 12. Exp Access to information/ 13. Exp Information literacy/ 14. Exp Information storage/ 15. Exp Information retrieval/ 16. Exp Information services/ 17. ((educat* or skill* or teach* or train* or coach* or instruct* )).ti,ab 18. Exp Health education/ 19. Exp Teaching materials/ 20. Exp Teaching/ 21. Exp Education, non professional/ 22. 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 or 21 23. Exp Internet/ 24. (Computer* or Internet* or online).ti,ab 25. ((World adj wide adj web) or (worldwide adj web).ti,ab 26. ((Web adj site*) or (web site*)).ti,ab 27. (Information adj technology).ti,ab 28. Exp Medical informatics/ 29. Exp nursing informatics/ 30. (Consumer adj health adj informatic*).ti,ab 31. Exp Educational technology/ 32. Exp Computer user training/ 33. Exp Computer assisted instruction/ 34. (“E‐health” or “electronic health” or ehealth).ti.ab 35. Exp User‐computer interface/ 36. Exp Hypermedia/ 37. (Blog* or weblog* or web‐log*).ti,ab 38. (“Bulletin board*” or bulletinboard* or “message board*” or messageboard* or forum*).ti,ab 39. (Search adj engine*).ti,ab 40. Exp Computer literacy/ 41. Exp Attitudes to computers/ 42. 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32 or 33 or 34 or 35 or 36 or 37 or 38 or 39 or 40 or 41 43. 4 and 22 and 42 44. (clinic* adj9 trial*).ti,ab 45. (singl* adj9 blind*).ti,ab 46. (doubl* adj9 blind*).ti,ab 47. (trebl* adj9 blind*).ti,ab 48. (tripl* adj9 blind*).ti,ab 49. (singl*adj9 mask*).ti,ab 50. (doubl* adj9 mask*).ti,ab 51. (trebl* adj9 mask*).ti,ab 52. (tripl* adj9 mask*).ti,ab 53. 45 or 46 or 47 or 48 or 49 or 50 or 51 or 52 54. (Random* or control* or intervention* or compar* or evaluat* or experiment* or placebo* or prospectiv* or volunteer*).ti,ab   55. (Research adj3 design).ti,ab 56. “Clinical trial”.pt 57. Exp Experimental studies/ 58. Exp random sample/ 59. Exp Prospective studies/ 60. “Time series”.ti,ab 61. (“Pre test” or pretest or “post test” or posttest).ti,ab 62. Exp Quasi‐experimental studies/ 63. 44 or 53 or 54 or 55 or 56 or 57 or 58 or 59 or 60 or 61 or 62 64. 43 and 63

Appendix 5. ERIC (CSA Illumina) search strategy

1. De= “Information needs” 2. De= “Information seeking” (explode) 3. De= “Information utilization” (explode) 4. De= “User needs (information)”or De= “User satisfaction (information)” or De= “Users (information)” 5. De= “Information literacy” 6. De= “Information skills” (explode) 7. De= “Access to information” 8. De= “Information services” (explode) 9. De= “Information management” 10. De= “Information sources” (explode) 11. KW= Communicat* 12. Or/1‐11 13. De= “Computers” 14. De= “Internet” 15. KW= (Computer* or Internet or online) 16. KW= ((Web site*) or (web site*) or (World wide web) or (worldwide web)) 17. De= “Information technology” 18. De= “Information Science” (explode) 19. KW= consumer* health informatic* 20. De= “Educational technology” (explode) 21. De= “Computer uses in education” (explode) 22. KW= ("E‐health" or "electronic health" or ehealth) 23. De= “Hypermedia” 24. KW= (bulletin board* or bulletinboard* or messageboard* or message board* or forum*) 25. De= “Search engines” 26. De= “Computer literacy” 27. De= “Computer attitudes” 28. De= “Computer Mediated Communication” (explode) 29. Or/13‐28  30. De= “Health” (explode) 31. KW= health* or clinic* 32. De= “Medicine” (explode) 33. KW= Medic* 34. De= “Health services” (explode) 35. De= “Health education” (explode) 36. Or/30‐35 37. 12 and 29 and 36

Appendix 6. LISA (CSA Illumina) search strategy

1. De= “Users” (explode) 2. KW= (Patient* or consumer* or carer* or caregiver* or lay* or user*). 3. OR/1‐2 4. De= “User behaviour” (explode) 5. De= “Consumer information” (explode) 6. De= “Information literacy” 7. KW = Communicat* 8. De= “Communication” (explode) 9. De= “Information communication” (explode) 10. De= “Access to information” 11. De= “Information storage and retrieval” (explode) 12. De= “Information services” (explode) 13. De= “Databases” (explode) 14. De= Information sources (explode) 15. KW= (educ* or skill* or teach* or train* or coach* or instruct*) 16. De= “Health Education” 17. De= “Teaching aids” (explode) 18. De= “Teaching methods” (explode) 19. De= “Teaching” (explode) 20. De= “Education” (explode) 21. De= “Adult Education” 22. De= “Training” (explode) 23. OR/4‐22 24. De= “Computers” (explode) 25. De= “Internet” (explode) 26. KW= (Computer* or Internet or online) 27. De= “World Wide Web” (explode) 28. KW= ((Web site*) or (web site*) or (“World wide web”) or (“worldwide web”)) 29. De= “Information Technology” (explode) 30. De= “Medical Informatics” 31. KW= (“consumer* health informatic*”) 32. De= “Educational technology” (explode) 33. KW= ("E‐health" or "electronic health" or ehealth) 34. De= “Computerized information storage and retrieval” (explode) 35. KW= (bulletin board* or bulletinboard* or messageboard* or “message board*” or forum*) 36. De= “Searching” (explode) 37. De= “Computer mediated communication” (explode) 38. Or/24‐37 39. De= “Health” (explode) 40. KW= health* 41. KW= ill* 42. KW= clinic* 43. De= “Medicine” (explode) 44. KW= Medic* 45. De= “Health Services” 46. Or/39‐45 47. 3 and 23 and 38 and 46

Appendix 7. Grey literature search strategy

Dissertation Abstracts

Search engine would only accept a limited length search string. Terms which contributed no/few additional results were therefore removed.

1. ((inform* or communicat* or educat* or skill* or teach* or train*)) AND ((Computer* or Internet or online or web* or informatic* or search*)) AND ((health* or medic* or clinic*)) AND (patient* or consumer* or carer* or caregiver* or lay*)

Index to Theses

Strategy had to be severely limited as the search engine would not accept a longer search. Search terms were tested to find a balance between an extensive search and the limitations of the database. General search terms were chosen to deliver the greatest number of results. Stemming was turned on.

1. (Internet or online or computer or web or informatic or search) and (health or medic*) and (information or communicat* or educate or skill or train)

ISI Conference Procedings

1. TS=(Patient* or consumer* or carer* or caregiver* or lay*) 2. TS=(Information SAME (need* OR seek* OR util* OR use* OR using)) 3. TS="Information literac*" 4. TS=(Information AND skill*) 5. TS=Communicat* 6. TS=(Information AND access) 7. TS=(Information AND dissemination) 8. TS=(Information AND (storage OR retrieval)) 9. TS=(Information AND service*) 10. TS=Database* 11. TS=(Educat* or skill* or teach* or train* or coach* or instruct*) 12. Or/2‐12 13. TS=(Computer* or Internet or online) 14. TS=((World wide web) OR (worldwide web)) 15. TS=((Web site*) or (web site*)) 16. TS=”Medical Informatic*” 17. TS=(Consumer* SAME health SAME informatic*) 18. TS=("E‐health" or "electronic health" or ehealth) 19. TS=Hypermedia 20. TS=(blog* or weblog* or web‐log*) 21. TS=(bulletin board* or bulletinboard* or messageboard* or message board* or forum*) 22. TS=search engine* 23. TS= "CMC" OR "electronic communication" 24. Or/13‐23 25. TS=health* 26. TS=Ill* 27. TS=Clinic* 28. TS=Medic* 29. Or/13‐28 30. 1 and 12 and 24 and 28

SIGLE

All searches in keyword.

1. ((Information) AND (need* or seek* or us* or util* or literac* or skill* or access or dissemination or storage or retrieval or service)) OR (communicat*) OR (Educat* or skill* or teach* or train* or coach* or instruct*) 2. (Computer* or Internet or online or "world wide web" or "worldwide web" or "web site" or web site or informatic* or "e‐health" or "electronic health" or ehealth or hypermedia or blog* or weblog* or web‐log* or bulletin board* or bulletinboard* or messageboard* or message board* or forum* or "search engine" or "electronic communication") 3. (health* or medic* or ill* or clinic*) 4. 1 and 2 and 3

Appendix 8. Clinical Trials search strategy

Controlled Clinical Trials

We decided to only search this database using computer related terms and hand‐search the results for relevant interventions. This was because the search engine would only accept a short single search term and we found that searching using general terms such as 'information' or 'communication' returned the entire database.

1. (computer% or Internet or online or web% or search% or informatic%)

UK Clinical Research Network

At the time we carried out this search, the search function was not working correctly, and the web site said that not all studies had been indexed. As there were a relatively small number of studies on the site, we therefore downloaded details of all the trials (in Excel format), removed studies that were classified as observational, and hand searched the rest.

Cochrane Central Register of Controlled Trials (CENTRAL) and Consumers and Communication Group specialised register, The Cochrane Library

1. (Patient* or consumer* or carer* or caregiver* or lay*): ti,ab,kw 2. (Information NEAR/4 (need* or seek* or us* or util* or literac*)): ti,ab,kw 3. Communicat*: ti,ab,kw 4. (Information AND (skill* or access or dissemination or storage or retrieval or service):ti,ab,kw 5. Database*:ti,ab,kw 6. (educat* or skill* or teach* or train* or coach* or instruct*): ti,ab,kw 7. Or/2‐6 8. (Computer* or Internet or online) 9. “World wide web” OR “Worldwide web”. ti,ab,kw 10. “Web site*” or web site*.ti,ab,kw 11. “Medical informatics*”.ti,ab,kw 12. “consumer* health informatic*”.ti,ab,kw 13. "E‐health" or "electronic health" or ehealth.ti,ab,kw 14. Hypermedia.ti,ab,kw 15. blog* or weblog* or “web‐log*”.ti,ab,kw 16. “bulletin board*” or bulletinboard* or messageboard* or “message board*” or forum*.ti,ab,kw 17. “Search engine*”.ti,ab,kw 18. "CMC" OR "electronic communication".ti,ab,kw 19. Or/8‐18 20. 1 and 7 and 19 21."SR‐COMMUN" in Clinical Trials 22. 21 and 22

Appendix 9. Differences between Protocol and Review

Developing the protocol we were expecting to identify several types and a considerable number of studies, so we described the methods for including and analysing them. The limitations we experienced in applying our protocol are described below:

1. The quality of quasi‐randomised controlled trials would have been assessed using the criteria for RCTs outlined above, omitting the assessment of randomisation.

2. The quality of ITS studies would have been assessed using the following criteria (EPOC 2002; EPOC 2007; Ramsay 2003):

   1. Protection against secular change, i.e. whether the intervention occurred independently of other changes.

   2. Appropriate data analysis. Appropriate methods are auto regressive integrated moving average (ARIMA) models or time series regression models with adjustment for serial correlation.

   3. Number of points pre‐ and post‐intervention. This will be recorded as adequate if a rationale for the number of points is stated or a sample size calculation was performed.

   4. Shape of the intervention effect. This will be recorded as adequate if the authors provide a rationale for the shape of the effect.

   5. Protection against detection bias. We will record separately whether the intervention itself is unlikely to affect data collection and whether the assessment of primary outcomes was blinded.

   6. Completeness of data set. This will be recorded as adequate if the data set covers 80 to 100% of the total number of participants or episodes of care in the study.

   7. Reliable primary outcome measures. As per CBA quality criteria.

3. Results from different study designs would be reported as follows. Individual RCTs, cluster RCTS, quasi‐randomised controlled trials and CBA studies as mean differences (continuous variables) or odds ratios (dichotomous variables) with 95% confidence intervals. ITS studies as mean differences in before and after values.

4. If there were sufficient studies the results would be presented as follows in a table to support a narrative synthesis. Data would be grouped within results tables with respect to study design and then by type of intervention. If there were sufficient appropriate studies they would also be categorised based on the four categories of internet health seeker, i.e. whether the intervention is aimed at people with long‐term or acute illnesses, carer, or the currently well. Within these categories the results would be further structured to reflect the comparisons detailed in the 'Types of Participant' and 'Types of Intervention' sections. We will separately present the results of studies that compare intervention to no intervention, those that compare intervention to other forms of consumer training (e.g. information literacy training) and those that compare two or more types of consumer internet skills interventions. We will also present separately the results of studies in people aged under or over 55 years, studies of hypothetical versus real information need, people searching for information for themselves versus for others, and studies exploring the immediate effects of training versus long‐term effects.

5. If the studies are sufficiently homogenous with respect to populations, inclusion criteria, interventions and/or outcomes, we would pool the data statistically using meta‐analysis. We would perform a formal random‐effects model meta‐analysis, which will report pooled mean differences (continuous variables using the same scale) or standardised mean differences (continuous variables using different scales) or odds ratios (dichotomous variables) and 95% confidence intervals. Numbers needed to treat (NNT) for benefit or harm will also be calculated if follow‐up periods are comparable. If necessary, we would account for the effects of clustering in cluster randomised trials by adjusting each trial to its 'effective sample size' using intra‐class coefficients where available, or external estimates from similar studies (Deeks 2006). We would analyse separately the comparisons detailed in the previous paragraph. Separate meta‐analyses will be carried out for each of the five primary outcomes and for the seven categories of secondary outcomes. The decision to carry out meta‐analyses will be made by consensus of all the review authors.

6. The authors of the one included RCT (Kalichman 2006) carried out an intention‐to‐treat analysis, and it is not appropriate to do so on a CBA study. Where appropriate we would have sought to carry out an intention‐to‐treat analysis where this had not been reported by the study authors. If the data is available in the paper, we would have carried out a full ITT analysis. If necessary, we will contact the authors to obtain the data necessary for this analysis. If sufficient data were not available, we would not impute the missing data (Deeks 2006). Instead we will carry out an available case analysis and consider the implications of the missing data in the review Discussion.

7. If a meta‐analysis was performed, statistical heterogeneity would have been examined using the I² statistic (Higgins 2002). This statistic gives the percentage of the variability in effect estimates that can be attributed to heterogeneity rather than chance. A value greater than 50% would be considered substantial heterogeneity (Deeks 2006). If statistical heterogeneity was found to be present, this would be further explored using analysis of a priori subgroup and sensitivity analyses. As age is known to predict internet use and is hypothesised to affect outcomes of the intervention, we would, if possible, conduct a subgroup analysis comparing participants aged less than 55 years with those aged over 55 years. We would also conduct sensitivity analyses by excluding unpublished trials (if there were any), and by excluding studies of poor quality or less rigorous study design. Less rigorous study designs would be considered to be non‐RCTs. RCTs would be separated into high, moderate and low risk of bias as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2008b), based on the number of quality criteria rated as adequate, inadequate or unclear for each study.

8. We would have investigated the effects of publication bias in the review using a funnel plot (Deeks 2006). We would then have discussed the results of the funnel plot and possible explanations for the results, which includes publication bias but also other sources of bias such as diverse methodological quality.

Data and analyses

Comparison 1.

Effects of interventions to enhance consumer skills compared with control

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Self‐efficacy for health information seeking	1	448	Mean Difference (IV, Fixed, 95% CI)	1.10 [0.64, 1.56]
2 Health information evaluation skills	1	448	Mean Difference (IV, Fixed, 95% CI)	0.60 [0.11, 1.09]
3 Number of times the patients discussed online information	1	448	Mean Difference (IV, Fixed, 95% CI)	0.70 [0.22, 1.18]
4 Affective depression (showing symptoms the previous 7 days)	1	448	Mean Difference (IV, Fixed, 95% CI)	‐0.10 [‐0.87, 0.67]
5 Total Internet use	1	448	Mean Difference (IV, Fixed, 95% CI)	3.50 [‐1.56, 8.56]
6 Used email	1	448	Mean Difference (IV, Fixed, 95% CI)	2.60 [‐0.12, 5.32]
7 Searched for online health information	1	448	Mean Difference (IV, Fixed, 95% CI)	1.5 [0.78, 2.22]
8 Searched for clinical trials	1	448	Mean Difference (IV, Fixed, 95% CI)	0.6 [‐0.01, 1.21]
9 Information seeking coping (likelihood of using strategy)	1	448	Mean Difference (IV, Fixed, 95% CI)	0.10 [‐0.21, 0.41]
10 Ways of coping ‐ Problem focused coping (likelihood of using strategy)	1	448	Mean Difference (IV, Fixed, 95% CI)	0.10 [‐0.04, 0.24]
11 Ways of coping ‐ Information coping (likelihood of using strategy)	1	448	Mean Difference (IV, Fixed, 95% CI)	0.20 [0.07, 0.33]
12 Ways of coping ‐ Social coping (likelihood of using strategy)	1	448	Mean Difference (IV, Fixed, 95% CI)	0.10 [‐0.04, 0.24]
13 Social support (intensity of social support)	1	448	Mean Difference (IV, Fixed, 95% CI)	2.10 [0.56, 3.64]
14 Readiness to adopt the internet for preventive health	1	22	Mean Difference (IV, Fixed, 95% CI)	34.19 [23.24, 45.14]

Analysis 1.10.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 10 Ways of coping ‐ Problem focused coping (likelihood of using strategy).

Analysis 1.11.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 11 Ways of coping ‐ Information coping (likelihood of using strategy).

Analysis 1.12.

Comparison 1 Effects of interventions to enhance consumer skills compared with control, Outcome 12 Ways of coping ‐ Social coping (likelihood of using strategy).

History

Protocol first published: Issue 2, 2008 Review first published: Issue 6, 2011

Differences between protocol and review

There are a number of differences between the protocol and this review. The changes mainly relate to data analysis and collection, and are a result of the small number and type of included studies. Future updates of the review may need to incorporate these methods if new studies are included. See Appendix 9.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Cortner 2006

Methods	Study design: 2 group controlled before and after study Location: 1 site in Minnesota, USA.
Participants	Description: People aged 50+ living in the community. Category of internet health seeker: General. N allocated to groups: 24 (12 intervention group, 12 control group). N completed study: 22 (11 intervention group, 11 control group). In both groups participants withdrew post‐allocation but pre‐intervention (no data available). Age: Range = 62 to 86, mean = 71.54. Gender: 18 Female; 4 Male. Ethnicity: Unknown. Health problems: None specified. Inclusion criteria: Aged 60+ years, living or working in the city of Maple Grove, Minnesota. Availability at the time of the classes. Basic internet knowledge.
Interventions	Description: Educational intervention, based on Diffusion of Innovation (Rogers 2003) and Stages of Change Theories (Prochaska 2001). Aim: To prepare older adults for adopting the internet for their preventive health information needs. Content: Week 1: How to search the Web for trustworthy health information; Evaluating health information on the World Wide Web. Week 2: How do you find reliable health web sites?; Visit to CDC’s interactive Growing Stronger Program; Importance of evidence‐based/best practice recommendations. Week 3: What to look for in evaluating the content on a health web site and use of an evaluation checklist; Using Search Engines; MedlinePlus. Week 4: www.health.gov and www.MyPyramid.gov; Mayo Clinic for interactive calculators; Special interests and questions. Format: Adult education internet classes administered to a group of 6 participants. Classes included 'hands on practice' of skills, but no further details were given about the format of classes. Source: Course developed by the study author. Setting: SeniorNet computer centre. Hypothetical/real information need: Hypothetical, although topics were chosen for relevance to seniors. Searching for self/others: Self. Delivery: Four weekly sessions of 90 minutes in length. Provider: Delivered by the lead researcher, with support from independent observers Intervention quality: Was the intervention delivered as intended? Yes, probably. Was the provider trained to deliver the intervention? Yes, although not a trained teacher. Did delivery of the intervention differ across settings or population? No. What was the evidence base for the intervention? Mainly theoretical, this was an exploratory study. Control group: Waiting list ‐ offered intervention after 1 month.
Outcomes	Outcome: Individual readiness to adopt the internet as a tool for preventive health information. Methods of assessing outcome: Self‐report questionnaire based on diffusion of innovation and stages of change ‐ 10 items, grouped into 5 constructs. Each item measured with a 10‐point semantic differential scale. Methods of follow‐up for non‐respondents: Not applicable. Timing of outcome assessment: Baseline and immediately post‐intervention (4 weeks). (Additional tests were administered at the start of each class but that data are not included in this review). Adverse events: None stated.
Notes	Contact with author: Yes  ‐ individual participant data obtained and used to calculate SD.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	No sequence generation.
Allocation concealment (selection bias)	High risk	No allocation concealment.
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding.
Incomplete outcome data (attrition bias) All outcomes	High risk	No.
Selective reporting (reporting bias)	Low risk	Outcome (stated in the 'purpose of the study' section) was referred to in the results section.
Other bias	High risk	Investigator bias ‐ investigator served as instructor/observer but supported by two unbiased observers.
Baseline measurements comparable?	Low risk	No significant differences between the groups with respect to age, gender, and pre‐test measurements.
Second site comparable?	Unclear risk	Cannot be determined from the paper.
Protection against contamination?	Unclear risk	Insufficient evidence to assess potential contamination between groups. It is possible that participants in the different groups had contact with each other as they were all recruited via the mailing list of a single Community Centre, but impossible to judge if the control group received any aspects of the intervention.
Outcome measures reliable?	Unclear risk	No assessment of reliability given. First use of the instrument.
Adequate follow‐up?	Low risk	92% of participants completed the study. 1 person withdrew from each group pre‐intervention.
Ethical approval obtained?	Low risk	Yes.
Outcome measures validated?	High risk	First use of the measure.

Kalichman 2006

Methods	Study design: 2 arm randomised controlled trial. Location: 1 site in Atlanta, USA.
Participants	Description: HIV positive consumers. Category of internet health seeker: Long‐term. Setting: AIDS service centre (community centre). N screened: 835. N eligible: 527. N excluded: 308. N refused: 79. N randomised to control: 231. N randomised to intervention: 217. Excluded post‐randomisation: none. Lost to follow‐up: 98 at the end of intervention, 123 at 9‐months follow‐up. Included in analysis: 448 (intention to treat, missing values estimated with a linear regression procedure for multiple imputations). Age: Mean = 42.5 years (SD = 6.9). Gender: 320 male; 116 female; 12 trans gender. Ethnicity: 399 African American; 36 White American; 13 other. Health problems: HIV positive. Stage of illness: Mean years since diagnosis = 10.2 (SD = 5.6). 54% had CD4 counts < 200; 28% had undetectable viral load. Treatment: 68% taking anti‐retroviral medications. Other social/demographic details: N = 354 had income <$10,000 per year Mean years of education = 12.2 (SD = 1.9). Inclusion criteria: HIV+; >75% on TOFHLA, mean = 85.5 (intervention, SD 15.6) and 83.3 (control, SD 20.3); used internet < 3 times in the last month.
Interventions	Description: Educational intervention. Skills based, with aim of developing self‐efficacy and skills for health information seeking on the basis of principles from social‐cognitive theory. (p545) Content: Introduction to computers/internet (including motivation), evaluating the quality of online information, using the internet to seek health information and support (including searching, evaluating quality, taking information to health professionals), skills for developing online social support and gaining personal internet access. Format: Groups of 6 to 10 mixed gender participants with 2 facilitators (at least 1 HIV positive peer counsellor). Delivered using facilitator modelling, small group practice and role plays. Source: Developed by the authors, piloted in previous studies. Setting: HIV+ service centre, in a specially set‐up community technology access centre. Hypothetical/real information need: Real ‐ content of intervention was specific to HIV. Searching for self/others: Self. Delivery: 8 x 120 minute sessions, twice weekly for 4 consecutive weeks. Provider: 2 facilitators (at least 1 HIV positive peer counsellor) per group. Intervention quality: Was the intervention delivered as intended? Not stated but delivery was monitored by post‐session briefings and weekly supervision of facilitators. Was the provider trained to deliver the intervention? Yes. Did delivery of the intervention differ across settings or population? Single setting. Standardised manuals were used. What was the evidence base for the intervention? Primarily theory driven. Evidence from previous work by the same authors showed that the intervention was plausible, and that use of the internet for health info is associated with self‐efficacy for info seeking, problem focused coping, and greater social support in people with HIV. Control: HIV information support group. Time and contact matched social support group. Sessions included information about HIV, management of health problems, nutrition, medication access and adherence strategies including requesting information from health profs. Participants met in a room in the same building as the intervention group, and had a tour of/access to the technology centre. Not skills based.
Outcomes	Outcome measures considered in this review: Self‐efficacy for information seeking Health information evaluation skills Internet use Information‐seeking coping Ways of coping Social support Affective depression Methods of assessing outcomes: Instructor‐guided self‐administration procedure Self‐efficacy for health information seeking ‐ 4‐item Likert scale. Health information evaluation skills ‐ calculation of the difference between participants' quality rating of the visited websites. Internet use  ‐ use of internet for any purpose, for health information, for email, for contact with health professionals. Information‐seeking coping ‐ adapted from Miller 1988, 3‐item Likert scale in response to a vignette about changing health status. Ways of coping ‐ measure drawn from Ways of Coping Inventory (Lazarus 1984), 17‐item Likert scale. Social support ‐ Social Support Questionnaire (Brock 1996), 15‐item Likert scale. Affective depression ‐ 6 items taken from the CES‐D (Radloff 1986). Methods of follow‐up for non‐respondents: Not stated. Timing of outcome assessment: All measures administered at baseline, 3, 6 and 9 months except group experience measures (immediate post intervention only) and internet skills evaluation assessment (baseline, 3 month follow up only). Adverse events: None stated.
Notes	Contact with author: Yes ‐ information obtained. Power calculation: Use of power calculation stated but details not given.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Generated by random number sequence.
Allocation concealment (selection bias)	Unclear risk	Condition coded label generated from the random number sequence ‐ no further information given.
Blinding (performance bias and detection bias) All outcomes	Low risk	Participants: Blinded to purpose of study and experimental manipulation. Providers: Intervention/control facilitators crossed groups so were unblinded. Assessors: Blinded to allocation.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	158/217 participants completed the intervention condition, 192/231 completed the control condition. No differences between conditions at follow‐ups. Substantial difference in numbers completing the intervention and control and information unavailable about reasons participants lost to follow‐up, but used intention‐to‐treat analysis with missing data imputed using linear regression.
Selective reporting (reporting bias)	Unclear risk	Outcomes in the methods section match those reported in the results. Non‐significant results are reported in the same table as significant results (given same weight). Unable to locate trial protocol. Insufficient evidence to assess risk of bias.
Other bias	Low risk	No other sources of bias could be identified.
Baseline measurements comparable?	Low risk	No significant differences between groups in age, education, literacy, HIV status, gender, ethnicity, income, employment, drug use history, sexuality.
Second site comparable?	Unclear risk	Establishment of an Community Technology Access Center (CTAC) for the Intervention group; the control group had access and guided tour to the CTAC but meet in a different room.
Protection against contamination?	Low risk	Potential contamination monitored in post‐session briefings and facilitator supervision.
Outcome measures reliable?	Unclear risk	Inconsistent reporting of reliability of the different measures.
Adequate follow‐up?	Unclear risk	73 % (intervention) and 83 % (control) completed study. Follow‐up at 3, 6 and 9 months.
Ethical approval obtained?	Unclear risk	Not mentioned in the publication.
Outcome measures validated?	Low risk	Half of the measures had been used in other studies or were validated instruments. The other measures had been developed and tested prior to use.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Eysenbach 2002c	Trial discontinued due to recruitment problems.
Masi 2003	Intervention involved training in use of WebTV; not finding, evaluating or using online health information.

Contributions of authors

The review was conceived and designed by JC, AM and AC. AC produced the first draft of the protocol. AC, JC, and AM substantially contributed to and edited later drafts of the protocol. AC and CU carried out the selection and data extraction, with EK acting as adjudicator. AC wrote the first draft of the review, with all other co‐authors contributing to later drafts. BL amended and finalised the review after peer review. The current version has been agreed by all co‐authors.

BL is guarantor for this review and JC will be responsible for leading any future update of this review.

Sources of support

Internal sources

• Department of Primary Care and Public Health, Imperial College London, London, UK.

  The review received a partial financial contribution from The Department of Primary Care and Public Health, Imperial College London. The Department of Primary Care & Public Health at Imperial College is grateful for support from the NIHR Collaboration for Leadership in Applied Health Research & Care (CLAHRC) Scheme, the NIHR Biomedical Research Centre scheme, and the Imperial Centre for Patient Safety and Service Quality

• Department of Family Medicine, University of Ljubljana, Slovenia.

  General support

External sources

• NHS Connecting for Health Evaluation Programme (NHS CFHEP 001), UK.

  <http://www.haps.bham.ac.uk/publichealth/cfhep>

Declarations of interest

None known

New