Do Internet interventions for consumers cause more harm than good? A systematic review

Abstract

Objective To systematically review the effect of consumer use of online health information on decision‐making, attitudes, knowledge, satisfaction and health outcomes and utilization.

Search strategy Electronic databases searched included the Cochrane Controlled Trials Register, MEDLINE, PREMEDLINE (to 14 March 2001), CINAHL, Australian Medical Index, Health and Society, National Institutes of Health Clinical Trials Database and CenterWatch.

Inclusion criteria All post‐1995 comparative studies (including controlled studies, before and after studies, and interrupted time series analyses) of Internet users vs. non‐Internet users and other communications mediums, and Internet characteristics such as e‐mail vs. other communication mediums, were included. Outcomes included consumer decision‐making, attitudes, knowledge, satisfaction and measurable changes in health status or health utilization.

Data extraction and synthesis One reviewer screened all papers then two reviewers independently assessed studies against the selection criteria and any discrepancies were resolved by discussion with a third reviewer. No attempt was made to combine the data for further statistical analysis.

Main results We identified 10 comparative studies. Studies evaluated the effectiveness of using the Internet to deliver a smoking cessation programme, cardiac and nutrition educational programmes, behavioural interventions for headache and weight loss, and pharmacy and augmentative services. All studies showed some positive effects on health outcomes, although the methodological quality of many studies was poor.

Conclusions Despite widespread consumer Internet use to obtain health‐care information, there is almost a complete lack of evidence of any effects this may have on health outcomes.

Introduction

In November 2000 more than 400 million people around the world accessed the Internet. ¹ The Internet is a communications medium that delivers information and services via text, graphics, sound and video. However, unlike other forms of communication the Internet is a global, open, public environment forum. The Internet also has multiple, discrete, characteristics such as the World Wide Web, e‐mail, chat rooms and discussion lists.

The Internet provides consumers with access to a wealth of health information and services whereby consumers can now access the same online information as health‐care providers. Consumers use the Internet‐based information to shape treatment preferences and select health‐care providers. ^{2 , 3} Consumers seek online consultations, support, self‐management and screening tools, and buy medicines online. ⁴ , 5, 6, ^–7 Health‐care services can now be delivered by virtual health‐care providers, located anywhere in the world, to consumers who never have to leave home. Pragmatically the Internet is becoming much more than just a communications medium.

Consumer use of the Internet has the potential to benefit or harm consumers who use it to manage their health. The quality of information and services on the Internet is variable and not regulated. ^{8 , 9} Legislation aimed at protecting consumers from fraudulent health‐care claims and practices lacks the development of technology and is difficult to enforce due to the global and open nature of the Internet. ¹⁰

In countries including Australia, Canada, Denmark, Finland, Iceland, Netherlands, Norway, Sweden and the United States of America (USA) more than 40% of the population is estimated to use the Internet. ¹ Two USA studies conducted in 1999 and 2000 found that 31% (aged less than 60 years, n = 1237) and 27% (n = 12751) of all American adults, respectively, sought health information via the Internet. ^{11 , 12} Furthermore, both studies determined that Internet use is associated with being younger, more affluent and better educated.

As Internet access increases and consumer demand for online health services and information grows there is an urgent need for information about whether this causes more harm than good. Our primary hypothesis is that consumer access to the extensive and variable quality of health information and services available via the Internet affects their decision‐making, attitudes, knowledge, satisfaction and health outcomes and utilization. Our secondary hypothesis is that the discrete communication features of the Internet also affect these outcomes.

The materials for this systematic review are all the post‐1995 comparative studies of consumer use of online health information and services. We examine Internet users vs. non‐users, Internet users vs. other communication mediums and Internet users using discrete characteristics of the Internet. For example, we are interested in studies that compare consumers who use online smoking cessation programmes vs. face‐to‐face counselling and consumers who communicate with doctors via e‐mail vs. the telephone.

Methods

Inclusion criteria

We include all comparative studies of consumers using the Internet to access healthcare information and services. We included controlled studies, before and after studies, and interrupted time series analyses of Internet users vs. non‐Internet users, Internet vs. other communication mediums and specific Internet characteristics such as e‐mail vs. other communication mediums. Outcomes of interest included consumer decision‐making, attitudes, knowledge, satisfaction and measurable changes in health status or health utilization.

Exclusion criteria

Non‐Internet computer‐based consumer information systems, e.g. information kiosks, Intranets and CD‐ROMs, were excluded. Whilst these might be good quality resources, their access is restricted and evaluating their use does not inform us about the risks and benefits of public access to the variable quality of information and services on the Internet. A suitable analogy is that studying the effects of access to a library is not the same as studying the effects of access to four specifically chosen books. We did not consider studies of Internet use by health professionals because we perceive them to have skills and training that enable them to interpret the quality of health information in a manner different to the general public. Studies pre‐1995 have been excluded because at that time the Internet was not widely used by the general public. In 1995 there were less than 10 million computers with registered Internet service provider addresses, compared with more than 100 million at the end of the year 2000. ¹³ The World Wide Web was developed in 1991 and the first commercial search engines only became available in 1995. ¹⁴

Identification of studies

Studies were identified from the Cochrane Controlled Trials Register (Cochrane Library Issue 1, 2001) and MEDLINE (1966, December, week 4, 2000), PREMEDLINE (14 March 2001), CINAHL (February 2001), Australian Medical Index (February 2001), Health and Society (January 2001) and the National Institutes of Health Clinical Trials Database (14 March 2001) available at http://www.clinicaltrials.gov and CenterWatch Clinical Trials Listing Services (14 March 2001) available at http://www.centerwatch.com/patient/trials.html.

The MEDLINE search terms included Internet, online, World Wide Web, web, e‐mail or e‐mail: or electronic mail, mail list, discussion list, chat room and newsgroup (tw). These terms were combined with study terms including evaluation studies, intervention studies, cohort studies, controlled studies or study, comparative study, before and after study, clinical trial and pilot projects (tw). The combined results were restricted to years 1995 to 14 March 2001. Searching was not restricted by language. The terms Internet, online, World Wide Web and web were limited to (ti) to reduce the number of citations from approximately 2500 to 597. The strategy designed for MEDLINE was adapted and used to search all other databases. Copies of the searches are available on request. All searches were conducted in March 2001.

Study selection and data extraction

Citations retrieved by the searches were downloaded via bibliographic software (EndNote 4.0) to form a single library from which duplicates were removed. The first reviewer screened each citation for possible inclusion according to the selection criteria and only those studies that did not meet the selection criteria were discarded. Full texts of the remaining citations were sought and independently appraised by two of us (TB and SM). Discrepancies were resolved by discussion with CS.

Ethics approval

The Standing Committee on Ethics in Research Involving Humans at Monash University granted ethics approval for this project and survey questions.

Statistical analysis

We intended to assess the relative contribution of included studies to describe the current state of knowledge and to generate hypotheses for future research using a random effects model, but the nature of the studies did not allow data pooling or further statistical analysis. ^{15 , 16}

Description of studies

Our search identified 599 citations from which we identified 10 comparative studies. ¹⁷ , 18, 19, 20, 21, 22, 23, 24, 25, ^–26 We also identified 44 descriptive studies (Appendix 1) and two case reports. ^{27 , 28} The majority of citations were excluded because they were concerned with `webs' of a non‐Internet nature or non‐consumers. Although it was not our intention to identify descriptive studies as part of this review we have included a list of such studies for interest, as they may be the precursors to future comparative studies. We also identified two further comparative studies; the first is in progress and will conclude in the later half of 2001, ²⁹ whilst the second study's results were reported in a press release but not in the medical literature. ³⁰

Results

All proposed outcomes and reported results of comparative studies are shown in 1, 2, 3Table 1.

Table 1.

 Characteristics of included studies

Table 2.

Table 1 (Continued)

Table 3.

Table 1 (Continued)

Takahashi demonstrated that the Internet is a medium in which effective smoking cessation intervention can be delivered. They reported that 52% of smokers and 43% of heavy smokers had quit 12 months after enrolling in an online smoking cessation programme. This programme was not limited to a restricted sample size and was accessible to all members of the public able to access the Internet in a convenient and timely manner.

Cellio, Winzelberg and Winett examined the effect of delivering education via the Internet on the eating habits, body image or physical activity of young women and all three studies found positive outcomes in the Internet intervention group. Scherrer‐Bannermann also examined the effect of delivering education via the Internet, comparing Web‐based and printed patient education manuals for consumers on cardiac surgery waiting lists. They reported that the Internet‐based intervention provided increased social support, decreased social anxiety and improved attitudes towards surgery.

Tate compared the use of a 6 month weight loss Internet‐based education programme and behaviour therapy intervention, comprising e‐mail‐facilitated weekly lessons, support and bulletin board, with an Internet‐based education programme only. They showed that the additional Internet‐based resources did improve the numbers of people achieving weight loss goals and mean weight loss 6 months post‐intervention.

The quality of the remaining four studies was poor. MacKinnon showed that e‐mail is a useful adjunct for some people with disabilities when using an augmentative communication service. Wagner compared the structural and cost outcomes of online and traditional USA pharmacies and showed that online pharmacies have higher consumer costs due to handling and delivery charges but more comprehensive consumer medicines information. However, the small sample sizes in the MacKinnon and Wagner studies do not allow robust conclusions to be drawn. Jones compared Internet users vs. outpatients seeking Viagra repeats; however, the control group of inner‐city outpatients is likely to have lower health, social and educational status than Internet users. Furthermore, consumers who experienced an adverse event are unlikely to request a repeat prescription. Strom conducted a controlled trial for self‐help treatment of recurrent headache for consumers randomized to an Internet‐based intervention or waiting list but 56% of participants dropped out of the study. Overall the results of these four studies are not reliable.

Discussion

The strongest finding to emerge from this systematic review is the lack of rigorous research regarding the effects of consumer Internet use on health outcomes. Given the widespread use of the Internet and the claims that are frequently made about its benefits as a tool for communicating health information and empowering consumers in health‐care decision‐making, this lack of research evidence is disturbing.

Firstly, this lack of research may be due to the provision of consumer Internet health‐care services being a recent and evolving phenomenon. Secondly, just as the Internet‐based services themselves are evolving, new research methodologies may be required to identify and track users prospectively in order to obtain health outcome measurements.

From the studies we did identify, there was some evidence that Internet‐based materials may provide consumers with necessary information and support in a timely manner to achieve positive health outcomes. Although the overall effectiveness of the Internet on health service utilization outcomes was not clear, the results indicate that the Internet may impact on the overall cost effectiveness of health service provision and workflow practices.

Conclusion

We need to know how and why consumers use the Internet for health‐care information and services, what subsequent actions they take, and how this affects their behaviour and health status. As increasing numbers of people use the Internet the public health benefits and risks of online health‐care should be examined by measuring changes in health status, health service utilization, costs, decision‐making models and consumer satisfaction. Indicators such as the number of adverse events, amount of quality online health information, number of licensed online health‐care providers and the implementation of online practice standards may also be used to evaluate the effectiveness of the Internet as a health‐care intervention in its own right.

At present, there is almost no evidence regarding the effect of consumer Internet use on health outcomes. Well‐designed controlled studies, instead of anecdotes and opinions, about the risks and benefits of using the Internet are urgently needed.

Acknowledgements

We wish to thank the Australian Commonwealth Department of Health and Aged Care for the provision of a Quality Use of Medicines Scholarship to Tracey Bessell.

Appendix 1

Descriptive studies

Bell DS, Kahn CE, Jr. Health status assessment via the World Wide Web. Proceedings/AMIA Annual Fall Symposium, 1996: 338–42.

Cimino JJ, Li J, Mendonca EA, Sengupta S, Patel VL, Kushniruk AW. An evaluation of patient access to their electronic medical records via the World Wide Web. Proceedings/AMIA Annual Symposium, 2000.

Culver JD, Gerr F, Frumkin H. Medical information on the Internet: a study of an electronic bulletin board [see comments]. Journal of General Internal Medicine, 1997; 12: 466–70.

Cunningham JA, Humphreys K, Koski‐Jannes A. Providing personalized assessment feedback for problem drinking on the Internet: a pilot project. Journal of Studies on Alcohol, 2000; 61: 794–798.

Curro V, Mauro V, Buonuomo PS et al. Accessing information on medical/health web sites by Italian families: a pilot study [letter]. Clinical Pediatrics, 2000; 39: 68–9.

D'Alessandro DM, Qian F, D'Alessandro MP et al. Performing continuous quality improvement for a digital health sciences library through an electronic mail analysis. Bulletin of the Medical Library Association, 1998; 86: 594–601.

Desai NS, Dole EJ, Yeatman ST, Troutman WG. Evaluation of drug information in an Internet newsgroup. Journal of the American Pharmaceutical Association, 1997; NS37: 391–4.

Dickerson SS, Flaig DM, Kennedy MC. Therapeutic connection: help seeking on the Internet for persons with implantable cardioverter defibrillators. Heart & Lung: Journal of Acute & Critical Care, 2000; 29: 248–55.

Dissanayake VH, Jayasekara RW. Pattern of use of medical information made available via an Internet website. Ceylon Medical Journal,1999; 44: 14–7.

Duffy CC, McLernon NF, D'Orsogna LJ et al. Internet use by radiation oncology patients: a pilot study [letter]. Medical Journal of Australia, 2000; 172: 350–1.

Ellis JE, Klock PA, Mingay DJ, Roizen MF. Use of electronic mail for postoperative follow‐up after ambulatory surgery. Journal of Clinical Anesthesia, 1999; 11: 136–9.

Eysenbach G. Diepgen TL. Patients looking for information on the Internet and seeking teleadvice: motivation, expectations, and misconceptions as expressed in E‐mail:s sent to physicians. Archives of Dermatology, 1999; 135: 151–6.

Eysenbach G, Diepgen TL. Evaluation of cyberdocs [letter]. Lancet, 1998; 352: 1526.

Feil EG, Glasgow RE, Boles S, McKay HG. Who participates in Internet‐based self‐management programs? A study among novice computer users in a primary care setting. Diabetes Educator, 2000; 26: 806–11.

Fernsler JI, Manchester LJ. Evaluation of a computer‐based cancer support network. Cancer Practice, 1997; 5: 46–51.

Finkelstein J, Cabrera MR, Hripcsak G. Internet‐based home asthma telemonitoring: can patients handle the technology? Chest, 2000; 117: 148–55.

Finkelstein J, Hripcsak G, Cabrera MR. Patients' acceptance of Internet‐based home asthma telemonitoring. Proceedings/AMIA Annual Symposium, 1998: 336–40.

Finn J. An exploration of helping processes in an online self‐help group focusing on issues of disability. Health & Social Work, 1999; 24: 220–31.

Gammon D, Rosenvinge JH. Is the Internet of any help for persons with serious mental disorders? Tidsskrift for Den Norske Laegeforening, 2000; 120: 1890–2.

Giordano NA. An investigation of the health concerns of the menopause discussion group on internet. Columbia University Teachers College, 1995; 230.

Gottlieb S. Study explores internet as a tool for care of diabetic patients [news]. Western Journal of Medicine, 2000; 173: 8–9.

Helwig AL, Lovelle A, Guse CE, Gottlieb MS. An office‐based Internet patient education system: a pilot study. Journal of Family Practice, 1999; 48: 123–7.

Henkel J. E‐mail:, snail mail, phone, or Fax:. Reporting adverse reactions and other product problems. FDA Consumer, 1998; 32: 7–9.

Johnson KB, Ravert RD, Everton A. Hopkins Teen Central: assessment of an internet‐based support system for children with cystic fibrosis. Pediatrics, 2001; 107(2).

Klemm P, Hurst M, Dearholt SL, Trone SR. Cyber solace: gender differences on Internet cancer support groups. Computers in Nursing, 1999; 17: 65–72.

Klemm P, Hurst M, Dearholt SL, Trone SR. Gender differences on Internet cancer support groups. Computers in Nursing, 1999; 17: 65–72.

Klemm P, Reppert K, Visich L. A nontraditional cancer support group. The Internet. Computers in Nursing, 1998; 16: 31–6.

Kraut R, Patterson M, Lundmark V et al. Internet paradox. A social technology that reduces social involvement and psychological well‐being? American Psychologist, 1998; 53: 1017–31.

Kushniruk AW, Patel VL, Cimino JJ. Evaluation of Web‐based patient information resources: application in the assessment of a patient clinical information system. Proceedings/AMIA Annual Symposium, 2000.

Lange A, van de Ven JP, Schrieken BA, Bredeweg B, Emmelkamp PM. Internet‐mediated, protocol‐driven treatment of psychological dysfunction. Journal of Telemedicine & Telecare, 2000; 6: 15–21.

Lehmann CU, Wang DJ, Kim GR, Johnson K. Study of users of the `Pediatric Internet' published… 2. Internet Medicine, 1998; 3.

Lehmann ED. Preliminary experience with the Internet release of AIDA – an interactive educational diabetes simulator. Computer Methods & Programs in Biomedicine, 1998; 56: 109–32.

Lyon E, Gilbert S. A pilot survey of user satisfaction with the online search service in South‐west Thames Regional libraries. Health Libraries Review, 1988; 5: 49–60.

Mandl KD, Feit S, Pena BM, Kohane IS. Growth and determinants of access in patient E‐mail: and Internet use. Archives of Pediatrics & Adolescent Medicine, 2000; 154: 508–11.

McKay HG, Feil EG, Glasgow RE, Brown JE. Feasibility and use of an Internet support service for diabetes self‐management. Diabetes Educator, 1998; 24: 174–9.

Pautler SE, Tan JK, Dugas GR et al. Use of the internet for self‐education by patients with prostate cancer. Urology, 2001; 57: 230–233.

Riva A, Smigelski C, Friedman R. WebDietAID: an interactive Web‐based nutritional counselor. Proceedings/AMIA Annual Symposium, 2000.

Sharf BF. Communicating breast cancer on‐line: support and empowerment on the Internet. Women & Health, 1997; 26: 65–84.

Van Brunt D. New technology. Internet‐based patient information systems: what are they, why are they here, how will they be used, and will they work? Journal of Rehabilitation Outcomes Measurement, 1998; 2: 58–63.

Vordermark D, Kolbl O, Flentje M. The Internet as a source of medical information. Investigation in a mixed cohort of radiotherapy patients. Strahlentherapie und Onkologie, 2000; 176: 532–535.

Weinberg N, Schmale J, Uken J, Wessel K. Online help: cancer patients participate in a computer‐mediated support group. Health & Social Work, 1996; 21: 24–9.

Weller AC. A study of remote users' satisfaction with online services before and after procedural modifications. Bulletin of the Medical Library Association 1985; 73: 352–7.

White MH, Dorman SM. Online support for caregivers. Analysis of an Internet alzheimer mailgroup. Computers in Nursing, 2000; 18: 168–76.

Widman LE, Tong DA. Requests for medical advice from patients and families to health‐care providers who publish on the World Wide Web [see comments]. Archives of Internal Medicine 1997; 157: 209–12.