Abstract
Background
Information seeking is an important coping mechanism for dealing with chronic illness. Despite a growing number of mental health websites, there is little understanding of how patients with bipolar disorder use the Internet to seek information.
Methods
A 39 question, paper-based, anonymous survey, translated into 12 languages, was completed by 1222 patients in 17 countries as a convenience sample between March 2014 and January 2016. All patients had a diagnosis of bipolar disorder from a psychiatrist. Data were analyzed using descriptive statistics and generalized estimating equations to account for correlated data.
Results
976 (81 % of 1212 valid responses) of the patients used the Internet, and of these 750 (77 %) looked for information on bipolar disorder. When looking online for information, 89 % used a computer rather than a smartphone, and 79 % started with a general search engine. The primary reasons for searching were drug side effects (51 %), to learn anonymously (43 %), and for help coping (39 %). About 1/3 rated their search skills as expert, and 2/3 as basic or intermediate. 59 % preferred a website on mental illness and 33 % preferred Wikipedia. Only 20 % read or participated in online support groups. Most patients (62 %) searched a couple times a year. Online information seeking helped about 2/3 to cope (41 % of the entire sample). About 2/3 did not discuss Internet findings with their doctor.
Conclusion
Online information seeking helps many patients to cope although alternative information sources remain important. Most patients do not discuss Internet findings with their doctor, and concern remains about the quality of online information especially related to prescription drugs. Patients may not rate search skills accurately, and may not understand limitations of online privacy. More patient education about online information searching is needed and physicians should recommend a few high quality websites.
Electronic supplementary material
The online version of this article (doi:10.1186/s40345-016-0058-0) contains supplementary material, which is available to authorized users.
Background
Information seeking is an important aspect of coping with chronic illness (Lambert and Loiselle 2007; Brashers et al. 2002) and many patients with bipolar disorder want to learn more (Hallett et al. 2013; Giacco et al. 2014). The majority of patients with bipolar disorder would prefer to learn about their illness through a face-to-face conversation with their physician, as do patients with other mental and physical illness (Hallett et al. 2013; Gaglio et al. 2012; Hesse et al. 2005; Horgan and Sweeney 2010). This is often not feasible due to the large number of questions that will arise over the long term and the limited access to the treating psychiatrist. Due to the unique properties of the Internet, both patients and providers view it as an important source of all medical information, including information about bipolar disorder. The Internet is available globally, can be accessed on the patient’s schedule and content can be read at the patient’s pace (Korp 2006). Another unique aspect is that healthcare web sites are often viewed by patients in many countries (Leon and Fontelo 2016; Heilman and West 2015). For healthcare providers, large numbers of patients can be reached at a reasonable cost with a website.
The goal of this study was to better understand online information seeking by patients with bipolar disorder. This was investigated using an anonymous patient survey completed by 1222 patients with bipolar disorder in 17 countries. Our initial findings, reported previously, were that the patients used the Internet at a percentage that was similar to the general public (81 %), and that 78 % of the Internet users looked online for information on bipolar disorder or 63 % of the total sample (Bauer et al. 2016). The patients who looked online for information also consulted medical professionals plus other information sources, such as printed media, physician handouts, television, and other patients with bipolar disorder. This analysis will focus on the patterns of online information seeking, including why and how patients with bipolar disorder find information online, what information they are looking for, and the impact of information seeking.
Methods
An anonymous, one-time survey was completed by patients with bipolar disorder. To maximize participation and minimize bias, the survey was paper-based and was translated into the local language. With a paper-based survey, patients without Internet skills or online access could be included. The treating psychiatrist provided the diagnosis, age of onset, and years of education for each patient. This study was approved by institutional review boards in accordance with local requirements.
Survey
The survey contained 39 questions and took about 20 min to complete. The complete survey in English is available in the Additional file 1. The survey topics included questions about demographics, living with bipolar disorder, Internet use, online information seeking, and participation in online support groups. Questions 21–35 about online information seeking were analyzed here. The questions on Internet access were analyzed previously (Bauer et al. 2016). The survey was translated into 12 local languages: Chinese, Danish, Finnish, French, German, Hebrew, Italian, Japanese, Polish, Portuguese, Spanish, and English (versions for US/Canada, UK and Australia). A convenience sample was collected between March 2014 and January 2016. In total, 1222 surveys were received: from Australia (N = 22), Brazil (N = 100), Canada (N = 109), Denmark (N = 209), Finland (N = 16), France (N = 50), Germany (N = 82), Hong Kong (N = 91), India (N = 30), Israel (N = 46), Italy (N = 80), Japan (N = 35), Malaysia (N = 25), Poland (N = 125), Spain (N = 82), UK (N = 50), and the US (N = 70). The treatment settings included private practice, university clinics, and community mental health centers.
Duplicate data entry was used for quality control with the paper-based surveys (Kawado et al. 2003; Neaton et al. 1990). Automated logic checking of numeric fields was implemented as appropriate. More details about the survey methodology and validation were published previously (Bauer et al. 2016).
Country variables
Since this was an international sample, country specific variables as well as the individual survey responses were included in the analysis. The county specific variables include the mean years of education for adults age ≥25 years (UNESCO 2015), education ratio (for those age ≥25 years, patient years of education compared to the mean years of education for general population), telecommunications data, and Hofstede’s cultural dimensions (Hofstsede 2016). The Hofstede’s power distance index (PDI) is a measure of social inequality where a lower number means that societies are questioning authority and striving to equalize the distribution of power.
Statistics
Explanatory models were estimated to gain insight into relationships among survey responses, external variables and demographics. The generalized estimating equation (GEE) statistical technique was selected to overcome the imbalance in the number of responses from the collection sites, and to account for the correlation in survey responses among the collection sites. The GEE models were estimated using a binomial distribution, independent working correlation matrix and a logit link function. Many individual and country specific variables were similar and correlated. The potential variables from univariate analyses that were significant at a level of 0.05 were entered into multivariate models, and the corrected quasi-likelihood independence model criterion was used to assist with multivariate model fitting (Pan 2001). The odds ratios and confidence intervals generated by the GEEs are reported.
Descriptive statistics are reported for demographic variables. Survey responses are also summarized as the mean value of the individual country means. SPSS version 23.0 was used for all analyses.
Results
The survey was completed by 1222 patients, with 51 more surveys received since the first analysis (Bauer et al. 2016). With 1212 valid responses, 976 (81 %) patients used the Internet. Of the 972 valid responses, 750 (77 %) used the Internet to look up information about bipolar disorder or 61 % of the total sample. The demographics for all patients who completed the survey and for the patients who used the Internet to find information about bipolar disorder are shown in Table 1. The 750 patients who used the Internet to find information on bipolar disorder had a mean age of 41 years ± 12.5, were 62 % female, and had a mean of 14 ± 3.0 years of education.
Table 1.
Variable | Value | All patients (N = 1222) | Used Internet to find out about bipolar disorder (BP) (N = 750) | ||
---|---|---|---|---|---|
N a | % | N a | % | ||
Diagnosis | BP I | 768 | 63.7 | 440 | 59.7 |
BP II | 380 | 31.6 | 256 | 34.7 | |
BP NOS | 57 | 4.7 | 41 | 5.6 | |
Gender | Female | 759 | 62.3 | 465 | 62.2 |
Male | 459 | 37.7 | 283 | 37.8 | |
Area of residence | Urban | 744 | 61.2 | 451 | 60.3 |
Suburban | 291 | 23.9 | 196 | 26.2 | |
Rural | 181 | 14.9 | 101 | 13.5 | |
Employment status | Full-time | 560 | 46.6 | 396 | 53.6 |
Not full-time | 641 | 53.4 | 343 | 46.4 | |
Marital status | Married | 593 | 48.9 | 359 | 48.3 |
Not married | 619 | 51.1 | 385 | 51.7 | |
Income group | Upper income | 80 | 6.6 | 60 | 8.1 |
Middle income | 594 | 49.1 | 363 | 48.9 | |
Lower income | 535 | 44.3 | 320 | 43.1 | |
Live alone | Yes | 299 | 24.7 | 190 | 25.5 |
No | 913 | 75.3 | 556 | 74.5 | |
Mood in last six months | Mostly normal | 581 | 47.9 | 317 | 42.6 |
Mostly not normal | 632 | 52.1 | 428 | 57.4 | |
BP interfered with regular activities | Frequently or sometimes | 768 | 63.2 | 524 | 70.1 |
Rarely or never | 448 | 36.8 | 223 | 29.9 | |
Confident managing living | Very confident | 454 | 37.5 | 259 | 34.8 |
Not very confident | 757 | 62.5 | 486 | 65.2 | |
Confident when to see doctor about BP | Very confident | 698 | 57.4 | 418 | 56.0 |
Not very confident | 518 | 42.6 | 328 | 44.0 |
Means | Na | Mean (SD) | Na | Mean (SD) | |
---|---|---|---|---|---|
Age | 1217 | 44.4 (13.8) | 748 | 41.1 (12.5) | |
Years of education | 1199 | 14.0 (3.2) | 737 | 14.4 (3.0) | |
Age of onset | 1201 | 27.1 (10.9) | 734 | 25.6 (10.1) | |
Years of Illness | 1194 | 17.4 (12.2) | 730 | 15.6 (11.6) |
aMissing values not included
Survey responses
The most common responses to the questions are shown as the mean value of the individual country means in Table 2. When looking for information about bipolar disorder, 89 % used a computer and 11 % used a smartphone or tablet. When online, 79 % started looking with a general search engine. 69 % of the patients rated their Internet search skills as basic or intermediate, and 31 % as expert. When searching, 16 % indicated that they always find what they are looking for, 57 % most of the time, 19 % half of the time, 8 % less than half, and 0.3 % never. The primary reasons that patients searched included side effects from prescription drugs (51 %), to learn anonymously (43 %), and because they need help coping (39 %). 62 % of the patients looked for information a couple of times a year, and 38 % looked monthly or more frequently.
Table 2.
Question number | Questiona | Valid responses N | Variable | Mean percentb (%) | SDc |
---|---|---|---|---|---|
21d | How do you access the Internet? | 607 | From a computer | 89 | 0.1221 |
607 | From a smartphone or tablet | 11 | 0.1221 | ||
22 | How do you rate your Internet search skills? | 746 | Basic or intermediate | 69 | 0.1288 |
746 | Expert | 31 | 0.1288 | ||
23 | How do you start looking for information about bipolar disorder? | 623 | General search engine | 79 | 0.0816 |
623 | Medical search engine | 6 | 0.0478 | ||
623 | Specific site on mental illness | 10 | 0.0640 | ||
623 | Other | 6 | 0.0475 | ||
24 | Do you find what you are looking for? | 733 | Always | 16 | 0.1094 |
733 | Most of the time | 57 | 0.1277 | ||
733 | About half the time | 19 | 0.0847 | ||
733 | Less than half the time | 8 | 0.0865 | ||
733 | Never | 0.3 | 0.0073 | ||
25 | Do you look for these topics relating to bipolar disorder?e | 742 | Prescription drug information | 73 | 0.1363 |
742 | Symptoms | 83 | 0.1173 | ||
742 | General course of illness | 58 | 0.1268 | ||
742 | Coping strategies | 52 | 0.1774 | ||
26 | Do you look for these topics relating to getting treatment for bipolar disorder?e | 542 | Clinic hours, location and directions | 52 | 0.2079 |
542 | Physician or therapist credentials | 31 | 0.1633 | ||
542 | Physician or therapist ratings | 24 | 0.1711 | ||
27 | How frequently do you search for information about bipolar disorder?e | 738 | Monthly or more | 38 | 0.1620 |
738 | Couple of times a year | 62 | 0.1620 | ||
28 | Why do you search for information about bipolar disorder on the Internet?e | 735 | To learn anonymously | 43 | 0.1600 |
735 | Side effects from prescription drugs | 51 | 0.1184 | ||
735 | Need help coping with the illness | 39 | 0.1706 | ||
29 | What are your favorite sources of information about bipolar disorder?e | 727 | Specific sites on mental health or bipolar disorder | 59 | 0.1744 |
727 | Online encyclopedia such as Wikipedia | 33 | 0.0843 | ||
727 | Government sponsored sites | 21 | 0.2168 | ||
30 | Do you discuss online information about bipolar disorder with your doctor? | 737 | Sometimes or frequently | 33 | 0.1178 |
737 | Rarely | 67 | 0.1178 | ||
31 | Does the online information help you cope with bipolar disorder? | 733 | Sometimes for frequently | 67 | 0.1477 |
733 | Rarely | 33 | 0.1477 | ||
32 | Do you attempt to verify online information? | 735 | Yes | 66 | 0.1158 |
33 | How do you attempt to verify online information?e,f | 428 | Discuss with doctor | 62 | 0.1633 |
428 | Discuss with family and friends | 37 | 0.1536 | ||
428 | Compare information on multiple sites | 56 | 0.1620 | ||
34 | How concerned are you about privacy and confidentiality? | 723 | Moderately or very concerned | 32 | 0.1788 |
723 | Slightly concerned | 68 | 0.1788 | ||
35 | Do you participate in patient support groups? | 746 | Yes | 20 | 0.0883 |
aSee survey for details and exact wording
bOverall mean of country means
cStandard deviation of country means
dSurvey questions 1–16 on demographics; questions 17-20 analyzed previously (Bauer et al. 2016)
eMore than one response allowed
fOnly completed by those who attempt to verify information (N = 485)
The patients looked frequently for a variety of topics related to bipolar disorder including symptoms (83 %), prescription drug information (73 %), general course of illness (58 %), coping strategies (52 %), and side effects from prescription drugs (51 %). The patients also looked for topics related to the healthcare system, although the percentage of missing responses differed among the countries. The patients looked frequently for clinic hours, location and driving instructions (52 %), physician credentials (31 %) and physician ratings (24 %). Patients looked less frequently for all remaining responses. For favorite sources of information, 59 % indicated a specific site on mental health or bipolar disorder, 33 % indicated Wikipedia and 21 % a government sponsored site. 66 % of the patients said they attempted to verify the information found online, primarily by discussing with a doctor (62 %) or comparing information on multiple sites (56 %).
67 % of the patients rarely or never discussed what they found online with their doctors, while 33 % sometimes or frequently did. 67 % of the patients said that information seeking online helped them to cope sometimes or frequently, while 33 % said rarely or never. Only 32 % of the patients were moderately or very concerned about privacy while 68 % were slightly or not concerned. Only 20 % of the patients read or participated in online support groups.
Explanatory models
The details for the explanatory models are shown in Table 3. To explain Internet skills self-rated as expert, the estimated coefficients from the best fitting model suggested that a 1 year increase in age will decrease the odds of having expert skills by 3 %, having Wikipedia as a favorite source will increase the odds by 49 %, attempting to verify the information found online will increase the odds by 72 %, being male will increase the odds by 89 %, and feeling very confident about when to see the doctor about bipolar disorder will increase the odds by 89 %.
Table 3.
Question number | Dependent variable | N | Independent variables | ||||
---|---|---|---|---|---|---|---|
Questiona | Answerb | Parameter | Significance | OR | 95 % CI | ||
22 | How do you rate your Internet search skills? | Expert | 725 | Intercept | 0.010 | 0.463 | 0.257, 0.835 |
Age | <0.001 | 0.970 | 0.956,0.984 | ||||
Male | 0.004 | 1.892 | 1.221, 2.931 | ||||
Very confident when to see the doctor | <0.001 | 1.891 | 1.354, 2.639 | ||||
Favorite source Wikipedia | <0.001 | 1.487 | 1.221,1.812 | ||||
Attempt to verify the online information | 0.003 | 1.717 | 1.198, 2.460 | ||||
24 | Do you find what you are looking for? | Always | 661 | Intercept | <0.001 | 0.203 | 0.127, 0.323 |
Consult more than one medical professional (e.g., counselor or psychologist) plus psychiatrist | 0.033 | 0.588 | 0.361, 0.958 | ||||
27 | How frequently do you search for information about bipolar disorder? | Monthly or more often | 703 | Intercept | 0.006 | 0.310 | 0.135, 0.709 |
Education ratio | 0.004 | 0.621 | 0.448, 0.858 | ||||
Male | 0.025 | 1.525 | 1.054, 2.207 | ||||
Mostly normal last 6 months | <0.001 | 0.395 | 0.262, 0.596 | ||||
Bipolar disorder interferes frequently or sometimes | <0.001 | 1.952 | 1.406, 2.711 | ||||
Search for side effects from prescription drugs | 0.021 | 1.465 | 1.060, 2.024 | ||||
Favorite source are specific sites on mental health | 0.021 | 1.292 | 1.039, 1.607 | ||||
Online information helps to cope | <0.001 | 2.699 | 2.008, 3.627 | ||||
Participate in online support groups | <0.001 | 2.286 | 1.552, 3.367 | ||||
29 | What are your favorite sources of information about bipolar disorder? | Specific sites on mental health or bipolar disorder | 642 | Intercept | <0.001 | 0.190 | 0.087, 0.415 |
Age | 0.001 | 1.022 | 1.009, 1.035 | ||||
Male | 0.002 | 0.729 | 0.595, 0.892 | ||||
Consult more than one medical professional | 0.001 | 1.515 | 1.197, 1.917 | ||||
Search monthly or more often | 0.019 | 1.383 | 1.054, 1.814 | ||||
Online information helps to cope | <0.001 | 1.598 | 1.284, 1.990 | ||||
Attempt to verify the online information | <0.001 | 2.670 | 2.070, 3.445 | ||||
29 | What are your favorite sources of information about bipolar disorder? | Wikipedia | 744 | Intercept | <0.001 | 0.293 | 0.211, 0.406 |
Male | 0.006 | 1.565 | 1.135, 2.158 | ||||
Expert search skills | <0.001 | 1.579 | 1.271, 1.963 | ||||
30 | Do you discuss online information about bipolar disorder with your doctor? | Sometimes or frequently | 715 | Intercept | <0.001 | 0.068 | 0.038, 0.123 |
Very confident when to see the doctor | 0.001 | 1.936 | 1.297, 2.891 | ||||
Online information helps to cope | <0.001 | 2.621 | 1.827, 3.761 | ||||
Attempt to verify the online information | <0.001 | 3.149 | 2.374, 4.177 | ||||
31 | Does the online information help you cope with bipolar disorder? | Sometimes or frequently | 711 | Intercept | 0.005 | 0.610 | 0.433, 0.861 |
Mostly normal last 6 months | <0.001 | 2.015 | 1.525, 2.663 | ||||
Always find what looking for | <0.001 | 2.637 | 1.706, 4.076 | ||||
Search monthly or more often | <0.001 | 2.290 | 1.659, 3.160 | ||||
Online information helps to cope | <0.001 | 1.684 | 1.276, 2.221 | ||||
Discuss findings with doctor | <0.001 | 3.023 | 2.024, 4.517 | ||||
34 | How concerned are you about privacy and confidentiality? | Moderately or very concerned | 723 | Intercept | <0.001 | 0.085 | 0.049, 0.147 |
Power Distance Index | <0.001 | 1.030 | 1.020, 1.040 | ||||
Search online to learn anonymously | 0.001 | 1.705 | 1.260, 2.308 |
aSee survey for details and exact wording
bMissing values not included
To explain if patients always find what they are looking for, the estimated coefficients from the best fitting model suggested that consulting another medical professional (e.g., primary care, psychologist or counselor) in addition to a psychiatrist will decrease the odds by 41 %. There was no association with expertise or education.
To explain if patients search monthly or more frequently, the estimated coefficients from the best fitting model suggested that a one unit increase in education ratio will decrease the odds by 38 %, having a mostly normal mood for the last 6 months will decrease the odds by 60 %, having a mental health site as a favorite source will increase the odds by 29 %, looking for drug side effects will increase the odds by 47 %, being male will increase the odds by 53 %, bipolar disorder interfering with life will increase the odds by 95 %, participating in online support groups will increase the odds by 129 %, and feeling the Internet helps to cope will increase the odds by 170 %.
There was no association between the favorite sources of online information and coming from a country where the primary language is English. To explain having a mental health site as a favorite source, the estimated coefficients from the best fitting model suggested that a 1 year increase in age will increase the odds by 2 %, being male will decrease the odds by 27 %, searching monthly or more frequently will increase the odds by 38 %, consulting another medical professional will increase the odds by 52 %, needing help coping will increase the odds by 60 %, and attempting to verify the information found online will increase the odds by 167 %. To explain having Wikipedia as a favorite source, the estimated coefficients from the best fitting model suggested that being male will increase the odds by 57 %, and having self-rated expert search skills will increase the odds by 58 %.
To explain if the patients discuss online information with their doctor, the estimated coefficients from the best fitting model suggested that feeling very confident when to see the doctor will increase the odds by 94 %, feeling the Internet helps to cope will increase the odds by 162 %, and attempting to verify the information found online will increase the odds by 215 %.
To explain if patients feel the information learned online helps to cope, the estimated coefficients from the best fitting model suggested that needing help coping will increase the odds by 68 %, having a mostly normal mood for the last 6 months will increase the odds by 102 %, searching monthly or more frequently will increase the odds by 129 %, always find what looking for will increase the odds by 164 %, and discussing findings with the doctor will increase the odds by 202 %.
To explain if patients are concerned about privacy, the estimated coefficients from the best fitting model suggested that a one unit increase in the country PDI will increase the odds by 3 %, and going online to learn anonymously will increase the odds by 71 %.
Discussion
Three reasons for online information seeking
There were three primary reasons why patients looked online for information about bipolar disorder: prescription drugs, perceived anonymity, and help coping. About half the patients looked online because of side effects from prescription drugs, in agreement with prior reports that patients with bipolar disorder want to know more about their drugs and especially side effects (Hallett et al. 2013; Bowskill et al. 2007). Many patients looked online to learn about bipolar disorder without revealing their identity. Although incorrect, the belief that one is anonymous online is a commonly perceived benefit of the Internet by those with a stigmatized illness (Berger et al. 2005; Pohjanoksa-Mäntylä et al. 2009; Chan et al. 2016). Patients also sought information online because they were having difficulty coping with the illness, confirming the need for online sources to help deal with the consequences of bipolar disorder.
Using a general search engine on a computer
The vast majority of patients started looking for information on bipolar disorder from a general search engine (79 %) on a laptop/desktop computer (89 %). This is consistent with prior findings that 80 % of the general public started looking for health information with a general search engine (Fox and Duggan 2013), and that general search engines are used much less frequently from a smartphone than from a computer (Arthur 2015; MacMillan 2015; Friedman 2015). Providers involved in developing online projects for bipolar disorder should consider how the targeted consumer device may impact use. About 43 % of total handsets in the world were smartphones in 2015, with an expected increase to 50 % by 2020 (Cisco 2016). In some countries such as the US, those with mental illness may have a much lower smartphone ownership than the general public (Klee et al. 2016; Miller et al. 2016).
General search engine background
Most patients start looking for information from a general search engine. Google dominates with a market share of ≥90 % in all but 3 of the 17 countries in this study, the exceptions being 87 % in Canada, 73 % in Hong Kong and 73 % in the US (Return on Now 2016). Search terms entered are brief and guided by auto-completion suggestions, with 57 % in Google US being 1–2 words, and 88 % being 4 words or less (Statistica 2016). Over 90 % of people select a website from the first page of search engine results, with 61 % of people selecting a web site from the top 3 results (Sharp 2014).
1/3 rated search skills as expert
In this study, about 2/3 of the patients rated their search skills as basic or intermediate, and 1/3 as expert. There was no association between education and self-rating as an expert. The factors that increased the odds of self-rating as an expert included being male, having confidence when to see the doctor about bipolar disorder, having Wikipedia as a favorite source, and attempting to verify the information. In prior research, self-reported computer skill ratings were found to be unreliable (Merritt et al. 2005), with university students tending to overestimate their abilities (Ivanitskaya et al. 2006), and females perceiving lower abilities than males at the same skill level (Bunz et al. 2007). There are several concerns about the accuracy of the self-rating of search expertise in this study.
A minority of patients (16 %) always found what they were looking for, but this was not associated with expertise, education or attempting to verify the information. However, always finding the answer was associated with not consulting more than one medical professional. There is concern that some patients may be satisfied with any answer, or use the Internet as a second opinion.
Although 1/3 of the patients rated their search skills as expert, it is quite difficult to search for medical information. In a recent study, no answer was found in the top three search results returned for 32 of 54 simple consumer mental health questions using the National Institute of Mental Health (NIMH) website (Crangle and Kart 2015). Patient questions are not answered due to a high degree of specificity, false presuppositions, and layman use of medical terminology (Crangle and Kart 2015; MacCray et al. 1999). In an analysis of consumer messages to MedlinePlus, the majority concerned specific medical questions rather than general information (Miller 2007). For perspective, physicians also report barriers to successful online searching including the need for specific information, too much information, selecting reliable sources, and time requirements (Clarke et al. 2013; Bennett et al. 2004; McKibbon and Fridsma 2006).
There was no association between concern about privacy and expertise. In this study, 43 % of patients believed they are anonymous online, a result similar to the 37 % found in a survey of the US general public (Rainie et al. 2013). Although the legal framework for online privacy varies internationally (Bowman 2016), this finding emphasizes an important need to educate patients at all levels of expertise about privacy on the Internet. Commercial firms that provide services at no charge, including search engines, social media, and many medical sites, generally make money by tracking all activities to sell targeted behavioral advertising, or by selling the tracked activities to third parties (Greengard 2012; Stark and Fins 2013; Glenn and Monteith 2014; FTC 2014; Eavis 2016; Rosenberg 2015). The data collected includes details about all Internet and smartphone activities including search terms, websites visited, email and social media content and metadata (data about data that provides context) (Libert 2015; NISO 2004). In a recent study of over 80,000 health-related websites in the US, over 90 % sent data to third parties, with 70 % including specific symptoms, treatments and diseases (Libert 2015). Those who use social media and online support groups owned by commercial organizations may not realize that privacy policies often give consent to the sale of patient created data (Li 2013; Lupton 2014; Glenn and Monteith 2014). Online medical data may be combined with other data for health risk predictions (Dixon and Gellman 2014), combined with genetic data for commercial research (Seife 2013), and de-identified data may be re-identified using very large, high-dimensional databases (Narayanan et al. 2016). Even though the legal framework is changing, such as with the general data protection regulation (GDPR) to be implemented by 2018 for the EU (European Parliament News 2016), patients of all search skill levels need to understand the data policies of websites they visit regularly for information on mental health.
Favorite sources are mental health sites and Wikipedia
For 59 % of the patients in this study, a mental health site was a favorite information source on bipolar disorder, while Wikipedia was a favorite source for 33 %. The factors that increased the odds for a mental health site included being female, needing help coping, consulting more than one medical professional, and attempting to verify the information. Being male or having expert search skills increased the odds that Wikipedia was a favorite source.
Types of mental health sites
Websites on mental health are owned by a variety of entities, including governments, professional organizations, and commercial businesses. The overall content of most Internet information related to affective disorders is generally rated as good (Grohol et al. 2014; Morel et al. 2008) although the quality may vary considerably (Reavley and Jorm 2011; Hasty et al. 2014; Barnes et al. 2009; Monteith et al. 2013). The content of government websites, such as from the US MedlinePlus and UK NHS are evaluated for accuracy and currency (MedlinePlus 2016; NHS 2016). Hospitals and health systems own many websites and understand the importance of accuracy, but also view these websites as marketing opportunities (Ford and Walls 2012). Website content from providers that offer specific treatments, such as substance abuse treatment centers, may be incomplete or imbalanced (Schenker and London 2015; Link et al. 2016).
Articles from Wikipedia, the open encyclopedia that invites user contributions, frequently appear in first page of results for medical searches (Laurent and Vickers 2009). The Wikipedia entry for bipolar disorder was viewed over 13 million times in 55 languages in 2013 (Heilman and West 2015). In 2013, there were less than 300 editors for the medical articles on Wikipedia, of which 82 % were male and 50 % were healthcare providers (Heilman and West 2015). About 85 % of the medical editors had a college degree, with editors of the non-English versions having a similar education level. As in this study, more males (56 %) than females (44 %) are readers of Wikipedia about all topics (Pande 2011).
As of 2015, the Google search engine returns pre-vetted facts at the top of the first page for many medical searches (Google 2015). Beyond this, results may be tailored to the user based on behavioral advertising algorithms (Monteith et al. 2013; Vaidhyanathan 2012). The results of medical searches from all major search engines overlap considerably (Wang et al. 2012).
Website language
There was no association between a native language other than English and the favorite information source in this study. Based on the top 10 million websites on the Internet, 53.6 % of all content is in English (W3 Techs 2016). The percent of content in German, Japanese, Spanish and French ranged between 4 and 6 % each, with a smaller percent in the other languages in this study. One consequence of the Internet is a global increase in teaching English as a second language. In 2014, 77 % of primary school students in the EU learned English as a foreign language, up from 35 % in 2000 (Eurostat 2016; Parker 2015).
Searching a couple times a year
Most of the patients in this study (62 %) looked online for information a couple of times a year, while 38 % looked monthly or more often. The odds of looking online more frequently were increased if bipolar disorder interfered with life, the patient was looking for drug side effects, felt the internet helps to cope, read or contributed to an online support group, or if male. The odds of looking online more frequently were decreased if patients were mostly euthymic, or if they were very educated. Both experiencing symptoms and participation in online support groups were previously noted to increase the frequency of information seeking (Rice 2006; Weaver et al. 2010). The very educated may prefer to consult with their doctor for individualized treatment (Bauer et al. 2016).
Seeking information on medical topics
The patients in this study most often sought information about medical topics, including symptoms (83 %), prescription drug information (73 %), general course of illness (58 %), coping strategies (52 %), and less frequently about nonmedical topics including clinic location, hours and directions (52 %), and physician credentials (31 %).
The strong patient interest in prescription drug information is important since both physical and mental side effects are associated with nonadherence (Bates et al. 2010; Johnson et al. 2007; Baldessarini et al. 2008; Szmulewicz et al. 2016). The accuracy of drug information on websites for consumers is of concern. Research has focused on Wikipedia, and compared the content for a variety of drugs to official product labels or professionally edited compendia. The drug information content on Wikipedia was mostly accurate but lacked details, was often incomplete (Kupferberg and Protus 2011; Clauson et al. 2008; Lavsa et al. 2011), and drug safety information may not be current (Koppen et al. 2015; Hwang et al. 2014). Some drug articles on Wikipedia were missing references (Lavsa et al. 2011) and unlike professional compendia, cited news stories as references (Koppen et al. 2015). Similarly, studies of accuracy of drug information on other consumer websites found generally good content quality but incomplete topic coverage (Sidhu et al. 2006; Ghoshal and Walji 2006).
Drug review websites based on patient ratings of drugs are also available online. Unlike standard compendia, patient review sites emphasize side effects and personal ratings of efficacy. These sites may over-represent the experiences of patients with acute exacerbations of illness, severe side effects or dissatisfaction with results (Chew and Khoo 2016; Hughes and Cohen 2011). Comparisons of drug efficacy ratings found that results varied among the patient review sites (Gidwani and Zulman 2015; Chew and Khoo 2016).
Patient education should emphasize that online drug information may be inaccurate or incomplete, even in compendia (Randhawa et al. 2016), and that patients should seek multiple sources. The emphasis on personal experience in drug review sites may be useful to some patients (Hughes and Cohen 2011; Chew and Khoo 2016), but patient education should also explain that these sites are not authoritative. Some patients may not realize that high quality drug information is available for free from government sources such as the US National Library of Medicine Drug Information Portal (NLM 2016).
Information seeking helps with coping
Online information seeking sometimes or frequently helped 67 % of the patients in this study to cope. In this study, the factors that increased the odds that information seeking helped to cope included being mostly euthymic, always finding what looking for, searching monthly or more frequently, needing help coping, and discussing findings with the doctor. In a prior study of high functioning patients with bipolar disorder, education about the disease was a key strategy for staying well (Murray et al. 2011).
Only 20 % of the patients read or participated in online support groups or forums, a rate similar to that for the general public (Fox 2011). In this study, participation in online support groups was not associated with information seeking helping patients to cope. This finding suggests that the information learned online is what helps most patients to cope, rather than emotional support or coping skills learned in online support groups. Prior research on online communities reported that information seeking was the most important activity, (Nambisan 2011; Meier et al. 2007; van Uden-Kraan 2009), although emotional support was also important in bipolar disorder (Bauer et al. 2013). The time length of most online psychological interventions is considerably shorter than for face-to-face programs and may not offer sufficient professional access for patients to learn coping skills (Hidalgo-Mazzei et al. 2015).
In this study, 67 % of the patients rarely or never discussed what they learned online with their doctor. About 60–80 % of general medical patients also do not discuss Internet health information with their doctor for reasons including time constraints, fear of insulting the doctor or being dismissed, information was self-explanatory, and a desire for self-management (Chung 2013; Russ et al. 2011; Hay et al. 2008; Kim and Kim 2009; Diaz et al. 2002; Imes et al. 2008). In this study, the factors that increased the odds of patients discussing online information with their doctor include feeling very confident about when to see the doctor, feeling the Internet helped to cope, and attempting to verify the information. Similarly, in prior research, patients who talked with their doctors felt the information from the Internet was of high quality (Diaz et al. 2002; Chung 2013).
The patients in this study had a mean of 17 years of illness, so it is likely that many searches were for specific questions rather than general information about bipolar disorder. Since most patients do not discuss the information learned online with their doctor, patient education about online health information seeking is important, and should be integrated into psychoeducation programs. There is a need to help patients to access high quality information, such as by providing a short list of recommended websites (Monteith et al. 2013). Website recommendations from physicians are favorably received by general medicine patients, and may increase communication about online information seeking (Coberly et al. 2010; Siegel et al. 2006). Patients should also understand that even a clear and accurate website will probably not provide answers to specific questions about their personal situation (Miller 2007).
Limitations
Since this survey was collected as a convenience sample, the survey participants do not reflect the demographic compositions of the countries which may bias the results. There was no follow-up discussion of the survey responses, and patient actual use of the Internet could not be verified. The patients not included in this study are those with bipolar disorder who do not seek professional help, who may be less educated or have a less stable living situation. Valid response rates varied among the questions analyzed. However, all results were consistent with prior research. These survey results are associations and do not establish causality between variables. This survey was administered by the treating psychiatrist so questions on patient satisfaction were omitted, although dissatisfaction is associated with increased online searching for health information (Tustin 2010). The survey did not address if patients visited sites in a language other than their native language or which specific websites were visited. The survey also did not address the use of problematic websites that encourage unwanted behaviors such as self-harm (Mitchell et al. 2014), issues related to online purchase of prescription or illegal drugs (Monteith et al. 2016; Orizio et al. 2011), or the impacts of direct to consumer pharmaceutical advertising where applicable (Mackey et al. 2015).
Conclusions
Online information seeking helped two thirds of the patients to cope with bipolar disorder. When considering all 1222 patients who completed the survey, about 41 % of all patients found help coping using the Internet, so other information sources remain important. Most patients do not discuss information learned online with their doctor. There is concern about the quality of website content especially related to prescription drugs. Patients may not rate search skills accurately, and may not understand online privacy issues. More patient education about online information seeking is needed, and physicians should recommend a few high quality websites about bipolar disorder to their patients.
Authors’ contributions
All authors contributed to and approved the final manuscript. Authors JC, RB, TG and MBa designed the study. Authors MA, RA, RB, MBe, YB, AB, LB, AMPC, EYWC, CC, SC, JC, RD, SD, AD, BE, AF, JGa, JGe, JGi, AGP, GMG, PG, HH, SH, CH, DHM, VK, GK, BL, CL, ERL, UL, RWL, AHL, BM, SM, RM, TN, REN, COD, YOk, YOs, PP, AR, PR, JKR, KSa, BS, ES, ÂMS, CS, AHS, KSu, AS, PT, YT, LT, EV, MV, BV, JV, and MZ were involved with data collection. Author TG provided data analysis. Authors RB, JC, TG, PCW and MBa were involved in the draft manuscript and initial review. All authors read and approved the final manuscript.
Acknowledgements
We thank Kerstin Heyde, Christiane Otto and Katharina Schultheiss for data entry support, and Daniela Jany for assisting with manuscript submission.
Competing interests
Michael Berk is supported by an NHMRC Senior Principal Research Fellowship (1059660), Eduard Vieta received a Grant from Telefonica.
Jörn Conell, Rita Bauer, Tasha Glenn, Martin Alda, Raffaella Ardau, Bernhard T Baune, Yuly Bersudsky, Amy Bilderbeck, Alberto Bocchetta, Letizia Bossini, Angela Marianne Paredes Castro, Eric Yat Wo Cheung, Caterina Chillotti, Sabine Choppin, Maria Del Zompo, Rodrigo Dias, Seetal Dodd, Anne Duffy, Bruno Etain, Andrea Fagiolini, Julie Garnham, John Geddes, Jonas Gildebro, Ana Gonzalez-Pinto, Guy M Goodwin, Paul Grof, Hirohiko Harima, Stefanie Hassel, Chantal Henry, Diego Hidalgo-Mazzei, Vaisnvy Kapur, Girish Kunigiri, Beny Lafer, Chun Lam, Erik Roj Larsen, Ute Lewitzka, Rasmus Licht, Anne Hvenegaard Lund, Blazej Misiak, Patryk Piotrowski, Scott Monteith, Rodrigo Munoz, Takako Nakanotani, René E Nielsen, Claire O’Donovan, Yasushi Okamura, Yamima Osher, Andreas Reif, Philipp Ritter, Janusz K Rybakowski, Kemal Sagduyu, Brett Sawchuk, Elon Schwartz, Ângela Miranda Scippa, Claire Slaney, Ahmad Hatim Sulaiman, Kirsi Suominen, Aleksandra Suwalska, Peter Tam, Yoshitaka Tatebayashi, Leonardo Tondo, Maj Vinberg, Biju Viswanath, Julia Volkert, Mark Zetin, Peter C Whybrow and Michael Bauer have no conflict of interest to declare.
Availability of data
The survey questionnaire is available in the Additional file 1. Online posting of complete datasets was not approved by ethics boards or the participants.
Additional file
Footnotes
An erratum to this article is available at http://dx.doi.org/10.1186/s40345-017-0082-8.
Contributor Information
Jörn Conell, Email: Joern.Conell@uniklinikum-dresden.de.
Rita Bauer, Email: rita.bauer@uniklinikum-dresden.de.
Tasha Glenn, Email: tglenn@chronorecord.org.
Martin Alda, Email: malda@dal.ca.
Raffaella Ardau, Email: ardau.raf@tiscali.it.
Bernhard T. Baune, Email: bernhard.baune@adelaide.edu.au
Michael Berk, Email: MIKEBE@BarwonHealth.org.au.
Yuly Bersudsky, Email: yuly@bgu.ac.il.
Amy Bilderbeck, Email: abilderbeck@p1vital.com.
Alberto Bocchetta, Email: bocchett@unica.it.
Letizia Bossini, Email: letizia.bossini@gmail.com.
Angela Marianne Paredes Castro, Email: angymarianne@hotmail.com.
Eric Yat Wo Cheung, Email: a9840748@yahoo.com.hk.
Caterina Chillotti, Email: cchillotti@aoucagliari.it.
Sabine Choppin, Email: sabine.choppin@gmail.com.
Maria Del Zompo, Email: delzompo@unica.it.
Rodrigo Dias, Email: dgdias@gmail.com.
Seetal Dodd, Email: SEETALD@BarwonHealth.org.au.
Anne Duffy, Email: annecduffy@yahoo.com.
Bruno Etain, Email: Bruno.Etain@inserm.fr.
Andrea Fagiolini, Email: andreafagiolini@gmail.com.
Julie Garnham, Email: Julie.Garnham@nshealth.ca.
John Geddes, Email: john.geddes@psych.ox.ac.uk.
Jonas Gildebro, Email: jonagild@rm.dk.
Ana Gonzalez-Pinto, Email: anapinto@telefonica.net.
Guy M. Goodwin, Email: guy.goodwin@psych.ox.ac.uk
Paul Grof, Email: paulgrof75@gmail.com.
Hirohiko Harima, Email: harimahirohiko@mac.com.
Stefanie Hassel, Email: hassel.stefanie@gmail.com.
Chantal Henry, Email: chantal.henry@inserm.fr.
Diego Hidalgo-Mazzei, Email: DAHIDALG@clinic.ub.es.
Vaisnvy Kapur, Email: vaisnvynm@gmail.com.
Girish Kunigiri, Email: Girish.Kunigiri@leicspart.nhs.uk.
Beny Lafer, Email: benylafer@gmail.com.
Chun Lam, Email: drlamchun@gmail.com.
Erik Roj Larsen, Email: eriklars@rm.dk.
Ute Lewitzka, Email: ute.lewitzka@uniklinikum-dresden.de.
Rasmus Licht, Email: RASMUS.LICHT@RN.DK.
Anne Hvenegaard Lund, Email: annmiels@rm.dk.
Blazej Misiak, Email: mblazej@interia.eu.
Patryk Piotrowski, Email: patryk.piotrowski@umed.wroc.pl.
Scott Monteith, Email: monteit2@msu.edu.
Rodrigo Munoz, Email: RMunozMD@aol.com.
Takako Nakanotani, Email: nakanotani-tk@igakuken.or.jp.
René E. Nielsen, Email: ren@rn.dk
Claire O’Donovan, Email: Claire.ODonovan@cdha.nshealth.ca.
Yasushi Okamura, Email: oka221kaku@yahoo.co.jp.
Yamima Osher, Email: yamy11@gmail.com.
Andreas Reif, Email: andreas.reif@kgu.de.
Philipp Ritter, Email: Philipp.Ritter@uniklinikum-dresden.de.
Janusz K. Rybakowski, Email: janusz.rybakowski@gmail.com
Kemal Sagduyu, Email: feelbetter@gmail.com.
Brett Sawchuk, Email: brettsawchuk@gmail.com.
Elon Schwartz, Email: lons6@optonline.net.
Ângela Miranda Scippa, Email: amscippa@gmail.com.
Claire Slaney, Email: Claire.slaney@nshealth.ca.
Ahmad Hatim Sulaiman, Email: hatim@um.edu.my.
Kirsi Suominen, Email: kirsisuominen@kolumbus.fi.
Aleksandra Suwalska, Email: asuwalska@umed.poznan.pl.
Peter Tam, Email: ocdocd24@gmail.com.
Yoshitaka Tatebayashi, Email: tatebayashi-ys@igakuken.or.jp.
Leonardo Tondo, Email: leonardot@aol.com.
Eduard Vieta, Email: EVIETA@clinic.ub.es.
Maj Vinberg, Email: Maj.Vinberg@regionh.dk.
Biju Viswanath, Email: bijuv1@gmail.com.
Julia Volkert, Email: Volkert_J@ukw.de.
Mark Zetin, Email: mzetinmd2@mac.com.
Iñaki Zorrilla, Email: i.zorrilla@gmail.com.
Peter C. Whybrow, Email: PWhybrow@MEDNET.ucla.edu
Michael Bauer, Email: Michael.Bauer@uniklinikum-dresden.de.
References
- Arthur C. Google’s growing problem: 50% of people do zero searches per day on mobile. 2015. https://www.theoverspill.wordpress.com/2015/10/19/searches-average-mobile-google-problem. Accessed 7 June 2016.
- Baldessarini RJ, Perry R, Pike J. Factors associated with treatment nonadherence among US bipolar disorder patients. Hum Psychopharmacol. 2008;23:95–105. doi: 10.1002/hup.908. [DOI] [PubMed] [Google Scholar]
- Barnes C, Harvey R, Wilde A, Hadzi-Pavlovic D, Wilhelm K, Mitchell PB. Review of the quality of information on bipolar disorder on the Internet. Aust NZ J Psychiatry. 2009;43:934–945. doi: 10.1080/00048670903179137. [DOI] [Google Scholar]
- Bates JA, Whitehead R, Bolge SC, Kim E. Correlates of medication adherence among patients with bipolar disorder: results of the Bipolar Evaluation of Satisfaction and Tolerability (BEST) Study: a nationwide cross-sectional survey. Prim Care Companion J Clin Psychiatry. 2010;12:333–338. doi: 10.4088/PCC.09m00883yel. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bauer R, Conell J, Glenn T, Alda M, Ardau R, Baune BT, et al. Internet use by patients with bipolar disorder: results from an international multisite survey. Psychiatry Res. 2016;242:388–394. doi: 10.1016/j.psychres.2016.05.055. [DOI] [PubMed] [Google Scholar]
- Bauer R, Bauer M, Spiessl H, Kagerbauer T. Cyber-support: an analysis of online self-help forums (online self-help forums in bipolar disorder) Nord J Psychiatry. 2013;67(3):185–190. doi: 10.3109/08039488.2012.700734. [DOI] [PubMed] [Google Scholar]
- Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians’ internet information-seeking behaviors. J Contin Educ Health Prof. 2004;24:31–38. doi: 10.1002/chp.1340240106. [DOI] [PubMed] [Google Scholar]
- Berger M, Wagner TH, Baker LC. Internet use and stigmatized illness. Soc Sci Med. 2005;61:1821–1827. doi: 10.1016/j.socscimed.2005.03.025. [DOI] [PubMed] [Google Scholar]
- Bowman CM. Basics of international privacy law for commercial litigators, part 1: the EU. The National Law Review. 2016. http://www.natlawreview.com/article/basics-international-privacy-law-commercial-litigators-part-1-eu. Accessed 7 June 2016.
- Bowskill R, Clatworthy J, Parham R, Rank T, Horne R. Patients’ perceptions of information received about medication prescribed for bipolar disorder: implications for informed choice. J Affect Disord. 2007;100:253–257. doi: 10.1016/j.jad.2006.10.018. [DOI] [PubMed] [Google Scholar]
- Brashers DE, Goldsmith DJ, Hsieh E. Information seeking and avoiding in health contexts. Hum Commun Res. 2002;28:258–271. doi: 10.1111/j.1468-2958.2002.tb00807.x. [DOI] [Google Scholar]
- Bunz U, Curry C, Voon W. Perceived versus actual computer-email-web fluency. Comput Human Behav. 2007;23:2321–2344. doi: 10.1016/j.chb.2006.03.008. [DOI] [Google Scholar]
- Chan JK, Farrer LM, Gulliver A, Bennett K, Griffiths KM. University students’ views on the perceived benefits and drawbacks of seeking help for mental health problems on the Internet: a qualitative study. JMIR Hum Factors. 2016;3(1):e3. doi: 10.2196/humanfactors.4765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chew SW, Khoo CS. Comparison of drug information on consumer drug review sites versus authoritative health information websites. J Assoc Inf Sci Technol. 2016;67:333–349. doi: 10.1002/asi.23390. [DOI] [Google Scholar]
- Chung JE. Patient-provider discussion of online health information: results from the 2007 health information national trends survey (HINTS) J Health Commun. 2013;18:627–648. doi: 10.1080/10810730.2012.743628. [DOI] [PubMed] [Google Scholar]
- Cisco. Cisco visual networking index: global mobile data traffic forecast update, 2015–2020 White Paper. 2016. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html. Accessed 7 June 2016.
- Clarke MA, Belden JL, Koopman RJ, Steege LM, Moore JL, Canfield SM, et al. Information needs and information-seeking behaviour analysis of primary care physicians and nurses: a literature review. Health Info Libr J. 2013;30:178–190. doi: 10.1111/hir.12036. [DOI] [PubMed] [Google Scholar]
- Clauson KA, Polen HH, Boulos MN, Dzenowagis JH. Scope, completeness, and accuracy of drug information in Wikipedia. Ann Pharmacother. 2008;42:1814–1821. doi: 10.1345/aph.1L474. [DOI] [PubMed] [Google Scholar]
- Coberly E, Boren SA, Davis JW, McConnell AL, Chitima-Matsiga R, Ge B, et al. Linking clinic patients to Internet-based, condition-specific information prescriptions. J Med Libr Assoc. 2010;98:160–164. doi: 10.3163/1536-5050.98.2.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Crangle CE, Kart JB. A questions-based investigation of consumer mental-health information. PeerJ. 2015;31(3):e867. doi: 10.7717/peerj.867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friedmann PD, Moulton AW. Patients’ use of the internet for medical information. J Gen Intern Med. 2002;17:180–185. doi: 10.1046/j.1525-1497.2002.10603.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dixon P, Gellman B. World privacy forum. The scoring of America: how secret consumer scores threaten your privacy and your future. 2014. https://www.worldprivacyforum.org/2014/04/wpf-report-the-scoring-of-america-how-secret-consumer-scores-threaten-your-privacy-and-your-future. Accessed 7 June 2016.
- Eavis P. How you’re making facebook a money machine. New York Times. 2016. http://www.mobile.nytimes.com/2016/04/30/upshot/how-youre-making-facebook-a-money-machine.html. Accessed 7 June 2016.
- European Parliament News. Q&A: new EU rules on data protection put the citizen back in the driving seat. 2016. http://www.europarl.europa.eu/news/en/news-room/20160413BKG22980/QA-new-EU-rules-on-data-protection-put-the-citizen-back-in-the-driving-seat. Accessed 7 June 2016.
- Eurostat. Foreign language learning statistics. 2016. http://www.ec.europa.eu/eurostat/statistics-explained/index.php/Foreign_language_learning_statistics. Accessed 7 June 2016.
- Ford EW, Walls VU. Effective US health system websites: establishing benchmarks and standards for effective consumer engagement/practitioner application. J Healthc Manag. 2012;57:47. [PubMed] [Google Scholar]
- Fox S. Peer-to-peer health care. Pew research. 2011. http://www.pewinternet.org/2011/02/28/peer-to-peer-health-care-2. Accessed 7 June 2016.
- Fox S, Duggan M. Health online. Pew research. 2013. http://www.pewinternet.org/Reports/2013/Health-online.aspx. Accessed 7 June 2016.
- Friedman A. The future of search engines is context. 2015. http://www.searchengineland.com/future-search-engines-context-217550. Accessed 7 June 2016.
- FTC (Federal Trade Commission). Data brokers: a call for transparency and accountability. 2014. https://www.ftc.gov/system/files/documents/reports/data-brokers-call-transparency-accountability-report-federal-trade-commission-may-2014/140527databrokerreport.pdf. Accessed 7 June 2016.
- Gaglio B, Glasgow RE, Bull SS. Do patient preferences for health information vary by health literacy or numeracy? A qualitative assessment. J Health Commun. 2012;17(sup3):109–121. doi: 10.1080/10810730.2012.712616. [DOI] [PubMed] [Google Scholar]
- Ghoshal M, Walji MF. Quality of medication information available on retail pharmacy web sites. Res Social Adm Pharm. 2006;2:479–498. doi: 10.1016/j.sapharm.2006.05.002. [DOI] [PubMed] [Google Scholar]
- Giacco D, Luciano M, Del Vecchio V, Sampogna G, Slade M, Clarke E, et al. Desire for information of people with severe mental illness. Soc Psychiatry Psychiatr Epidemiol. 2014;49:1883–1891. doi: 10.1007/s00127-014-0901-5. [DOI] [PubMed] [Google Scholar]
- Gidwani R, Zulman D. Association between acute medical exacerbations and consuming or producing web-based health information: analysis from Pew survey data. J Med Internet Res. 2015;17:e145. doi: 10.2196/jmir.3801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glenn T, Monteith S. Privacy in the digital world: medical and health data outside of HIPAA protections. Curr Psychiatry Rep. 2014;16:1–1. doi: 10.1007/s11920-014-0494-4. [DOI] [PubMed] [Google Scholar]
- Google. A remedy for your health-related questions: health info in the Knowledge Graph. 2015. https://www.googleblog.blogspot.co.uk/2015/02/health-info-knowledge-graph.html. Accessed 7 June 2016.
- Greengard S. Advertising gets personal. Commun ACM. 2012;55:18–20. doi: 10.1145/2240236.2240243. [DOI] [Google Scholar]
- Grohol JM, Slimowicz J, Granda R. The quality of mental health information commonly searched for on the Internet. Cyberpsychol Behav Soc Netw. 2014;17:216–221. doi: 10.1089/cyber.2013.0258. [DOI] [PubMed] [Google Scholar]
- Hallett C, Gupta S, Priebe S. What do outpatients with schizophrenia and mood disorders want to learn about their illness? Psychiatr Serv. 2013;64:764–769. doi: 10.1176/appi.ps.201200382. [DOI] [PubMed] [Google Scholar]
- Hasty RT, Garbalosa RC, Barbato VA, Valdes PJ, Jr, Powers DW, Hernandez E, et al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc. 2014;114:368–373. doi: 10.7556/jaoa.2014.035. [DOI] [PubMed] [Google Scholar]
- Hay MC, Cadigan RJ, Khanna D, Strathmann C, Lieber E, Altman R, et al. Prepared patients: internet information seeking by new rheumatology patients. Arthritis Rheum. 2008;59:575–582. doi: 10.1002/art.23533. [DOI] [PubMed] [Google Scholar]
- Heilman JM, West AG. Wikipedia and medicine: quantifying readership, editors, and the significance of natural language. J Med Internet Res. 2015;17(3):e62. doi: 10.2196/jmir.4069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med. 2005;165:2618–2624. doi: 10.1001/archinte.165.22.2618. [DOI] [PubMed] [Google Scholar]
- Hidalgo-Mazzei D, Mateu A, Reinares M, Matic A, Vieta E, Colom F. Internet-based psychological interventions for bipolar disorder: review of the present and insights into the future. J Affect Disord. 2015;188:1–3. doi: 10.1016/j.jad.2015.08.005. [DOI] [PubMed] [Google Scholar]
- Hofstede Cultural Dimensions. 2016. https://www.geert-hofstede.com/national-culture.html. Accessed 7 June 2016.
- Horgan A, Sweeney J. Young students’ use of the internet for mental health information and support. J Psychiatr Ment Health Nurs. 2010;17:117–123. doi: 10.1111/j.1365-2850.2009.01497.x. [DOI] [PubMed] [Google Scholar]
- Hughes S, Cohen D. Can online consumers contribute to drug knowledge? A mixed-methods comparison of consumer-generated and professionally controlled psychotropic medication information on the internet. J Med Internet Res. 2011;13:e53. doi: 10.2196/jmir.1716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hwang TJ, Bourgeois FT, Seeger JD. Drug safety in the digital age. N Engl J Med. 2014;370:2460–2462. doi: 10.1056/NEJMp1401767. [DOI] [PubMed] [Google Scholar]
- Imes RS, Bylund CL, Sabee CM, Routsong TR, Sanford AA. Patients’ reasons for refraining from discussing internet health information with their healthcare providers. Health Commun. 2008;23:538–547. doi: 10.1080/10410230802460580. [DOI] [PubMed] [Google Scholar]
- Ivanitskaya L, O’Boyle I, Casey AM. Health information literacy and competencies of information age students: results from the interactive online Research Readiness Self-Assessment (RRSA) J Med Internet Res. 2006;8(2):e6. doi: 10.2196/jmir.8.2.e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson FR, Özdemir S, Manjunath R, Hauber AB, Burch SP, Thompson TR. Factors that affect adherence to bipolar disorder treatments: a stated-preference approach. Med Care. 2007;45:545–552. doi: 10.1097/MLR.0b013e318040ad90. [DOI] [PubMed] [Google Scholar]
- Kawado M, Hinotsu S, Matsuyama Y, Yamaguchi T, Hashimoto S, Ohashi Y. A comparison of error detection rates between the reading aloud method and the double data entry method. Control Clin Trials. 2003;24:560–569. doi: 10.1016/S0197-2456(03)00089-8. [DOI] [PubMed] [Google Scholar]
- Kim J, Kim S. Physicians’ perception of the effects of internet health information on the doctor–patient relationship. Inform Health Soc Care. 2009;34:136–148. doi: 10.1080/17538150903102422. [DOI] [PubMed] [Google Scholar]
- Klee A, Stacy M, Rosenheck R, Harkness L, Tsai J. Interest in technology-based therapies hampered by access: a survey of veterans with serious mental illnesses. Psychiatr Rehabil J; 2016 [Epub ahead of print]. [DOI] [PubMed]
- Koppen L, Phillips J, Papageorgiou R. Analysis of reference sources used in drug-related Wikipedia articles. J Med Libr Assoc. 2015;103:140. doi: 10.3163/1536-5050.103.3.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korp P. Health on the Internet: implications for health promotion. Health Educ Res. 2006;21:78–86. doi: 10.1093/her/cyh043. [DOI] [PubMed] [Google Scholar]
- Kupferberg N, Protus BM. Accuracy and completeness of drug information in Wikipedia: an assessment. J Med Libr Assoc. 2011;99:310–313. doi: 10.3163/1536-5050.99.4.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lambert SD, Loiselle CG. Health information—seeking behavior. Qual Health Res. 2007;17:1006–1019. doi: 10.1177/1049732307305199. [DOI] [PubMed] [Google Scholar]
- Laurent MR, Vickers TJ. Seeking health information online: does Wikipedia matter? J Am Med Inform Assoc. 2009;16:471–479. doi: 10.1197/jamia.M3059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lavsa SM, Corman SL, Culley CM, Pummer TL. Reliability of Wikipedia as a medication information source for pharmacy students. Curr Pharm Teach Learn. 2011;3:154–158. doi: 10.1016/j.cptl.2011.01.007. [DOI] [Google Scholar]
- Leon SA, Fontelo P. MedlinePlus en Espanol and Spanish-speakers. AMIA Annu Symp Proc; 2016:1028. [PubMed]
- Li J. Privacy policies for health social networking sites. J Am Med Inform Assoc. 2013;20:704–707. doi: 10.1136/amiajnl-2012-001500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Libert T. Privacy implications of health information seeking on the web. Commun ACM. 2015;58:68–77. doi: 10.1145/2658983. [DOI] [Google Scholar]
- Link TW, Hefner JL, Ford EW, Huerta TR. An effective web presence for substance abuse treatment facilities. Subst Use Misuse. 2016;51:664–667. doi: 10.3109/10826084.2015.1131296. [DOI] [PubMed] [Google Scholar]
- Lupton D. The commodification of patient opinion: the digital patient experience economy in the age of big data. Sociol Health Illn. 2014;36:856–869. doi: 10.1111/1467-9566.12109. [DOI] [PubMed] [Google Scholar]
- Mackey TK, Cuomo RE, Liang BA. The rise of digital direct-to-consumer advertising? Comparison of direct-to-consumer advertising expenditure trends from publicly available data sources and global policy implications. BMC Health Serv Res. 2015;15:1. doi: 10.1186/s12913-015-0885-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacMillan D. Mobile search tops at Google. Oct 8 2015. Wall Street Journal. (WSJ.D). 2015. http://blogs.wsj.com/digits/2015/10/08/google-says-mobile-searches-surpass-those-on-pcs. Accessed 7 June 2016.
- McCray AT, Loane RF, Browne AC, Bangalore AK. Terminology issues in user access to Web-based medical information. In: Proceedings of the AMIA symposium. Bethesda: American Medical Informatics Association; 1999. p. 107. [PMC free article] [PubMed]
- McKibbon KA, Fridsma DB. Effectiveness of clinician-selected electronic information resources for answering primary care physicians’ information needs. J Am Med Inform Assoc. 2006;13:653–659. doi: 10.1197/jamia.M2087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- MedlinePlus. About MedlinePlus. 2016. https://www.nlm.nih.gov/medlineplus/aboutmedlineplus.html. Accessed 7 June 2016.
- Meier A, Lyons E, Frydman G, Forlenza M, Rimer B. How cancer survivors provide support on cancer-related Internet mailing lists. J Am Med Inform Assoc. 2007;9:e12. doi: 10.2196/jmir.9.2.e12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Merritt K, Smith D, Renzo JC. An investigation of self-reported computer literacy: is it reliable. Issues Inf Syst. 2005;6:289–295. [Google Scholar]
- Miller CJ, McInnes DK, Stolzmann K, Bauer MS. Interest in use of technology for healthcare among veterans receiving treatment for mental health. Telemedicine and e-health. Telemed J E Health. 2016. [Epub ahead of print]. [DOI] [PubMed]
- Miller N. Analysis of user messages to MedlinePlus. gov. J Med Libr Assoc. 2007;95:81. [PMC free article] [PubMed] [Google Scholar]
- Mitchell KJ, Wells M, Priebe G, Ybarra ML. Exposure to websites that encourage self-harm and suicide: prevalence rates and association with actual thoughts of self-harm and thoughts of suicide in the United States. J Adolesc. 2014;37:1335–1344. doi: 10.1016/j.adolescence.2014.09.011. [DOI] [PubMed] [Google Scholar]
- Monteith S, Glenn T, Bauer M. Searching the internet for health information about bipolar disorder: some cautionary issues. Int J Bipolar Disord. 2013;1:22. doi: 10.1186/2194-7511-1-22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Monteith S, Glenn T, Bauer R, Conell J, Bauer M. Availability of prescription drugs for bipolar disorder at online pharmacies. J Affect Disord. 2016;193:59–65. doi: 10.1016/j.jad.2015.12.043. [DOI] [PubMed] [Google Scholar]
- Morel V, Chatton A, Cochand S, Zullino D, Khazaal Y. Quality of web-based information on bipolar disorder. J Affect Disord. 2008;110:265–269. doi: 10.1016/j.jad.2008.01.007. [DOI] [PubMed] [Google Scholar]
- Murray G, Suto M, Hole R, Hale S, Amari E, Michalak EE. Self-management strategies used by ‘high functioning’ individuals with bipolar disorder: from research to clinical practice. Clin Psychol Psychother. 2011;18:95–109. doi: 10.1002/cpp.710. [DOI] [PubMed] [Google Scholar]
- Nambisan P. Information seeking and social support in online health communities: impact on patients’ perceived empathy. J Am Med Inform Assoc. 2011;18:298–304. doi: 10.1136/amiajnl-2010-000058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Narayanan A, Huey J, Felten EW. A precautionary approach to big data privacy. In: Gutwirth S, Leenes R, De Hert P, editors. Data protection on the move. Berlin: Springer; 2016. pp. 357–385. [Google Scholar]
- Neaton JD, Duchene AG, Svendsen KH, Wentworth D. An examination of the efficiency of some quality assurance methods commonly employed in clinical trials. Stat Med. 1990;9:115–123. doi: 10.1002/sim.4780090118. [DOI] [PubMed] [Google Scholar]
- NHS Choices: How we work. 2015. http://www.nhs.uk/aboutNHSChoices/aboutnhschoices/Aboutus/Pages/Editorialpolicy.aspx. Accessed 7 June 2016.
- NISO (National Information Standards Organization). Understanding Metadata 2004. http://www.niso.org/publications/press/UnderstandingMetadata.pdf. Accessed 7 June 2016.
- NLM. Drug information portal. 2016. http://www.druginfo.nlm.nih.gov/drugportal/jsp/drugportal/nlmDrugResources.jsp?APPLICATION_NAME=drugportal. Accessed 7 June 2016.
- Orizio G, Merla A, Schulz PJ, Gelatti U. Quality of online pharmacies and websites selling prescription drugs: a systematic review. J Med Internet Res. 2011;13:e74. doi: 10.2196/jmir.1795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pan W. Akaike’s information criterion in generalized estimating equations. Biometrics. 2001;57:120–125. doi: 10.1111/j.0006-341X.2001.00120.x. [DOI] [PubMed] [Google Scholar]
- Pande M. Wikimedia Foundation. Average Wikipedia reader is 36 years old. 2011. http://www.blog.wikimedia.org/2011/10/21/average-wikipedia-reader-is-36-years-old. Accessed 7 June 2016.
- Parker B. More than any other foreign language, European youths learn English. Per Research. 2015. http://www.pewresearch.org/fact-tank/2015/10/08/more-than-any-other-foreign-language-european-youths-learn-english. Accessed 7 June 2016.
- Pohjanoksa-Mäntylä M, Saari JK, Närhi U, Karjalainen A, Pylkkänen K, Airaksinen MS, et al. How and why do people with depression access and utilize online drug information: a qualitative study. J Affect Disord. 2009;114:333–339. doi: 10.1016/j.jad.2008.06.024. [DOI] [PubMed] [Google Scholar]
- Rainie L, Kiesler S, Kang R, Madden M, Duggan M, Brown S, et al. Anonymity, privacy, and security online. Pew Res Cent. 2013;5:5. [Google Scholar]
- Randhawa AS, Babalola O, Henney Z, Miller M, Nelson T, Oza M, et al. A collaborative assessment among 11 pharmaceutical companies of misinformation in commonly used online drug information compendia. Ann Pharmacother. 2016;50:352–359. doi: 10.1177/1060028016635196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reavley NJ, Jorm AF. The quality of mental disorder information websites: a review. Patient Educ Couns. 2011;85:e16–e25. doi: 10.1016/j.pec.2010.10.015. [DOI] [PubMed] [Google Scholar]
- Return on Now. 2015 search engine market share by country. 2016. http://www.returnonnow.com/internet-marketing-resources/2015-search-engine-market-share-by-country. Accessed 7 June 2016.
- Rice RE. Influences, usage, and outcomes of Internet health information searching: multivariate results from the Pew surveys. Int J Med Inform. 2006;75:8–28. doi: 10.1016/j.ijmedinf.2005.07.032. [DOI] [PubMed] [Google Scholar]
- Rosenberg E. The business of Google. Investopedia. 2015. http://www.investopedia.com/articles/investing/020515/business-google.asp. Accessed 7 June 2016.
- Russ H, Giveon SM, Granek Catarivas M, Yaphe J. The effect of the Internet on the patient-doctor relationship from the patient’s perspective: a survey from primary care. Isr Med Assoc J. 2011;13:220. [PubMed] [Google Scholar]
- Schenker Y, London AJ. Risks of imbalanced information on US Hospital Websites. JAMA Intern Med. 2015;175:441–443. doi: 10.1001/jamainternmed.2014.7400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seife C. 23andMe is terrifying, but not for the reasons the FDA thinks. Sci Am. 2013. http://www.scientificamerican.com/article/23andme-is-terrifying-but-not-for-the-reasons-the-fda-thinks/.
- Sharp E. The first page of Google, by the numbers. 2014. http://www.protofuse.com/blog/first-page-of-google-by-the-numbers/. Accessed 7 June 2016.
- Sidhu H, Raynor DK, Knapp P. Evaluation of website medicines information content, in comparison with official patient information. Int J Pharm Pract. 2006;14:189–195. doi: 10.1211/ijpp.14.3.0005. [DOI] [Google Scholar]
- Siegel ER, Logan RA, Harnsberger RL, Cravedi K, Krause JA, Lyon B, et al. Information Rx: evaluation of a new informatics tool for physicians, patients, and libraries. Inf Serv Use. 2006;26:1–10. doi: 10.3233/isu-2006-26101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stark M, Fins JJ. Engineering medical decisions. Camb Q Healthc Ethics. 2013;22:373–381. doi: 10.1017/S0963180113000224. [DOI] [PubMed] [Google Scholar]
- Statistica. Average number of search terms for online search queries in the United States as of January 2016. http://www.statista.com/statistics/269740/number-of-search-terms-in-internet-research-in-the-us/. Accessed 7 June 2016.
- Szmulewicz A, Samamé C, Caravotta P, Martino DJ, Igoa A, Hidalgo-Mazzei D, et al. Behavioral and emotional adverse events of drugs frequently used in the treatment of bipolar disorders: clinical and theoretical implications. Int J Bipolar Disord. 2016;4(1):6. doi: 10.1186/s40345-016-0047-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tustin N. The role of patient satisfaction in online health information seeking. J Health Commun. 2010;15:3–17. doi: 10.1080/10810730903465491. [DOI] [PubMed] [Google Scholar]
- UNESCO Institute for Statistics. 2015. Education. http://www.data.uis.unesco.org/Index.aspx?queryid=242. Accessed 7 June 2016.
- Vaidhyanathan S. The Googlization of everything: (and why we should worry). 1st ed. Berkeley: University of California Press; 2012.
- van Uden-Kraan CF, Drossaert CH, Taal E, Seydel ER, van de Laar MA. Participation in online patient support groups endorses patients’ empowerment. Patient Educ Couns. 2009;74:61–69. doi: 10.1016/j.pec.2008.07.044. [DOI] [PubMed] [Google Scholar]
- W3Techs. Usage of content languages for websites. 2016. http://w3techs.com/technologies/overview/content_language/all. Accessed 7 June 2016.
- Wang L, Wang J, Wang M, Li Y, Liang Y, Xu D. Using internet search engines to obtain medical information: a comparative study. J Med Internet Res. 2012;14:e74. doi: 10.2196/jmir.1943. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weaver JB, III, Mays D, Weaver SS, Hopkins GL, Eroglu D, Bernhardt JM. Health information-seeking behaviors, health indicators, and health risks. Am J Public Health. 2010;100:1520–1525. doi: 10.2105/AJPH.2009.180521. [DOI] [PMC free article] [PubMed] [Google Scholar]