Abstract
Background
The number of individuals looking for health information on the Internet continues to expand. The purpose of the study was to understand the prevalence of major depression, among English-speaking individuals worldwide looking for information on depression online.
Methods
An automated online Mood Screener website was created and advertised via Google AdWords, for one year. Participants (N = 24,965) completed a depression screening measure and received feedback based on their results. Participants were then invited to participate in a longitudinal mood screening study.
Results
Of the 24,965 who completed the screening, 66.6% screened positive for current major depression, 44.4% indicated current suicidality, and 7.8% reported a recent (past two weeks) suicide attempt. Of those consenting to participate in the longitudinal study (n = 1,327, from 86 countries), 77.4% screened positive for past depression, 64.6% reported past suicidality, and 17.5% a past suicide attempt. Yet, only 25% of those screening positive for current depression, and only 37.2% of those reporting a recent suicide attempt are in treatment.
Conclusions
Many of the consumers of Internet health information may genuinely need treatment and are not “cyberchondriacs”. Online screening, treatment, and prevention efforts may have the potential to serve many currently untreated clinically depressed and suicidal individuals.
Keywords: depression rates, cyberchondriasis, major depression, internet screening, telecare
In the past several years, the Internet has become the de facto primary resource for health information. A large proportion of individuals use the Internet to search for information regarding health conditions and symptoms.[1,2] With the rapid proliferation of websites such as Wikipedia, MayoClinic, or WebMD, among many others, individuals can research their health-related concerns at their convenience, without having to travel to a doctor’s office.
Several nationwide surveys periodically assess the online health information-seeking behavior of the American public: the Pew Internet & American Life Project (PI&ALP); the NCI Health Information National Trends Survey (HINTS), conducted biannually, in 2003, 2005, and 2007; and the Harris Poll, 2010 being the most recent. The results from these polls suggest that the majority of Americans have used the Internet to look for health information. According to the 2005 HINTS, 58% of 3,244 Internet users polled looked for health information for themselves.[3] The PI&ALP,[1] surveying 2,928 adults, reported that 80% of American users have searched for information related to health for themselves or others. The 2010 Harris Poll,[4] surveying 1,066 adults, found that 88% of those surveyed and connected to the Internet have looked for health-related information for themselves or others, which was the highest number registered by the Harris Poll since they began surveying on the subject of Internet health information seeking in the United States. A third of all individuals connected to the Internet said that they look for health information “Often”.
The national polls also identified several patterns regarding the types of users looking for health information. The PI&ALP[1] reported that more educated individuals and women were more likely to look for health information, and older individuals were somewhat less likely to do so. Similar results were obtained in the 2005 HINTS[3], and the 2007 HINTS data:[5] women, as well as younger and more educated individuals were more likely to seek health information. The 2007 HINTS data also revealed racial and ethnic differences in using Internet for health. Caucasians tended to use the Internet to seek health information more frequently than other groups.[6] However, even though Latinos are less likely to have access to the Internet, among Internet users, Latinos were the most likely to use the Internet to keep track of their health and to communicate with providers; they were also more likely to say that doing so is important.[7]
Looking for health information also appears to influence behavior. Over half of the individuals surveyed had said that they have discussed the information obtained online with their doctors.[4] The PI&ALP found that over half (53%) of those surveyed reported that information they have obtained has affected their decisions about their health, ranging from finding new questions to ask their doctors to choosing a particular treatment.[1]
The popularity of Internet health information led some to be concerned about “cyberchondriasis” unfounded concerns about benign symptoms fueled by overly enthusiastic search for and/or misinterpretation of online health information.[8,9] Cyberchondriasis might lead one to be unduly anxious about one’s health, prompting either self-treatment for a nonexistent condition or requesting specific treatments from a provider.[10–12] Indeed, some have found that excessive use of the Internet for health information results in increased anxiety and mood disturbances.[13]
With physicians and mental health professionals becoming wary of Internet-informed patients,[10,14] it is tempting to label Internet health seekers as “cyberchondriacs”. However, it is possible that many of these individuals are genuinely concerned about their health for legitimate reasons. For instance, one survey study used a symptom level measure to screen a large sample of US participants interested in a depression screening provided by a health insurance company online portal, and found that over half of visitors had high levels of depressive symptoms.[15] Van Ameringen and colleagues[16] recruited participants interested in anxiety screening from their clinic website; using both a screening instrument and a symptom level measure, they found that a third of participants screened positive for depression. These studies provide evidence suggesting that those wishing to be screened for depression are more likely to be experiencing significant depressive symptoms than the general population.
The current study was undertaken to understand the prevalence of major depression and depressive symptoms in a worldwide Internet community, among English speakers searching for information on depression online. Specifically, we examined the characteristics of those looking for depression-related information, to determine whether those interested in specific disorder-related information do in fact have clinically significant symptoms of that disorder, or are merely “cyberchondriacs” overly concerned about minimal symptoms. This study improved on previous research in several significant ways. First, we used a screening measure specifically designed to screen for the presence of a Major Depressive Episode according to the DSM-IV[17] criteria, rather than a symptom severity measure. Second, we recruited a very large worldwide sample of participants, rather than focusing on North American participants. Third, we recruited individuals via a search engine, rather than a clinic page or a health site, to attract a broad sample of participants who are exploring depression-related information, rather than those who are already familiar with a particular clinic or health-related website (and who may presumably be more depressed). Based on previous screening studies, we hypothesized that the rates of depression in our sample, which also consisted of those interested in depression screening, would be higher than that in the general population.
Methods
Participants
In one year, over 50,000 people visited the Mood Screener website, and 32,836 were screened for eligibility (approximately 90 per day). The only eligibility criterion was being 18 years of age; ineligible participants (n=5,177) were 15.4 (SD = 1.39) years old, after excluding unlikely values (ages less than 10, n = 43). 27,659 individuals were eligible to participate. Eligible participants were predominantly Caucasian (62.9%); 18.0% were of Asian descent, 4.3% -- of African descent. Other races were represented at less than 2% each, and 11.1% of participants indicated “Other” as their race. Eligible participants’ mean age was 32.1 (SD = 12.1, range: 18–92), and 63.8% were women.
Materials
Demographics - 1
Prior to proceeding to the screening questionnaire, participants were asked about their age, gender, and race.
Demographics - 2
Those participants agreeing to participate in a more comprehensive study were asked about their country of residence and birth, their living situation (alone or with others), education, employment status, marital status, income, and subjective social status, as assessed by the “social ladder”[18] which asks participants to estimate their position in society relative to others. These participants were also asked whether they have sought treatment for their depression in the past, and whether they are (or were) in therapy or taking antidepressant medications.
The MDE Screener[19] is an 18-item self-report measure designed to screen for the presence of current and past major depressive episodes (MDEs). It rates the presence of nine symptoms of depression according to the DSM-IV[20] over a period of two weeks or more, and assesses whether Criterion C (significant impairment in functioning) is met within the same time span. Participants are asked about the presence of symptoms over the past two weeks (to identify a “Current MDE”), as well as symptoms occurring in any two-week period in their lifetime, excluding the past two weeks (to identify history of depression). The screener has been shown to have good agreement with the PRIME-MD[21,22] and with clinician-administered diagnostic interviews[23].
Study design (Figure 1)
Participants were recruited using a worldwide Google AdWords campaign. This report focuses on the first year of recruitment (12/09/2009 to 12/08/2010). Searching for keyword terms such as “depression symptoms”, “sad mood”, or “am I depressed” triggered one of several ads for a “Free Online Depression Screener.” Those clicking on the ad were taken to the landing page of the research website (https://ihrc.ucsf.edu/Collector/Survey.ashx?Name=Mood_Screener_Survey_1). The landing page informed them that the screener is part of a research study, and provided information about the limits to their confidentiality. Participants then provided their age to determine eligibility, as well as race and gender. Beginning in mid-April 2010, eligible participants were presented an “honesty question” in addition to the existing MDE screener, to explore if the high rates of depression and suicide attempts in our sample were due to participants’ spurious responses. The question asked whether the participants’ responses are “accurate”, or whether they are simply “testing the site.” Upon completing the “Current” MDE Screener, participants were offered personalized feedback on the results. The feedback contained a brief explanation of the individuals’ symptom level. Participants indicating a high symptom level (5+ symptoms) or meeting Criterion C were prompted to consult with a mental health professional, if one is available. Participants who were suicidal (wanting to die, thinking about death, or making a suicide attempt) were offered additional feedback expressing concern, and urging them to immediately seek help by consulting with their provider or going to a hospital. Additionally, all pages contained a link to befrienders.org, an international and multilingual online database of suicide hotlines.
Participants were then offered the opportunity to participate in a monthly rescreening study. Interested participants provided their email address (to enable future contact and to prevent multiple participation) and signed consent. Consenting participants provided additional demographic information as described above, completed the “Lifetime” portion of the MDE screener, and were, once again, presented individualized feedback based on their responses to the screener. These participants were emailed monthly invitations to re-screen their mood. The data presented herein is limited to the initial screening and excludes the monthly follow-ups, as they are not yet available for this sample.
Results
Main sample characteristics (n = 27,659)
Of the 24,965 participants who provided enough data to evaluate the presence of a current major depressive episode (<2 missing answers to the MDE screener; no missing answers to questions necessary to identify an MDE, i.e., depressed mood or anhedonia, and Criterion C), 66.6% screened positive for an MDE. The mean number of symptoms in our sample was 6.49 (SD = 2.64). Of participants providing data, approximately three-quarters (73.2%) indicated that their symptoms have interfered with their lives “a lot”. Almost half of the participants providing data (44.4%) indicated some degree of suicidality, ranging from wishing to die to an actual suicide attempt, and 7.8% indicated a recent (in the past two weeks) suicide attempt.
Since April 12, 2010, an “honesty” probe was included in the survey: “How accurate are your answers? (Your answer will not affect your ability to participate, or the feedback you will receive).”, with answer options “I am trying to answer accurately.” and “I am just testing the site, so my answers may not be accurate”. The vast majority of participants (96.4%, n = 18,031) recruited since that date reported answering the screening questions accurately. The rates of depression and symptoms reported by those answering accurately were virtually identical to that of the full sample: 66.4% screened positive for current MDE, with the average number of symptoms being 7.01 (SD = 2.00), 73.1% indicated that their symptoms interfered with their lives “a lot”, 47.9% indicated some degree of suicidality, and 7.7% reported a recent suicide attempt.
Characteristics of participants consenting to monthly rescreening (n = 1,371)
Though all participants were offered to participate in the monthly rescreening study, only 5% did so. Consenting participants represented 86 countries. The largest proportion of participants were from United Kingdom (n = 384), followed by India (n = 244) and South Africa (n = 150); 15.2% of participants reported being born in a country other than the country of their current residence. Most participants (81.9%) were urban dwellers, employed at least part-time (63.6%), and either married or in a relationship (54.1%). Participants reported completing 14.7 (SD = 3.65) years of education. In regards to the social ladder, participants placed themselves, on average, just under the middle rating: 4.58 (SD = 2.26, range: 1–10) in their community, and 4.86 (SD = 2.25, range 1–10) in their country.
There were no differences between those consenting to participate in a monthly follow-up and the rest of the sample in terms of gender; however, there were significant differences in race and age, with the sample agreeing to monthly follow-ups being more diverse (50.4% Caucasian vs. 63.7%, chi-square = 99.82, p < .001), and slightly older (33.0 vs. 32.1 years old, t(27,657) = 2.81, p < 0.01), though the significance of the latter difference is likely due to the large size of the sample. Consenting participants indicated more severity, across all parameters: more current symptoms (m = 7.35 vs. 6.44, t(1671.11) = 17.17, p < 0.001); greater likelihood of symptoms affecting current functioning, (77.4% vs. 72.9%, chi-square = 13.06, p < 0.001); greater incidence of current depression (72% vs. 66.3%, chi-square = 19.06, p < 0.001); higher incidence of suicidality (57.3% vs. 43.8%, chi-square = 96.24, p < 0.001); and more recent suicide attempts (10.9% vs. 7.7%, chi-square = 18.55, p < 0.001). Consenting participants were also more likely to report being honest (99.0% vs. 96.2%, chi-square = 20.86, p < 0.001).
Participants reported high levels of lifetime (any time aside from the past two weeks) depressive symptoms. Mean number of lifetime symptoms was 6.47 (SD = 3.25), and 80.4% reported significant interference of symptoms with daily functioning. Three quarters (77.4%) of participants screened positive for lifetime MDE. In regards to suicidality, 64.6% endorsed some level of past suicidality, and 17.5% reported attempting suicide in the past.
Approximately half of the consenting participants (53.1%) reported having sought help for their depression at some point in their lives. Antidepressant medication was more prevalent among participants than talk therapy, with 43.5% of the consenting participants reporting having taken antidepressants (20.4% currently), but only 31.8% reported having been in therapy (7.9% currently). Of the people who screened positive for current depression, only 25% were currently in treatment (23.1% taking antidepressants, 8.8% in therapy). Of the people who reported attempting suicide in the past two weeks, only 37.2% reported being currently in treatment (35.0% taking antidepressants, 15.3% in therapy).
Discussion
During our year-long study, we have been able to attract over 50,000 participants to our site and screen 24,965 people for the presence of major depression. Further, we have been able to enroll 1,371 people – a diverse international sample – into a longitudinal study. The rate of traffic on our Mood Screener website indicates both the high interest of the worldwide Internet community in obtaining health information and services on the internet, as well as the effectiveness of using the Internet to attract and recruit study participants. The very high rates of depression observed in our sample was not anticipated, however, and suggests that the majority of those looking for depression information online might be doing so for a good reason.
Several results observed in our data were quite troubling, and deserve further study. Our study only included adults of 18 years or older, which was the only exclusion criterion, as required by our institutional IRB. Yet, over 5,000 individuals, which represented 15% of those screened for eligibility, were unable to use the screening site because of this requirement. Thus, there are a considerable number of children and adolescents who are actively looking for information on depression online and are wishing to screen themselves for presence of depression. This suggests a clear and urgent need to engage in widely disseminated screening and prevention efforts aimed specifically at children and adolescents. It also highlights an ethical dilemma: Is it preferable to withhold readily accessible depression screening tools from young people who may be at risk, or to provide such tools to minors who may not have the maturity to use the information properly and who may even be unnecessarily upset by such information? Depression is a recurrent disorder, and having a single episode dramatically increases the chances of future episodes.[17,24] Preventing the first episode in childhood or adolescence may protect an individual from a lifetime of struggle with depression, and, given the devastating effect of depression, will have a considerable positive impact on public health.
The high rate of depression found in our sample suggests that the vast majority of those who go online in search of health information are not merely “cyberchondriacs”, who are overly concerned about relatively benign symptoms. Indeed, a third of the sample screened positive for current major depressive episode, and three quarters screened positive for lifetime major depression. These rates are several times higher than would be expected in the general population. It is especially important to note that less than half of participants had ever been treated for depression, which echoes the Houston et al (2001) findings, and only a quarter of participants screening positive for current depression were receiving treatment currently. Even more troubling was our finding that only 37.2% of the participants reporting attempting suicide in the past two weeks are in treatment currently. Put in another way, over 60% of people who have attempted suicide are currently without mental health care.
Just as having a previous depressive episode is a powerful predictor of future episodes, a past suicide attempt is a powerful predictor of future attempts (and, ultimately, of a completed suicide). The very high rate of untreated depression and of suicide attempts in our sample should serve as a strong call for providing prevention and treatment services for individuals at risk.[25] However, it should also be clear that conventional methods of prevention and treatment delivery – mood management training, face-to-face therapy, and/or medication – might only help a fraction of those in need of screening and services. Even if it would be possible to train and deploy enough providers to treat all 16,000 individuals screening positive for depression on our site, there would still be hundreds of thousands of individuals undergoing similar screenings on other sites, who are also in need of services. And even if it might be possible to train thousands more mental health practitioners, some individuals would be reluctant to seek services due to fear of stigma or inability to pay for such services. Thus, it is necessary to offer access to prevention and treatment in a medium where sufferers are already seeking information – on the Internet. Internet interventions, including interventions for major depression, are already being tested, and some have attained promising results.[26,27] However, greater efforts are needed to create, develop, test, disseminate, and maintain these interventions, to offer access to currently underserved individuals and communities and to help reduce health disparities worldwide.[28] Development and proliferation of such services will make a considerable contribution to public health.[29]
The high traffic experienced by our website is indicative of the interest in using the Internet as a resource for health information. Although it can be extremely useful to have a research tool as powerful as the Internet at one’s disposal, it can also be harmful and dangerous. The vast majority (85%) of those finding health information online believe that it is reliable[4] and do not verify the source of the information.[1] In other words, in the eyes of the majority of consumers of this information, an opinion voiced by someone in a blog may be just as valid as a reputable site relaying state-of-the-science information. It is therefore necessary to institute and widely adopt a system of health ratings, so that a user can quickly and easily verify the reputability of the information source and, therefore, the trustworthiness of the message.
There are several limitations to this study. Participants were screened for MDE via an automatic online screener, rather than being given a diagnostic interview by a live mental health professional. Although the MDE Screener has good psychometric properties and high concordance with live interviewers, the results only offer a provisional screening, and should not be considered a definitive diagnosis. However, administering the study online, rather than with live personnel, allowed us to attract and screen people who, according to our data, are either unwilling or unable to see a mental health provider. Although MDE screener has been validated for concordance with clinicians’ assessments, including via non-traditional (computerized) administration [22,23], it was not specifically validated for use on the Internet. However, there is a growing body of evidence that there are few if any differences between Internet and paper-and-pencil administrations of health-related questionnaires [30–40]. All participants in this study were those searching for information related to major depression. Insofar as our results are not based on random sampling of the Internet community, they may generalize only to those interested in being screened for depression, and are likely overestimates of the rates of depression for the English-speaking Internet community. It is possible that some participants experienced symptoms of depression as a result of bereavement or a general medical condition. Though it is possible that some participants answered randomly, or experimented with responses, given the results of our “honesty” probe and our high sample size, it is unlikely that such instances of problematic data were numerous enough to skew our results substantially. Though it is possible that some responses represent multiple completions by the same individual or individuals, it is highly unlikely that such responses account for a large proportion of our sample, given that those who were interested in being rescreened could sign up for a study that offered this service.
The influence of the Internet on health systems and on public health is likely to continue to grow and expand. Although some small proportion of those seeking health information online may indeed be “cyborchondriacs”, many, if not most may have legitimate undertreated symptoms. Internet screenings may potentially reach an underserved population not represented in either clinics or face-to-face screening studies. To the extent that this population adds to the global burden of disease and reflects a substantial number of individuals who are suffering from serious depression, it is important to develop online resources for these individuals, such as evidence-based Internet interventions to prevent or manage depression.
Acknowledgments
This work was supported by Robert Wood Johnson Health and Society Scholars Seed Grant (Leykin, P.I.), by an infrastructure grant from the University of California Committee on Latino Research to the University of California/San Francisco General Hospital Latino Mental Health Research Program (Muñoz, P.I.), by an NIMH Training grant (T32 MH19391) to Yan Leykin (Nancy Adler, P.I.), and by NIMH grant 5K08MH091501 (Leykin, P.I.). The authors thank the Center for Health and Community (Nancy Adler, Director) for providing office space and additional resources. Special thanks go to Google, Inc. for awarding us an AdWords grant (Muñoz, P.I.), which provided us with the ability to recruit participants worldwide using Google sponsored links, and to the Brin Wojcicki Foundation for additional funding.
References
- 1.Fox S. Online health search 2006: Most internet users start at a search engine when looking for health information online, very few check the source and date of the information they find: Pew Internet & American Life Project. 2006 http://www.pewinternet.org/~/media/Files/Reports/2006/PIP_Online_Health_2006.pdf.pdf.
- 2.Powell JA, Darvell M, Gray JAM. The doctor, the patient and the world-wide web: How the internet is changing healthcare. J R Soc Med. 2003;96:74. doi: 10.1258/jrsm.96.2.74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Atkinson NL, Saperstein SL, Pleis J. Using the internet for health-related activities: Findings from a national probability sample. J Medical Internet Res. 2009;11:e4. doi: 10.2196/jmir.1035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Poll Harris. “cyberchondriacs” on the rise? Those who go online for healthcare information continues to increase. 2010 http://www.harrisinteractive.com/vault/HI-Harris-Poll-Cyberchondriacs-2010-08-04.pdf.
- 5.Koch-Weser S, Bradshaw Y, Gualtieri L, Gallagher S. The internet as a health information source: Findings from the 2007 Health Information National Trends Survey and implications for health communication. J Health Commun. 2010;15:279–293. doi: 10.1080/10810730.2010.522700. [DOI] [PubMed] [Google Scholar]
- 6.Hou J, Shim M. The role of provider-patient communication and trust in online sources in internet use for health-related activities. J Health Commun. 2010;15:186–199. doi: 10.1080/10810730.2010.522691. [DOI] [PubMed] [Google Scholar]
- 7.Wen KY, Kreps G, Zhu F, Miller S. Consumers’ perceptions about and use of the internet for personal health records and health information exchange: Analysis of the 2007 Health Information National Trends Survey. J Medical Internet Res. 2010;12:e73. doi: 10.2196/jmir.1668. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Taylor H. Explosive growth of a new breed of Cyberchondriacs. Executive Summary, The Harris Poll. 1999;11 http://www.harrisinteractive.com/vault/Harris-Interactive-Poll-Research-EXPLOSIVE-GROWTH-OF-A-NEW-BREED-OF-CYBERCHONDRIACS-1999-02.pdf.
- 9.“Cyberchondria” Hits web users. BBC News. 2001 April 13; http://news.bbc.co.uk/2/hi/health/1274438.stm.
- 10.Keller GL, Padala PR, Petty F. Clinical pearls to manage cyberchondriacs. Primary Care Companion to The J Clin Psychiatry. 2008;10:75. doi: 10.4088/pcc.v10n0113e. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Smith PK, Fox AT, Davies P, Hamidi-Manesh L. Cyberchondriacs. International Journal of Adolescent Medicine and Health. 2006;18:209. [PubMed] [Google Scholar]
- 12.White RW, Horvitz E. Cyberchondria: Studies of the escalation of medical concerns in web search. ACM Transactions on Information Systems (TOIS) 2009;27:1–37. [Google Scholar]
- 13.Bessière K, Pressman S, Kiesler S, Kraut R. Effects of internet use on health and depression: A longitudinal study. J Medical Internet Res. 2010;12:e6. doi: 10.2196/jmir.1149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ahmad F, Hudak PL, Bercovitz K, Hollenberg E, Levinson W. Are physicians ready for patients with internet-based health information? J Medical Internet Res. 2006;8:e22. doi: 10.2196/jmir.8.3.e22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Houston TK, Cooper LA, Vu HT, Kahn J, Toser J, Ford DE. Screening the public for depression through the internet. Psychiatr Serv. 2001;52:362. doi: 10.1176/appi.ps.52.3.362. [DOI] [PubMed] [Google Scholar]
- 16.Van Ameringen M, Mancini C, Simpson W, Patterson B. Potential use of internet based screening for anxiety disorders: A pilot study. Depress and Anxiety. 2010;27:1006–1010. doi: 10.1002/da.20723. [DOI] [PubMed] [Google Scholar]
- 17.APA. Diagnostic and statistical manual of mental disorders. Washington DC: Author; 2000. [Google Scholar]
- 18.Adler NE, Epel E, Castallazzo G, Ickovics J. Relationship of subjective and objective social status with psychological and physiological functioning in preliminary data in healthy white women. Health Psychol. 2000;19:586–592. doi: 10.1037//0278-6133.19.6.586. [DOI] [PubMed] [Google Scholar]
- 19.Muñoz RF. The major depression episode (MDE) screener. 1998 http://www.medschool.ucsf.edu/latino/manuals.aspx.
- 20.APA. Diagnostic and statistical manual of mental disorders. Washington DC: Author; 1994. [Google Scholar]
- 21.Spitzer RL, Williams JB, Kroenke K, Linzer M, DeGruy FV, Hahn SR, Brody D, Johnson JG. Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA. 1994;272:1749–1756. [PubMed] [Google Scholar]
- 22.Munoz RF, McQuaid JR, Gonzalez GM, Dimas J, Rosales VA. Depression screening in a women’s clinic: Using automated Spanish-and English-language voice recognition. J Consult Clin Psychol. 1999;67:502–510. doi: 10.1037//0022-006x.67.4.502. [DOI] [PubMed] [Google Scholar]
- 23.Vázquez FL, Muñoz RF, Blanco V, López M. Validation of Muñoz’s mood screener in a nonclinical Spanish population. European Journal of Psychological Assessment. 2008;24:57–64. [Google Scholar]
- 24.Muñoz RF, Le HN, Clarke GN, Barrera AZ, Torres LD. Preventing first onset and recurrence of major depressive episodes. In: Gotlib IH, Hammen CL, editors. Handbook of depression. 2. New York: Guilford; 2008. pp. 533–553. [Google Scholar]
- 25.National Research Council and Institute of Medicine. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: The National Academies Press; 2009. [PubMed] [Google Scholar]
- 26.Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the internet: Randomised controlled trial. BMJ. 2004;328:265. doi: 10.1136/bmj.37945.566632.EE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med. 2003;33:217–27. doi: 10.1017/s0033291702007225. [DOI] [PubMed] [Google Scholar]
- 28.Muñoz RF. Using evidence-based internet interventions to reduce health disparities worldwide. J Med Internet Res. 2010;12:e60. doi: 10.2196/jmir.1463. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health. 1999;89:1322–1327. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Holländare F, Andersson G, Engström I. A comparison of psychometric properties between Internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J Medical Internet Res. 2010;12:e49. doi: 10.2196/jmir.1392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Spek V, Nyklícek I, Cuijpers P, Pop V. Internet administration of the Edinburgh Depression Scale. J Affect Disorders. 2008;106:301–5. doi: 10.1016/j.jad.2007.07.003. [DOI] [PubMed] [Google Scholar]
- 32.Andersson G, Kaldo-Sandström V, Ström L, Strömgren T. Internet administration of the Hospital Anxiety and Depression Scale in a sample of tinnitus patients. J Psychosom Res. 2003;55:259–62. doi: 10.1016/s0022-3999(02)00575-5. [DOI] [PubMed] [Google Scholar]
- 33.Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Medical Internet Res. 2004;6:e29. doi: 10.2196/jmir.6.3.e29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Vallejo MA, Jordán CM, Díaz MI, Comeche MI, Ortega J. Psychological assessment via the internet: a reliability and validity study of online (vs paper-and-pencil) versions of the General Health Questionnaire-28 (GHQ-28) and the Symptoms CheckList-90-Revised (SCL-90-R) J Medical Internet Res. 2007;9:e2. doi: 10.2196/jmir.9.1.e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Coles ME, Cook LM, Blake TR. Assessing obsessive compulsive symptoms and cognitions on the internet: evidence for the comparability of paper and Internet administration. Behavior Res Ther. 2007;45:2232–40. doi: 10.1016/j.brat.2006.12.009. [DOI] [PubMed] [Google Scholar]
- 36.Basnov M, Kongsved SM, Bech P, Hjollund NH. Reliability of short form-36 in an Internet- and a pen-and-paper version. Informatics for Health and Social Care. 2009;34:53–8. doi: 10.1080/17538150902779527. [DOI] [PubMed] [Google Scholar]
- 37.Eaton DK, Brener ND, Kann L, Denniston MM, McManus T, Kyle TM, Roberts AM, Flint KH, Ross JG. Comparison of paper-and-pencil versus Web administration of the Youth Risk Behavior Survey (YRBS): risk behavior prevalence estimates. Evaluation Rev. 2010;34:137–53. doi: 10.1177/0193841X10362491. [DOI] [PubMed] [Google Scholar]
- 38.van de Looij-Jansen PM, de Wilde EJ. Comparison of Web-Based versus Paper-and-Pencil Self-Administered Questionnaire: Effects on Health Indicators in Dutch Adolescents. Health Serv Res Epub. 2008 doi: 10.1111/j.1475-6773.2008.00860.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Howell RT, Rodzon KS, Kurai M, Sanchez AH. A validation of well-being and happiness surveys for administration via the Internet. Behav Res Meth. 2010;42:775–84. doi: 10.3758/BRM.42.3.775. [DOI] [PubMed] [Google Scholar]
- 40.Fortson BL, Scotti JR, Del Ben KS, Chen YC. Reliability and validity of an Internet traumatic stress survey with a college student sample. Journal Trauma Stress. 2006;19:709–20. doi: 10.1002/jts.20165. [DOI] [PubMed] [Google Scholar]