Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 1.
Published in final edited form as: J Psychiatr Res. 2022 Nov 25;157:112–118. doi: 10.1016/j.jpsychires.2022.11.024

Digitally Filling the Access Gap in Mental Health Care: An Investigation of the Association between Rurality and Online Engagement with Validated Self-report Screens across the United States

Elad Yom-Tov 1,2, Damien Lekkas 3,4, Michael V Heinz 3,5, Theresa Nguyen 6, Paul J Barr 3,7, Nicholas C Jacobson 3,4,5,8
PMCID: PMC9898139  NIHMSID: NIHMS1854509  PMID: 36462251

Abstract

Mental health disorders are highly prevalent, yet few persons receive access to treatment; this is compounded in rural areas where mental health services are limited. The proliferation of online mental health screening tools are considered a key strategy to increase identification, diagnosis, and treatment of mental illness. However, research on real-world effectiveness, especially in hard to reach rural communities, is limited. Accordingly, the current work seeks to test the hypothesis that online screening use is greater in rural communities with limited mental health resources. The study utilized a national, online, population-based cohort consisting of Microsoft Bing search engine users across 18 months in the United States (representing approximately one-third of all internet searches), in conjunction with user-matched data of completed online mental health screens for anxiety, bipolar, depression, and psychosis (N=4354) through Mental Health America, a leading non-profit mental health organization in the United States. Rank regression modeling was leveraged to characterize U.S. county-level screen completion rates as a function of rurality, health-care availability, and sociodemographic variables. County-level rurality and mental health care availability alone explained 42% of the variance in MHA screen completion rate (R2=0.42, p<5.0*10−6). The results suggested that online screening was more prominent in underserved rural communities, therefore presenting as important tools with which to bridge mental health-care gaps in rural, resource-deficient areas.

Keywords: mental health, online screen, health-care access, health-care equity, digital health, internet search behavior

1. Introduction

Mental health disorders are highly prevalent in the United States, with estimated 12 month prevalences of 21% and 5.6% for any mental illness and a serious mental illness, respectively (NIMH, 2022). Moreover, mental disorders such as depression and anxiety, are among the twenty most common causes of disability worldwide (Global Burden of Disease Collaborative Network, 2020) and have been estimated to cost more than $200 billion annually in the United States (Roehrig, 2016). Despite these numbers, less than half of individuals with serious mental disorder receive stable treatment, in part driven by lack of insight1* and inaccessibility of services (Kessler et al., 2001). Consequently, there is an appreciable national treatment gap2†, with 271 mental health workers (including 10.5 psychiatrists) per 100,000 persons in the U.S. (World Health Organization, 2017). Individuals who live in rural locations across the U.S. are particularly impacted, as these regions are 4.7 times more likely to have mental health treatment gaps2 compared with more urbanized areas (Merwin et al., 2003). Factors including underdeveloped infrastructure, poverty, and higher rates of substance use (Thomas et al., 2012) contribute to both treatment challenges and observed gaps within these rural settings. Primary care providers (PCPs) could help fill the higher necessity for mental health treatment in rural America; however, there is a growing shortage of PCPs in these areas (Nielsen et al., 2017) alongside an estimated 66–98% misdiagnosis rate for common mental disorders in primary care (Vermani et al., 2011). For further statistics on the state of mental health-care provider shortages, interested readers are encouraged to peruse the data warehouse of the Health Resources & Services Administration (https://data.hrsa.gov). This invaluable resource provides a frequently updated and detailed collection of geographically contextual information on mental health-care resources across the U.S..

Rapid advances in the emergent field of digital health have presented the potential for the translation of technological innovation into timely and affordable options for information acquisition, monitoring, intervention, and treatment (Bell et al., 2017; Bush et al., 2014; Kenny et al., 2016; Kim et al., 2016; Kushner and Sharma, 2020; Sheikh et al., 2021; Yim et al., 2020). Among its manifold tools, digital health has seen the implementation of online mental health screening resources (hereafter, “screens”) to improve awareness and detection of mental illness. Screens therefore afford the possibility of a direct link to further mental health-care services. This mechanism may be particularly invaluable in rural communities – locales that despite mental illness prevalence comparable to more metropolitan areas, (Morales et al., 2020) are wholly characterized by a critical lack of mental health services (Andrilla et al., 2018). It is estimated that as many as 65% of nonmetropolitan counties do not have psychiatrists, (Andrilla et al., 2018) and over 60% of rural Americans live in designated mental health provider shortage areas (Health Resources & Services Administration, 2022). Compounding this issue, rural cultural factors have been shown to impact help seeking behaviors in remote and rural areas. Beliefs regarding work, entitlement, and mental health treatment in rural communities impact perceptions of treatment “deservingness” and present barriers to care (Snell-Rood and Carpenter-Song, 2018). Further, life in rural areas may necessitate a higher level of stoicism and selfreliance, which may, in effect, raise the threshold for help-seeking to “dire necessity”, limiting the potential for intervention during early stages of psychiatric illness (Fuller et al., 2000). In addition, qualitative research has revealed that rural gossip networks and the increased social visibility within rural communities may compound worries about mental health stigma and subsequent social alienation, therefore limiting assessment and treatment (Aisbett et al., 2007). Screens may serve to address both geographic inaccessibility and privacy concerns surrounding mental health diagnosis and treatment in rural regions of the country.

Uptake of mental health resources first depends on a recognition of experiencing mental illness. To this end, screens allow individuals to answer questions about symptoms they are experiencing and provide preliminary diagnoses. Framed within the Transitional Care Model (Hirschman et al., 2015), online screens not only present opportunities for risk assessment and informed awareness, but may promote timely movement into a treatment setting. In this light, screens can be thought of as far-reaching digital resources that may instigate care-seeking behavior and increase receptivity to other digital and in-person treatment options.

Research has suggested both receptivity to, and utility of, digital screens. One study found that linking decision support of PHQ-9-defined depression outcomes to digitallyadministered screens was viewed positively by clinically depressed individuals and healthy controls, as well as by clinicians (Dannenberg et al., 2019). Another work explored online completion of the Pediatric Symptom Checklist-Youth Form (PSC-Y) and found that many adolescents use the internet to learn about mental health – a very high percentage of which might be at risk, and a majority of those at-risk reporting plans to seek help as a consequence of their results (Murphy et al., 2018). This popularity and reach is complemented by evidence of efficacy, with implementations of validated screens for psychosis showing moderate sensitivity and specificity in the detection of clinically high-risk individuals (McDonald et al., 2019).

Because of the unregulated nature of the internet at large, there are many websites that advertise and host mental health self-administered questionnaires and corpuses of information on symptomatology, risk factors, and therapies that are not professionally curated. The inclusion of false or misleading information thereby represents considerable risk to potentially vulnerable individuals. Unlike such websites, Mental Health America (MHA; https://mhanational.org/), the leading non-profit mental health organization in the United States, provides a collection of free and clinically validated mental health screening tools that span a variety of mental disorders. These screens are well-utilized, approximating 3,000 uses per day and totaling around one million completed screens each year (Mental Health America, 2020). Given the functional and temporal importance of online screens as one preliminary source for information-seeking behavior, screens represent a potential key step in the broader digital answer to mental health-care access inequity. Accordingly, further investigation into screen use patterns across the country is warranted. The reputation and quality of the resources provided by MHA makes analysis of their website’s traffic and use patterns especially valuable in pursuit of this goal.

Despite being potentially important initial digital resources, little is known about the sociodemographics and environmental factors of those who utilize screens unprompted. One older, open participation study reported age, gender, and ethnicity of individuals (N=24,479) who completed an internet-based screen of the Centers for Epidemiological Studies Depression (CESD) scale (Houston et al., 2001). To the authors’ knowledge, no other studies conducted on the general population exist with reported demographic information. Additionally, most studies on mental health screens are characterized by targeted recruitment to test the digital efficacy of a screen or to use a screen to characterize the clinical profile of a population of interest (Donker et al., 2011; Farvolden et al., 2003; McDonald et al., 2019).

The current research aimed to leverage sociodemographic information and MHA screen completion data collected for users across an 18-month time span in conjunction with their respective Microsoft Bing search activity to explore associations among rurality, mental health-care availability, county-level sociodemographics and patterns of online mental health screen use. This research also sought to provide a summary of MHA user-specific sociodemographic characteristics across utilization of specific screens. From this, the study tested the following hypotheses: county-level MHA screen use rates will be (i) proportionately higher among those living in areas that are more rural, and (ii) proportionately higher among those living in areas that have more limited access to traditional in-person treatment options.

2. Materials and Methods

2.1. Study Cohort and Data Set

The current study utilized data from Microsoft Bing and Mental Health America (MHA). The market share of Bing in the U.S. is approximately 37% (Microsoft, 2022). Past research has found strong correspondence between U.S. population statistics and those of Bing users (Yom-Tov, 2017) and between Bing and Google users (Rosenblum and Yom-Tov, 2017).

Queries submitted to the Bing search engine from users in the United States during 18 months beginning on January 1, 2020 were first extracted from Bing’s records. For each query, information on (i) date and time, (ii) text of the query, (iii) anonymized user identification, (iv) pages clicked by the user, and (v) the county from which the user submitted their query was obtained. Second, records of all anxiety, bipolar, depression, and psychosis screens completed by individuals on the MHA website who were directly referred via Bing during the same 18-month time frame were extracted from MHA’s records. Each screen is free, confidential, anonymous, and based on one of the following scientifically and psychometrically validated questionnaires: GAD-7 (Spitzer et al., 2006), MDQ (Hirschfeld, 2002), PHQ-9 (Kroenke et al., 2001), and PQ-B (Loewy et al., 2011). Relevant to the current study, each MHA record consisted of (i) screen type, (ii) date and start time of the screen, (iii) user gender, (iv) user race, (v) user household income, and (vi) indications of current and past mental health diagnosis and treatment.

MHA data were then linked to Bing user identifiers by finding a user who clicked on the same webpage as that of the screen within five minutes of the start time indicated by MHA. From these users, the time of each Bing query relative to the time of the MHA screen was computed, and data was discarded if more than one access from Bing was found or if multiple screens from the same topic were started on MHA within the same five-minute time window (this was to ensure that we could uniquely identify each person). Primary outcomes and predictors (see 2.2) were derived at the county level using user location information available from Bing.

This study was approved by the Institutional Review Board at Dartmouth College (Committee for the Protection of Human Subjects: STUDY00032145).

2.2. Measures

2.2.1. County-Level Variables for Modeling

Percentage of Persons Accessing MHA.

The primary outcome measure of the study was the number of users from each county who completed an online mental health screen through MHA, divided by the total number of Bing users in that county.

Health-care Availability.

Health-care availability per county was taken from information provided by the Health Resources & Services Administration (Health Resources & Services Administration, 2022) and presented through the Rural Health Information Hub (Rural Health Information Hub, 2021). Data consists of county-level mental health shortage classifications updated for 2021: “None of county is a shortage area”, “Part of county is a shortage area”, and “Whole county is a shortage area”.

Rurality.

Rurality designations for each county were classified using the 2013 CDC NCHS Urban-Rural Classification Scheme for Counties (Rothwell et al., 2014). Categories in order of increasing rurality included: (i) large central metro, (ii) large fringe metro, (iii) medium metro, (iv) small metro, (v) micropolitan, and (vi) non-core.

Median Household Income.

Per-county median household income estimates were based on the 2020 edition of the Small Area Income and Poverty Estimates Program (SAIPE) conducted by the United States Census Bureau (United States Census Bureau, 2021).

Race.

Race was operationalized as the percentage of people in each county who identified as White and those who identified as Black according to the United States Census Bureau (United States Census Bureau, 2021).

Religiosity.

Religiosity estimates reflected per-county values reported in the 2010 U.S. Religion Census Religious Congregations and Membership Study (Grammich et al., 2019).

2.2.2. MHA User-Level Sociodemographic Variables

Self-report sociodemographic attributes on 11,564 users who completed a screen through MHA included: (i) gender, (ii) race/ethnicity, (iii) household income, (iv) ever diagnosis of a mental health disorder, (v) ever treatment for a mental disorder, and (vi) current treatment for a mental disorder. A “Not provided” category for each attribute was used to manage missing data. These variables were used for descriptive statistical reporting and not for modeling purposes.

2.3. Planned Analysis

Associations of county-level mental health-care availability categories with differences in the average percentage of Bing users who completed an MHA screen were first analyzed using the Kruskal-Wallis nonparametric one-way analysis of variance test. Similarly, associations of rurality categories with differences in the average percentage of Bing users who completed an MHA screen were also analyzed via Kruskal-Wallis. To ascertain the relationship between county-level mental health-care availability and rurality, the Spearman correlation was calculated. Following this, two primary rank regression models were employed to predict the percentage of Bing users per county who completed an MHA screen. The primary outcome data was determined, via the Lilliefors test, to not be normally distributed (p<0.001), thus rank regression was selected (Chen et al., 2014). The first model included rurality and mental health-care availability as independent variables. The second model was a robustness check of the first model and added county-level sociodemographic information as independent variables. Ancillary models were tested for the significance of interaction between rurality and availability. Additional ancillary models also checked the comparability of predictive performance when the rate of county-level MHA screen completions was quantified on a per screen type basis.

Lastly, MHA self-report sociodemographic and clinical attributes were presented as raw counts and percentage of users with each associated attribute value. These results are stratified for each of anxiety, bipolar, depression, and psychosis screens. 𝝌2 tests with Bonferroni correction were performed to quantify significant differences among the values of each sociodemographic attribute.

3. Results

N=4354 MHA users were matched to their Bing user identifiers. The most commonly completed screens were depression (n=2070), anxiety (n=204), bipolar (n=152), and psychosis (n=100). There were 605 counties with at least one access to MHA. Figure 2 illustrates county distribution along with respective relative proportions of screen completions by screen type. County representation in the data covers the full geographic range of the U.S. and includes counties across the rural-urban spectrum. Among those that include more rural areas of the country include (i) the northeastern corner of Minnesota (Lake county), (ii) northeastern Arizona tribal lands (Navajo and Apache counties), (iii) eastern Maine (Washington county), and (iv) several pockets of the Midwest. The average percentage of Bing users who completed an MHA screen in counties labeled as having no shortage was 3.1*10−5%, compared to 2.6*10−5% in counties with partial shortage, and 6.7*10−5% in counties which are entirely a shortage area. The differences were statistically significant for shortage areas (Kruskal-Wallis, p<10−10) and for rurality (Kruskal-Wallis, p<10−10).

Figure 2.

Figure 2.

Open in a new tab

Per County Completed MHA Screens across the United States

Note. Each map panel illustrates the percentage of users per county who completed the associated online mental health screen for (A) anxiety, (B) bipolar disorder, (C) psychosis, and (D) depression. Darker shades of red indicate higher percentages of people within the county who completed the screen out of the total population of Bing users. Data for all screen types consists of several rural areas of the country including (i) northeastern Minnesota, (ii) northeastern Arizona, (iii) eastern Maine, and (iv) several pockets of the Midwest. However, counties across the full extent of the rural-urban spectrum and all corners of the country are represented. n = 605 counties.

Increased county rurality and decreased availability predicted higher MHA screen uses. As shown in Table 1, the first regression model explained 42% of the variance in MHA screen completion rate (R2=0.42), with both rurality (p<1.0*10−10) and mental health-care availability (p<5.0*10−6) statistically significantly correlated with screen use rates. Figure 1 shows the average percentage of MHA screen completions for each combination of rurality and availability category, with non-core and whole county shortage associated with the highest rate of MHA screen uses. Moreover, differences in screen completion rates as a function of county-level health-care availability are less pronounced among less rural counties, illustrating that a lack of mental health-care resources may have greater implications for online screen use in more rural areas of the country. The Spearman correlation between rurality and health-care availability was found to be 0.47 (p<10−10).

Table 1.

Outcome Results of Regression Models

Model 1 (R2 = 0.42; P<10−10)
Variable Coefficient (β) 95% CI P-value
Healthcare availability 36.5 [21.0, 52.1] 5.0 × 10−6
Rurality 62.8 [55.3, 70.4] 1.0 × 10−10
Model 2 (R2 = 0.44; P<1Q−10)
Variable Coefficient (β) 95% CI P-value
Healthcare availability 33.1 [17.4, 48.8] 4.0 × 10−5
Rurality 56.5 [48.4, 64.7] 1.0 × 10−10
Median income (USD) −0.0006 [0.0013, 0.0001] NS
Race - White (%) 1.59 [−0.56, 2.62] NS
Race - Black (%) 0.59 [−0.73, 1.90] NS
Religiosity 0.03 [−0.05, 0.11] NS
Open in a new tab

Note. Model 1 utilized health-care availability and rurality as predictors of all MHA screen completions per county. Model 2 additionally included county-level sociodemographic information to predict total MHA screen completions. All sociodemographic variables were non-significant (NS). n = 605 counties.

Figure 1.

Figure 1.

Open in a new tab

Average Rate of Access from Bing to MHA Screens across Counties as a Function of County-Specific Health-care Availability and Rurality Classification

Note. The y-axis represents the average percentage of Bing users who completed a mental health screen on MHA. Averages were calculated across all counties that belonged to a respective mental health-care availability (x-axis) and rurality (z-axis) stratum. Lowest availability is designated by “Whole”, while highest availability is designated by “None”. Lowest rurality (i.e., highest urbanicity) is designated by “Large central metro”, while highest rurality (i.e., lowest urbanicity) is designated by “Non-core”. Data shows a general trend toward higher average screen completions across counties with lower availability and higher rurality. Impact of county mental health-care availability on screen completion rates is more pronounced in more rural counties compared with more urban counties, with micropolitan (orange bars) and non-core (yellow bars) counties exhibiting the greatest differences among None/Part/Whole county availability stratifications.

The second regression model, which additionally incorporated county-level sociodemographics, explained 44% of the variance (R2=0.44); however, no county-level sociodemographic variables were significant. Similar results were obtained for the above models when MHA screen completion rates for each specific screen type were analyzed separately. Moreover, models that included the interaction term of rurality and availability did not reach statistically significantly higher R2.

As expressed in Table 2, statistical analyses on MHA sociodemographic data across 11,564 users (including the 4,354 users with matched Bing user identifier data) reflected values for gender, ever diagnosed with a mental disorder, ever received treatment for a mental disorder, and currently receiving treatment for a mental disorder as significantly different across screen types (p<0.05 with Bonferroni correction).

Table 2.

Sociodemographic Profile of Users Completing MHA Screens

Sociodemographic and Clinical Factors Count (%) of MHA users by Screen Type
Attribute Value Anxiety Bipolar Depression Psychosis Total
Gender* Female 681 (71) 594 (64) 5348 (58) 196 (47) 6819 (59)
Male 223 (23) 262 (28) 3346 (36) 173 (42) 4004 (35)
Other 30 (3) 37 (4) 411 (4) 27 (7) 505 (4)
Not provided 30 (3) 30 (3) 159 (2) 17 (4) 236 (2)
Race/Ethnicity White (non-Hispanic) 461 (48) 495 (54) 3904 (42) 213 (52) 5073 (44)
Asian/Pacific Islander 75 (8) 60 (7) 1000 (11) 21 (5) 1156 (10)
Hispanic or Latino 81 (8) 75 (8) 859 (9) 33 (8) 1048 (9)
Asian 94 (8) 52 (6) 824 (9) 14 (3) 984 (9)
Black or African American (non-Hispanic) 51 (5) 46 (5) 672 (7) 30 (7) 799 (7)
Other 126 (13) 127 (14) 1372 (15) 61 (15) 1686 (14)
Not provided 76 (8) 68 (7) 633 (7) 41 (10) 818 (7)
Household Income (USD) Less than $20,000 176 (18) 209 (23) 2140 (23) 83 (20) 2608 (23)
$20,000 - $39,999 108 (11) 132 (14) 1156 (12) 73 (18) 1469 (13)
$40,000 - $59,999 87 (9) 118 (13) 860 (9) 35 (8) 1100 (10)
$60,000 - $79,999 56 (6) 76 (8) 645 (7) 21 (5) 798 (7)
$80,000 - $99,999 53 (5) 58 (6) 476 (5) 20 (5) 607 (5)
$100,000 - $149,999 69 (7) 53 (6) 609 (7) 20 (5) 751 (6)
$150,000+ 59 (6) 47 (5) 608 (7) 27 (7) 741 (6)
Not provided 356 (37) 230 (25) 2770 (30) 134 (32) 3490 (30)
Ever diagnosed with a mental disorder* Yes 183 (19) 424 (46) 2123 (23) 184 (45) 2914 (25)
No 732 (76) 466 (5) 6809 (73) 209 (51) 8216 (71)
Not provided 49 (5) 33 (4) 332 (4) 20 (5) 434 (4)
Ever received treatment for a mental disorder* Yes 222 (23) 432 (47) 2346 (25) 188 (46) 3188 (28)
No 708 (73) 460 (5) 6602 (71) 206 (5) 7976 (69)
Not provided 34 (4) 31 (3) 316 (3) 19 (5) 400 (3)
Currently receiving treatment for a mental disorder* Yes 95 (1) 248 (27) 1041 (11) 101 (24) 1485 (13)
No 120 (12) 169 (18) 1237 (13) 78 (19) 1604 (14)
Not provided 749 (78) 506 (55) 6986 (75) 234 (57) 8475 (73)
Open in a new tab

Note.

Statistically significant (P<0.05 with Bonferroni correction) differences are denoted by an asterisk (*); statistical significance was determined via 𝝌2 test, excluding the “Not provided” category.

N=11,564 users.

4. Discussion

The current work leveraged Bing search activity data, in combination with online mental health screen completion and user demographic information available through MHA, to interrogate the associations among rurality, mental health-care availability, and online screen use from a large, nationally representative cohort of internet users across the United States. In addition, this study reported the sociodemographics of online screen use. Supporting this study’s hypothesis that county-level MHA screen use rates will be higher among those living in more rural areas as well as in areas with more limited access to in-person mental health treatment options, modeling results showed significant positive associations between the rate of MHA screen completions and both county-level rurality (β=62.8, P<1.0*10−10) and availability (β=36.5, P<5.0*10−6). These associations strongly suggested that screens, being more frequently utilized in these areas, serve as an especially important tool for individuals who live in remote areas and/or have limited access opportunities to seek information and help regarding their mental health.

MHA sociodemographic results indicated a broad uptake of online screens across multiple racial-ethnic groups, with historically marginalized populations making up nearly half of online screening survey use (see Table 2). Given the disparate access to mental health services across racial-ethnic groups, with historically marginalized groups having lower access (Cook et al., 2017), online platforms such as MHA may be an important initial step in reaching persons who would otherwise not have traditional resources. In addition, higher utilization by women across all analyzed screens was found. This is consistent with evidence that depression and anxiety are more common in women than men (Eaton et al., 2012) and broadly speaks to more prominent mental health-related help-seeking behaviors in women (Mackenzie et al., 2006). However, the results were inconsistent with evidence for a nearly-even gender distribution in bipolar disorder (Diflorio and Jones, 2010).

Relating to the recognized mental health treatment gaps in rural areas (Merwin et al., 2003), strong associations between online screen use and both county-level rurality and mental health care availability were found (Figure 1). The models implicated areas that are both rural and have low mental health care availability as showing the strongest association with online screen use. In contrast, metropolitan areas with low mental health resource shortage showed the lowest utilization of online screen access. This finding supports the hypothesis that online tools, like those provided by MHA, have an important role in bolstering mental health-care access in low resource areas. Furthermore, a majority of individuals completing MHA screens did not yet have a psychiatric diagnosis (Table 2), suggesting an increased importance of these online tools as resources of information in the early course of illness.

Though our work shows rural use of online screens, it is important to consider mental health assessment and treatment disparities are multifactorial in etiology, and arise not only from resource shortages, but also from rural cultural factors. Therefore, a multifaceted approach is warranted; any intervention which is to be optimally effective must address both problems of availability and those rural cultural factors, which impact diagnosis and treatment of mental disorders. Qualitative research has demonstrated stigma (though not unique to rural areas) and reduced confidentiality (Aisbett et al., 2007) as concerns of rural community dwellers (Nicholson, 2008). Fuller et al. suggest an increased stoic, self-reliant approach to problems in rural communities, which may limit early help-seeking behaviors (Fuller et al., 2000). Hence, provision of resources is required, but not sufficient to fully address disparities between urban and rural communities. We suggest that campaigns aimed at addressing stigma and other cultural barriers to effective mental health assessment in tandem with resource provision will be necessary to optimize uptake of mental health resources.

The current work represents the first large-scale, naturalistic, and demographically untargeted3‡ study to identify and characterize person- and region-specific factors associated with online mental health screening tool utilization in the United States. The work was strengthened by the use of a large, temporally-anchored dataset derived in tandem by both Bing and MHA. These data sources allowed for unique framing and interrogation of individual online mental health screen use. Though this study has important strengths, it has limitations which motivate the need for further research. First, while the work explored and quantified the importance of factors associated with online screen use, it did not establish causal associations between these factors and online screen use nor address the question of impact nature or magnitude on the populations of interest. Second, though the sample size was large and naturalistic, there were relatively fewer samples for persons having completed anxiety, bipolar, and psychosis screens compared to depression. Third, relating to county-level sociodemographics, only racial data on White and Black percentages were used. While there are certainly more nuanced mental health-related differences by race and ethnicity that warrant careful consideration, these differences unfortunately could not be modeled based on the county-level data available. Fourth, this work unfortunately could not incorporate county-level age in the regression models nor report on the age sociodemographics of MHA users due to a lack of data availability. Last, this research operated under the implicit hypothesis that low availability is an effective proxy for individual online screen use rationale. Further research is needed to characterize the direct motives for individuals in their utilization of these tools.

Through statistical support of the hypothesized positive association between MHA screen use and under-resourced areas, this research suggested that online screens fill an important gap in the heterogeneous mental health access infrastructure of the country. Thus, it is the task for future endeavors to more precisely define the potential benefits and effects that such tools may have on the population at large. Such fruitful research avenues may include (i) modeling the trajectories of how online search behavior changes as a result of completing and receiving feedback from a screen, (ii) reporting how search trajectories relate to subsequent in-person, professional help-seeking, and (iii) comparing search trajectories across regions of differential rurality and mental health-care availability. In addition, studies that aim to further characterize the clinical and sociodemographic profiles of individuals who access online screens, and thus more fully contextualize the aforementioned behavioral trajectories, would greatly enhance understanding of this digital resource and allow for further targeted deployment and refinement.

Acknowledgments

This work was supported by an institutional grant from the National Institute on Drug Abuse [grant number NIDA-5P30DA02992610].

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

The authors declare no conflict of interest.

1*

Lack of perceived need for treatment, represented here as ‘insight’, was shown by Kessler et al. (2001) to be a highly prevalent reason for not seeking treatment (55% among nonpatients with 12-month serious mental illness).

2†

‘Treatment gap’ signifies shortage areas, defined by Merwin et al. (2003) to include those areas with a population to mental health professional ratio of 6,000:1.

3‡

The study was demographically untargeted in that data was selected naturalistically as a population of all those using the Bing Search data.

References

  1. Aisbett DL, Boyd CP, Francis KJ, Newnham K, 2007. Understanding barriers to mental health service utilization for adolescents in rural Australia. Rural and Remote Health. [PubMed] [Google Scholar]
  2. Andrilla CHA, Patterson DG, Garberson LA, Coulthard C, Larson EH, 2018. Geographic Variation in the Supply of Selected Behavioral Health Providers. American Journal of Preventive Medicine, The Behavioral Health Workforce: Planning, Practice, and Preparation 54, S199–S207. 10.1016/j.amepre.2018.01.004 [DOI] [PubMed] [Google Scholar]
  3. Bell IH, Lim MH, Rossell SL, Thomas N, 2017. Ecological momentary assessment and intervention in the treatment of psychotic disorders: A systematic review. PS 68, 1172–1181. 10.1176/appi.ps.201600523 [DOI] [PubMed] [Google Scholar]
  4. Bush NE, Ouellette G, Kinn J, 2014. Utility of the T2 mood tracker mobile application among army warrior transition unit service members. Military Medicine 179, 1453–1457. 10.7205/MILMED-D-14-00271 [DOI] [PubMed] [Google Scholar]
  5. Chen T, Tang W, Lu Y, Tu X, 2014. Rank regression: An alternative regression approach for data with outliers. Shanghai Arch Psychiatry 26, 310–315. 10.11919/j.issn.1002-0829.214148 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cook BL, Trinh N-H, Li Z, Hou SS-Y, Progovac AM, 2017. Trends in Racial-Ethnic Disparities in Access to Mental Health Care, 2004–2012. PS 68, 9–16. 10.1176/appi.ps.201500453 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dannenberg MD, Bienvenida JCM, Bruce ML, Nguyen T, Hinn M, Matthews J, Bartels SJ, Elwyn G, Barr PJ, 2019. End-user views of an electronic encounter decision aid linked to routine depression screening. Patient Education and Counseling 102, 555–563. 10.1016/j.pec.2018.10.002 [DOI] [PubMed] [Google Scholar]
  8. Diflorio A, Jones I, 2010. Is sex important? Gender differences in bipolar disorder. International Review of Psychiatry 22, 437–452. 10.3109/09540261.2010.514601 [DOI] [PubMed] [Google Scholar]
  9. Donker T, van Straten A, Marks I, Cuijpers P, 2011. Quick and easy self-rating of Generalized Anxiety Disorder: Validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Research 188, 58–64. 10.1016/j.psychres.2011.01.016 [DOI] [PubMed] [Google Scholar]
  10. Eaton NR, Keyes KM, Krueger RF, Balsis S, Skodol AE, Markon KE, Grant BF, Hasin DS, 2012. An invariant dimensional liability model of gender differences in mental disorder prevalence: evidence from a national sample. J Abnorm Psychol 121, 282–288. 10.1037/a0024780 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Farvolden P, McBride C, Bagby RM, Ravitz P, 2003. A web-based screening instrument for depression and anxiety disorders in primary care. Journal of Medical Internet Research 5, e896. 10.2196/jmir.5.3.e23 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Fuller J, Edwards J, Procter N, Moss J, 2000. How Definition of Mental Health Problems Can Influence Help Seeking in Rural and Remote Communities. Australian Journal of Rural Health 8, 148–153. 10.1046/j.1440-1584.2000.00303.x [DOI] [PubMed] [Google Scholar]
  13. Global Burden of Disease Collaborative Network, 2020. Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 396, 1204–1222. 10.1016/S0140-6736(20)30925-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Grammich C, Hadaway K, Houseal R, Jones D, Krindatch A, Stanley R, Taylor RH, 2019. U.S. Religion Census Religious Congregations and Membership Study, 2010 (Metro Area File) 10.17605/OSF.IO/9AMDJ [DOI] [Google Scholar]
  15. Health Resources & Services Administration, 2022. Data Downloads [WWW Document]. Data Downloads. URL https://data.hrsa.gov/data/download#SHORT (accessed 2.5.22).
  16. Hirschfeld RMA, 2002. The Mood Disorder Questionnaire: A Simple, Patient-Rated Screening Instrument for Bipolar Disorder. Prim Care Companion J Clin Psychiatry 4, 9–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hirschman K, Shaid E, McCauley K, Pauly M, Naylor M, 2015. Continuity of care: The transitional care model. The Online Journal of Issues in Nursing 20. 10.3912/OJIN.Vol20No03Man01 [DOI] [PubMed] [Google Scholar]
  18. Houston TK, Cooper LA, Vu HT, Kahn J, Toser J, Ford DE, 2001. Screening the public for depression through the internet. PS 52, 362–367. 10.1176/appi.ps.52.3.362 [DOI] [PubMed] [Google Scholar]
  19. Kenny R, Dooley B, Fitzgerald A, 2016. Ecological momentary assessment of adolescent problems, coping efficacy, and mood states using a mobile phone app: An exploratory study. JMIR Mental Health 3, e6361. 10.2196/mental.6361 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kessler RC, Berglund PA, Bruce ML, Koch JR, Laska EM, Leaf PJ, Manderscheid RW, Rosenheck RA, Walters EE, Wang PS, 2001. The prevalence and correlates of untreated serious mental illness. Health Serv Res 36, 987–1007. [PMC free article] [PubMed] [Google Scholar]
  21. Kim J, Lim S, Min YH, Shin Y-W, Lee B, Sohn G, Jung KH, Lee J-H, Son BH, Ahn SH, Shin S-Y, Lee JW, 2016. Depression screening using daily mental-health ratings from a smartphone application for breast cancer patients. Journal of Medical Internet Research 18, e5598. 10.2196/jmir.5598 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kroenke K, Spitzer RL, Williams JBW, 2001. The PHQ-9. J Gen Intern Med 16, 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kushner T, Sharma A, 2020. Bursts of activity: Temporal patterns of help-seeking and support in online mental health forums, in: Proceedings of The Web Conference 2020. ACM, Taipei Taiwan, pp. 2906–2912. 10.1145/3366423.3380056 [DOI] [Google Scholar]
  24. Loewy RL, Pearson R, Vinogradov S, Bearden CE, Cannon TD, 2011. Psychosis Risk Screening with the Prodromal Questionnaire – Brief version (PQ-B). Schizophr Res 129, 42–46. 10.1016/j.schres.2011.03.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Mackenzie CS, Gekoski WL, Knox VJ, 2006. Age, gender, and the underutilization of mental health services: The influence of help-seeking attitudes. Aging & Mental Health 10, 574–582. 10.1080/13607860600641200 [DOI] [PubMed] [Google Scholar]
  26. McDonald M, Christoforidou E, Van Rijsbergen N, Gajwani R, Gross J, Gumley AI, Lawrie SM, Schwannauer M, Schultze-Lutter F, Uhlhaas PJ, 2019. Using online screening in the general population to detect participants at clinical high-risk for psychosis. Schizophrenia Bulletin 45, 600–609. 10.1093/schbul/sby069 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mental Health America, 2020. About MHA Screening [WWW Document]. About MHA Screening. URL https://screening.mhanational.org/about-mha-screening/ (accessed 1.4.22). [Google Scholar]
  28. Merwin E, Hinton I, Dembling B, Stern S, 2003. Shortages of rural mental health professionals. Archives of Psychiatric Nursing 17, 42–51. 10.1053/apnu.2003.1 [DOI] [PubMed] [Google Scholar]
  29. Microsoft, 2022. Microsoft Search Network Data [WWW Document]. Microsoft Advertising. URL https://about.ads.microsoft.com/en-us/insights/planning-tools/microsoft-search-network-https://www.zotero.org/google-docs/?eMVHmQdata (accessed 2.12.22).
  30. Morales DA, Barksdale CL, Beckel-Mitchener AC, 2020. A call to action to address rural mental health disparities. J Clin Transl Sci 4, 463–467. 10.1017/cts.2020.42 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Murphy JM, Nguyen T, Lucke C, Chiang C, Plasencia N, Jellinek M, 2018. Adolescent self-screening for mental health problems: Demonstration of an internet-based approach. Acad Pediatr 18, 59–65. 10.1016/j.acap.2017.08.013 [DOI] [PubMed] [Google Scholar]
  32. Nicholson LA, 2008. Rural mental health. Advances in Psychiatric Treatment 14, 302–311. 10.1192/apt.bp.107.005009 [DOI] [Google Scholar]
  33. Nielsen M, D’Agostino D, Gregory P, 2017. Addressing Rural Health Challenges Head On. Mo Med 114, 363–366. [PMC free article] [PubMed] [Google Scholar]
  34. NIMH, 2022. Mental Illness [WWW Document]. National Institute of Mental Health (NIMH). URL https://www.nimh.nih.gov/health/statistics/mental-illness (accessed 3.6.22). [Google Scholar]
  35. Roehrig C, 2016. Mental Disorders Top The List Of The Most Costly Conditions In The United States: $201 Billion. Health Affairs 35, 1130–1135. 10.1377/hlthaff.2015.1659 [DOI] [PubMed] [Google Scholar]
  36. Rosenblum S, Yom-Tov E, 2017. Seeking web-based information about attention deficit hyperactivity disorder: Where, what, and when. Journal of Medical Internet Research 19, e6579. 10.2196/jmir.6579 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rothwell CJ, Madans JH, Arispe IE, 2014. 2013 NCHS Urban-Rural Classification Scheme for Counties (No. Series 2, Number 166), Vital and Health Statistics U.S. Department of Health and Human Services, Hyattsville, MD. [Google Scholar]
  38. Rural Health Information Hub, 2021. Map of Health Professional Shortage Areas: Mental Health, by County, 2022 - Rural Health Information Hub [WWW Document]. RHIhub Rural Health Information Hub. URL https://www.ruralhealthinfo.org/charts/7 (accessed 2.12.22). [Google Scholar]
  39. Sheikh M, Qassem M, Kyriacou PA, 2021. Wearable, environmental, and smartphone-based passive sensing for mental health monitoring. Frontiers in Digital Health 3, 33. 10.3389/fdgth.2021.662811 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Snell-Rood C, Carpenter-Song E, 2018. Depression in a depressed area: Deservingness, mental illness, and treatment in the contemporary rural U.S. Soc Sci Med 219, 78–86. 10.1016/j.socscimed.2018.10.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Spitzer RL, Kroenke K, Williams JBW, Löwe B, 2006. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 166, 1092–1097. 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
  42. Thomas D, MacDowell M, Glasser M, 2012. Rural Mental Health Workforce Needs Assessment - a national survey [WWW Document]. 10.22605/RRH2176 [DOI] [PubMed] [Google Scholar]
  43. United States Census Bureau, 2021. Small Area Income and Poverty Estimates (SAIPE) Program [WWW Document]. URL https://www.census.gov/programs-surveys/saipe.html (accessed 2.5.22).
  44. Vermani M, Marcus M, Katzman MA, 2011. Rates of Detection of Mood and Anxiety Disorders in Primary Care: A Descriptive, Cross-Sectional Study. Prim Care Companion CNS Disord 13. 10.4088/PCC.10m01013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. World Health Organization, 2017. Mental Health ATLAS 2017 Member State Profile.
  46. Yim SJ, Lui LMW, Lee Y, Rosenblat JD, Ragguett R-M, Park C, Subramaniapillai M, Cao B, Zhou A, Rong C, Lin K, Ho RC, Coles AS, Majeed A, Wong ER, Phan L, Nasri F, McIntyre RS, 2020. The utility of smartphone-based, ecological momentary assessment for depressive symptoms. Journal of Affective Disorders 274, 602–609. 10.1016/j.jad.2020.05.116 [DOI] [PubMed] [Google Scholar]
  47. Yom-Tov E, 2017. Predicting drug recalls from internet search engine queries. IEEE J Transl Eng Health Med 5, 4400106. 10.1109/JTEHM.2017.2732945 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES