Estimating the Prevalence of Mental and Substance Use Disorders: A Systematic Approach to Triangulating Available Data to Inform Health Systems Planning

Abstract

Objective:

To estimate the prevalence of specific mental and substance use disorders (MSUDs), by age and sex, as a first step toward informing needs-based health systems planning by decision-makers.

Methods:

We developed a conceptual framework and a systematic methodology for combining available data sources to yield prevalence estimates for specific MSUDs. Data sources used included published, peer-reviewed literature from Canada and comparable countries, Canadian population survey data, and health administrative data from British Columbia. Several well-established methodologies including systematic review and meta-analyses of published prevalence estimates, modelling of age- and sex-specific distributions, and the Global Burden of Disease severity distribution model were incorporated in a novel mode of triangulation.

Results:

Using this novel approach, we obtained prevalence estimates for 10 MSUDs for British Columbia, Canada, as well as prevalence distributions across age groups, by sex.

Conclusion:

Obtaining reliable assessments of disorder prevalence and severity is a useful first step toward rationally estimating service need and plan health services. We propose a methodology to leverage existing information to obtain robust estimates in a timely manner and with sufficient granularity to, after adjusting for comorbidity and matching with severity-specific service bundles, inform need-based planning efforts for adult (15 years and older) mental health and substance use services.

Introduction

Health systems planning is a critical component of health policy. Traditional efforts have relied on previous utilization trends (“utilization-based planning”), which do not capture the full disease burden and associated needs. Researchers, policy-makers, and decision-makers have argued for planning based on a rational, evidence-based analysis of population health requirements or needs-based planning. ^{1 –6} This is particularly important in the area of mental health and substance use due to the fact that the burden of these disorders has traditionally been underestimated. The services required have been underfunded both globally and in Canada due to complex issues including stigma and political considerations. ^7,8 Indeed, in Canada, the proportion of the mental health and substance use disorder burden has been estimated to triple the proportion of the efficiently allocated spending in the same set of disorders. ⁹

A needs-based approach requires an accurate understanding of how many people are affected. However, simply knowing, the number of prevalent cases is insufficient. Additional details, such as how the distribution of a disorder varies by age, sex, and severity, are required to adequately design and size services. In order to develop such detailed estimates, we set out to design and apply a framework of methodological and data source “triangulation.” Triangulation has a variety of definitions but usually implies taking results from different approaches to strengthen the validity of the conclusions. ¹⁰ In order to obtain the best estimate of the age by sex distribution of prevalence for mental and substance use disorders (MSUDs) for the adult population of British Columbia (BC), Canada, we triangulated different data sources and techniques. To summarize the available peer-reviewed evidence, we conducted systematic reviews and meta-analyses of published prevalence estimates. To model age- and sex-specific distributions, we triangulated available sources of data, and to obtain severity fractions, we drew from the Global Burden of Disease (GBD) model.

The goal is to provide a method by which timely, detailed, and evidence-based estimates for MSUDs can be derived or updated using existing data, and used to support health system planning, after adjustment for comorbidity and matching with the required level of services.

Methods

As the quality and quantity of available data vary by disorder, we developed a systematic approach that allowed for the incorporation of the broadest range of sources but could still be implemented in the absence of 1 or more specific sources. Where possible, we systematically integrated published peer-reviewed data, national survey data, and health administrative data in order to arrive at a set of age- and sex-specific prevalence estimates for each of 10 MSUDs. A stepwise approach was used for each disorder. We first estimated 12-month prevalence, then calculated age- and sex-specific estimates based on distributions obtained from published data and administrative sources. These distributions were adjusted to match the estimated 12-month prevalence. When existing evidence or approaches were insufficient to follow the systematic approach, we developed ad hoc methodologies in consultation with content experts. The details of disorder-specific approaches can be found in the accompanying papers. We now present a step-by-step overview of the process.

MSUDs: Classification and Selection

This study focuses on 10 MSUDs as defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), which were prioritized by the BC Ministry of Health: (i) depressive disorders, (ii) anxiety disorders, (iii) attention deficit hyperactivity disorder, (iv) bipolar disorders, (v) schizophrenia spectrum disorders, (vi) eating disorders, (vii) intellectual disabilities, (viii) opioid use disorders, (ix) alcohol use disorders, and (x) cannabis use disorders. ¹¹

Calculating 12-month Prevalence: Systematic Review, Quality Assessment, and Meta-analysis

For the purpose of health systems planning, 12 months is an appropriate period to ground estimates of prevalence (as opposed to lifetime or point prevalence) since it coincides with standard fiscal and financial planning periods. We performed a systematic literature search in the electronic databases PubMed, PsycINFO, and Web of Science. The structure of our search terms was as follows:

(Headings or terms for all subtypes included in the DSM-5 grouping of (specific) disorder joined by the Boolean OR)

AND (Headings or terms related to population epidemiology measures joined by the Boolean OR)

AND (Headings or terms identifying the specific regions of interest)

We limited our search to studies from Canada and comparable settings (i.e., Australia, New Zealand, Western Europe, United States of America). Initial results were assessed by our research team. To be considered for further analysis, studies had to meet the inclusion criteria listed in Table 1. Previous versions of DSM were included in specific circumstances to capture all the relevant high-quality studies. Each study underwent a quality assessment process in which a subset of the research team (including epidemiologists, psychiatrists, and public health specialists) reviewed content and reached consensus on whether each study met the outlined inclusion criteria and on whether the inclusion criteria were appropriate for the specific disorder. Of note, within each DSM-5 disorder grouping, study quality varies widely, so after presenting the results of the quality assessment to the full team, a collective decision was made as to which of the data points would be captured by our meta-analytic estimate. The decision depended on data availability and epidemiological significance. For example, in the case of anxiety disorders, the literature reported 2 different types of estimates: an aggregate estimate that included all anxiety disorders and individual estimates by subtype (general anxiety disorder, panic disorder, specific phobia, etc.). Our meta-analysis included estimates for “all anxiety disorders.”

Table 1.

Inclusion and Exclusion Criteria.

Study Features

Studies focusing on the general population of BC, Canada, the United States, Western European countries, Australia, and New Zealand Uses diagnostic survey tools, such as Composite International Diagnostic Interview or Structured Clinical Interview for DSM, based on standardized diagnostic criteria, such as ICD or DSM, for case identification. ○ For Canadian studies, studies that use DSM-III and onwards are included ○ For international prevalence studies, studies that use DSM-IV and onwards are included ○ For incidence data, which is scarcer, studies that use DSM-III and onwards are included regardless of country Community surveys using probability sampling methods or the entire population of a confined region Outcome measure is a population epidemiological measure such as prevalence or incidence

Participant Features

Age 15 or older General population (i.e., not nursing homes, prison populations, depressive symptomatology in populations with cancer, injury—some kind of specialized group)

Note. BC = British Columbia; DSM = Diagnostic and Statistical Manual of Mental Disorders; ICD = International Statistical Classification of Diseases and Related Health Problems.

Data were extracted to a summary table in order to capture authors, year of publication, site, study type, study name, year of study, age range, subtype of disorder included, sample size, sampling method, instrument, and outcome measures. Additional tables and graphs were created to capture subtypes of disorders and other information of relevance to age- and/or sex-specific prevalence. The main outcome was 12-month prevalence for each selected MSUD. Secondary outcome measures included age- and sex-specific prevalence, if provided.

After data extraction, a meta-analysis was performed on the 12-month prevalence estimates. When insufficient studies with 12-month prevalence were available, we included point, 3-month, or 6-month prevalence to ensure sufficient sample size for a meta-analysis. We examined whether there were any effects on prevalence estimates by geographic region (i.e., whether the Canadian estimate was significantly different from the United States estimate or the estimates from other world regions) or by period (12 months vs. less than 12 months). No period effect was detected for any disorder. When a significant regional difference was detected, the Canadian estimate was preferred. For each MSUD, a random effects model was used to generate 12-month prevalence and 95% confidence intervals. Meta-analytic results were compared through the use of a forest plot. This result became the 12-month prevalence upon which all further calculations were anchored. Statistical analyses were performed using the R Package. ¹² Meta-analyses were carried out using the metaprop function in the R package meta. ¹³

Generating Age- and Sex-specific Prevalence Distributions

A systematic procedure was used to generate age- and sex-specific prevalence through triangulation of sources. The procedure was consistent across all disorders, though the inputs differed slightly depending on data availability. Four sources of data were considered for each disorder.

Population-level Survey Data

The Canadian Community Health Survey (CCHS)–Mental Health supplement (conducted in 2002 and 2012) is a representative community survey that provides age- and sex-specific estimates for some MSUDs. Much of these data are publicly available through Statistics Canada sources. ¹⁴ While data can be broken down to the provincial level, differences between BC and national results were not statistically significant, so the national results, which are less variable across age groups, were used.

Administrative Data

Previously analyzed age- and sex-specific results from administrative data in BC were available for some MSUDs. Relevant administrative data used here were obtained from physician billing data, where every instance of billed service delivery performed by a physician must include a primary diagnosis, and hospital discharge data that include primary as well as secondary diagnoses. Similar sources exist in all Canadian provinces.

Other Data Sources

For some disorders, age- and/or sex-specific distributions were found through the systematic review and through the Canadian Alcohol and Other Drug Use Monitoring Survey (CADUMS). ^15,16 For each MSUD, age- and sex-specific estimates obtained through the systematic literature review, CCHS, local administrative data, or CADUMS were collected. To avoid excessively weighting data from a particular source, median values were obtained using a 2-step process. First, if we had multiple measures within a source (e.g., several age–sex estimates from studies obtained through the literature review), we took the median at each age and sex point within a source. Second, we then took the median across the sources. This produced a median age- and sex-specific estimate of annual prevalence. Age was categorized into 5-year age groups (i.e., 15 to 19,…85 to 89, and 90+). The distribution across ages for a given disorder could be quite variable and hence epidemiologically implausible, so we smoothed the distributions using the locally estimated scatterplot smoothing (LOESS) function in the R statistical package, ¹² which applies a non-parametric function based on a polynomial regression using local fitting. ¹⁷

Adjusting Estimates of Age- and Sex-specific Prevalence to the Expected Prevalence

The curves obtained through the procedure described above reflected the expected shape of the prevalence curve over age categories by sex, but their magnitude did not necessarily match the expected prevalence obtained through the meta-analysis. In our view, the overall meta-analytic estimate was more valid than the estimate derived from the initial age- and sex-specific curves (which were derived from fewer, more fragmented, and heterogeneous sources). Thus, as a final step, the curves (as a whole) were adjusted up or down so that the total expected number of cases in BC, given the age- and sex-population estimates that were produced by the curves matched (within 0.1%) the expected number of cases in BC based on the 12-month prevalence estimate from the meta-analysis. This was done through the use of a small adjustment factor repeatedly applied to the age- and sex-specific curves until the overall number of cases predicted matched (within 0.1%) the expected number of cases obtained from the meta-analytic result.

Figure 1 presents an example including all the curves sequentially obtained for anxiety disorders. Plotted on the figure are the averages from the 3 data sources with age- and sex-specific data (epidemiological studies, administrative data, and CCHS population surveys). The dashed line represents the average of these 3 sources smoothed using the LOESS procedure. The thick solid line presents the final prevalence estimate, which is the LOESS line adjusted (upwards in this case) so that the total estimated number of anxiety cases in BC based on the values that make up the curve equals the estimated number of cases in BC based on the prevalence rate determined by the meta-analysis.

Figure 1.

Prevalence rates from data sources used in generating the age- and sex-specific prevalence distribution curve. Note. Data for anxiety disorders are shown. In the label, “Epidemiological studies” refers to studies published in the peer-reviewed literature. “Population Surveys” refers to data obtained from Canadian Community Health Survey and Canadian Alcohol and Other Drug Use Monitoring Survey (not peer-reviewed).

Severity Distribution: Severity-adjusted Number of Cases

The number of cases predicted for each disorder was stratified into severity clusters following the GBD framework, which provides severity fractions for the majority of human disorders. ^{18 –20} Examples of the types of severity are as follows: (i) asymptomatic, (ii) mild, (iii) moderate, and (iv) severe, but important disorder-specific variations exist. Of note, the “asymptomatic” or subthreshold fraction represents the fraction of people that, despite meeting criteria for a 12-month disorder, will present at any given time a clinically improved picture with minimal functional impairment (due to, e.g., treatment-related or spontaneous improvement). Considering this fraction is important for health systems planning because their service needs are below those with more severe presentation but still above those with no 12-month disorder (e.g., a periodic visit with a general physician to monitor for treatment effectiveness and prevent relapse). In instances where the GBD severity distribution was not available for a given disorder, an alternate method was developed through a scoping review of the literature focused on disorder-specific severity splits (see Online Appendix, Table 1 for full list). To calculate the number of individuals for each severity category, we multiplied the total number of expected cases in BC for each disorder by the corresponding severity fraction.

Results

Overall, 217 studies were identified through the systematic search and reviewed in detail, with 180 of them “passing” the quality review and yielding data for subsequent calculations Future papers focusing on substance use disorders and mental disorders will present the full details and list of studies. Meta-analytic results were obtained for all 10 MSUDs. However, with opioid use disorders, we observed a time-trend in prevalence that led to a restriction of the inclusion criteria to the most recent studies and a variation in the estimated prevalence (departures from the systematic procedure are explained in detail in the aforementioned papers focusing on substance use disorders and mental disorders). For depressive disorders and anxiety disorders, there were sufficient studies to perform a meta-analysis by geographic region (Canada, United States, or “Other”), and statistically significant differences were found. For major depressive disorder, the Canadian estimate was lower and had less heterogeneity than both the Other and the United States estimates, so it was considered the most appropriate estimate. In the case of anxiety disorders, the Canadian prevalence estimate was also lower than the Other and U.S. prevalence estimates (though only significantly lower than the U.S. estimate). Our approach prioritized local data whenever it was available and of high quality, so we found the Canada estimate alone to be preferred in these cases. The remaining conditions did not have enough Canadian studies to stratify the meta-analyses by geographic region. Sufficient studies were available to perform subgroup analyses for depressive, anxiety, alcohol-related, cannabis-related, and schizophrenia spectrum disorders. As a result, more than 1 prevalence estimate is available for these groups. The estimated annual prevalence results for each disorder grouping are summarized in Table 2. The age and sex distributions were estimated as described above for all disorders and will be presented in the aforementioned papers. Figure 2 shows the age- and sex-specific results for anxiety disorders as an exemplar. With respect to the severity distributions, the GBD framework provided fractions for 9 of the 10 disorders, and an ad hoc method was developed for eating disorders. Online Appendix, Figure 1 presents an area plot of the BC-wide age-specific case estimates (by sex) for anxiety disorders, with the severity-specific estimates presented within the overall area.

Table 2.

Estimated Prevalence of Mental Health and Substance Use Disorders.

DSM-5 Disorder Grouping	Meta-analysis Prevalence Estimate (Cases/100)
	Prevalence (%)	95% CI
Depressive disorders
Major depressive disorder only	5.8	4.9%	6.7%
Canadian estimates	4.1	3.8%	4.4%
Depressive disorders	6.4	5.2%	7.8%
Dysthymia	0.8	0.4%	1.3%
Anxiety disorders
Any anxiety	9.8	6.6%	13.6%
Canadian estimates	6.9	3.5%	11.4%
Any anxiety less OCD/PTSD	7.9	5.0%	11.3%
By type
Panic	1.7	1.3%	2.2%
Agoraphobia	1.1	0.7%	1.6%
Social	4.4	3.4%	5.7%
Specific	6.4	4.6%	8.5%
Generalized anxiety disorder	2.3	1.9%	2.8%
Separation anxiety disorder	1.3	0.8%	1.9%
Alcohol-related disorders
Alcohol Use Disorder	5.4	3.3%	8.0%
Opioid-related disorders
Prescription opioid use disorder	0.9	0.8%	1.0%
Cannabis-related disorders
Cannabis use disorder	1.6	1.3%	2.1%
Attention deficit hyperactivity disorders
Attention deficit hyperactivity disorder	3.6	3.0%	4.3%
Bipolar and related disorders
Bipolar I	0.59	0.43%	0.78%
Bipolar II	0.69	0.42%	1.0%
Bipolar I and II	0.93	0.60%	1.3%
Schizophrenia spectrum and other psychotic disorders
Schizophrenia	0.36	0.17%	0.62%
Schizophrenia spectrum disorders	0.55	0.38%	0.74%
Feeding and eating disorders
Anorexia and bulimia nervosa	0.33	0.22%	0.46%
Intellectual disabilities
Intellectual disabilities	0.60	0.42%	0.81%

Note. DSM = Diagnostic and Statistical Manual of Mental Disorders; OCD = obsessive-compulsive disorder; PTSD = post-traumatic stress disorder.

Figure 2.

Step 8: Predicted age- and sex-specific prevalence distribution curve and predicted number of cases in British Columbia population. Note. Data corresponding to anxiety disorders are shown.

Discussion

We propose a systematic approach that derives a set of MSUD prevalence estimates based on a triangulation methodology that utilizes information from a variety of sources to provide timely, valid epidemiological estimates as a first step to inform health planning efforts. This approach uses existing data (instead of gathering primary data) to produce comprehensive and detailed prevalence estimates combining scientific rigour with the pragmatic time constraints of decision-making.

This methodology may offer a first step toward filling an important health planning and policy gap. Decision-makers often base their choices on a combination of political considerations and readily available evidence, highlighting the need to collate, integrate, and present relevant data in a timely and accessible manner. In the absence of actionable evidence, decision-making can default to political considerations driven by advocacy, lobbying, and (when available) raw service utilization data. The result is an incomplete and inaccurate picture from a scientific and public health perspective. The approach outlined here maximizes the use of already existing data, reducing the time required to obtain available evidence.

Our approach builds on other national and international needs-based planning approaches for MSUDs. ^2,21,22 We have expanded and adapted them to our context (i.e., to our decision-makers’ needs and our data sources) while also providing a systematic approach that can be applied in any jurisdiction with relatively minor adaptations. Indeed, Figure 3A can function as an algorithm to apply this method through sequential steps of identification, assessment, and integration of available evidence. In data-rich jurisdictions, local prevalence estimates and age- and sex-specific curves will be available, and straightforward summary measures and modelling will yield comprehensive results. In data-deprived settings, the incorporation of a systematic review will yield inputs from more or less comparable regions, which could be integrated with the sparse local data during the triangulation phase. In the absence of any local evidence, the results presented here can be used as a starting point, and then adjusted up or down using other sources of evidence, such as how social determinants impact the distribution of MSUDs in the region under consideration.

Figure 3.

(A). Analytical framework for estimating age-, sex-, and severity-adjusted prevalence in mental health and substance use disorders (phase 1 of a needs-based planning approach). (B). Analytical framework for estimating service needs in terms of fulltime equivalents for human resources and beds/days for hospital-based services and structured day programs (phases 1 and 2 of a needs-based planning approach).

Figure 3.

(Continued)

This approach has important limitations. First, primary collection of local data through representative population-level surveys with structured screening tools is, in general, the gold standard and should be pursued in the long term. However, these endeavours depend on the availability of significant resources (both economic and human), which often make them inadequate for timely response to planning needs. Our approach does not replace, but complements (and relies upon for validity), such primary sources. Second, our standard methodology may fail to capture disorder-specific and/or local circumstances that require adaptation of the method, such as was the case here with the time-trend detected for opioid use disorders. Thus, this approach requires careful and informed review of the results it produces. A key feature is its flexibility and ability to accommodate disorder-specific or local adaptations. Third, each data source has specific limitations. For example, administrative data reflect help-seeking (or service utilization) estimates rather than population prevalence. Misdiagnosed, undiagnosed, or untreated cases will not appear in these data. As a result, it usually fails to capture the earlier or milder stages of most common disorders. Epidemiological studies can take a long time to undertake, and thus, the data may be dated or not reflective of present-day frameworks. Routine surveys (like the CCHS) are limited in what can be examined, given the demands placed by disciplines other than mental health. The CADUMS survey was discontinued in 2012, so its results would not capture more recent trends. Our approach provides several safeguards against this type of source-specific validity issue: (a) triangulation of sources can mitigate bias inherent in a single source; (b) our approach grounds the overall estimate on the meta-analytic estimate; hence, the overall number is not affected by the potential lack of cases in the administrative data; and (c) disorder-specific analysis of the shape of the age-specific curve offers an opportunity to detect inconsistencies and apply an adjustment factor to age-specific data points. Other limitations (shared by most approaches) should also be noted. Unless regularly updated, results become dated as newer evidence becomes available. This is particularly true for disorders that are more susceptible to shifts in prevalence over time. The GBD severity distribution offers a useful, evidence-based, and relatively comprehensive set of disorder-specific severity splits (including an asymptomatic/subthreshold fraction), which allows for our model to eventually match the level of need with the intensity of services required. However, it also has limitations: the severity distributions are constant across age groups and settings, instead of capturing potentially relevant age- and location-specific patterns (e.g., in jurisdictions where proven treatments are insufficiently available, the distribution may shift toward the more severe end of the spectrum). Finally, though much faster than primary data gathering, implementing this approach takes some time. The work related to each disorder grouping took approximately 6 weeks to complete (i.e., generating the literature review, conducting the quality assurance, doing the meta-analysis, estimating the age–sex distribution, and writing the results). Disorders that required deviation from the methodological framework can be expected to take longer as a new approach is conceptualized and executed.

It should be noted that this is only the first step of the needs-based planning process. Of note, these estimates have not been adjusted for comorbidity. This is the object of the next step of this needs-based planning project, which also involves the design of the ideal suites of services for each disorder at each level of severity, the adjustment of the suites of services for meaningful comorbidities, and the development of conversion formulae to consolidate all the suites of services into a set of service-specific benchmarks (see Figure 3B). Once evidence-based benchmarks are obtained, a gap analysis would indicate which are the largest shortfalls in coverage for the estimated need, and a prioritization exercise could be undertaken based on various considerations such as adjusting for help-seeking behaviours, focusing on high-priority or underserved sub-populations, most disabling or lethal disorders, and so on.

In summary, we propose a methodology that can begin to fill a gap with respect to the data that policy- and decision-makers require to allocate resources in a rational manner. We highlight that this is only a first step and that many other steps are needed (as described in detail in Figure 3B) before this effort can help identify the true population needs with the ultimate goal of increasing coverage, improving quality of services, and improving mental and substance use outcomes.