Association between prescriber practices and major depression treatment outcomes

Abstract

Practice variability may represent an opportunity to improve care by identifying the differences in outcomes associated with differences in practice. To characterize differences in depression treatment outcomes among individual providers in outpatient psychiatry practices and primary care practices, we examined a longitudinal cohort derived from outpatient electronic health records from two academic medical centers and six community hospitals in Eastern Massachusetts. This cohort included antidepressant-treated individuals with an ICD-9/10 diagnosis of major depressive disorder, and deidentified health care providers treating at least 10 such patients per year between 2008 and 2022. We examined the association between individual provider prescribing characteristics and proportions of treated patients who do not follow up after initial antidepressant prescription or who achieve a stable ongoing prescription. In binomial regression models, among 104 psychiatrists, greater heterogeneity in antidepressant prescribing and lesser proportion of serotonin reuptake inhibitors (SSRIs)¹ prescribed were associated with greater rates of achieving stability (for heterogeneity, adjusted odds ratio AOR, 1.55 [95 % CI, 1.22 – 2.06]; for proportion of SSRIs, AOR, 0.01 [95 % CI, 0.00–0.59]). Among 369 primary care physicians, greater volume of depression encounters per year, but not prescribing heterogeneity, was associated with greater rates of achieving stability (for encounters, AOR, 2.15 [95 % CI, 1.61 – 2.89]; for heterogeneity, AOR, 0.99 [95 % CI, 0.85 – 1.15]). Primary care and psychiatry predictors are not the same and therefore suggest potentially distinct strategies to improve clinical outcomes in each setting. Trial Registration: N/A

1. Introduction

Clinical practice variability has emerged as an opportunity to identify strategies to improve outcomes that does not require the development of novel therapeutic or diagnostic tools [1], [2]. Instead, understanding whether and why differences in practice associate with differential outcomes may allow identification of potential strategies that practices could adopt to improve clinical outcomes.

Depression treatment outcomes are known to exhibit substantial heterogeneity. A widely cited rule of thumb notes that 1/3 of individuals improve with initial treatment, and another 1/3 with multiple next-step treatments [3]. This variability in response to standard treatments is typically ascribed to individual level differences, prompting efforts to identify clinical subtypes [4] or predictive biomarkers [5], [6]. However, at least some of this variability may well arise from differences in clinician practices.

Within psychiatry, such practice variation has rarely been studied. An analysis of commercial claims data found that individuals diagnosed with depression by psychiatrists were more likely to receive treatment of any kind, and treatment perceived as adequate, than individuals diagnosed by primary care physicians [7]. Similar results emerged from National Ambulatory Medical Care Surveys data comparing 1300 primary care depression visits to 2418 psychiatry depression visits [8], as well as a survey comparing psychiatrists to primary care providers in their approach to treatment of a hypothetical mild major depressive episode [9]. While these studies compared primary and specialty care, they did not examine variability within psychiatric or primary care practices, nor associations with outcomes.

In the present study, we examined two large groups of clinicians spanning two health systems – one drawn from general outpatient psychiatry, the other from primary care psychiatry. We aimed to understand the extent to which outcomes inferred from electronic health records [10], [11] varied among individual providers in these groups, and then to quantify the extent to which such variability was explained by quantifiable features of individual clinician practices. We hypothesized that clinicians with greater clinical volume, and greater range of prescribing to facilitate treatment personalization, would on average have greater treatment stability and fewer treatment discontinuations among their patients.

2. Materials and methods

2.1. Study design and cohort development

We included outpatient visits from the electronic health records of 2 academic medical centers and 6 community hospitals, and their affiliated sites, in Eastern Massachusetts between March 1, 2008, the start of routine e-prescribing across the hospital systems, and April 27, 2022. Each provider was assigned a unique anonymized identifier, linked to all ICD-9 and 10 diagnostic codes, procedure codes, and prescriptions associated with that provider.

We restricted the provider cohort to those billing for at least 10 individuals with major depressive disorder (MDD) defined by diagnostic codes (Supplemental Table 1) per year, averaged over the time period in which the provider appears in the data set, and at least 100 diagnostic codes billed in total. To account for the transition from ICD-9 to ICD-10, and reduce dimensionality of diagnostic codes, we aggregated codes up to the 530 clinical categories defined by Clinical Classifications Software Refined (CCSR, version 2022–1) [12]. As in prior work, we then generated high-dimensional representations for each provider based on counts of CCSR codes, reduced to two dimensions via UMAP [13], [14] and then clustered into ten groups using a Gaussian Mixture Model [15].

This study protocol was approved by the appropriate institutional committee under protocol #2007P002138 with a waiver of informed consent, as no contact with human subjects was required, data were deidentified, and consent would not be feasible at this scale.

2.2. Definitions of provider features and patient outcomes

For each provider, we calculated the total number of antidepressant (AD) prescriptions recorded, then divided by the provider’s number of total diagnostic codes as a proxy for practice volume. We also examined proportion of total prescriptions representing SSRI, selective norepinephrine reuptake inhibitor (SNRI), atypical antidepressant, or tricyclic antidepressant (TCA). Because augmenting agents are not routinely used in primary care settings, we chose to focus solely on primary antidepressants. We generated a measure of heterogeneity of medications used by each provider by determining the number of different ADs accounting for 75% of all AD prescriptions for each provider. In essence, this represents the breadth or range of prescribing for a given provider.

Treatment outcomes were defined for all individuals with at least one diagnostic code of MDD, excluding those with codes for schizophrenia or bipolar disorder. As in prior work [11], we examined treatment stability, reflecting the proportion of patients who remain on a stable AD treatment, and treatment dropout, reflecting the proportion of patients who do not return after an initial prescription. Although clinical trial outcomes such as response and remission are challenging to reliably define for individuals using solely coded clinical data, we applied a simplifying but face-valid assumption that successful treatments continue uninterrupted over time with repeated prescriptions. Specifically, stability captures continuation of a given AD in the 180-day period after index prescription, with at least one recorded prescription in each 90d period. An index prescription was defined as the first AD prescription within the MGB health system, regardless of prior antidepressant treatment or subsequent prescription of a new AD. Only the initial AD prescription within the MGB healthcare system was included in the data analysis to ensure that no patient was counted more than once.

Dropout reflects absence of any further AD prescriptions or other psychiatric treatment in the 90-day window after index prescription. We excluded patients completely lost to follow-up by excluding those with no facts in the electronic health record during the subsequent 90-day period. Outcomes were attributed to the provider recording the initial AD prescription.

2.3. Analysis

We used standard univariate measures to describe distributions among the providers in each of the 2 cohorts – outpatient psychiatry and primary care. To examine the association between provider-level characteristics and the two outcomes (proportion of individuals with stable treatment, and proportion with dropout), we used binomial regression – first univariate, then adjusted for all features of prescribing and clinical volume. The threshold for statistical significance was considered to be 2-tailed p of 0.05. As a hypothesis-generating experiment – i.e., to identify possible differences for further study – we elected a priori not to correct for testing multiple hypotheses.

3. Results

The outpatient psychiatry provider cohort include 104 providers who treated a total of 145,140 patients (see CONSORT diagrams, Supplemental Figures 1 and 2). On average, these providers treated 201 individuals per year (Table 1). In binomial regression examining association between psychiatric provider practice characteristics and outcomes (Fig. 1 and Supplemental Figure 3), we found that greater heterogeneity in prescribing was associated with greater rates of stability (AOR, 1.55 [95% CI, 1.22 – 2.06]), as was lesser proportion of SSRI prescriptions (AOR, 0.01 [95% CI, 0.00 – 0.59]). Lower volume of antidepressant prescribing per year was also significantly associated with greater stability (AOR, 0.35 [95% CI, 0.10 – 0.80]). However, not all patients met inclusion criteria for stability or dropout and were thus excluded from outcome analyses, resulting in a difference between the providers included in the study cohort and those included in analyses for specific treatment outcomes.

Table 1.

Characteristics of prescribing clinicians.

	Primary Care Providers	Psych Providers
Count (%)	369 (78.01%)	104 (21.99%)
	Mean (SD)	Mean (SD)
Antidepressant heterogeneity	2.19 (1.17)	2.90 (1.93)
Antidepressants/year	8.61 (9.25)	15.11 (36.83)
MDD patients/year	106.67 (175.46)	57.49 (61.44)
MDD encounters/year	50.40 (83.30)	106.43 (199.25)
Psychiatry encounters/year	129.60 (207.43)	194.99 (287.59)
Total encounters/year	610.76 (761.69)	201.35 (289.95)
% SSRI	58.74 (35.49)	46.74 (35.29)
% SNRI	15.66 (26.85)	22.39 (30.06)
% TCA	8.48 (19.76)	7.08 (18.75)
% Other antidepressant	9.06 (19.68)	10.29 (24.30)

Fig. 1.

Adjusted binomial regression model analyzing psychiatric provider practice characteristics associated with treatment stability.

We also examined regression models for proportion of dropout (Fig. 2 and Supplemental Figure 4), using the same features. In these models, lower volume of antidepressant prescribing was associated with lesser rates of drop out (AOR, 0.66 [95% CI, 0.42 – 0.98]). No other prescribing or volume characteristics were significantly associated with early discontinuation. We excluded patients completely lost to follow-up by excluding those with no facts in the electronic health record during the subsequent 90-day period.

Fig. 2.

Adjusted binomial regression model analyzing psychiatric provider practice characteristics associated with treatment dropout.

We next sought to understand whether similar patterns emerged in the primary care provider cohort. This group included 369 providers who treated a total of 268,018 patients. (see CONSORT diagrams, Supplemental Figures 1 and 2). On average, these providers treated 611 individuals per year (Table 1). In a regression model for treatment stability, greater number of depression encounters was associated with greater rates of depression stability (AOR, 2.15 [95% CI, 1.61 – 2.89]), but no significant association with prescribing heterogeneity was detected in adjusted regression models (AOR, 0.99 [95% CI, 0.85 – 1.15]); (Fig. 3 and Supplemental Figure 5).

Fig. 3.

Adjusted binomial regression model analyzing primary care providers practice characteristics associated with treatment stability.

For dropout, fewer depression encounters were associated with lower dropout rates (AOR, 0.36 [95% CI, 0.30 – 0.42]); (Fig. 4 and Supplemental Figure 6), as were lesser proportion of SSRI, tricyclic, and other antidepressant prescribing (for SSRI, AOR, 0.68 [95% CI, 0.48 – 0.95]; for tricyclic, AOR, 0.42 [95% CI, 0.27 – 0.66]; for other antidepressants, AOR, 0.57 [95% CI, 0.37 – 0.89]). Conversely, greater heterogeneity of prescribing was associated with greater rates of discontinuation (AOR, 1.12 [95% CI, 1.04 – 1.21]).

Fig. 4.

Adjusted binomial regression model analyzing primary care providers practice characteristics associated with treatment dropout.

4. Discussion

In these two cohorts of providers treating major depressive disorder, we identified distinct correlates of achieving stable antidepressant treatment. Notably, heterogeneity of antidepressant prescribing was associated with greater likelihood of stability only among psychiatric providers; among primary care providers, this feature exhibits no association with stability, and indeed was associated with greater rates of dropout. Conversely, volume of visits for major depression had no significant association with outcome among psychiatric providers, while it associated with greater likelihood of stability, and lesser likelihood of dropout, among primary care providers. We hypothesize that a greater range of prescribing among individuals with less psychopharmacologic training may lead to greater discontinuation because of challenges in managing side effects or adjusting dose. Future work will be needed to examine this possibility directly.

Our work is difficult to compare directly to prior studies, which have generally sought to compare clinical populations rather than capture variability within them. Those analyses generally found greater rates of antidepressant prescribing among psychiatrists compared to primary care physicians [16]. We did not directly compare these two groups, except for descriptive purposes, reasoning that the patients treated in these settings are likely to be quite different. For example, individuals with more severe depression, or greater comorbidity, would be likely to be referred for specialty care rather than remaining in a primary care setting. One prior investigation compared providers from different kinds of practices, contrasting staff/group-model Managed Care Organizations (MCOs) to network-model organizations; this study found differences in propensity to refer out to specialists, rather than treating without referral [17].

Our work may suggest opportunities to improve the standard of care for depression treatment and indicates that distinct strategies may be prioritized in different settings. Among outpatient psychiatrists, one conjecture for why greater prescribing variability was associated with better outcomes is that patients in specialist care need more specific treatments, and thus providers who used their full range of options to identify treatments got better results. While the underlying mechanism of association merits further study, our work suggests that there may be an opportunity for educational interventions promoting comfort with a broader range of prescribing to facilitate greater rates of achieving stable treatment. (If prescribing heterogoneity were solely a proxy for treating more severely ill individuals requiring more complex treatment regimens, we might have expected the opposite finding, with lower stability associated with greater heterogeneity). Conversely, among primary care clinicians, outcomes were more positive among providers with greater volume of major depressions visits -- both in terms of achieving stability and minimizing dropout. Greater heterogeneity was actually associated with higher dropout rates. While these associations cannot establish causation, this finding may suggest the value of augmenting training in depression treatment among primary care physicicans, or interventions that could increase visit frequency in this setting. That is, primary care physicians who see individuals with majore depression more frequently may achieve better results, consisitent with quality guidelines for follow-up that may not be fully implemented [18], [19]. On the other hand, in this group of clinicians, in contrast to psychiatry, treatment heterogeneity was not significantly associated with stability in adjusted models. This result suggests that simply broadening pharmacologic options for primary care physicians (rather than, for example, focusing on strategies to improve adherence for first-line treatments, or facilitating referral for more difficult-to-treat patients) may not be a useful strategy.

This study has multiple limitations. While it includes a broad group of clinics across two health systems, the extent to which this region will generalize to others in the US and internationally remains to be determined. Furthermore, it does not include the more sensitive outcome measures typically available in clinical trials; on the other hand, we elected to focus on measures which are face-valid in terms of relevance to clinicians and patients, and available in any electronic health record. These measures were also demonstrated to be modestly predictable on the basis of patient-level data in prior work [8], [9], suggesting that provider-level data would also be informative. While our investigation aimed to examine differences among providers, certain aspects of provider-level data, including level of training, were inaccessible due to IRB limitations. Furthermore, patient characteristics that were not analyzed, such as insurance status, could have influenced physician type as well as drop out and treatment outcomes. Results were also not adjusted for multiple comparisons using a Bonferroni-corrected threshold and thus observed associations should be treated as hypotheses warranting further investigation.

Our work may suggest opportunities to improve the standard of care for depression treatment and indicates that distinct strategies may be prioritized in different settings. Among outpatient psychiatrists, one conjecture for why greater prescribing variability was associated with better outcomes is that patients in specialist care need more specific treatments, and thus providers who used their full range of options to identify treatments got better results. While the underlying mechanism of association merits further study, our work suggests that there may be an opportunity for educational interventions promoting comfort with a broader range of prescribing to facilitate greater rates of achieving stable treatment. (If prescribing heterogeneity were solely a proxy for treating more severely ill individuals requiring more complex treatment regimens, we might have expected the opposite finding, with lower stability associated with greater heterogeneity). Conversely, among primary care clinicians, outcomes were more positive among providers with greater volume of major depression visits – both in terms of achieving stability and minimizing dropout. Greater heterogeneity was actually associated with higher dropout rates. While these associations cannot establish causation, this finding may suggest the value of augmenting training in depression treatment among primary care physicians, or interventions that could increase visit frequency in this setting. That is, primary care physicians who see individuals with major depression more frequently may achieve better results, consistent with quality guidelines for follow-up that may not be fully implemented [16], [17]. On the other hand, in this group of clinicians, in contrast to psychiatry, treatment heterogeneity was not significantly associated with stability in adjusted models. This result suggests that simply broadening pharmacologic options for primary care physicians (rather than, for example, focusing on strategies to improve adherence for first-line treatments, or facilitating referral for more difficult-to-treat patients) may not be a useful strategy. However, specific care location could associate with outcome differences. Because there is not 1:1 relationship between providers and locations, or between patients and locations, providers in these networks may work in multiple clinics and patients may interact with the systems in multiple settings (e.g., primary care, cardiology, outpatient psychiatry). This is why we elected to focus on individual provider-level characteristics, rathern than properties of an individual clinic or setting, reasoning that these characteristics may be more straightforward to identify and modify. However, future work should further investigate the association between properties of individual settings and differences in patient outcomes.

Overall, we identified associations between variations in practice and clinical outcomes which differ for two groups of providers. Future work should examine whether altering these practice characteristics can improve outcomes. As all of our measures and outcomes are readily derived from electronic health records, investigating potential interventions either naturalistically or with randomized designs should be highly feasible.

Funding

This work was supported by the National Institutes of Health [R01MH123804-03; Dr. Perlis and Dr. Doshi-Velez]. The sponsors did not contribute to any aspect of study design, data collection, data analysis or data interpretation. The authors had the final responsibility for the decision to submit for publication.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:Roy H. Perlis reports financial support was provided by National Institutes of Health. Roy H. Perlis reports a relationship with Vault Health that includes: consulting or advisory and equity or stocks. Roy H. Perlis reports a relationship with Belle Artificial Intelligence that includes: consulting or advisory. Roy H. Perlis reports a relationship with Swan AI Studios that includes: consulting or advisory. Roy H. Perlis reports a relationship with Mila Health that includes: consulting or advisory. Roy H. Perlis reports a relationship with Alkermes Inc that includes: consulting or advisory. Roy H. Perlis reports a relationship with Genomind that includes: consulting or advisory. Roy H. Perlis reports a relationship with Takeda Pharmaceuticals America Inc that includes: consulting or advisory. Roy H. Perlis reports a relationship with Circular Genomics that includes: consulting or advisory and equity or stocks. Roy H. Perlis reports a relationship with Psy Therapeutics that includes: consulting or advisory and equity or stocks. JAMA Network - Open (Service as an Associate Editor) RHP If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

Supplementary material

.