Using mHealth technologies to improve the identification of behavioral health problems in urban primary care settings

Abstract

Introduction:

Behavioral health disorders remain under recognized and under diagnosed among urban primary care patients. Screening patients for such problems is widely recommended, yet is challenging to do in a brief primary care encounter, particularly for this socially and medically complex patient population.

Methods:

In 2013, intervention patients at an urban Connecticut primary clinic were screened for post-traumatic stress disorder, depression, and risky drinking (n = 146) using an electronic tablet-based screening tool. Screening data were compared to electronic health record data from control patients (n = 129) to assess differences in the prevalence of behavioral health problems, rates of follow-up care, and the rate of newly identified cases in the intervention group.

Results:

Results from logistic regressions indicated that both groups had similar rates of disorder at baseline. Patients in the intervention group were five times more likely to be identified with depression (p < 0.05). Post-traumatic stress disorder was virtually unrecognized among controls but was observed in 23% of the intervention group (p < 0.001). The vast majority of behavioral health problems identified in the intervention group were new cases. Follow-up rates were significantly higher in the intervention group relative to controls, but were low overall.

Conclusion:

This tablet-based electronic screening tool identified significantly higher rates of behavioral health disorders than have been previously reported for this patient population. Electronic risk screening using patient-reported outcome measures offers an efficient approach to improving the identification of behavioral health problems and improving rates of follow-up care.

Introduction

For much of the United States, primary care settings have become the first and only entryway for addressing the needs of the one in four adults with a diagnosable behavioral health condition.^1,2 The World Health Organization and the United States Surgeon General have called for the integration of behavioral health treatment into primary care as the most efficient and effective way of addressing the treatment gap for the estimated 50% of behavioral health conditions that remain undiagnosed.^3–5 This is particularly true in urban clinics that disproportionately serve racial and ethnic minorities, the poor, immigrants, and the chronically ill^1,2,6 and where behavioral health disorders may be both more prevalent and less recognized and treated. While 9.5% of the US population over age 18 has a mood disorder,^7–9 rates of major depressive disorder may be as high as 18.9% in urban health clinics.^10,11 African Americans living in the inner city have high rates of exposure to severe trauma, with 43% of patients in community health center clinics found to have post-traumatic stress disorder (PTSD),¹² and patients with chronic diseases such as diabetes may have even higher rates of depression (18%–31%).¹³

Despite high rates of these conditions, recent studies indicate that behavioral health problems remain largely unrecognized and undiagnosed in underserved minority patients seen in primary care. Recent studies have found that 30%–66% of people in primary care centers with clinical depression remained undiagnosed^14–16 and that primary care providers arrived at a depression diagnosis significantly less often for African Americans and Medicaid patients compared to White patients.¹⁷ While racial and ethnic minorities generally have lower rates of alcohol consumption than Whites,¹⁸ African Americans and Hispanics with alcohol dependency were less likely to seek or receive treatment, in part because they were less likely to have a primary care provider and were more likely to be under- or uninsured.^19,20

One approach to improving the identification and treatment of common, under-recognized health conditions is to implement systematic screening procedures (or processes). The United States Preventive Services Task Force has published screening recommendations in primary care practice for 71 problems for adults, five of which involve behavioral health problems or disorders.²¹ Such information, if available to the clinician at the time of the encounter, could facilitate the delivery of targeted interventions to prevent or ameliorate various negative outcomes. For example, screening for risky drinking in both community and clinic settings has been shown to be an effective approach to linking at-risk individuals to treatment and in reducing alcohol consumption.^21–23 Similar results have been observed with community and clinic-based screening for depression.²⁴ However, administering recommended screenings, in addition to providing care, requires significant time and staffing resources. It is estimated that if primary care providers screened all of their patients for health problems as recommended, an additional 7 h a day would be spent addressing patients’ preventive services alone.^25,26

While some screening protocols, like those for cancer, diabetes, or asthma, require diagnostic tests or imaging, behavioral health risks and symptoms require information that must come directly from the patient (patient-reported outcomes), relating to patient symptoms (e.g. feelings of depression or anxiety), behavior (e.g. diet, exercise, smoking, or drinking), or experience (e.g. exposure to violence, poverty).^27–29 For these risks and conditions, patient involvement in providing information using structured screening approaches is crucial. Particularly outside the United States, mHealth initiatives are emerging tools for this type of screening, well accepted by both patients and providers, in the identification and management of chronic and complex illness and can facilitate patient involvement in providing risk information that is responsive to patient experiences and needs, while eliminating issues of provider knowledge and attitudes, accounting for time and personnel scarcity, while aiding clinical decision making.^30–34

This article presents an innovative approach to the collection of patient-reported data on behavioral health risks among patients in an urban, underserved clinic setting. The purpose of this study is to compare screening results to data derived from chart reviews of patients seen prior to the deployment of the screening intervention to determine (1) the rates of unrecognized and undiagnosed depression, PTSD, and risky drinking (referred to, collectively, as “behavioral health problems” for the purposes of this study) in this patient population and (2) whether increased recognition of behavioral health problems in the encounter was associated with appropriate treatment and follow-up of identified patients.

Methods

Study design and setting

We used a quasi-experimental design to assess the rates of a positive screen of depression, PTSD, and risky drinking among adult primary care patients at a Federally Qualified Health Center (FQHC) in southern Connecticut that serves a largely urban population living below the federal poverty line, most of whom used public insurance. On-site behavioral healthcare was available several days a week through referrals to a staff psychiatrist or a psychologist at that site. These three behavioral health problems were selected by clinic administration and staff for this project as being the most urgent within their clinic population.

For the intervention condition, 146 clinic patients seen between September and December 2013 were asked by receptionists to complete the electronic risk screening questionnaire in the waiting room after checking in for their appointment. This electronic questionnaire system, designed by OpenClinica, LLC, and Dimagi, Inc., allowed self-reporting by patients using a touch-screen tablet that supported English and Spanish versions of the instrument, as well as audio versions, with disposable headphones, for patients with low literacy or vision impairment. Patients were told they could complete the screen in the waiting room or in a private examination room, if they preferred, for additional privacy. Screening responses were transmitted through a secure wireless network to a secure server where they were automatically scored. Results and recommendations for referral or further follow-up were summarized on the tablet and made available to the clinician at the time of the encounter. Medical staff received training on interpreting the results of the mental health screening tools used for this project, how to discuss the results with patients to determine whether the results were accurate and clinically relevant, and the expected follow-up intervention (or lack of intervention if the results were determined to be inaccurate) and its documentation in patient charts.

Screenings were completed with 146 of 314 eligible patients (see Figure 1 for an intervention flow chart). All patients who had not previously completed the screening questionnaire and presented to the clinic for a physical examination, an urgent care visit, walk-in visit, routine follow-up, or well visit were eligible for screening. Patients did not participate in the study if they spoke a language other than English or Spanish, had a disability that prevented completion of the screen, or were in acute distress. Patients who did not complete the screening were typically seen during periods when the clinic was very busy or when receptionist coverage was limited.

Figure 1.

Flow chart of clinic patients during study period.

Results for patients in the intervention group were compared to a control group of 129 primary care patients who were seen at the same clinic in 2 weeks of August 2013 prior to the implementation of the risk screening intervention. Measures of depression, PTSD, and problem drinking were collected from systematic reviews of patients’ medical records.

There was no evidence to suggest that these three behavioral health disorders or their symptoms might vary seasonally due to the timing of the control group (August) and of the intervention group (September–December).

Measures

The measures and data sources for the intervention and control conditions are summarized in Table 1. For the intervention group, the primary outcome measures consisted of screening results obtained during the target appointment. Depression was assessed with the 9-item Patient Health Questionnaire (PHQ-9),³⁵ which yields a score that indicates minimal, mild, moderate, moderately severe, and severe depression, over the previous 2 weeks. Participants whose scores ranged from the mild to severe range were considered depressed in this study. PTSD was assessed using the four questions from the PTSD module of My Mood Monitor (M-3),³⁶ which was created for primary care providers as an electronic screen specifically for patients with co-morbid disorders. Results from this screening instrument consist of a binary measure of PTSD. Risky drinking was assessed using the 3-item Alcohol Use Disorders Identification Test–Consumption (AUDIT-C),³⁷ which yields measures of heavy episodic or “binge” drinking as well as excess alcohol consumption on a weekly basis. All three behavioral health measures were selected primarily because they were validated in this population and are used commonly in primary care settings, but also because they are brief and easy to understand and answer in a population with diverse literacy skills.

Table 1.

Measures and data sources table.

Experimental group	Problem in 6 months prior	Target appointment	Tx/FU 6 months following
Intervention	• EHR:   Any mention of problem in Problem List or notes in any appointment in last 6 months   Follow-up care recommended	• Screening results:   PHQ-9   My Mood Monitor   AUDIT-C • EHR:   Screening results noted by MA   Follow-up care recommended	• EHR:   Any mention of problem in problem list or notes in any appointment in 6 months post target appointment   Any mention of problem in problem list or notes in any appointment in 6 months post target appointment
Control	• EHR:   Any mention of problem in Problem List or notes in any appointment in last 6 months   Follow-up care recommended	• EHR:   Any mention of problem in Problem List or notes   Follow-up care recommended	• EHR:   Any mention of problem in problem list or notes in any appointment in 6 months post target appointment   Any mention of problem in problem list or notes in any appointment in 6 months post target appointment

EHR: electronic health record; PHQ-9: Patient Health Questionnaire-9; AUDIT-C: Alcohol Use Disorders Identification Test–Consumption; MA: Medical Assistant.

For the control condition, the presence or absence of depression, PTSD, or alcohol problems in the target appointment was determined based on the problem list and clinical notes sections of the patient’s electronic health record (EHR). Any mention of these problems or disorders during the target appointment (e.g. “X reported feeling depressed” in the clinical notes, or mention of depression in the problem list) resulted in the patient scoring positive for this problem.

For patients in both the intervention and control, the EHR was systematically reviewed (1) to determine rates of PTSD, depression, and risky drinking in the 6 months prior to the target appointment and (2) to document any treatment or follow-up for these problems in the 6 months following the target appointment. Any mention of these problems in either the problem list or in the progress notes for any appointment occurring in the 6 months prior to the target appointment resulted in a positive score for the problem. A patient was considered to have received treatment or follow-up for each of these problems if there was a notation that the provider had counseled the patient, scheduled a follow-up appointment for the problem, made a referral to another provider, or prescribed medication.

EHR review procedures

The project manager and trained research assistants reviewed all records using dual data entry techniques such that a minimum of two reviewers abstracted every chart. In accordance with accepted procedures,^38–41 cases were defined as patients having been seen in the clinic in August 2013, for the control condition, or in September–November 2013, in the intervention condition. Both reviewers examined the 129 charts of the control condition and 146 of the intervention condition. The senior author and the research team adjudicated the small number of coding discrepancies between reviewers.

Prior to resolution of coding discrepancies, inter-rater reliabilities were calculated for the list and note entries for each diagnosis (alcohol problems, PTSD, Depression). Because identification of and treatment for these three disorders were relatively rare, we used the Gwet AC1⁴² to assess inter-rater agreement. In situations involving low prevalence, traditional measures of reliability, such as Cohen’s Kappa, can result in very low, or even negative, values when overall agreement is very high.^42,43 Inter-rater reliabilities for all six disorder/entry-type combinations and the three list-note composites ranged between 0.97 and 1.0 indicating a very high level of reliability.

Analysis

SPSS 22.0 statistical analysis software (SPSS, INC., Chicago, IL) was used to analyze all quantitative data. Logistic regression models for the binary outcome variables were used to examine the intervention effects for each disorder. For the logistic regression analysis predicting identification of PTSD in the target appointment, a Firth bias correction using a penalized likelihood estimation method was used to address separation issues.⁴⁴ All significance tests were two-tailed (p < 0.05).

Power analyses were performed using SAS Version 9.2. Given the baseline incidence of these conditions, and with a power of 0.8, we anticipated being able to detect a 10% difference in prevalence between groups of 150 or greater. Post hoc power analysis indicated that recruited sample sizes were adequate to demonstrate power greater than 0.8.

Results

The demographic characteristics of participants are presented in Table 2. Overall, patients in the intervention and control conditions were similar. A majority of patients were Hispanic. Two-thirds of patients were female, with half of patients receiving Medicare or Medicaid and a quarter uninsured. Most had been patients in the clinic for longer than 6 months and had been seen by a clinic provider in the previous 6 months. Differences in age distribution of intervention and control patients were observed, with a greater proportion of younger patients in the intervention group. Demographic characteristics of patients seen in the clinic during the intervention period who did not complete the risk screening assessment are also presented in Table 2. Those who did and did not complete the screening during the intervention period shared a similar demographic profile, but there were significantly more African Americans in the Not Screened group.

Table 2.

Patient demographics by experimental group.

	Control n = 129		Intervention n = 146		Not screened N = 168		Interv v. Control	Screened v. Not Screened
	N	%	N	%	N	%	p-value	p-value
Gender							.44	.75
Male	40	31	52	35.6	57	33.7
Female	89	69	94	64.4	111	66.3
Age							.05	.2
18–24	10	7.8	24	16.4	13	7.7
25–29	6	4.7	18	12.3	22	13.1
30–39	35	27.1	41	28.1	45	26.8
40–49	31	24	26	17.8	35	20.1
50–64	33	25.6	29	19.9	40	23.8
65+	12	9.3	8	5.5	13	7.7
Race/ethnicity							.38	.04
White	43	33.3	36	24.7	26	17.3
African American	15	11.6	20	13.7	35	20.8
Hispanic/Latino	68	52.7	87	59.6	97	57.7
Other	3	2.3	3	2.1	10	6
Insurance							.13	.19
Public	74	57.4	70	48	93	55.4
Private	19	14.7	35	24	27	16
Self-pay	36	27.9	41	28	48	28.6
Patient status							.55	.38
Existing	118	91.5	130	89	144	85.7
New	11	8.5	16	11	24	14.3

Significance determined by chi-square analysis. Significant p-values at the <.05 level are in bold italics.

Table 3 contrasts rates of behavioral health problems identified during the target appointment through patient screening (intervention) with those identified by providers during the clinical encounter and recorded in the patient’s chart (control). Using logistic regression analysis with age as a covariate indicates that much higher rates of behavioral health problems were identified through patient screening than were identified by providers during the encounter. Depression was more than five times more likely to be identified among intervention as opposed to control patients (odds ratio (OR) = 5.3, 95% confidence interval (CI) = 2.5, 11.3). PTSD was unrecognized among patients in the control condition, yet its prevalence in the intervention condition exceeded 28% (OR = 105.6, 95% CI = 6.5, >999). Alcohol abuse was 3.5 times more likely to be identified among intervention patients, but this difference was not statistically significant. There was no statistical difference between the intervention and control groups in terms of identification of behavioral health disorders in the 6 months prior to the target appointment.

Table 3.

Prevalence of behavioral health problems in target appointment and in 6 months prior to target appointment.

	Target appointment			6 Months prior
	I	II	III	IV	V	VI
	Intervention n = 146	Control n = 129	OR (95% CI)	Intervention n = 146	Control n = 129	OR (95% CI)
Alcohol abuse	8.2% (12)	2.3% (3)	3.50 (0.95, 12.92)	2.7% (4)	4.7% (6)	0.65 (0.17, 2.40)
Depression	29.5% (43)	7.8% (10)	5.33 (2.50, 11.33)	15.8% (23)	18.6% (24)	0.94 (0.49, 1.80)
PTSD	28.1% (41)	0	105.6 (6.48, >999)	2.1% (3)	0	7.47 (0.44, 128.4)

OR: odds ratio; CI: confidence interval; PTSD: post-traumatic stress disorder. Significant p-values at the <.05 level are in bold italics.

Odds ratios and 95% confidence intervals were derived from logistic regression models with age as a covariate. Because of the lack of PTSD cases identified in the control condition, logistic regression analyses were conducted using the Firth correction for this outcome.³³

Although patient screening resulted in much higher rates of identification of those with behavioral health problems, rates of documentation in patients’ EHRs and rates of follow-up care remained very low. Screening results for one-third of all patients in the intervention group were not entered into the patient records. Among these patients, only one patient, who had screened positive for depression, had that finding noted, and none of these patients received any follow-up care. Table 4 presents rates of EHR documentation and follow-up care among the two-thirds of intervention patients whose screening results were entered into their medical records and reviewed by their provider. Among those screening positive for any of the three behavioral health disorders, few patients had their results included in the notes section of the EHR during the target appointment. Even fewer had their screening result recorded in the EHR problem list, or received follow-up treatment. Among patients screening positive for depression, 39% had their screening results included in the notes section of the EHR during the target appointment, 9% had their screening result included in the problem list, and only 18% were provided any follow-up care. In contrast, patients screening positive for alcohol problems and PTSD were much less likely to have had their screening results included in either the EHR notes or list, and only two patients screening positive for PTSD received follow-up care. It should be emphasized that these low rates of documentation and follow-up were observed despite the fact that the vast majority, of the cases identified through screening were “new” cases andwere not previously reflected in the patient’s EHR.

Table 4.

Identification of behavioral health problem and follow-up among those identified.

Behavioral health problem	% Positive in target appointment (n = 98)	% of those identified that were new cases	For those patients identified with the problem in the target appointment
			% in notes	% in list	% with follow-up
Alcohol	6.1% (6)	100% (6)	16.7% (1)	0	0
Depression	33.7% (33)	78.8% (26)	39.4% (13)	9.1% (3)	18.2% (6)
PTSD	32.7% (32)	96.9% (31)	12.5% (4)	6.3% (2)	6.3% (2)

PTSD: post-traumatic stress disorder; EHR: electronic health record.

Data in this table are restricted to patients whose screening results were entered into the EHR at the time of the encounter.

Discussion

In this study, we demonstrate that automated, tablet-based screenings using validated, patient-reported outcomes are effective in identifying three behavioral health disorders in an urban, underserved population, many of which were previously unrecognized. To our knowledge, this is the first study within the United States demonstrating the effectiveness of electronic, patient-reported screening for identifying behavioral health problems in an urban safety-net primary care setting.

Although this approach holds great potential for assisting primary care clinicians in addressing the significant and often unrecognized behavioral health needs of patients that are typically seen in FQHCs and other safety net settings, our results also indicate that screening alone is not sufficient to ensure that patients receive adequate follow-up care, which is consistent with the findings of previous research. Even in a clinic routinely screening for, and addressing, behavioral health issues, newly identified behavioral health issues were not included in many patient’s problem list, a critical portion of the medical record for codifying and monitoring patient health.^45,46 Moreover, though some patients might not have required any follow-up care, that decision was often not noted within patient records. For those who did receive a positive screening result and who did not have a previous diagnosis, few of these patients were referred for treatment, despite on-site behavioral health specialists in this clinical setting.

This study has several limitations. First, the small sample size of patients in a single clinical setting limits the generalizability of these findings, and we were only successful in screening 47% of the eligible patients. Second, while our intervention group was demographically comparable to control patients, we had limited information on patients who declined to use the electronic tablet screening and why. Clinic staff was responsible for asking patients to complete the screening questionnaire, and information on reasons for refusal was not systematically collected. Although this approach replicated the real-life workflow of the clinic, we cannot be sure that there are not some significant demographic or health differences in patients who were not screened. Third, the use of the patient chart as a proxy for the content of the clinical encounter can be problematic, as it may not fully reflect the content of the clinical encounter, including whether patients felt they could and would be truthful about risk factors.^28,29 Finally, our screening measure for PTSD, derived from the M-3, has shown strong sensitivity and specificity in primary care samples but limited positive predictive value.³⁶

Although screening is a critical first step for identifying patients at risk, it alone is not sufficient to ensure adequate care. Future studies should seek to understand how such information can be integrated into a clinical workflow that supports clinicians in both recognizing and responding to the behavioral health needs of patients. Despite its limitations, this study presents a promising approach to identifying previously unrecognized behavioral health problems in a challenging patient population and lays the groundwork for future work to improve clinical outcomes within both medical and behavioral health settings. By collecting patient-reported outcome measures during patients’ idle time in the waiting room, this approach addresses screening requirements in an efficient and pragmatic way and yields health information that is unlikely to be generated in a typical clinical encounter.