Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Apr 1.
Published in final edited form as: J Affect Disord. 2018 Jan 3;230:50–55. doi: 10.1016/j.jad.2017.12.068

Measurement of Depression Treatment among Patients Receiving HIV Primary Care: Whither the Truth?

Bethany L DiPrete 1,*, Brian W Pence 1, David J Grelotti 2, Bradley N Gaynes 3
PMCID: PMC5889117  NIHMSID: NIHMS937723  PMID: 29407538

Abstract

Background

Prescription records, manual chart review, and patient self-report are each imperfect measures of depression treatment in HIV-infected adults.

Methods

We compared antidepressant prescription records in an electronic data warehouse with antidepressant treatment and psychotherapy identified via manual chart review and self-report for patients at 6 academic HIV treatment centers. We examined concordance among these three sources, and used latent class analysis (LCA) to estimate sensitivity and specificity of each measure.

Results

In our charts sample (n=586), 59% had chart indication of “any depression treatment” and 46% had a warehouse prescription record. Antidepressant use was concordant between charts and data warehouse for 77% of the sample. In our self-report sample (n=677), 52% reported any depression treatment and 43% had a warehouse prescription record. Self-report of antidepressant treatment was consistent with prescription records for 71% of the sample. LCA estimates of sensitivity and specificity for “any depression treatment” were 67% and 90% (warehouse), 87% and 75% (self-report), and 96% and 77% (chart).

Limitations

There is no gold standard to measure depression treatment. Antidepressants may be prescribed to patients for conditions other than depression. The results may not be generalizable to patient populations in non-academic HIV clinics. Regarding LCA, dependence of errors may have led to overestimation of sensitivity and specificity.

Conclusions

Prescription records were largely concordant with self-report and chart review, but there were discrepancies. Studies of depression in HIV-infected patients would benefit from using multiple measures of depression treatment or correcting for exposure misclassification.

Keywords: Depression, HIV, Antidepressive agents, Self Report, Electronic Health Records, Misclassification

Introduction

Depression is the most common mental health comorbidity among persons living with HIV (PLWH), with estimates of 20-30% affected (Bing et al., 2001; Ciesla and Roberts, 2001; Pence et al., 2006). Depression in PLWH is associated with adverse health and behavior outcomes, including reduced adherence to antiretroviral treatment (ARV) (Gonzalez et al., 2011; Mugavero et al., 2006), reduced CD4 counts, increased viral load (Pence et al., 2007), and increased rates of AIDS-related morbidity and mortality (Gonzalez et al., 2011; Horberg et al., 2008; Todd et al., 2017). However, depression is under-diagnosed and frequently untreated in HIV-infected patients (Asch et al., 2003; O’Cleirigh et al., 2015; Pence et al., 2012). Treatment interventions for depression, including treatment with antidepressants and mental health counseling, are shown to be effective in HIV-infected individuals (Himelhoch and Medoff, 2005; Himelhoch et al., 2007). Some studies suggest that depression treatment improves HIV medication adherence and viral suppression (Safren et al., 2016; Sin and DiMatteo, 2014; Tsai et al., 2010), although the evidence remains mixed (Pence et al., 2015; Pyne et al., 2011; Tsai et al., 2013). A better understanding of the impact of depression treatment on HIV outcomes in large, generalizable populations is crucial for defining the importance of improved mental health access for people living with HIV.

One avenue for gathering large-scale evidence on the impact of depression treatment on HIV outcomes is to leverage electronic health records (EHRs). EHRs make it possible to use prescription records to characterize depression treatment on a large scale and are increasingly used in depression research (Cholera et al., 2017; Zuniga et al., 2015). However, depression treatment characterized through EHR prescription records may be inconsistent with more labor-intensive measurement approaches, such as manual chart review or patient self-report. Electronic medication histories may be incomplete or contain discrepancies (Coletti et al., 2015; Lee et al., 2014; Linsky and Simon, 2013; Stewart and Lynch, 2012), especially among individuals taking many medications (Coletti et al., 2015; Lee et al., 2014; Linsky and Simon, 2013) or among those who receive prescriptions from offsite providers (Madden et al., 2016). Additionally, electronic prescription data may not capture patients who receive mental health counseling without antidepressant treatment, or may capture patients who are prescribed antidepressants for conditions other than depression, such as patients prescribed antidepressants for anxiety or smoking cessation. Therefore, a measure of depression treatment indicated solely by an electronic prescription record could result in misclassification of depression treatment for HIV-infected individuals, affecting the validity of conclusions.

Provider notes may capture patient information regarding counseling or other treatment that is not distinguished in electronic prescription records, and serve to provide context for the patient narrative (Kobus et al., 2013; Turchin et al., 2009). However, provider notes in the EHR may not be comprehensive due to factors such as patient volume or the type of information sought (Kobus et al., 2013; Pakhomov et al., 2008; Turchin et al., 2009), and chart abstraction can be labor-intensive and challenging to standardize. Patient report has been shown to be a useful supplement to EHR data (Coletti et al., 2015; Kozak et al., 2012; Pakhomov et al., 2008), as documentation of treatment for depression in EHRs is less robust than medical documentation in other areas of healthcare, and clinical information on mental health is frequently missing (Kobus et al., 2013; Madden et al., 2016; Tomines et al., 2013). While electronic prescription records may miss individuals with prescriptions from an offsite clinic or those who receive counseling without antidepressants, patient reports provide the opportunity to capture this information. But patient reports may contain inaccuracies or be subject to missing information because of patient unwillingness to report particular types of sensitive information or because patients may be unfamiliar with their diagnosis or treatment regimens (Lee et al., 2014; Pakhomov et al., 2008; Stewart and Lynch, 2012).

Each of these sources may provide imperfect information for some portion of patients receiving treatment for depression when used alone as a measure of treatment. Electronic prescription records housed in data warehouses are increasingly used in depression research (Bengtson et al., 2016b; Cholera et al., 2017), but little information exists regarding how well this data source captures patients receiving depression treatment. Thus, consideration must be taken to understand what is being measured and what patient data is missed by each data source when conducting research on depression treatment among PLWH using electronic prescription data. This study seeks to measure how accurately each data source, and electronic prescription records in particular, can identify patients receiving treatment for depression, including antidepressants or mental health counseling. To this end, this paper evaluates concordance of depression treatment as measured from EHR antidepressant prescription data, manual chart review including provider notes, and patient self-report to inform research practices on depression treatment among PLWH.

Methods

Study population and data collection

Data come from the Center for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS) observational clinical cohort. CNICS is a collaboration of eight academically affiliated HIV clinics across the United States (Kitahata et al., 2008). Each clinic has established a database capturing information routinely collected by electronic health records and other institutional data systems. To date, the CNICS cohort includes over 32,000 HIV-infected adults in routine HIV clinical care. Nearly all patients consent to have their data captured. Data are de-identified and uploaded to a central CNICS repository on a quarterly basis. CNICS data elements include demographics, medications, health care utilization, clinical diagnoses, laboratory values and vital signs, vital status, antiretroviral resistance, and biologic specimens. CNICS sites have also integrated patient-reported outcomes (PROs) into routine clinical care to assess depression and anxiety, adherence, drug and alcohol use, sexual risk behaviors, symptom burden, physical activity level, body morphology, and quality of life. Data quality procedures have been previously described (Kitahata et al., 2008). Data collection procedures are approved by the institutional review boards (IRBs) at each site, and all participants provide written informed consent. Ethical approval for these analyses was provided by the IRB at the University of North Carolina at Chapel Hill.

For the purposes of this validation study, two new PRO items were integrated at 6 CNICS sites for 54-288 patients per site between 2013 and 2015 (total n: 677). The new PRO items, specifying the past three months, asked participants (1) whether they had seen any providers (psychiatrist; other doctor who prescribes antidepressant medications; therapist, counselor, or psychologist; alternative provider; or other provider) for mental health counseling, antidepressants, or other treatment and (2) whether they had taken antidepressants, herbs, homeopathic medicines, or other medicines for depression. Patients answered yes or no to each type of provider and each type of medication.

At each of the 6 sites, 46-50 patients selected randomly from those who had completed the new PROs (sample A) and 48-50 patients selected randomly from all patients keeping an HIV primary care appointment in the prior 12 months (sample B) were identified for manual chart review (n=586 total). Trained and experienced abstractors reviewed medical records and chart notes from HIV and on-site behavioral health providers for the 12-month period prior to the PRO (for sample A) or prior to the attended appointment which led to the patient’s inclusion in the sample (for sample B). Abstractors recorded any indication that the patient was receiving antidepressant treatment, counseling for depression, or mental health treatment of unknown type during the review period, including indications that the patient was receiving treatment offsite (e.g. from a community mental health agency). A chart indication of mental health treatment was classified as being of unknown type if the patient was recorded as receiving mental health treatment, but the provider had not made any note regarding the specific type of treatment the patient was receiving. Treatment was classified according to whether it was being provided in the HIV clinic, in the larger health system whose records would be captured in the CNICS data warehouse, or outside of the health system. Abstractors were instructed not to include meetings with clinic social workers unless there was a clear indication that the purpose of the meeting was to address depression, since social workers at these clinics frequently meet with patients to address a wide range of social service needs.

Measures

Depression was measured via the Patient Health Questionnaire-9 (PHQ-9), (Kroenke et al., 2001) which was included as part of the PRO administration. The PHQ-9 assesses presence of the nine DSM-V criteria symptoms for depression in the past two weeks and is well-validated among HIV-infected adults (Crane et al., 2010). A PHQ-9 score ≥10 (on a scale of 0-27) is indicative of probable major depressive disorder (Kroenke and Spitzer, 2002; Kroenke et al., 2001; Mulvaney-Day et al., 2017).

Our goal was to compare depression treatment measured from antidepressant prescriptions captured in the CNICS data warehouse to depression treatment measured from the PROs and chart review. Antidepressant prescription in the data warehouse was defined as a record of a prescription of any selective serotonin reuptake inhibitor, serotonin-norepinephrine reuptake inhibitor, or other second-generation antidepressant (bupropion or mirtazapine) that was active during the time period covered by the PRO (3 months) or the chart review (12 months). For the PRO and chart review data, we constructed separate measures of any antidepressant prescription, any mental health counseling, and any depression treatment (medication or counseling). For chart review data, we created these measures for both the full 12-month period, to compare to the data warehouse, and for only the 3 months corresponding to the PRO recall period.

Analyses

Table 1 displays the two-way comparisons considered between each of the three data sources: data warehouse, PROs, and chart records. As each of these data sources is likely to have strengths and weaknesses, we did not consider any one of them to be a reference standard.

Table 1.

Details of measures compared

Sources compared Time frame N Constructs compared
Data warehouse and Chart review Past 12 months 586 Warehouse: Antidepressant prescription, compared to
Chart: Antidepressant prescription
Warehouse: Antidepressant prescription, compared to
Chart: Counseling
Warehouse: Antidepressant prescription, compared to
Chart: Any depression treatment
Data warehouse and PROs Past 3 months 677 Warehouse: Antidepressant prescription, compared to
PRO: Antidepressant prescription
Warehouse: Antidepressant prescription, compared to
PRO: Counseling
Warehouse: Antidepressant prescription, compared to
PRO: Any depression treatment
Chart review and PROs Past 3 months 301 Chart: Antidepressant prescription, compared to
PRO: Antidepressant prescription
Chart: Counseling, compared to
PRO: Counseling
Chart: Any depression treatment, compared to
PRO: Any depression treatment
Open in a new tab

For participants with both warehouse and PRO information (n=677), we compared (1) warehouse versus PRO indication of antidepressant prescription, (2) warehouse record of antidepressant prescription versus PRO indication of counseling and (3) warehouse record of antidepressant prescription versus PRO indication of any depression treatment.

For participants with both warehouse and chart review information (n=586), we compared (1) warehouse versus chart records of antidepressant prescription, (2) warehouse record of antidepressant prescription versus chart record of counseling, and (3) warehouse record of antidepressant prescription versus chart record of any depression treatment. We examined these comparisons in the overall sample as well as separately in the subsamples that were selected from PRO completers (sample A) and from all clinic attendees (sample B).

Finally, for participants with both PRO and chart review information (n=301), we compared (1) PRO versus chart record of antidepressant prescription, (2) PRO versus chart record of counseling, and (3) PRO versus chart record of any depression treatment.

As each of these three data sources is likely error-prone, we applied latent class analysis (LCA) to estimate the sensitivity and specificity of any depression treatment measured from each data source relative to the latent, or not directly measured, construct of true depression treatment status. LCA is used when multiple imperfect measures can be leveraged to triangulate on the desired but unmeasured true construct (Walter and Irwig, 1988), and has been used previously to estimate the performance of mental health measures among PLWH (Pence et al., 2009). LCA was fit using the disk operating system program Latent.exe (developed by Steven Walter, walter@mcmaster.ca; personal communication, May 28, 2007). All other analyses were conducted using Stata 14 (StataCorp: College Station, TX).

Results

Participants in the PRO (n=677) and chart review (n=586) samples were majority male (81% and 81%, respectively), white non-Hispanic (51% and 53%), on ART (91% and 89%), had median CD4 counts >500 cells/mm3 (59% and 59%), and had an undetectable viral load (74% and 73%). The median age in each sample was 47 years. The prevalence of depression in each sample was high: 42% of patients in the PRO sample and 45% in the chart review sample had a PHQ9 ≥10 (Table 2).

Table 2.

Characteristics of the study samples

Characteristic PRO sample*
N=677
N(%) or median (IQR)		 Chart review sample†
N=586
N(%) or median (IQR)
Age 47 (37, 53) 47 (39, 53)
Gender
 Male 551 (81.4) 472 (80.6)
 Female 126 (18.6) 111 (18.9)
Race/ethnicity
 White, non-Hispanic 342 (51.1) 307 (53.2)
 Black, non-Hispanic 259 (38.7) 183 (31.7)
 Hispanic 47 (7.0) 62 (10.6)
 Other 21 (3.1) 25 (4.3)
On ART 609 (90.5) 507 (88.6)
CD4 count, cells/mm3
 ≤200 72 (10.6) 65 (11.2)
 201-500 204 (30.2) 171 (29.4)
 >500 400 (59.2) 346 (59.5)
Viral load
 Undetectable, <50 copies/mL 498 (73.7) 427 (73.4)
 Detectable, ≥50 copies/mL 178 (26.3) 155 (26.6)
PHQ9 ≥10 269 (41.6) 241 (44.9)
Open in a new tab
*

Missing data: race/ethnicity 1.2%, on ART 0.6%, CD4 0.2%, viral load 0.2%, PHQ9 4.3%

†

Missing data: age 0.5%, gender 0.5%, race/ethnicity 1.5%, ART 2.4%, CD4 0.7%, viral load 0.7%, PHQ9 8.2%

Among participants who completed the PRO (n=677), 52% reported any depression treatment in the past 3 months, including antidepressant treatment alone (14%), counseling alone (8%), or both (30%) (Table 3). Of these same participants, 43% had a record of an antidepressant in the CNICS data warehouse during the corresponding 3-month window. Of the participants with chart abstraction data (n=586), 59% had a record of any treatment for depression during the 12-month chart abstraction period, including antidepressant treatment alone (42%), counseling alone (4%), or both (13%), and 46% had a CNICS warehouse record of an antidepressant during the corresponding 12-month period (Table 3). Three percent of patients were classified as receiving mental health treatment of unknown type at some point during follow up (data not shown).

Table 3.

Measures of depression treatment frequency

Depression treatment measure N (%)
Patient-reported outcome (PRO): Past 3 months* (n=677)
 Any depression treatment 354 (52.3)
  Antidepressants alone 96 (14.2)
  Counseling alone 57 (8.4)
  Both antidepressants and counseling 201 (29.9)
Chart review: Past 12 months (n=586)
 Any depression treatment 346 (59.0)
  Antidepressants alone 244 (41.6)
  Counseling alone 25 (4.3)
  Both antidepressants and counseling 77 (13.1)
CNICS data warehouse†
In PRO sample (n=677): Any antidepressant, past 3 months 285 (43.3)
In chart review sample (n=586): Any antidepressant, past 12 months 262 (45.9)
Open in a new tab
*

Missing data: self-reported antidepressant use 0.6%

†

Missing data: PRO sample 2.8%, chart sample 2.6%

Overall, chart records and the data warehouse had consistent information regarding antidepressant use for 77% (442) of patients in the past 12 months (Table 4). Of patients with a chart indication of an antidepressant prescription, 72% were classified in the data warehouse as taking an antidepressant; of those with no chart report of an antidepressant, 84% were classified as not taking an antidepressant. Individuals receiving only offsite antidepressant treatment accounted for 6% of individuals with a chart indication of antidepressant use. Thus, most discrepancies between chart report of antidepressant use and warehouse prescriptions were not due to offsite prescriptions. Concordance between chart record of “any depression treatment” and the warehouse classification was very similar to the concordance between chart record of antidepressant treatment and the warehouse classification. Although 17% of participants had a chart record of counseling, few were receiving counseling without also having a record of an antidepressant (Table 3). Concordance between warehouse and chart review was similar when considering the two different samples of patients selected for chart review (data not shown).

Table 4.

Concordance of measures of depression treatment

Data warehouse: Any antidepressant?*
N (Row %)
Yes No
Chart review: Past 12 months
Any antidepressant?
 Yes 221 (71.5)   88 (28.5)
 No   41 (15.7) 221 (84.4)
Any counseling?
 Yes   62 (62.6)   37 (37.4)
 No 200 (42.4) 272 (57.6)
Any depression treatment?
 Yes 226 (67.7) 108 (32.3)
 No   36 (15.2) 201 (84.8)
Patient-reported outcome: Past 3 months
Any antidepressant?
 Yes 189 (66.6)   95 (33.5)
 No   95 (25.7) 275 (74.3)
Any counseling?
 Yes 149 (58.9) 104 (41.1)
 No 136 (33.6) 269 (66.4)
Any depression treatment?
 Yes 209 (61.3) 132 (38.7)
 No   76 (24.0) 241 (76.0)
Open in a new tab
*

Missing data: Chart sample 2.6%, PRO sample 2.8%

Patient-report of antidepressant use was less consistent than chart records with warehouse data (Table 4). Of patients who self-reported antidepressant use, 67% were classified as taking an antidepressant in the data warehouse; and of those who did not self-report taking an antidepressant, 74% did not have an antidepressant prescription record in the data warehouse. Mental health counseling was reported by 38% (253) of patients and of those, 41% had no record of an antidepressant prescription in the data warehouse. Overall, concordance of self-report of any depression treatment with warehouse prescription records was slightly less than concordance of self-report of antidepressant treatment with warehouse prescription records: 61% of patients reporting “any depression treatment” had a warehouse antidepressant prescription, but 76% of patients who denied treatment had no evidence of a prescription record in the data warehouse.

Additionally, we compared PROs with chart review data for the 301 patients who completed the PRO and for whom a chart review was completed (Table 5). This comparison was the least consistent with respect to antidepressant use. Of patients who reported antidepressant use, 57% had a chart record of antidepressant treatment; of those who denied antidepressant treatment, 78% had no chart record of an antidepressant prescription. Discrepancies between PROs and chart data stemmed more from patients who self-reported antidepressant use but had no evidence of antidepressant treatment in the chart record (24%) than from those who denied antidepressant treatment but had a chart record of treatment (10%). PROs and chart data were less consistent for mental health counseling. While there was concordance between patient self-report and chart record of mental health counseling for 56% of patients, most patients who reported counseling had no indication in the chart record (84%). Overall, patient report of “any depression treatment” was consistent with the chart record for the majority of patients (65%). Among patients who self-reported depression treatment, 58% had a chart indication of treatment; of those who denied treatment, 77% had no chart record of depression treatment.

Table 5.

Concordance of PRO and chart review (n=301)

Chart Review
N (Row %)
Yes No
PRO
Any antidepressant?*
 Yes 94 (56.6) 72 (43.4)
 No 30 (22.4) 104 (77.6)
Any counseling?
 Yes 25 (16.3) 128 (83.7)
 No 3 (2.0) 145 (98.0)
Any depression treatment?
 Yes 112 (58.0) 81 (42.0)
 No 25 (23.2) 83 (76.9)
Open in a new tab
*

Missing data: PRO 0.3%

In the subsample of patients represented in all three data sources (n=292), LCA estimates of sensitivity and specificity for capturing the true latent construct of any depression treatment were 67% (95% CI: 0.62-0.71) and 90% (0.85-0.94) for prescription records in the CNICS data warehouse, 87% (0.84-0.91) and 75% (0.69-0.81) for self-report of any depression treatment, and 96% (0.93-0.98) and 77% (0.70-0.84) for chart indication of any depression treatment. When warehouse, self-report, and chart review were limited to measures of antidepressant treatment only, LCA estimates of sensitivity and specificity for measurement of any antidepressant use were 69% (0.64-0.73) and 91% (0.87-0.95) for the warehouse, 81% (0.77-0.85) and 87% (0.83-0.92) for patient self-report, and 92% (0.89-0.95) and 84% (0.77-0.90) for chart indication (Table 6).

Table 6.

Latent class analysis estimates of sensitivity and specificity of each measure (n=292)

Any Depression Treatment
Measure Estimated Sensitivity 95% C.I. Estimated Specificity 95% C.I.
Warehouse 0.67 0.62 – 0.71 0.90 0.85 – 0.94
PRO 0.87 0.84 – 0.91 0.75 0.69 – 0.81
Chart 0.96 0.93 – 0.98 0.77 0.70 – 0.84
Antidepressant Treatment
Measure Estimated Sensitivity 95% C.I. Estimated Specificity 95% C.I.
Warehouse 0.69 0.64 – 0.73 0.91 0.87 – 0.95
PRO 0.81 0.77 – 0.85 0.87 0.83 – 0.92
Chart 0.92 0.89 – 0.95 0.84 0.77 – 0.90
Open in a new tab

Discussion

In this study, we compared the concordance of antidepressant treatment as captured in electronic health record databases with manual chart review and patient self-report of depression treatment. The majority of patients had consistent treatment information in the medical chart, CNICS data warehouse, and patient reports. Chart records and the data warehouse were most consistent—three quarters of patients with chart abstraction data had both warehouse prescription records of antidepressants and provider notes documenting treatment of depression with antidepressants, or had no evidence of antidepressant treatment in both data sources. Yet, discrepancies were present in each data source comparison. Over a quarter of patients who self-reported antidepressant treatment or who had documentation of antidepressants in the medical chart did not have a warehouse prescription record in the corresponding time window. In comparisons between chart data and warehouse data, the majority of inconsistent data stemmed from patients without a prescription record but with chart evidence of antidepressant treatment. This is consistent with previous evidence of inconsistencies and missing data from medication records (Coletti et al., 2015; Lee et al., 2014; Linsky and Simon, 2013; Stewart and Lynch, 2012).

LCA results suggested that chart indication of depression treatment had the highest sensitivity, whereas warehouse prescription records had the lowest. Warehouse prescription records had the highest specificity while patient self-report had the lowest. These estimates are consistent with the hypothesis that warehouse prescription records underestimate depression treatment by missing individuals receiving treatment offsite or psychotherapy alone, but are less likely than chart records or PROs to misclassify untreated individuals as receiving depression treatment.

Increasingly, studies are using routinely captured electronic medical record data to address questions such as the effect of depression treatment on clinical outcomes (Bengtson et al., 2016a; Sowa et al., 2016). Thus, it is important to be able to account for misclassification errors stemming from the use of electronic medical records. Yet few studies have examined the accuracy of measurement of depression treatment in such large databases (Kobus et al., 2013; Madden et al., 2016). To our knowledge, this study is among the first to compare multiple measures of depression treatment in HIV-infected patients from different documentation sources. Each data source provides rich patient data, but each is also an imperfect representation of a patient’s full medical picture. Warehouse prescription records provide vast quantities of data on large numbers of patients that are easily accessible by researchers. However, prescription records may have inconsistencies with PROs and chart reviews, which are more labor-intensive. Inconsistencies may arise because prescriptions for antidepressants might be provided for reasons other than depression treatment (e.g., bupropion for smoking cessation), and prescription records may not document the rationale for the treatment or capture prescriptions obtained offsite. Prescription records also do not capture mental health counseling and do not account for patient non-adherence. Conversely, patient reports are imperfect because the patient’s definition of treatment in self-report may conflict with the study definition. For example, patients may have a broader definition of counseling and thus reported counseling that was not in fact evidence-based, sufficient-dose counseling for depression. Similarly, HIV providers may record evidence of depression treatment in their chart note based on conversations with their patients, but the definition of treatment may again not match the study definition of treatment. However, both chart notes and patient self-report may provide evidence of offsite management of depression or evidence-based counseling that may not be captured in the data warehouse.

This study highlights that defining depression treatment as treatment with antidepressants may miss those patients who receive treatment in the form of mental health counseling without antidepressants or who receive treatment offsite. Studies have shown that counseling is an effective form of treatment for depression among patients infected with HIV (Himelhoch et al., 2007; Sin and DiMatteo, 2014); however, not all counseling is protocolled, evidence-based therapy. Only 4% of participants based on chart review and 8% based on self-report were receiving counseling without medication, and the concordance of warehouse prescriptions with “any depression treatment” was similar to its concordance with antidepressant treatment alone. LCA estimates suggested that although warehouse prescription records have high specificity, they have only moderate sensitivity. Thus, to reduce misclassification errors, future studies would benefit from using multiple measures of depression treatment to identify HIV-infected patients who are being treated for depression. The present study provides a basis for applying quantitative bias analysis methods to correct for measurement error in studies that examine depression treatment and health outcomes among PLWH. While there is no gold standard to measure depression treatment, and thus no perfect estimate of misclassification, the information presented in this paper may provide guidance on the choice of probabilities of misclassification in future studies.

Several study limitations exist that must be considered when interpreting the results of this study. First, there is no gold standard for sources of data on depression treatment. Data warehouse prescription records, manual chart reviews, and patient reports are three imperfect sources. Patients may receive prescriptions for antidepressants for conditions other than depression, thus would be misclassified as receiving depression treatment. Second, there was inconclusive information about prescription dates in the data warehouse for a small subset of patients, who thus were not included in analyses. These patients did not have information about day or month of the antidepressant prescription, so we could not verify whether their prescription was active during the defined chart or PRO window. Self-report of treatment for depression was restricted to antidepressant use or contact with a mental health professional, thus responses indicating alternative treatments such as acupuncture or herbal remedies were excluded. This was deemed a reasonable exclusion as a minimal number of patients reported these alternative treatments, but further analyses may benefit from including alternative therapies in the definition of treatment interventions for depression. Regarding LCA, the validity of the sensitivity and specificity estimates is based on the assumption that classification errors (false positives and false negatives) are independent between the three tests, an assumption likely violated in this situation; such dependence of errors may have led to overestimation of sensitivity and specificity (Torrance-Rynard and Walter, 1997). Finally, while the study includes patients from 6 sites across the United States, the results may not be generalizable to patient populations in non-academic HIV clinics.

In summary, we found that prescription records are largely in agreement with patient self-report and manual chart review as measures of depression treatment, but some differences between the measures were noted. Defining treatment for depression as the use of antidepressants may result in misclassification of some patients who receive offsite medications or mental health counseling alone; however, in this study the proportion receiving counseling alone was small. To improve measurement of depression treatment, future studies should consider using multiple measures of depression treatment or applying quantitative bias analysis to correct for exposure misclassification.

Highlights.

  • There was concordance between the three data sources for the majority of patients.

  • Prescription records may miss those receiving offsite treatment or counseling alone.

  • Studies should use multiple measures of treatment or correct for misclassification.

Acknowledgments

BWP and BNG contributed to the acquisition of data. BWP conceived and designed the experiments. BLD and BWP analyzed the data. BLD drafted the manuscript. BWP, BNG, and DJG assisted with the interpretation of the data and critically revised the manuscript for important intellectual content. All authors take responsibility for and approve the final version of the manuscript. We thank the National Institutes of Mental Health [grant number R01MH100970] and the National Institute of Allergy and Infectious Diseases [grant numbers R24AI067039 and P30 AI50410] for their support of this work.

We thank the National Institutes of Mental Health [grant number R01MH100970] and the National Institute of Allergy and Infectious Diseases [grant numbers R24AI067039 and P30 AI50410] for their support of this work.

Role of the Funding Source:

This work was supported by the National Institutes of Mental Health [grant number R01MH100970 to BWP] and by the National Institute of Allergy and Infectious Disease [grant numbers R24AI067039, P30AI50410].

Funding:

This work was supported by the National Institutes of Mental Health [grant number R01MH100970 to BWP] and by the National Institute of Allergy and Infectious Disease [grant numbers R24AI067039, P30AI50410].

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributors:

BWP and BNG contributed to the acquisition of data. BWP conceived and designed the experiments. BLD and BWP analyzed the data. BLD drafted the manuscript. BWP, BNG, and DJG assisted with the interpretation of the data and critically revised the manuscript for important intellectual content. All authors take responsibility for and approve the final version of the manuscript.

Institutional Review Board:

Data collection procedures are approved by the institutional review boards (IRBs) at each Center for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS) site. Ethical approval for these analyses was provided by the Institutional Review Board at the University of North Carolina at Chapel Hill.

Conflict of interest:

Brian W. Pence has received a speaking honorarium from MSD. All other authors declare that they have no conflicts of interest.

References

  1. Asch SM, Kilbourne AM, Gifford AL, Burnam MA, Turner B, Shapiro MF, Bozzette SA, Consortium, H. Underdiagnosis of depression in HIV: who are we missing? J Gen Intern Med. 2003;18:450–460. doi: 10.1046/j.1525-1497.2003.20938.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bengtson AM, Pence BW, Crane HM, Christopoulos K, Fredericksen RJ, Gaynes BN, Heine A, Mathews WC, Moore R, Napravnik S, Safren S, Mugavero MJ. Disparities in Depressive Symptoms and Antidepressant Treatment by Gender and Race/Ethnicity among People Living with HIV in the United States. PLOS ONE. 2016a;11:e0160738. doi: 10.1371/journal.pone.0160738. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bengtson AM, Pence BW, Gaynes BN, Quinlivan EB, Heine AD, O’Donnell JK, Crane HM, Mathews WC, Moore RD, Westreich D, O’Cleirigh C, Christopoulos K, Mimiaga MJ, Mugavero MJ. Improving Depression Among HIV-Infected Adults: Transporting the Effect of a Depression Treatment Intervention to Routine Care. J Acquir Immune Defic Syndr. 2016b;73:482–488. doi: 10.1097/QAI.0000000000001131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, Turner BJ, Eggan F, Beckman R, Vitiello B, Morton SC, Orlando M, Bozzette SA, Ortiz-Barron L, Shapiro M. Psychiatric Disorders and Drug Use Among Human Immunodeficiency Virus–Infected Adults in the United States. Archives of General Psychiatry. 2001;58:721–728. doi: 10.1001/archpsyc.58.8.721. [DOI] [PubMed] [Google Scholar]
  5. Cholera R, Pence BW, Bengtson AM, Crane HM, Christopoulos K, Cole SR, Fredericksen R, Gaynes BN, Heine A, Mathews WC, Mimiaga MJ, Moore R, Napravnik S, O’Clerigh C, Safren S, Mugavero MJ. Mind the Gap: Gaps in Antidepressant Treatment, Treatment Adjustments, and Outcomes among Patients in Routine HIV Care in a Multisite U.S. Clinical Cohort. PLoS One. 2017;12:e0166435. doi: 10.1371/journal.pone.0166435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Ciesla JA, Roberts JE. Meta-Analysis of the Relationship Between HIV Infection and Risk for Depressive Disorders. AJP. 2001;158:725–730. doi: 10.1176/appi.ajp.158.5.725. [DOI] [PubMed] [Google Scholar]
  7. Coletti DJ, Stephanou H, Mazzola N, Conigliaro J, Gottridge J, Kane JM. Patterns and predictors of medication discrepancies in primary care. J Eval Clin Pract. 2015;21:831–839. doi: 10.1111/jep.12387. [DOI] [PubMed] [Google Scholar]
  8. Crane PK, Gibbons LE, Willig JH, Mugavero MJ, Lawrence ST, Schumacher JE, Saag MS, Kitahata MM, Crane HM. Measuring depression levels in HIV-infected patients as part of routine clinical care using the 9-item patient health questionnaire (PHQ-9) AIDS care. 2010;22:874–885. doi: 10.1080/09540120903483034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Gonzalez JS, Batchelder AW, Psaros C, Safren SA. Depression and HIV/AIDS Treatment Nonadherence: A Review and Meta-analysis. J Acquir Immune Defic Syndr. 2011;58 doi: 10.1097/QAI.0b013e31822d490a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Himelhoch S, Medoff DR. Efficacy of Antidepressant Medication among HIV-Positive Individuals with Depression: A Systematic Review and Meta-Analysis. AIDS Patient Care and STDs. 2005;19:813–822. doi: 10.1089/apc.2005.19.813. [DOI] [PubMed] [Google Scholar]
  11. Himelhoch S, Medoff DR, Oyeniyi G. Efficacy of group psychotherapy to reduce depressive symptoms among HIV-infected individuals: a systematic review and meta-analysis. AIDS Patient Care STDS. 2007;21:732–739. doi: 10.1089/apc.2007.0012. [DOI] [PubMed] [Google Scholar]
  12. Horberg MA, Silverberg MJ, Hurley LB, Towner WJ, Klein DB, Bersoff-Matcha S, Weinberg WG, Antoniskis D, Mogyoros M, Dodge WT, Dobrinich R, Quesenberry CP, Kovach DA. Effects of Depression and Selective Serotonin Reuptake Inhibitor Use on Adherence to Highly Active Antiretroviral Therapy and on Clinical Outcomes in HIV-Infected Patients. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2008;47:384–390. doi: 10.1097/QAI.0b013e318160d53e. [DOI] [PubMed] [Google Scholar]
  13. Kitahata MM, Rodriguez B, Haubrich R, Boswell S, Mathews WC, Lederman MM, Lober WB, Van Rompaey SE, Crane HM, Moore RD, Bertram M, Kahn JO, Saag MS. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. Int J Epidemiol. 2008;37:948–955. doi: 10.1093/ije/dym231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kobus AM, Harman JS, Do HD, Garvin RD. Challenges to depression care documentation in an EHR. Fam Med. 2013;45:268–271. [PubMed] [Google Scholar]
  15. Kozak MS, Mugavero MJ, Ye J, Aban I, Lawrence ST, Nevin CR, Raper JL, McCullumsmith C, Schumacher JE, Crane HM, Kitahata MM, Saag MS, Willig JH. Patient Reported Outcomes in Routine Care: Advancing Data Capture for HIV Cohort Research. Clin Infect Dis. 2012;54:141–147. doi: 10.1093/cid/cir727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Kroenke K, Spitzer RL. The PHQ-9: A new depression diagnostic and severity measure. Psychiat Ann. 2002;32:509–515. [Google Scholar]
  17. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lee KP, Nishimura K, Ngu B, Tieu L, Auerbach AD. Predictors of Completeness of Patients’ Self-reported Personal Medication Lists and Discrepancies With Clinic Medication Lists. Ann Pharmacother. 2014;48:168–177. doi: 10.1177/1060028013512109. [DOI] [PubMed] [Google Scholar]
  19. Linsky A, Simon SR. Medication discrepancies in integrated electronic health records. BMJ Quality & Safety. 2013;22:103–109. doi: 10.1136/bmjqs-2012-001301. [DOI] [PubMed] [Google Scholar]
  20. Madden JM, Lakoma MD, Rusinak D, Lu CY, Soumerai SB. Missing clinical and behavioral health data in a large electronic health record (EHR) system. Journal of the American Medical Informatics Association. 2016:ocw021. doi: 10.1093/jamia/ocw021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Mugavero M, Ostermann J, Whetten K, Leserman J, Swartz M, Stangl D, Thielman N. Barriers to Antiretroviral Adherence: The Importance of Depression, Abuse, and Other Traumatic Events. AIDS Patient Care and STDs. 2006;20:418–428. doi: 10.1089/apc.2006.20.418. [DOI] [PubMed] [Google Scholar]
  22. Mulvaney-Day N, Marshall T, Downey Piscopo K, Korsen N, Lynch S, Karnell LH, Moran GE, Daniels AS, Ghose SS. Screening for Behavioral Health Conditions in Primary Care Settings: A Systematic Review of the Literature. J Gen Intern Med. 2017 doi: 10.1007/s11606-017-4181-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. O’Cleirigh C, Magidson JF, Skeer MR, Mayer KH, Safren SA. Prevalence of Psychiatric and Substance Abuse Symptomatology Among HIV-Infected Gay and Bisexual Men in HIV Primary Care. Psychosomatics. 2015;56:470–478. doi: 10.1016/j.psym.2014.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Pakhomov S, Jacobsen SJ, Chute CG, Roger VL. Agreement between Patient-reported Symptoms and their Documentation in the Medical Record. Am J Manag Care. 2008;14:530–539. [PMC free article] [PubMed] [Google Scholar]
  25. Pence BW, Gaynes BN, Adams JL, Thielman NM, Heine AD, Mugavero MJ, McGuinness T, Raper JL, Willig JH, Shirey KG, Ogle M, Turner EL, Quinlivan EB. The effect of antidepressant treatment on HIV and depression outcomes: results from a randomized trial. AIDS. 2015;29:1975–1986. doi: 10.1097/QAD.0000000000000797. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Pence BW, Miller WC, Gaynes BN. Prevalence estimation and validation of new instruments in psychiatric research: an application of latent class analysis and sensitivity analysis. Psychol Assess. 2009;21:235–239. doi: 10.1037/a0015686. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Pence BW, Miller WC, Gaynes BN, Eron JJ. Psychiatric Illness and Virologic Response in Patients Initiating Highly Active Antiretroviral Therapy. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2007;44:159–166. doi: 10.1097/QAI.0b013e31802c2f51. [DOI] [PubMed] [Google Scholar]
  28. Pence BW, Miller WC, Whetten K, Eron JJ, Gaynes BN. Prevalence of DSM-IV-Defined Mood, Anxiety, and Substance Use Disorders in an HIV Clinic in the Southeastern United States. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2006;42:298–306. doi: 10.1097/01.qai.0000219773.82055.aa. [DOI] [PubMed] [Google Scholar]
  29. Pence BW, O’Donnell JK, Gaynes BN. Falling through the cracks: The gaps between depression prevalence, diagnosis, treatment, and response in HIV care. AIDS. 2012;26:656–658. doi: 10.1097/QAD.0b013e3283519aae. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Pyne JM, Fortney JC, Curran GM, Tripathi S, Atkinson JH, Kilbourne AM, Hagedorn HJ, Rimland D, Rodriguez-Barradas MC, Monson T, Bottonari KA, Asch SM, Gifford AL. Effectiveness of collaborative care for depression in human immunodeficiency virus clinics. Archives of internal medicine. 2011;171:23–31. doi: 10.1001/archinternmed.2010.395. [DOI] [PubMed] [Google Scholar]
  31. Safren SA, Bedoya CA, O’Cleirigh C, Biello KB, Pinkston MM, Stein MD, Traeger L, Kojic E, Robbins GK, Lerner JA, Herman DS, Mimiaga MJ, Mayer KH. Cognitive behavioural therapy for adherence and depression in patients with HIV: a three-arm randomised controlled trial. Lancet HIV. 2016;3:e529–e538. doi: 10.1016/S2352-3018(16)30053-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Sin NL, DiMatteo MR. Depression Treatment Enhances Adherence to Antiretroviral Therapy: A Meta-Analysis. Ann Behav Med. 2014;47:259–269. doi: 10.1007/s12160-013-9559-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Sowa NA, Bengtson A, Gaynes BN, Pence BW. Predictors of depression recovery in HIV-infected individuals managed through measurement-based care in infectious disease clinics. J Affect Disord. 2016;192:153–161. doi: 10.1016/j.jad.2015.12.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Stewart AL, Lynch KJ. Identifying discrepancies in electronic medical records through pharmacist medication reconciliation. Journal of the American Pharmacists Association. 2012;52:59–68. doi: 10.1331/JAPhA.2012.10123. [DOI] [PubMed] [Google Scholar]
  35. Todd JV, Cole SR, Pence BW, Lesko CR, Bacchetti P, Cohen MH, Feaster DJ, Gange S, Griswold ME, Mack W, Rubtsova A, Wang C, Weedon J, Anastos K, Adimora AA. Effects of Antiretroviral Therapy and Depressive Symptoms on All-Cause Mortality Among HIV-Infected Women. Am J Epidemiol. 2017;185:869–878. doi: 10.1093/aje/kww192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Tomines A, Readhead H, Readhead A, Teutsch S. Applications of electronic health information in public health: uses, opportunities & barriers. EGEMS (Wash DC) 2013;1:1019. doi: 10.13063/2327-9214.1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Torrance-Rynard VL, Walter SD. Effects of dependent errors in the assessment of diagnostic test performance. Stat Med. 1997;16:2157–2175. doi: 10.1002/(sici)1097-0258(19971015)16:19<2157::aid-sim653>3.0.co;2-x. [DOI] [PubMed] [Google Scholar]
  38. Tsai AC, Karasic DH, Hammer GP, Charlebois ED, Ragland K, Moss AR, Sorensen JL, Dilley JW, Bangsberg DR. Directly observed antidepressant medication treatment and HIV outcomes among homeless and marginally housed HIV-positive adults: a randomized controlled trial. Am J Public Health. 2013;103:308–315. doi: 10.2105/AJPH.2011.300422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tsai AC, Weiser SD, Petersen ML, Ragland K, Kushel MB, Bangsberg DR. A marginal structural model to estimate the causal effect of antidepressant medication treatment on viral suppression among homeless and marginally housed persons with HIV. Arch Gen Psychiatry. 2010;67:1282–1290. doi: 10.1001/archgenpsychiatry.2010.160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Turchin A, Shubina M, Breydo E, Pendergrass ML, Einbinder JS. Comparison of Information Content of Structured and Narrative Text Data Sources on the Example of Medication Intensification. Journal of the American Medical Informatics Association : JAMIA. 2009;16:362–370. doi: 10.1197/jamia.M2777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Walter SD, Irwig LM. Estimation of test error rates, disease prevalence and relative risk from misclassified data: a review. J Clin Epidemiol. 1988;41:923–937. doi: 10.1016/0895-4356(88)90110-2. [DOI] [PubMed] [Google Scholar]
  42. Zuniga JA, Yoo-Jeong M, Dai T, Guo Y, Waldrop-Valverde D. The Role of Depression in Retention in Care for Persons Living with HIV. AIDS Patient Care and STDs. 2015;30:34–38. doi: 10.1089/apc.2015.0214. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES