Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Sep 3.
Published in final edited form as: AIDS Behav. 2013 Oct;17(8):2781–2791. doi: 10.1007/s10461-012-0342-7

Routine Depression Screening in an HIV Clinic Cohort Identifies Patients with Complex Psychiatric Co-morbidities Who Show Significant Response to Treatment

Joseph E Schumacher 1,, Cheryl McCullumsmith 2, Michael J Mugavero 3, Paige E Ingle-Pang 4, James L Raper 5, James H Willig 6, Zhiying You 7, D Scott Batey 8, Heidi Crane 9, Sarah T Lawrence 10, Charles Wright 11, Glenn Treisman 12, Michael S Saag 13
PMCID: PMC4559144  NIHMSID: NIHMS717215  PMID: 23086427

Abstract

This study described characteristics, psychiatric diagnoses and response to treatment among patients in an outpatient HIV clinic who screened positive for depression. Depressed (25 %) were less likely to have private insurance, less likely to have suppressed HIV viral loads, had more anxiety symptoms, and were more likely to report current substance abuse than not depressed. Among depressed, 81.2 % met diagnostic criteria for a depressive disorder; 78 % for an anxiety disorder; 61 % for a substance use disorder; and 30 % for co-morbid anxiety, depression, and substance use disorders. Depressed received significantly more treatment for depression and less HIV primary care than not depressed patients. PHQ-9 total depression scores decreased by 0.63 from baseline to 6-month follow-up for every additional attended depression treatment visit. HIV clinics can routinely screen and treat depressive symptoms, but should consider accurate psychiatric diagnosis as well as co-occurring mental disorders.

Keywords: Depression, HIV, Clinic, Treatment, Cohort

Background

Persons living with HIV/AIDS (PLWHA) have high rates of co-morbid depression for a complex blend of pre-existing and illness specific reasons [1, 2]. While persons with depression or other psychiatric illness may be more vulnerable to high risk behaviors contributing to HIV transmission, HIV infection can contribute to unique psychosocial stressors as well as neurobiological changes in the brain caused by HIV or antiretroviral treatment side effects [3-5]. PLWHA, as compared to persons with other chronic diseases, experience more stigma and discrimination [6], punishment beliefs [7], and social isolation/loneliness [8].

Prevalence and Predictors of Psychiatric Disorders Among PLWHA

Psychiatric disorders, generally, and depressive disorders specifically, are more common among PLWHA compared to the general population in the U.S. [9]. Among PLWHA in care, both psychiatric disorders and substance use have been found to be very common in both national and regional patient populations. A nationally representative probability sample from the United States of 2,864 adults receiving HIV care in 1996 showed that approximately 48 % had any psychiatric disorders in the previous 12 months, including major depression (36 %), dysthymia (27 %), generalized anxiety disorder (16 %), and panic attack (11 %) [9], compared to a prevalence in the general population of 8 % for major depression and 2 % for generalized anxiety disorder [3]. A large study from an HIV cohort in the Southeast United States found a 12 month prevalence of 39 % with mood/anxiety diagnoses, and 21 % with substance use diagnoses, including 8 % with co-morbid mood and substance use diagnoses. Of patients with mood/anxiety diagnoses, 76 % had clinically relevant depression and 11 % had post traumatic stress disorder (PTSD) [2]. Among 350 patients attending two county-based HIV primary care clinics in Southern California, 38 % of HIV-infected patients met screening criteria for depression, 34 % for PTSD, and 43 % for anxiety. Thirty-eight percent screened positive for two or more disorders [1]. Among 1,774 patients in the University of Washington HIV-cohort during 2004, 63 % had a mental illness diagnosis (including mood, anxiety, psychotic, or personality disorders), 45 % had a substance use disorder, and 38 % had both [10]. Major depressive disorder is the most common psychiatric co-morbidity of HIV infection, with estimates of lifetime prevalence of up to 45 % [11].

Predictors of depression, addiction and other mental disorders among PLWHA has been associated with unemployment, unstable housing, detectable viral load and smoking [12.] Greater emotional distress, substance use, and a higher number of baseline stressful experiences were significantly associated with reporting a greater number of incident stressful experiences and any traumatic experiences among persons with HIV [13.] While the diagnosis of a life-threatening illness can be an extremely stressful, traumatic experience, the quality of social support, patients’ coping strategies and several indicators of mental and physical health were consistently associated with post-traumatic growth [14.]

Screening for Depression Using the PHQ-9 Among Patients Receiving HIV Care

The Patient Health Questionnaire (PHQ-9) is a reliable and valid measure of depression severity [15, 16]. The feasibility of depression screening using the PHQ-9 on touch-screen-based tablets was demonstrated in a busy HIV clinic [17]. The PHQ-9 has been used to assess the prevalence of depressive symptoms as a part of standard care in a cross-sectional study among PLWHA at an urban HIV clinic [18]. The PHQ-9 has also been used for timely detection and intervention to address suicidal ideation among patients in HIV care [19]. Although the PHQ-9 may be a useful instrument to incorporate into routine clinical care for the detection and treatment of depression among PLWHA, potential challenges include time and resources constraints, lack of expertise in psychopathology, and lack of available mental health treatment services [20].

Treatment for Depression

Olatunji et al. [21] reviewed existing literature on psychosocial and psychopharmacologic treatment of depression in the context of HIV and concluded that such treatments of depression appear to be effective for individuals with HIV. Safren et al. [22] demonstrated the positive effect of a specific cognitive behavioral treatment for HIV treatment adherence and depression. They reported improvements in medication adherence and depression and improvements in plasma HIV RNA concentrations. Ferrando [23] added that devising a treatment plan for depression, clinicians should take into account stage of HIV illness, co-morbid illnesses such as Hepatitis B and C, the potential for drug interactions with antiretroviral therapy, and other medications used to treat HIV and patient preference. Soller [24] found 46.6 % of low-income HIV-infected patients with psychiatric comorbidity saw a psychiatrist and/or were prescribed a psychiatric medication.

The UAB 1917 Clinic Depression Study

The purpose of this study was to investigate routine depression screening in an HIV Clinic cohort and identify patients with complex psychiatric co-morbidities who show significant response to treatment. The UAB (University of Alabama at Birmingham) 1917 HIV Outpatient Clinic (1917 Clinic) provides comprehensive outpatient health care to PLWHA in Birmingham, Alabama, where an increasingly high prevalence of HIV has been observed [25]. Among 1,244 active patients at the 1917 Clinic in 2006, 52 % (647) had a documented mental health need in their medical record. While mental health services were available at the clinic by referral, depression screens had not been routine processes in daily clinic operations.

Methodology

Study Setting

This study was conducted among patients in the UAB 1917 Clinic Cohort, a longitudinal observational cohort of HIV-infected patients who receive primary care from the University of Alabama at Birmingham at the 1917 Clinic in Birmingham, Alabama. The 1917 Clinic provides comprehensive, interdisciplinary HIV medical care and is one of eight clinical sites within the Center for AIDS Research Network of Integrated Clinical Systems (CNICS), a National Institutes of Health-funded (R24) collaborative research infrastructure that integrates a broad range of clinical data collected through point-of-care electronic medical record (EMR) systems across multiple academic medical centers engaged in the longitudinal care of individuals with HIV/AIDS [26].

Study Participants

HIV-infected patients over 18 years of age who attended the 1917 Clinic for a routinely scheduled primary care appointment between November 1, 2008 and July 17, 2010 were eligible for participation. Patients were approached by a research assistant in the waiting room before their clinic appointment and invited to participate in the study. Patients who declined to provide informed consent (N = 9) were not included in the study. After obtaining informed consent, the research assistant gave the participants brief instructions on the use of the touch screens computers and asked them to complete the assessment. Patients were asked to complete a brief assessment of patient reported outcomes (PROs) every four to six months which included the PHQ-9 to screen for depression. Assessments were done at the time of scheduled routine clinical care appointments; none were done at the time of urgent care or walk-in same-day appointments (“sick calls”). Patients also had to complete two PRO assessments at least four to 6 months apart during the 21-month study period to be included in this study. The study protocol was approved by the University of Alabama at Birmingham Institutional Review Board.

Design

This study used a prospective observational, quasi-experimental design. Patients completed a clinical assessment that included the PHQ-9 to screen for depression symptoms during their primary care visits, usually every 4–6 months. HIV and depression treatment visits were collected from the EMR for a period of 6 months from baseline PRO assessment. Patients were stratified and analyzed by depression status by baseline PRO assessment.

Clinical Assessment of PROs

Patients at CNICS sites completed a PROs battery of clinical assessments with a series of standardized and validated instruments measuring clinically relevant domains such as depression, anxiety, current symptoms, antiretroviral medication adherence, alcohol, tobacco, and drug use, and sexual risk behavior on touch-screen computers or tablets using an open-source, web-based survey software application. The administration of this PRO battery took a mean of 10.0 min (SD 4.4 min) and a median of 9.6 min, range of 7–18 min, and mean completion time for the PHQ-9 was less than 1 min.

The clinical assessment was integrated into routine clinical care at UAB in April 2008. The PRO assessments were administered using touch-screen desktop computers in the clinic waiting or examination rooms. Those with vision impairment or inadequate literacy or inability to complete the assessment were aided by research assistants from the UAB Center for AIDS Research Behavioral Science Core.

The integration of the PRO clinical assessment into the clinic workflow was done carefully to avoid the disruption of routine clinic operations through supervised negotiation of the process. Touch-screen computers were available where patients passively waited during regular visits (e.g., waiting and examination rooms). Upon arrival to the scheduled clinic visit, each patient was given a four-digit temporary code to access their PRO session (ticket number) or was logged in by a staff member. Patients could pause and rejoin the PRO assessment on the same or different computers by reentering their ticket number [19]. All computers were linked to the cohort network server, and no identifying or assessment data was stored on any touch-screen computer.

Clinical Assessment Instrument Scoring

The PHQ-9 from the Primary Care Evaluation of Mental Disorders (PRIME-MD) Patient Health Questionnaire was used to screen for symptoms of depression as part of every clinical assessment [27]. Reliability and validity of the PHQ-9 have been established [15, 27-29], as well as responsiveness to psychopharmacological depression treatment [30]. Patients indicate for each of nine depressive symptoms, whether during the previous 2 weeks the symptom has bothered them “not at all,” “several days,” “more than half the days,” or “nearly every day.” Symptoms are consistent with DSM-IV criteria for major depression. Standard PHQ-9 scores range from 0-27 and are categorized as: none (0–4 points), mild (5–9 points), moderate (10–14 points), moderately-severe (15–19 points), and severe (≥20 points) depressive symptom severity. A positive screen for depression was defined as a PHQ-9 total score ≥10 (negative screen for depression was a total score of <10) [15, 27, 31]. A depression change score was calculated (PHQ-9 total score at six-month follow-up minus PHQ-9 total score at baseline) where a negative change score would indicate a decrease in the PHQ-9 total score or a reduction in symptoms of depression over time.

Anxiety was measured using the PHQ-5 [29]. Patients are asked if they have had any feelings of sudden fear or panic in the prior 4 weeks. If yes, they are then asked four yes/no questions about the presence of panic attacks or anxiety symptoms. Patients were classified as having no anxiety if they answered no to the first question, anxiety symptoms if they answered yes to the first question, and a panic syndrome if they answered yes to all of the panic/anxiety questions.

Alcohol use was evaluated using the abbreviated version of the alcohol use disorders identification test consumption questions (AUDIT-C) [32]. The AUDIT-C scores were calculated by summing the scores for each AUDIT-C question (0–4 points each). Consistent with previous reports, we used a score of 0 to define patients as no risk, 1–4 as low risk, and 5 or more for at-risk for alcohol abuse [29, 32, 33].

Substance abuse (crack or cocaine, amphetamines, opioids, and injection drug use) was categorized into never, historical, or current (within the previous three months) per the responses to the alcohol, smoking, and substance involvement screening test (ASSIST) [34, 35]. HIV antiretroviral therapy (ART) adherence was measured using an item from the ACTU-4 [36]. Patients were categorized as “ART adherent” if they reported missing no doses in the past 2 weeks and “ART not adherent” if they missed one or more doses in the past 2 weeks.

Mental Disorders

The Structured Clinical Interview for DSM-IV (SCID) was used to identify and characterize the presence mental disorders for descriptive purposes among screened depressed patients with a PHQ-9 scores ≥10. A SCID trained psychologist approached these patients in the clinic after they completed the baseline PRO and administered the SCID for axis I disorders using a standard protocol for administration [37] to a convenience sample of 66 patients. Only 66 of the 399 patients who screened positive for depression were willing and able to be scheduled for this two hour interview during their clinic visit. DSM-IV axis I diagnoses that were found in this sample were categorized into the following groups according to the DSM-IV classification system: anxiety (Anx), bipolar disorder (Bip), depression (Dep), psychotic (Psy), and substance use (Sub). Too few other diagnoses met criteria for categorization.

Electronic Medical Record and Billing Systems

Demographic and clinical characteristics and treatment utilization information were obtained from the 1917 Clinic Cohort EMR. The 1917 Clinic Cohort EMR system conducts 100 % quality assurance reviews [38].

Treatment for Depression and Treatment Variables

Patients who screened positive for depression were referred to psychiatric assessment and psychotherapy and pharmacotherapy treatment at the 1917 Clinic. Depressive symptoms were treated by a psychiatrist according to standard clinical practice. Standard care for mental disorders at the clinic included similar treatment services offered at the request of the medical treatment provider.

Treatment utilization for depression and HIV primary care was recorded by the UAB Health System IDX billing system as number of arrived visits for HIV primary care, sick-call, psychological treatment, and psychiatric treatment visits. Depression treatment variable was calculated as the total number of psychiatric or psychological (psychotherapy) visits attended within the six month observation period.

Analysis

We defined two time points to capture a treatment observation period for study patients: baseline (the patient’s first PRO within the study period or study entry) and follow-up (6 months later with a window of −2 months and +12 months around the patient’s exact 6 month point). Data utilized for the baseline and follow-up comparison was taken from two PRO administrations. The PRO was administered to patients (with consent) in the clinic routinely at each primary care visit. These visits varied by their clinic provider appointments, not by scheduled research follow-up appointments. Clinic visits typically spanned 4–6 months, some longer, and some patients returned after a year or not at all. Treatment utilization data was abstracted from baseline over 6 months for all patients, regardless of when their PRO was administered. Loss to follow-up was defined as those patients with a baseline PRO who did not return for a primary care visit during the follow-up window. Descriptive statistics such as mean and standard deviation and proportion were provided as appropriate. Tables and figures were employed to present the results. The χ2 test was performed for comparison of proportions and the two-sample t test for means. The Sattertwaite method was used in case of unequal variance. Linear regression models was used to examine the association between change in depression score (PHQ-9 total score follow-up minus baseline) and depression treatment. Analyses were adjusted for other covariates of interest.

Results

Study Sample

Figure 1 shows that there were a total of 1,579 patients who had primary care visits at the clinic and were approached to complete the PRO assessment battery during the 21-month study period. Of these, nine patients declined to participate, leaving enrollment of 1,570 into the study that completed the PRO battery at least once (at baseline). From the baseline PRO, 399 (25 %) of the 1,570 enrolled patients screened positive for depression on the PHQ-9 (“depressed”) and 1,171 (75 %) were “not depressed.” There were a total of 750 patients (247 depressed and 503 not depressed) with only one PRO (at baseline) who were lost to follow up as a result of not having a follow-up PRO within the follow-up window. As such, there were a total of 820 patients (152 depressed and 668 not depressed) patients who had a second PRO within the follow-up window.

Fig. 1.

Fig. 1

Flow diagram of enrollment, stratification (by depression screening), follow-up, and analysis

Demographic, Depression and Other Clinical Characteristics

Table 1 shows the demographic and clinical characteristics for the total sample by baseline depression screening results. The study sample was mostly male (79 %), African American or other race (52 %), uninsured (55 %), receiving ART (81 %), and had a suppressed HIV viral load (52 %). Mean depression and anxiety baseline scores were in the mild severity category. Among all participants, 12 % were at risk for alcohol problems, 5 % reported current substance abuse, and 30.7 % prior substance abuse.

Table 1.

Demographic and clinical characteristics of study sample

Variable Total sample (N = 820) Not depressed (N = 668) Depressed (N = 152) Test statistics* P*
Age, mean years (SD) 43.9 (10.3) 44.0 (10.5) 43.5 (8.9) 0.68 0.4976
Sex
 Female 170 (20.7) 139 (20.8) 31 (20.4)
 Male 650 (79.3) 529 (79.2) 121 (79.6) 0.0129 0.9096
Race
 AA or other 426 (52.0) 355 (53.1) 71 (46.7)
 Caucasian 394 (48.1) 313 (46.9) 81 (53.3) 2.0530 0.1519
Insurance
 Other 452 (55.1) 346 (51.8) 106 (69.7)
 Private 368 (44.9) 322 (48.2) 46 (30.3) 16.1107 <0.0001
CD4 cell count, mean cells/mm3 (SD) 497.9 (298.4) 501.7 (291.2) 481.4 (328.3) 0.76 0.4502
Receiving ART
 No 154 (18.8) 125 (18.7) 29 (19.1)
 Yes 663 (80.9) 540 (80.8) 123 (80.9) 0.0064 0.9361
HIV load
 <50 copies/mL (suppressed) 428 (52.2) 361 (54.0) 67 (44.1)
 ≥50 copies/mL (unsuppressed) 392 (47.8) 307 (46.0) 85 (55.9) 4.9259 0.0265
Depression score (PHQ-9), mean (SD) 4.1 (5.8) 1.6 (2.5) 14.7 (4.1) 37.96 <0.0001
Anxiety score (PHQ-9A), mean (SD) 0.71 (1.6) 0.35 (1.16) 2.3 (2.3) 10.11 <0.0001
Anxiety symptoms (PHQ-9A)
 No anxiety 674 (82.9) 601 (90.0) 73 (48.0)
 Panic syndrome 66 (8.1) 22 (3.3) 44 (29.0)
 Anxiety symptoms 73 (8.9) 39 (5.8) 34 (22.4) 165.5032 <0.0001
Alcohol risk score (AUDIT-C), mean (SD) 1.8 (2.1) 1.8 (2.0) 1.7 (2.2) 0.49 0.6234
Alcohol abuse risk (AUDIT-C)
 At risk 101 (12.3) 85 (12.7) 16 (10.5)
 Low risk 413 (50.4) 341 (51.1) 72 (47.4)
 No risk 293 (35.7) 232 (34.7) 61 (40.1) 1.8323 0.4001
Substance abuse risk (ASSIST)
 Current 41 (5.0) 24 (3.4) 17 (11.2)
 Prior 252 (30.7) 190 (28.4) 62 (40.8)
 Never 516 (62.9) 447 (66.9) 69 (45.4) 29.7984 <0.0001
*

Based on the χ2 test for proportion and the t test for mean

At baseline depressed and not depressed patient groups had significantly different insurance status (χ2 = 16.1107, P < 0.0001), HIV viral loads (t = 4.9259, P = 0.0265), and rates of anxiety (χ 2 = 165.5032, P < 0.0001), and substance abuse (χ2 = 29.7984, P < 0.0001). Depressed patients were less likely to have private insurance, less likely to have suppressed HIV viral loads, had more anxiety symptoms, and were more likely to report current substance abuse than non-depressed patients.

Description of Mental Disorders

Among 66 patients who had PHQ-9 scores of 10 or greater at baseline and who underwent SCID diagnostic interviews, criteria for 236 separate axis I disorders were met across all patients interviewed. All but one patient had at least one axis I mental disorder, and most patients had two or more axis I diagnoses, and 19 % did not meet criteria for any depressive disorder. The most common DSM-IV axis I diagnoses (prevalence of 10 or more diagnoses) by classification category were 300.02 (generalized anxiety disorder) and 309.81 (post traumatic stress disorder) for anxiety disorders; 296.32 (major depression, recurrent, moderate) and 296.34 (major depression, recurrent, severe, with psychotic features) for depressive disorders; and 303.90 (alcohol dependence) and 304.20 (cocaine dependence) for substance use disorders.

Figure 2 shows the percentage of patients with diagnoses in each diagnostic category. Depressive disorders were diagnosed in 81 % of patients, 78 % were diagnosed with anxiety disorders, and 61 % with substance use disorders. Additionally, 12 % had a psychotic disorder diagnosis, and 13 % had a bipolar disorder diagnosis.

Fig. 2.

Fig. 2

Percentage of 66 patients with depression by PHQ-9 screen with DSM-IV Axis mental disorders by category. Categories include anxiety (Anx), bipolar disorder (Bip), depression (Dep), psychotic (Psy), and substance use (Sub). N = 66

Figure 3 shows the percentage of patients with co-occurring mental disorders. Most patients interviewed met criteria for more than one DSM-IV axis I mental disorders (92.8 %). Nearly one-third of patients (30.4 %) were diagnosed with co-morbid anxiety and depressive and substance use disorders. Over a quarter (28.9 %) of the sample was diagnosed with anxiety plus depressive disorders. few patients were diagnosed with depression only (4.4 %) or a substance use disorder only (1.5 %), and no patients were diagnosed with anxiety or psychotic disorders only. Five patients met criteria for diagnoses in four diagnostic categories.

Fig. 3.

Fig. 3

Percentage of 66 screened (PHQ-9) depressed patients with DSM-IV Axis mental disorders by category combination. Combinations include anxiety plus depression (A + D), anxiety plus depression plus substance use (A + D + S), and depression plus substance use (D + S)

Treatment Exposure and Utilization

Mental health and HIV treatment exposure and utilization during the patient’s 6 month period after baseline assessment for the total sample and by depression status is shown in Table 2. Mean number of visits scheduled and arrived and percentage of visits arrived are shown in four treatment categories (HIV primary care, psychiatry, psychology, and sick-calls) compared by depression screening groups. For mental health treatment, the results revealed that over a 6 month observation period, patients from the total sample averaged 2.9 arrived visits for psychological services and 1.2 visits for psychiatric services. Percentage of screened depressed and not depressed patients who received any psychiatric or psychological treatment is shown in Fig. 3. Depressed patients received significantly more treatment for depression than not depressed patients (t = −2.88, P = 0.0041).

Table 2.

Mental health and HIV Treatment exposure and utilization during the patient’s 6 month treatment period for the total sample and compared by depression screening status

Service Total sample (N = 820) Not depressed (N = 668) Depressed (N = 152) t value* P*
Psychological services
Mean (SD) arrived visits 2.9 (3.0) 2.4 (2.8) 3.3 (3.1) −1.59 0.1118
Psychiatric services
Mean (SD) arrived visits 1.2 (1.3) 0.9 (0.9) 1.5 (1.5) −2.88 0.0041
HIV primary care
Mean (SD) arrived visits 2.3 (1.2) 2.6 (1.2) 2.2 (1.1) 3.74 0.0002
Sick call
Mean (SD) arrived visits 1.2 (0.7) 1.2 (0.8) 1.2 (0.7) −0.33 0.7391
*

Based the two-sample t test and the Sattertwaite method was used in case on unequal variance

Patients arrived for an average of 2.3 HIV primary care visits and 1.2 sick-call visits. Depressed patients attended significantly fewer HIV primary care visits than not depressed patients (t = 3.74, P = 0.0002). No differences were found in sick-call visits by depression screening status (t = −0.33, P = 0.7391).

Relationship Between Depression Treatment and Depression

Table 3 shows the univariate and multivariate relationships between depression treatment (number of attended psychiatric and psychological treatment visits) and change in PHQ-9 total score from baseline to follow-up (negative numbers indicate reduction in depression symptoms). The univariate relationship (regression coefficient = −0.6304) between change in depression symptoms and depression treatment was significant (t = −7.45, P < 0.0001) indicating that an increase in the number of depression treatment visits was associated with a reduction in depression. The magnitude and significance of this relationship did not change as a result of adjusting for baseline demographics (age, sex, race, and insurance), ART adherence, and HIV disease status (CD4 count at baseline). These findings suggest that there is a −0.6304 reduction in PHQ-9 total depression score from baseline to follow-up for every additional attended depression treatment visit.

Table 3.

Association between change in depression score (PHQ-9 total score follow-up minus baseline) and depression treatment with/without adjustment for other covariates, using linear regression model

Univariate
DFa model
DF + ART
DF + ART + CD4
Reg. coef t value P value Reg. coef t value P value Reg. coef t value P value Reg. coef t value P value
Depression treatment visits −0.6304 −7.45 <0.0001 −0.6227 −7.32 <0.0001 −0.6236 −7.34 <0.0001 −0.6235 −7.33 <0.0001
Age at baseline 0.0041 0.26 0.7937 0.0046 0.29 0.7681 0.0046 0.28 0.7758
Female versus male 0.3469 0.86 0.3884 0.3492 0.86 0.3883 0.3501 0.85 0.3938
White versus other −0.0928 −0.28 0.7809 −0.1331 −0.40 0.6909 −0.1318 −0.39 0.6967
Private insurance versus other 0.4106 1.28 0.2020 0.3985 1.24 0.2166 0.3991 1.22 0.2222
ART medication use 0.1442 0.55 0.5821 0.1463 0.56 0.5776
CD4 at baseline 0.0000 0.02 0.9811
a

DF demographic factors, ART 0 if not taking medication at baseline and follow-up, 1 if taking medication at baseline or follow-up, and 2 if taking medication at baseline and follow-up

Discussion

Screening for Depression

Depression screening using the PHQ-9 was implemented into clinical care. It was brief and had minimal patient burden. It was self-administered using touch-screen computers and offers a variety of continuous and categorical severity scoring and diagnostic impressions. Most (74.2 %) of those screening positive for depression were accurately predicted as having a depressive disorder diagnosis by the SCID gold standard, however many also had other comorbid conditions. These data indicate that while screening for depression indicates patients who have a concerning mental illness diagnosis, automatic treatment for depression based on screening alone would not be accurate and in some cases (psychotic disorders or bipolar disorders) would be harmful to patient care.

Advancement of depression screening process among persons with HIV is underway. Crane et al. [39] reports that little is known about the psychometric properties of depression screening instruments among PLWHA. Cella et al. [40] suggests that PROs are essential when evaluating many new treatments in health care; yet, current measures have been limited by a lack of precision, standardization, and comparability of scores across studies and diseases. The patient-reported outcomes measurement information system (PROMIS) provides item banks that offer the potential for efficient (minimizes item number without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly studied PROs [41].

Prevalence of Depression and Other Mental Disorders

This study found 25.4 % of patients at the 1917 HIV Clinic screened positive for depressive symptoms (moderate severity or greater on the PHQ-9). Previous estimates within this same clinic by Lawrence et al. [19] in a study of suicide risk, depression was less pronounced (12 %) while a comparison sample at University of Washington was somewhat greater (38 %). Screening criteria for depression was met for 38 % from another sample of HIV patients in two county-based HIV primary care clinics in California [1]. Those estimates appear to be closer to the upper end of the range of estimates of lifetime prevalence of depression estimated 4–45 % among persons with HIV published by Basu et al. [11].

Pence et al. [2] validated screening of mental disorders using the substance abuse-mental illness symptoms screener among 148 patients of 1,125 patients with a reference standard diagnostic tool, the SCID. They found 39 % had a mood/anxiety diagnosis, 21 % had a substance use diagnosis, and 8 % had both mood/anxiety and substance use disorders. Of patients with a mood/anxiety diagnosis, 76 % had clinically relevant depression, and 11 % had PTSD.

In the present study, we assessed mental disorders in a convenience sample of those who screened positive for depression. In this sample, 78 % had some anxiety disorder (10 % had PTSD), 81 % had a depressive disorder, 61 % had some substance use disorder, and 93 % had two or more diagnoses. These findings suggest that the prevalence of verified mental disorders is large among patients who screen positive for depression. The administration of the SCID in this study supports the presence of an extensive and complex diagnostic profile of patients who report depression symptoms during care. While anxiety, depression, and substance use were the most common and frequently co-occurring diagnoses, 12 % had even more serious psychotic and 13 % had bipolar diagnoses. These findings suggest that those who screen positive for depression in similar contexts are likely to represent a more complex diagnostic picture than depression only as implied by a positive screen for depression. Available structured clinical interviews and relevant treatment services in such environments would better help identify, describe, and treat these patients.

Treatment of Depression and HIV

The present research has important implications for screening and assessment for depression and other mental disorders, but also informs treatment response. In the present study, screened depressed patients received significantly more treatment for depression, and depression treatment visits were associated with a reduction in depression. This study demonstrated a −0.6304 unit reduction in PHQ-9 total depression score from baseline to follow-up for every additional attended depression treatment visit. Despite offering depression treatment to all depressed patients, only 46 % received any depression treatment. Reasons for this may have been that the patients may have already completed some psychotherapy for depression or receiving psychotherapy or pharmacotherapy elsewhere not measured in this study. Screened depressed patients also received fewer HIV primary care visits than their counterparts. This finding is consistent with other research linking depression to HIV treatment non-adherence [42-45.]

Strengths, Limitations and Future Research

The strengths of this study include the systematic collection of data within a real world HIV treatment clinic, utilization of standardized instruments, inclusion of almost all patients during the study period, access to data collected by electronic medical record clinical cohort with monitored quality assurance, and a naturalistic research context with depressed and non depressed patients with varying levels of depression and HIV treatment participation.

There were also limitations. First, the definition of depression based on a screening instrument cut off is arbitrary and may lack sensitivity. Most screening instruments are developed to capitalize on sensitivity while sacrificing a relatively small degree of specificity (i.e., not miss true cases). Indeed, the PHQ-9 seemed to over estimate the presence of depression diagnoses. While this is the most practical strategy, as opposed to a more involved diagnostic interview, more sensitive classification strategies may enhance the internal and external validity of this research.

Second, the investigation of depression treatment exposure was observational as opposed to deliberate random assignment to treatment groups. While this study was an example of innovative comparative effectiveness research, future research using random assignment and controlling for access to and measurement of depression treatment would provide greater experimental rigor.

Third, the findings from this study are from one clinic in the Southeastern U.S. Its generalization to other clinics or other patients is limited to a single site. Cross site research using the CNICS infrastructure would enhance generalizability and provide geographical, cultural, and more diverse (MSM, IDU, etc.) comparisons.

Fourth, not designating a structured six month follow-up PRO interview, as opposed to natural occurrence at scheduled primary HIV care visits, resulted in a significant number of missing follow-up assessments. A differential loss to follow-up between screened depressed (62 %) and not depressed (43 %) reduces statistical power and further limit the generalizability and could have biased the results. Substantial loss to follow-up was due to due to failure of the patients to complete a second PRO assessment within the follow-up window because of not being scheduled or not making their scheduled primary provider appointment. Research on treatment engagement and compliance by this investigative team has identified various factors associated with missed visits [42, 44, 46, 47.] There was no difference, however, in baseline PHQ-9 total depression scores between depressed and not depressed groups not followed. Future research may consider conducting scheduled follow-up assessments for this purpose, despite the difficulty with scheduling and logistics for patients who have barriers to attending the clinic.

Finally, this study does not address the relationship between depression, depression treatment, and HIV treatment and disease outcomes. These findings will be reported in the next paper.

Conclusions

This study demonstrates significant benefits in access to depression treatment and resulting decrease in depression scores after integration of a routine assessment for depression screening in a real world outpatient HIV clinic where mental health services had been available on a referral basis. Pence [20] states that future research should prioritize the evaluation of mental health interventions that are cost-effective and feasible for widespread integration into HIV clinical care; the impact of such interventions on ART adherence and clinical outcomes; and interventions to identify individuals with histories of traumatic experiences and to elucidate the mechanisms through which such histories pose barriers to effective HIV treatment. Our results indicate that routine depression screening in an HIV clinic can identify patients at risk with a variety of co-morbid psychiatric illnesses. Further, these patients were able to be specifically engaged in psychiatric and psychological care through recruitment into this study. The findings of this study contribute to need, strategies, benefits and outcomes of screening and treatment for depression among persons receiving treatment for HIV [39].

Acknowledgments

This research was sponsored by a Grant from Boehringer-Ingelheim. The authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICMJE) and were fully responsible for all aspects of manuscript development.

Contributor Information

Joseph E. Schumacher, Email: jschum@uab.edu, Division of Preventive Medicine, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA.

Cheryl McCullumsmith, Division of Consultation & Liaison, Department of Psychiatry, University of Alabama at Birmingham, Birmingham, AL, USA.

Michael J. Mugavero, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

Paige E. Ingle-Pang, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

James L. Raper, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

James H. Willig, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

Zhiying You, Division of Preventive Medicine, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA.

D. Scott Batey, Behavioral Science Core, Center for AIDS Research, University of Alabama at Birmingham, Birmingham, AL, USA.

Heidi Crane, Department of Medicine, University of Washington, Seattle, WA, USA.

Sarah T. Lawrence, Behavioral Science Core, Center for AIDS Research, University of Alabama at Birmingham, Birmingham, AL, USA

Charles Wright, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA.

Glenn Treisman, Department of Psychiatry, Johns Hopkins University, Baltimore, MD, USA.

Michael S. Saag, Division of Infectious Diseases, Department of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA

References

  • 1.Israelski DM, Prentiss DE, Lubega S, Balmas G, Garcia P, Muhammad M, et al. Psychiatric co-morbidity in vulnerable populations receiving primary care for HIV/AIDS. AIDS Care. 2007;19(2):220–5. doi: 10.1080/09540120600774230. [DOI] [PubMed] [Google Scholar]
  • 2.Pence BW, Miller WC, Whetten K, Eron JJ, Gaynes BN. Prevalence of DSM-IV-defined mood, anxiety, and substance use disorders in an HIV clinic in the Southeastern United States. J Acquir Immune Defic Syndr. 2006;42(3):298–306. doi: 10.1097/01.qai.0000219773.82055.aa. [DOI] [PubMed] [Google Scholar]
  • 3.Brinkman K, ter Hofstede HJ, Burger DM, Smeitink JA, Koopmans PP. Adverse effects of reverse transcriptase inhibitors: mitochondrial toxicity as common pathway. AIDS. 1998;12(14):1735–44. doi: 10.1097/00002030-199814000-00004. [DOI] [PubMed] [Google Scholar]
  • 4.Vinet-Oliphant H, Alvarez X, Buza E, Borda JT, Mohan M, Aye PP, et al. Neurokinin-1 receptor (NK1-R) expression in the brains of SIV-infected rhesus macaques: implications for substance P in NK1-R immune cell trafficking into the CNS. Am J Pathol. 2010;177(3):1286–97. doi: 10.2353/ajpath.2010.091109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Weiser SD, Wolfe WR, Bangsberg DR. The HIV epidemic among individuals with mental illness in the United States. Curr HIV/AIDS Rep. 2004;1(4):186–92. doi: 10.1007/s11904-004-0029-4. [DOI] [PubMed] [Google Scholar]
  • 6.Vanable PA, Carey MP, Blair DC, Littlewood RA. Impact of HIV-related stigma on health behaviors and psychological adjustment among HIV-positive men and women. AIDS Behav. 2006;10(5):473–82. doi: 10.1007/s10461-006-9099-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Safren SA, Radomsky AS, Otto MW, Salomon E. Predictors of psychological well-being in a diverse sample of HIV-positive patients receiving highly active antiretroviral therapy. Psychosomatics. 2002;43(6):478–85. doi: 10.1176/appi.psy.43.6.478. [DOI] [PubMed] [Google Scholar]
  • 8.McDowell TL, Serovich JM. The effect of perceived and actual social support on the mental health of HIV-positive persons. AIDS Care. 2007;19(10):1223–9. doi: 10.1080/09540120701402830. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, et al. Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry. 2001;58(8):721–8. doi: 10.1001/archpsyc.58.8.721. [DOI] [PubMed] [Google Scholar]
  • 10.Tegger MK, Crane HM, Tapia KA, Uldall KK, Holte SE, Kitahata MM. The effect of mental illness, substance use, and treatment for depression on the initiation of highly active anti-retroviral therapy among HIV-infected individuals. AIDS Patient Care STDS. 2008;22(3):233–43. doi: 10.1089/apc.2007.0092. [DOI] [PubMed] [Google Scholar]
  • 11.Basu S, Chwastiak LA, Bruce RD. Clinical management of depression and anxiety in HIV-infected adults. AIDS. 2005;19(18):2057–67. doi: 10.1097/01.aids.0000182518.84407.32. [DOI] [PubMed] [Google Scholar]
  • 12.Protopopescu C, Raffi F, Brunet-Francois C, Salmon D, Verdon R, Reboud P, et al. Incidence, medical and socio-behavioural predictors of psychiatric events in an 11-year follow-up of HIV-infected patients on antiretroviral therapy. Antivir Ther. 2012;17(6):1079–83. doi: 10.3851/IMP2102. [DOI] [PubMed] [Google Scholar]
  • 13.Reif S, Mugavero M, Raper J, Thielman N, Leserman J, Whetten K, et al. Highly stressed: stressful and traumatic experiences among individuals with HIV/AIDS in the deep South. AIDS Care. 2011;23(2):152–62. doi: 10.1080/09540121.2010.498872. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Barskova T, Oesterreich R. Post-traumatic growth in people living with a serious medical condition and its relations to physical and mental health: a systematic review. Disabil Rehabil. 2009;31(21):1709–33. doi: 10.1080/09638280902738441. [DOI] [PubMed] [Google Scholar]
  • 15.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Martin A, Rief W, Klaiberg A, Braehler E. Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population. Gen Hosp Psychiatry. 2006;28(1):71–7. doi: 10.1016/j.genhosppsych.2005.07.003. [DOI] [PubMed] [Google Scholar]
  • 17.Crane HM, Lober W, Webster E, Harrington RD, Crane PK, Davis TE, et al. Routine collection of patient-reported outcomes in an HIV clinic setting: the first 100 patients. Curr HIV Res. 2007;5(1):109–18. doi: 10.2174/157016207779316369. [DOI] [PubMed] [Google Scholar]
  • 18.Shacham E, Nurutdinova D, Satyanarayana V, Stamm K, Overton ET. Routine screening for depression: identifying a challenge for successful HIV care. AIDS Patient Care STDS. 2009;23(11):949–55. doi: 10.1089/apc.2009.0064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Lawrence ST, Willig JH, Crane HM, Ye J, Aban I, Lober W, et al. Routine, self-administered, touch-screen, computer-based suicidal ideation assessment linked to automated response team notification in an HIV primary care setting. Clin Infect Dis. 2010;50(8):1165–73. doi: 10.1086/651420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Pence BW. The impact of mental health and traumatic life experiences on antiretroviral treatment outcomes for people living with HIV/AIDS. J Antimicrob Chemother. 2009;63(4):636–40. doi: 10.1093/jac/dkp006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Olatunji BO, Mimiaga MJ, O’Cleirigh C, Safren SA. Review of treatment studies of depression in HIV. Top HIV Med. 2006;14(3):112–24. [PubMed] [Google Scholar]
  • 22.Safren SA, O’Cleirigh C, Tan JY, Raminani SR, Reilly LC, Otto MW, et al. A randomized controlled trial of cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected individuals. Health Psychol. 2009;28(1):1–10. doi: 10.1037/a0012715. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Ferrando SJ, Freyberg Z. Treatment of depression in HIV positive individuals: a critical review. Int Rev Psychiatry. 2008;20(1):61–71. doi: 10.1080/09540260701862060. [DOI] [PubMed] [Google Scholar]
  • 24.Soller M, Kharrazi N, Prentiss D, Cummings S, Balmas G, Koopman C, et al. Utilization of psychiatric services among low-income HIV-infected patients with psychiatric comorbidity. AIDS Care. 2011;23(11):1351–9. doi: 10.1080/09540121.2011.565024. [DOI] [PubMed] [Google Scholar]
  • 25.Alabama Department of Public Health. [2010];Alabama Department of Public Health: HIV/AIDS slides 2005. 2005 Available from: http://www.adph.org/aids/HIVAIDS%20slides%202005.pdf.
  • 26.Kitahata MM, Rodriguez B, Haubrich R, Boswell S, Mathews WC, Lederman MM, et al. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. Int J Epidemiol. 2008;37(5):948–55. doi: 10.1093/ije/dym231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA. 1999;282(18):1737–44. doi: 10.1001/jama.282.18.1737. [DOI] [PubMed] [Google Scholar]
  • 28.Lowe B, Grafe K, Zipfel S, Spitzer RL, Herrmann-Lingen C, Witte S, et al. Detecting panic disorder in medical and psychosomatic outpatients: comparative validation of the hospital anxiety and depression scale, the patient health questionnaire, a screening question, and physicians’ diagnosis. J Psychosom Res. 2003;55(6):515–9. doi: 10.1016/s0022-3999(03)00072-2. [DOI] [PubMed] [Google Scholar]
  • 29.Spitzer RL, Williams JB, Kroenke K, Hornyak R, McMurray J. Validity and utility of the PRIME-MD patient health questionnaire in assessment of 3000 obstetric-gynecologic patients: the PRIME-MD Patient Health Questionnaire Obstetrics-Gynecology Study. Am J Obstet Gynecol. 2000;183(3):759–69. doi: 10.1067/mob.2000.106580. [DOI] [PubMed] [Google Scholar]
  • 30.Lowe B, Schenkel I, Carney-Doebbeling C, Gobel C. Responsiveness of the PHQ-9 to psychopharmacological depression treatment. Psychosomatics. 2006;47(1):62–7. doi: 10.1176/appi.psy.47.1.62. [DOI] [PubMed] [Google Scholar]
  • 31.Kroenke K, Strine T, Spitzer R, Williams J, Berry J, Mokdad A. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114:163–73. doi: 10.1016/j.jad.2008.06.026. [DOI] [PubMed] [Google Scholar]
  • 32.Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. Alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol use disorders identification test. Arch Intern Med. 1998;158(16):1789–95. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
  • 33.Bradley KA, Bush KR, Epler AJ, Dobie DJ, Davis TM, Sporleder JL, et al. Two brief alcohol-screening tests from the alcohol use disorders identification test (AUDIT): validation in a female veterans affairs patient population. Arch Intern Med. 2003;163(7):821–9. doi: 10.1001/archinte.163.7.821. [DOI] [PubMed] [Google Scholar]
  • 34.WHO ASSIST Working Group. The alcohol, smoking and substance involvement screening test (ASSIST): development, reliability and feasibility. Addiction. 2002;97(9):1183–94. doi: 10.1046/j.1360-0443.2002.00185.x. [DOI] [PubMed] [Google Scholar]
  • 35.Newcombe DA, Humeniuk RE, Ali R. Validation of the World Health Organization alcohol, smoking and substance involvement screening test (ASSIST): report of results from the Australian site. Drug Alcohol Rev. 2005;24(3):217–26. doi: 10.1080/09595230500170266. [DOI] [PubMed] [Google Scholar]
  • 36.Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG) AIDS Care. 2000;12(3):255–66. doi: 10.1080/09540120050042891. [DOI] [PubMed] [Google Scholar]
  • 37.First M, Spitzer R, Gibbon M, Williams J. Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition (SCID-I/P), biometrics research. New York: New York State Psychiatric Institute; 2002. [Google Scholar]
  • 38.Willig JH, Abroms S, Westfall AO, Routman J, Adusumilli S, Varshney M, et al. Increased regimen durability in the era of once-daily fixed-dose combination antiretroviral therapy. AIDS. 2008;22(15):1951–60. doi: 10.1097/QAD.0b013e32830efd79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Crane PK, Gibbons LE, Willig JH, Mugavero MJ, Lawrence ST, Schumacher JE, et al. Measuring depression levels in HIV-infected patients as part of routine clinical care using the nine-item Patient Health Questionnaire (PHQ-9) AIDS Care. 2010;22(7):874–85. doi: 10.1080/09540120903483034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63(11):1179–94. doi: 10.1016/j.jclinepi.2010.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Gibbons LE, Feldman BJ, Crane HM, Mugavero M, Willig JH, Patrick D, et al. Migrating from a legacy fixed-format measure to CAT administration: calibrating the PHQ-9 to the PROMIS depression measures. Qual Life Res. 2011;20(9):1349–57. doi: 10.1007/s11136-011-9882-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Kempf MC, McLeod J, Boehme AK, Walcott MW, Wright L, Seal P, et al. A qualitative study of the barriers and facilitators to retention-in-care among HIV-positive women in the rural southeastern United States: implications for targeted interventions. AIDS Patient Care STDS. 2010;24(8):515–20. doi: 10.1089/apc.2010.0065. [DOI] [PubMed] [Google Scholar]
  • 43.Kozak MS, Mugavero MJ, Ye J, Aban I, Lawrence ST, Nevin CR, et al. Patient reported outcomes in routine care: advancing data capture for HIV cohort research. Clin Infect Dis. 2012;54(1):141–7. doi: 10.1093/cid/cir727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Moneyham L, McLeod J, Boehme A, Wright L, Mugavero M, Seal P, et al. Perceived barriers to HIV care among HIV-infected women in the Deep South. J Assoc Nurses AIDS Care. 2010;21(6):467–77. doi: 10.1016/j.jana.2010.03.003. [DOI] [PubMed] [Google Scholar]
  • 45.Mugavero M, Ostermann J, Whetten K, Leserman J, Swartz M, Stangl D, et al. Barriers to antiretroviral adherence: the importance of depression, abuse, and other traumatic events. AIDS Patient Care STDS. 2006;20(6):418–28. doi: 10.1089/apc.2006.20.418. [DOI] [PubMed] [Google Scholar]
  • 46.Mugavero MJ, Lin HY, Allison JJ, Giordano TP, Willig JH, Raper JL, et al. Racial disparities in HIV virologic failure: do missed visits matter? J Acquir Immune Defic Syndr. 2009;50(1):100–8. doi: 10.1097/QAI.0b013e31818d5c37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Mugavero MJ, Lin HY, Allison JJ, Willig JH, Chang PW, Marler M, et al. Failure to establish HIV care: characterizing the “no show” phenomenon. Clin Infect Dis. 2007;45(1):127–30. doi: 10.1086/518587. [DOI] [PubMed] [Google Scholar]

RESOURCES