How Well Do Clinicians Estimate Patients' Adherence to Combination Antiretroviral Therapy?

Abstract

OBJECTIVE

Adherence to combination antiretroviral therapy is critical for clinical and virologic success in HIV-infected patients. To combat poor adherence, clinicians must identify nonadherent patients so they can implement interventions. However, little is known about the accuracy of these assessments. We sought to describe the accuracy of clinicians' estimates of patients' adherence to combination antiretroviral therapy.

SETTING

Public HIV clinic.

DESIGN

Prospective cohort study. During visits, we asked clinicians (nurse practitioners, residents and fellows, and their supervising attending physicians) to estimate the percentage of antiretroviral medication taken by patients over the last 4 weeks and predicted adherence over the next 4 weeks. Adherence was measured using electronic monitoring devices, pill counts, and self-reports, which were combined into a composite adherence measure.

PATIENTS AND PARTICIPANTS

Clinicians estimated 464 episodes of adherence in 82 patients.

RESULTS

Among the 464 adherence estimates, 264 (57%) were made by principal care providers (31% by nurse practitioners, 15% by fellows, 6% by residents, and 5% by staff physicians) and 200 (43%) by supervising attending physicians. Clinicians' overestimated measured adherence by 8.9% on average (86.2% vs 77.3%). Greater clinician inaccuracy in adherence prediction was independently associated with higher CD4 count nadir (1.8% greater inaccuracy for every 100 CD4 cells, P = .005), younger patient age (3.7% greater inaccuracy for each decade of age, P = .02), and visit number (P = .02). Sensitivity of detecting nonadherent patients was poor (24% to 62%, depending on nonadherence cutoff). The positive predictive value of identifying a patient as nonadherent was 76% to 83%.

CONCLUSIONS

Clinicians tend to overestimate medication adherence, inadequately detect poor adherence, and may therefore miss important opportunities to intervene to improve antiretroviral adherence.

Highly active antiretroviral therapy (HAART) has markedly reduced morbidity and mortality from HIV infection.^1–3 However, the emergence of drug-resistant virus has dampened hopes that HAART invariably provides long-term suppression of viral replication.⁴ Inadequate patient adherence to HAART is an important cause of failure to achieve suppression of detectable plasma viremia^5,6 and there are concerns that poor adherence leads to the development of drug-resistant HIV.^4,7–10

Clinicians' assessment of patient adherence plays an important role in the care of HIV-infected patients. Clinicians employ their estimates to identify nonadherent patients who may benefit from adherence interventions. In addition, physicians have used such estimates of adherence to withhold HAART from patients whom they fear will develop viral resistance.¹¹ Physicians' adherence estimates even have been used as measures of adherence to antiretroviral medication in clinical studies.^12–15 National guidelines emphasize that physicians should use their estimate of medication adherence when deciding whether to initiate HAART treatment in asymptomatic persons.^7,16 However, our understanding of clinicians' ability to assess patients' adherence to HAART is limited.^5,6,9,10

In other diseases, studies suggest that physicians, nurses, and other health care workers have difficulty estimating patients' adherence to medical therapy.^17–25 However, most studies comparing clinicians' estimates of adherence to measured adherence values were done prior to the introduction of modern electronic devices that measure adherence. To date, there are little data evaluating the accuracy of clinicians' estimates of adherence to HAART.

We investigated clinicians' estimates of their patients' past and future adherence to combination antiretroviral therapy. We hypothesized that despite heightened awareness of the importance of HAART adherence, clinicians would have difficulty estimating adherence and identifying nonadherent patients.

METHODS

Subjects

All patients were enrolled in the ADEPT (ADherence and Evaluation of Protease Therapy) cohort of HIV-infected patients at a county hospital, university-associated HIV clinic. This clinic cares for over 800 HIV-infected persons and handles approximately 4,000 patient visits per year. ADEPT is a prospective observational study of the relationship between adherence and virologic outcomes among patients recently started on HAART, which has been described in detail elsewhere.²⁶ Clinicians chose HAART regimens and did not receive study-collected adherence information. HIV-infected adults followed at the clinic were eligible to enroll in ADEPT if they began therapy with a protease inhibitor–containing regimen within the past 6 months or were switched to a new antiretroviral regimen containing ≥3 new medications in the past month. Sixty percent of eligible patients enrolled in ADEPT. Patients visited a study nurse every 4 weeks (a “wave”) during the 48-week study period and were asked to bring their pill bottles for assessment of adherence. Clinicians were asked to estimate adherence to HIV medications when subjects attended regularly scheduled clinic visits, which occurred every 1 to 3 months. Approval for the study was obtained from the institutional review board at Harbor-UCLA Medical Center.

Four types of data were collected. First, self-reports of adherence were elicited in face-to-face interviews with subjects at study entry and then during waves 2 and 6. Second, clinical data were collected by abstraction of the medical record. Third, clinician estimates of adherence were elicited by a self-administered survey provided by research nurses during study clinic visits. Finally, at each wave adherence was measured by pill counts and MEMS (Medication Event Monitoring System, Aprex Corporation, Union City, Calif), an electronic pill bottle cap that records each time a patient opens his/her bottle.

Clinician Estimates of Past and Future Adherence

Before each regularly scheduled clinic visit, a research nurse placed 2 Provider Questionnaires in enrolled patients' charts. One questionnaire was designated for the principal care provider (PCP), the clinician who first saw the patient and who spent the majority of time with the patient. A second questionnaire was designated for the supervising attending physician. The PCP (a nurse practitioner, second- or third-year medical resident, fellow, medical student, or staff physician) presented each patient's case to an attending physician (except staff physicians, who were not required to present cases). All clinic patients were assigned a PCP. Most were assigned to a nurse practitioner and a minority were assigned to a fellow or a staff physician. These clinicians were all continuity clinic physicians. A small minority of patients were assigned to a rotating medical resident slot; residents each dedicated 4 sessions to this clinic during residency training. Attending physicians reviewed each patient's case but spent a smaller and variable amount of time in the exam room with the patient. To enhance continuity, patients were preferentially scheduled to attend clinic on days on which the attending physician who knew them was present.

The provider questionnaire asked clinicians to evaluate the patients' adherence to antiretroviral medications. Clinicians were asked to indicate “What percentage of time do you think this patient took their (antiretroviral) medications as prescribed?” during the past 4 weeks by marking an “X” on a linear analog scale ranging from 0% to 100%. Additionally, we elicited clinician certainty by asking them to insert hash marks on the scale around their adherence point estimate to indicate the smallest and largest percentages of time that they thought a patient might take their medications as prescribed. Using the same method, clinicians were then asked to estimate how adherent the patient would be over the next 4 weeks.

Adherence Measurements

Adherence was measured by three methods: MEMS, pill count, and patient self-report. MEMS bottle caps were placed on patients' protease inhibitor–containing pill bottle(s). For patients taking a nonprotease inhibitor regimen, the device was placed on the bottle of the most frequently dosed antiretroviral medication. Data from MEMS were retrieved every 4 weeks at the study visit. Adherence was computed as the number of doses taken per day divided by the number of doses prescribed summed over a 4-week period and expressed as a percentage. Pill count was performed on each antiretroviral medication at wave visits and the number of missed doses was calculated based on the remaining versus expected number of pills. Self-report asked about doses missed during the 1-week period prior to the interview.

Adherence was measured using a Composite Adherence Score, described in detail elsewhere.²⁶ MEMS was the primary source of information for the composite adherence score.²⁶ When MEMS data were missing or unreliable, the Composite Adherence Score was derived from pill count (first choice) or self-report (second choice) values, and calibrated to the MEMS metric. The Composite Adherence Score, which was calculated in 4-week blocks of time, correlates better than MEMS-, pill count-, or self-report–derived adherence with plasma HIV RNA measurements.²⁶ If the time period of the clinician estimate did not match the study wave and the Composite Adherence Score was derived from MEMS alone, the Composite Adherence Score values applying to the exact dates were used for comparison to clinician estimates. If the Composite Adherence Score was pill count or self-report derived, adherence was assumed to be constant over the wave, with adherence computed proportionate to the portion of the waves covered by the estimate. In other words, if the previous 4-week period spanned 2 waves with pill count–derived adherence of 80% and 90% for 1 and 3 weeks, respectively, adherence was calculated to be [(.80)*1 + (.90)*3]/4 or 87.5%.

Comparison of Clinician Estimate with Measured Adherence

We analyzed the accuracy of clinicians' estimates by calculating the difference between estimated adherence and measured adherence for the same time period. This Adherence Difference was calculated for each clinician estimate as follows: Adherence Difference = clinician-estimated adherence minus measured adherence. Theoretically, Adherence Difference could range from –100% to 100%. Adherence Difference provides information on the directionality and magnitude of the difference; a positive Adherence Difference represents an overestimation of adherence and a negative Adherence Difference represents an underestimate. When Adherence Difference is summed over several estimates, underestimations can cancel out overestimations to yield a mean close to zero. We also calculated the absolute value of the Adherence Difference as a measure of the overall accuracy of the estimates. The theoretical range of the Absolute Adherence Difference is 0% to 100%; a higher value represents a more inaccurate estimate, regardless of whether the guess is an under- or overestimation. Adherence Difference and Absolute Adherence Difference were calculated for the four weeks prior to and following the clinic visit.

Measure of Clinicians' Confidence in Their Estimates

The confidence of the clinician's guess was measured by calculating the width of the hash marks around the adherence estimate. To counteract a ceiling effect, if a hash mark was at 100% and the distance from the point estimate to this hash mark was less than the distance to the lower hash mark, the confidence width was calculated as twice the distance between the lower hash mark and the point estimate.

Clinician and Patient Factors Hypothesized to Be Associated with Adherence Difference and Absolute Adherence Difference

We hypothesized that clinicians would perceive adherence to be highest among patients with socioeconomic profiles similar to their own (e.g., more educated, English-speaking, Asian or Caucasian race, and older age), as observed in one previous study.²⁷ We hypothesized that these demographics would be predictors of higher adherence estimates, more inaccurate estimates (larger Absolute Adherence Difference) and larger overestimations (positive Adherence Difference), given clinicians' propensity to overestimate adherence. We also hypothesized that clinicians would overestimate adherence more among persons with less severe illness, which is associated with less adherence.²⁸ Disease severity was measured by CD4 count nadir and highest viral load at study entry, and a modified Boston Opportunistic Disease Survival Score (a measure of severity of illness developed for patients with AIDS).²⁹ We also hypothesized that clinician type (PCP or attending) would be associated with accuracy; attending physicians have been demonstrated to overestimate adherence to a larger degree than clinicians with less experience.²² In an exploratory fashion, we investigated the association of patient gender, HIV risk factor (injection drug use, male-male sex), history of alcohol or substance use, history of psychiatric disease, visit number, and protease inhibitor–naive status with clinician estimates. In addition, we hypothesized that clinicians would tend to guess that poor adherence would improve and thus estimates of future adherence would be even more inaccurate than estimates of past adherence. We based this on our observations that clinicians use education as a means of improving adherence, yet we believed it unlikely that relatively simple interventions would significantly improve adherence.³⁰ Finally, we hypothesized that certainty of the adherence estimate would not be associated with accuracy, given previous reports of the poor association between certainty and accuracy of an adherence estimate.^17,19

Analyses

We computed Adherence Difference and Absolute Adherence Difference for each patient and the mean across all patients. Then we examined bivariate associations between hypothesized predictors described above and Adherence Difference and Absolute Adherence Difference using generalized estimating equations (GEEs), which adjust for intrapatient, intraclinician, and intravisit correlation.³¹ We hypothesized that predictors of the Adherence Difference and the Absolute Adherence Difference would not differ between estimates of past or future adherence; therefore, we restricted our subgroup analyses to estimates of past adherence.

Next we developed multivariate models of Adherence Difference and Absolute Adherence Difference using the GEE approach.^31,32 Both models used identical predictors, incorporating all variables with a P value of ≤.20 from bivariate analyses. The diagnostic “test characteristics” of clinician estimates for identifying nonadherence over the past month were compared, defining nonadherence as <80%, <90%, and <95% adherence. We chose these cut-off values based on the minimal levels of adherence that may be needed to achieve a high likelihood of suppressing detectable viral replication.⁶ Because the number of visits varied among patients, a weight inversely related to the number of clinician visits was assigned to each patient in the sensitivity and specificity computation.

Clinician estimates of adherence over the past month and future month were compared using only those cases in which both estimates as well as past and future adherence measures were available. We compared past and future mean adherence estimates and past and future adherence measurements graphically and used a pair-wise t test. Because of the clinical importance of past and future adherence among patients for whom clinician estimate of past adherence was ≤85%, we evaluated each of these comparisons separately for this subgroup (n = 175).

RESULTS

Study Sample

During the study period, 718 Provider Questionnaires were distributed during 359 clinic visits and 595 were returned completed (83% response rate). Among returned surveys, clinician-estimated adherence could not be analyzed for 131 surveys for the following reasons: the patient was off HAART at some point during the previous 4 weeks (24 surveys); the clinician reported not knowing the patient well enough to estimate adherence (6 surveys, all attending physicians); measured adherence was missing for at least a portion of the period (80 surveys); and 1 practitioner acted as both the PCP and the attending (21 surveys). In the latter case, we analyzed only 1 of the 2 Provider Questionnaires from the visit and considered the provider a PCP. Missing adherence information was due to missed study visits or subject failure to return the pill bottles at study visits. Therefore, we analyzed 464 adherence estimates made by 42 different providers (27 residents, 6 fellows, 5 attendings, and 4 nurse practitioners). Two hundred sixty-four (57%) adherence estimates were made by PCPs: nurse practitioners 144 (31%); fellows 70 (15%); residents 29 (6%); and staff physicians 21 (5%). The remaining estimates were performed by supervising attending physicians (n = 200, 43%). Of the 464 measures of adherence from 558 four-week periods, 62% (n = 337) were derived from MEMS, 36% (n = 203) from pill count, and 1% (n = 8) from self-report.

Sample Characteristics

The 82 subjects had a mean age of 37 years. Fifty percent of subjects were Hispanic and 27% African American. Subjects were largely poor (65% reported an annual income ≤$10,000); nearly two thirds were protease inhibitor naive at study entry, and about one third had a CD4 nadir below 200. Demographic and clinical characteristics are displayed in Table 1.

Table 1.

Characteristics of Study Subjects (N = 82)

Characteristic	Value
Age, y
Mean (SD)	37.4 (7.9)
Range	22 to 63
Gender, n (%)
Male	64 (78)
Female	18 (22)
Education, n (%)
Not high school graduate	33 (40)
High school graduate	40 (49)
College graduate	9 (11)
Household income,*n (%)
<$10,000	33 (65)
>$10,000	18 (35)
Race/ethnicity, n (%)
African American	22 (27)
Caucasian	12 (15)
Hispanic	41 (50)
Other/mixed race	7 (9)
Primary language, n (%)
English	52 (63)
Spanish	30 (37)
HIV risk factors,†n (%)
Injection drug use	14 (17)
Male–male sex	31 (38)
Other	41 (50)
Naive to protease inhibitors at study enrollment, n (%)
Yes	52 (63)
No	30 (37)
Severity of illness category,‡n, (%)
1 (asymptomatic)	4 (5)
2	46 (56)
3	16 (20)
4 (severe disease)	16 (20)
History of alcohol/substance abuse, n (%)	30 (37)
History of psychiatric disease, n (%)	19 (23)
Protease inhibitor naïve, n (%)	53 (63)
CD4 count nadir
Mean (SD)	163 (192)
Range	0 to 1,130
Peak viral load (HIV RNA, log10)
Mean (SD)	3.98 (1.37)
Range	2.00 to 6.13
Number of visits with clinic provider during study period, n (%)
1	15 (18)
2	18 (22)
3	18 (22)
≥4	31 (38)

^*Information on income was not available for all patients.

^†Patients may have >1 risk factor for HIV.

^‡As measured by Boston Opportunities Disease Survival Score.²⁹

Clinicians' Estimates of Adherence and Measured Adherence

Mean clinician estimated adherence was 86.2%; (SD 16.5%, median 90.0%) and mean measured adherence among the 464 four-week periods of adherence was 77.3% (SD 24.6%, P < .0001; median 86.6%) (Fig. 1). Mean Adherence Difference was 8.9% (SD 21.8%) and mean Absolute Adherence Difference was 15.7% (SD 21.8%). Overall, 279 of the estimates (60%) were overestimates, 188 (39%) were underestimates, and 2 were exactly correct. The magnitude of overestimation was greater than that of underestimation; the mean overestimation of adherence was 20.5% whereas the mean underestimation was 8.7% (P < .0001). Adherence Difference was greater than +10% in 162 (35%) of cases, greater than +20% in 96 (21%) of cases, less than –10% in 45 cases (10%), and less than –20% in 17 (3.7%) of cases. Adherence Difference was between +5% and –5% in 165 (36%) cases and was between +10% and –10% in 257 (55%).

FIGURE 1.

Plot of Adherence Difference (estimated adherence minus measured adherence) versus measured adherence.

In bivariate analysis, higher CD4 count nadir and visit number were associated with more inaccurate estimates (larger Absolute Adherence Difference) (Table 2). Bivariate predictors of overestimation (Adherence Difference) were higher CD4 count nadir, PCP clinician, and visit number. The mean Adherence Difference in all subgroups was positive, demonstrating that overestimation (higher mean Adherence Difference) was more common than underestimation. For all subgroups in Table 2, clinician estimates ranged between 83% and 91% while measured adherence had a much greater spread, ranging from 64% to 84%.

Table 2.

Factors Associated with the Adherence Difference and Absolute Adherence Difference^*,^† (N = 464)

	Number of Observations	Mean Adherence Estimate, %	Mean Adherence, %	Mean Absolute AD, %	P Value†	Mean AD, %	P Value†
All estimates	464	86.2	77.3	15.7		8.9
Patient factors
Age
18 to 35	176	84.4	72.3	19.7	.15	12.0	.53
36 to 40	178	85.4	78.1	14.7		8.3
>40	110	88.6	83.9	11.6		4.7
Gender
Male	358	86.4	79.4	14.0	.09	6.9	.09
Female	106	85.3	70.1	21.4		15.1
Education
Not high school grad	210	82.6	72.5	19.2	.41	10.0	.97
High school grad	210	89.4	81.8	14.7		7.6
College grad	44	87.7	78.3	20.3		9.4
Ethnicity
African American	112	84.9	71.3	17.2	.62	13.6	.50
Caucasian	60	83.1	74.5	12.0		8.7
Hispanic	266	87.1	80.4	16.2		6.8
Other/mixed race	26	88.5	78.3	19.0		10.2
Primary language
English	251	85.6	74.9	15.4	.82	10.7	.29
Spanish	213	88.6	80.1	16.1		6.7
Injection drug use
Yes	67	82.5	75.3	14.2	.63	7.2	.65
No	297	86.8	77.6	16.0		9.1
Male-male sex
Yes	168	87.4	78.5	15.3	.70	8.9	.47
No	296	85.4	76.6	16.0		8.8
History of alcohol/substance abuse
Yes	164	86.0	76.2	18.1	.76	9.9	.85
No	300	86.3	77.9	14.5		8.4
History of psychiatric disease
Yes	122	83.2	76.3	12.1	.16	6.9	.40
No	342	87.2	77.6	16.9		9.6
Protease inhibitor naive
Yes	319	85.0	76.8	16.3	.74	8.3	.56
No	145	88.6	78.5	14.6		10.1
Severity of illness‡
1 (asymptomatic)	15	88.7	74.7	19.6	.64	13.9	.94
2	247	85.5	76.2	15.4		9.3
3	85	87.7	80.6	15.4		7.1
4 (severe disease)	117	86.6	77.5	16.3		8.5
CD4 count nadir
<100	276	87.0	81.2	13.6	.001	5.8	.002
100 to 350	126	85.4	75.3	14.1		10.1
>350	62	84.0	63.9	27.3		20.1
HIV RNA (log10)
<3	103	91.3	84.2	16.3	.34	7.1	.93
3 to 4.5	199	83.8	71.1	21.1		12.6
>4.5	234	84.9	76.1	16.2		8.7
Visit to provider during study period
1 (first)	143	83.8	71.4	17.8	.04	14.4	.008
2	117	87.8	81.4	12.7		6.3
3	79	86.5	84.0	13.7		13.7
≥4	125	87.2	76.0	17.5		11.2
Clinician factors
Type of clinician
Principal care provider	264	87.2	76.6	16.4	.50	10.6	.04
Attending	200	84.8	78.2	14.9		6.5
Confidence in adherence estimate§
<12% (more confident)	202	90.8	81.6	13.4	.20	9.3	.17
12 to 18%	58	84.4	76.7	17.2		7.7
>18% (less confident)	113	80.4	70.5	19.3		9.9

^*Adherence Difference (AD) = clinician estimated adherence minus measured adherence. Absolute Adherence Difference defined as absolute value of Adherence Difference.

^†Bivariate associations measured using generalized estimating equations.

^‡As measured by modified Boston Opportunistic Disease Survival score²⁹ (see text).

^§Confidence is measured as the distance between the hash marks around the clinician adherence point estimate. Smaller value signifies greater confidence in the adherence estimate.

We performed longitudinal multivariate models for Absolute Adherence Difference and Adherence Difference that included age, gender, CD4 count nadir, visit number, and confidence in the adherence estimate. These models also included a cubic term suggested by the bivariate association between visit number and Absolute Adherence Difference. Significant predictors of Absolute Adherence Difference were higher CD4 count nadir (P = .005), younger patient age (P = .02), and visit number (P = .02). For every increase of 100 in the patient's CD4 count nadir, inaccuracy increased by 1.8%; for every decade of age, Absolute Adherence Difference decreased by 3.7%. Greater inaccuracy was associated with increasing visit number. The relationship between Absolute Adherence Difference and visit number, fit with a cubic term, demonstrated that Absolute Adherence Difference was high at visit 1, decreased at visit 2, and then tended to increase at each subsequent visit. In the multivariate model for Adherence Difference, CD4 count nadir (P = .008), principal care provider (P = .04), and visit number (P = .002) were significant predictors of greater overestimation. For every increase of 100 in the CD4 count nadir, clinician overestimation increased by 2.4%. PCPs tended to overestimate adherence more than attending physicians (by 4.0%) and greater inaccuracy was associated with increasing visit number. As in the previous multivariate model, a cubic term for visit number best explained the relationship with Adherence Difference, which was increased at visit 1, decreased at visit 2, and then tended to progressively increase at subsequent visits.

“Test Characteristics” of Clinician Estimates of Nonadherence

Sensitivity of detecting nonadherent patients in our sample was 24% when nonadherence was defined as taking <80% of medications. Defining nonadherence as taking <90% or <95% of medications, sensitivity was 38% and 62%, respectively. (Table 3) The positive predictive values of estimates of nonadherence were 76%, 83% and 82%, when using nonadherence cut offs of 80%, 90% and 95%, respectively. Negative predictive values were 60%, 47% and 31%, respectively, for these cut offs.

Table 3.

Test Characteristics of Clinician Estimates of Medication Nonadherence^*

Definition of Nonadherence	Sensitivity, %†	Specificity, %†	PPV, %†	NPV, %†
<80% adherence	24	87	76	60
<90% adherence	38	87	83	47
<95% adherence	62	55	82	31

^*Test characteristics based on 464 estimates of adherence among 82 patients.

^†Results shown are weighted for number of visits with each clinician (see text). Original (unweighted) values presented in parenthesis.

PPV, positive predictive value; NPV, negative predictive value.

Estimates of Future Adherence and Comparisons with Past Adherence

We analyzed 402 visits for which there were complete data on past and future estimated and measured adherence. Future adherence was incalculable for the remaining 62 visits because the patient was taken off HAART after the visit (n = 9) or values for adherence were missing (n = 53). The mean estimate of future adherence (89.8%) significantly exceeded the mean estimate of past adherence (87.0%, P < .001). (Fig. 2) The bottom graph in Figure 2 presents past and future measured adherence. Future measured adherence (81.5%) significantly exceeded past measured adherence (79.2%, P =.009).

FIGURE 2.

Both graphs are based on 402 observations of clinician-estimated adherence and measured adherence over the past 4 weeks and next 4 weeks. Overlapping data points in the upper graph give the illusion of having fewer data points than the lower graph.

Among the subset of surveys in which clinicians estimated that the patient had poor (≤85%) past adherence (n = 175), mean estimate of future adherence significantly exceeded mean estimate of past adherence (86.1% vs 81.3%, P < .001) and mean adherence over the future 4 weeks exceeded the past 4 weeks' adherence (67.3% vs 59.1%, P < .001).

DISCUSSION

A crucial element predicting therapeutic success for persons taking antiretroviral therapy is antiretroviral adherence. We designed a study to evaluate clinicians' ability to assess adherence to HAART. Therapeutic decisions in HIV-infected patients, such as when to start or change regimens, hinge in part on subjective assessment of adherence to HAART.^7,16 We found that clinicians managing HIV-infected persons were inaccurate when assessing adherence, tended to overestimate adherence, and failed to identify many patients with suboptimal adherence.

Our study demonstrates deficits in clinician estimates of patients' adherence consistent with previous studies in non-HIV samples.^{17–21,23–25} Our data move beyond the prior work by comparing clinician estimates to a composite objective adherence measure that is related to virologic outcome²⁶ and by evaluating estimates of both past and future adherence. We demonstrate that estimation errors are due primarily to patient-based variation in adherence that is not accounted for by clinicians, and that these errors occur within a backdrop of overall overestimation. In our population, the positive predictive value of identifying a nonadherent patient was reasonable (76% to 83%), depending on the cut off used to distinguish adherence from nonadherence; however, the sensitivity of clinician estimates to detect nonadherent patients was poor (24% to 62%). Thus, clinicians did not identify many nonadherent patients. These results reinforce previous observations that clinician suspicion that a patient is nonadherent is often correct but estimates of adequate adherence are frequently erroneous.^17–20

The magnitude of adherence overestimation in our study is less than previously described.¹⁷ This attenuated overestimation occurred despite employing electronic measures of adherence that tend to give lower values for measured adherence.^33–35 Whether the clinicians' more realistic global assessments of medication adherence are related to the heightened awareness of the importance of adherence in the HIV provider community or is a result of higher adherence by HIV-infected persons is not clear. Measured adherence in our study is higher than electronically measured adherence to thrice a day medication previously reported for other chronic diseases.^34–36 This suggests that increased adherence may explain at least part of the narrowed gap between estimated and measured adherence.

The bivariate analyses demonstrate that clinicians do not differentiate among patient characteristics in estimating adherence. For example, clinicians were more accurate at estimating adherence in patients with lower CD4 count nadir because clinicians routinely overestimate adherence and persons with high CD4 count nadirs adhere less well. This suggests that clinicians overlook important relationships between adherence and markers such as disease severity despite previous observations showing associations between less severe disease and lower adherence.^28,37,38 There may be additional implications of the association between higher CD4 count and inaccuracy and overestimation of adherence estimates, which was also seen in the multivariate model. Since clinicians failed to appreciate the association between early disease and poor adherence, patients with early HIV infection starting HAART may need to be aggressively targeted with interventions to improve adherence to minimize the potential for developing drug resistance. While national guidelines explicitly state that decisions to initiate HAART should be made in part on estimates of patient adherence,^7,16 the difficulty clinicians have in predicting adherence in less symptomatic patients should be acknowledged in these documents.

The association between visit number and accuracy of adherence estimations also has important implications. Previous investigation suggested that overestimates of adherence may be more common among clinicians who had increased familiarity with a patient.²² Our study demonstrated a complex relationship between accuracy of guess and visit number. Large Adherence Differences and Absolute Adherence Differences were seen at visit 1 and at later visits (i.e., visit 4 and later). We hypothesize that inaccuracy at visit 1 may have been due to clinicians' reluctance to ascribe high adherence to a patient with whom they were relatively unfamiliar. Inaccuracy at later visits is probably due to decreasing adherence that occurred unbeknown to providers. Several investigations have demonstrated that adherence to HAART tends to decrease over time.^26,39 These data suggest that longer patient-provider relationship may breed a complacency toward efforts to detect new difficulties with adherence.

We found estimates of future adherence exceeded past adherence estimates, even in patients perceived to have poor past adherence. However, we also found that adherence after an estimate exceeded adherence before an estimate. This latter observation needs to be considered in the context of the fact that adherence in this cohort, as in others, declined over time.^26,39 These findings may suggest that clinicians undertook effective interventions to improve adherence at clinic visits. Alternately, the finding could simply represent continued clinician overestimation in the setting of “regression to the mean” of actual adherence, which was particularly poor during the wave in question, or a “Hawthorne effect.”

Our study has limitations. First, the study was conducted at only 1 clinic and there are differences between this sample and the national patient population (i.e., relatively high proportion of Hispanic patients and persons from lower socioeconomic status). Second, we have little information on the 40% of patients who refused participation in the study; the sample studied may not be reflective of all eligible clinic patients. Because the clinic is academically affiliated and residents and fellows rotate through the clinic, continuity of care may be more fragmented than at nonacademic sites. Some have argued that provider characteristics themselves influence adherence⁴⁰; we could not evaluate this due to the small number of core providers. Additionally, despite a large number of analyzable observations (464), we were relatively underpowered, due to intracorrelations at the patient, provider, and visit level, to study relationships between patient characteristics and prediction accuracy (e.g., between age and Adherence Difference). Finally, some of our estimates were not analyzable, most commonly because patients missed study visits, although the proportion is comparable to investigations employing similar methodology.^19,23,25 Given that some investigators have found an association between nonadherence and missing clinic visits,⁴¹ we suspect that those missing study visits may have had a lower adherence than our mean measured adherence of 77%. When measured adherence was missing, the mean estimate of adherence was 72%, about 14% lower than the mean adherence estimate when adherence was measurable (86%). If adherence among those patients without adherence measurements was similar to our mean measured adherence, then our results may underestimate clinician accuracy. However, if adherence among those with missing measurements was very poor, our clinicians may have been less accurate than we describe.

There are numerous strengths to our investigation. Our investigation was based on adherence measured from electronic bottle caps, pill counts, and self-reported adherence, combined into a composite adherence score. Our adherence measure has a stronger correlation with objective measures (viral load suppression) than any of the three component measures alone,²⁶ thus increasing the validity of our observations. Additionally, the longitudinally collected adherence estimates amount to the largest assembly of adherence estimates to date. Finally, we have investigated accuracy of adherence estimates in a chronic disease in which adherence is of paramount importance to physicians and guidelines base critical medical decisions on clinician-perceived adherence.^7,16

In summary, our results have several important implications. First, clinicians should view their assessments of patients' adherence with caution. Estimates of nonadherence are helpful, but estimates of adequate adherence may be inaccurate. Better methods for identifying nonadherent patients are needed. Interventions or educational programs aiming to improve poor adherence may be needed regardless of perceived adherence, given that subjective detection of poor adherence is problematic. Future research to evaluate methods of integrating objective measures into clinical practice is warranted. Finally, subjective assessments of adherence, which have been employed in multiple investigations of HIV-infected persons as a surrogate adherence measure,^12–15 are flawed and should not be used when other more objective measures are available. There is an urgent need for developing accurate means of assessing adherence in patients on HAART and a need for interventions to improve antiretroviral adherence.