Evaluating the utility of provider-recorded clinical status in the medical records of HIV-positive adults in a limited-resource setting

Abstract

Provider-reported summaries of clinical status may assist with clinical management of HIV in resource poor settings if they reflect underlying biological processes associated with HIV disease progression. However, their ability to do so is rarely evaluated. Therefore, we aimed to assess the relationship between a provider-recorded summary of clinical status and indicators of HIV progression. Data were abstracted from 201 randomly selected medical records at a large HIV clinic in the Dominican Republic. Multivariable logistic regressions were used to examine the relationship between provider-assigned clinical status and demographic (gender, age, nationality, education) and clinical factors (reported medication adherence, CD4 cell count, viral load). The mean age of patients was 41.2 (SD = ±10.9) years and most were female (n = 115, 57%). None of the examined characteristics were significantly associated with provider-recorded clinical status. Higher CD4 cell counts were more likely for females (OR = 2.2 CI: 1.12–4.31) and less likely for those with higher viral loads (OR = 0.33 CI: 0.15–0.72). Poorer adherence and lower CD4 cell counts were significantly associated with higher viral loads (OR = 4.46 CI: 1.11–20.29 and 6.84 CI: 1.47–37.23, respectively). Clinics using provider-reported summaries of clinical status should evaluate the performance of these assessments to ensure they are associated with biologic indicators of disease progression.

Introduction

Though incidence is declining, the Caribbean still has the highest prevalence of human immunodeficiency virus (HIV) in the western hemisphere.^1,2 Prevalence is unequally distributed across the region (ranges from <0.1% to 3%) and approximately 12,000 AIDS-related deaths are reported annually.^3–5 The persistent threat of HIV-related illness has led to an amplified international response focused on improving prevention and treatment.^6–8 As a result, antiretroviral therapy (ART) can now significantly increase life expectancy, decrease mortality, and prevent transmission when optimal adherence is obtained.^9–12 Treatment is most effective when initiated in the early stages of infection; however, many individuals with HIV in the Caribbean present for treatment in advanced disease states leading to high mortality rates in the first three months of treatment.^4,13–15 This pattern is observed in the Dominican Republic, where high early mortality rates (34.2 deaths/100 person-years) from HIV have been recorded even though 78% (around 20,000 people) of those eligible for ART receive it.^4,16

In limited-resource settings such as the Dominican Republic, deciding when to initiate ART, change treatment based on the emergence of resistance, and confirming treatment failure can be dependent on a combination of laboratory and clinical factors, as well as what is available and/or cost-effective.^17,18 Routine laboratory testing of both immunologic and virologic markers is critical to prevent early treatment adjustment to second line therapy, correctly identify patients in treatment failure, and improve patient survival rates.^17–20 Medical record or patient monitoring systems are necessary to effectively track these laboratory results as well as to inform providers regarding HIV treatment needs, rates of treatment adherence, and to monitor patient's clinical status. In 2006, the WHO created guidelines for designing patient monitoring systems for HIV care in limited-resource settings to ensure patient monitoring is sufficient and reliable.^21,22 These guidelines which have since been updated, outline the documentation necessary for effective clinical management and to satisfy national reporting standards. The medical record system used at Clínica de Familia in La Romana, Dominican Republic, was designed using the WHO guidelines and implemented in 2011.²² One of the indicators located in the medical records is a provider-assigned summary of clinical status. This indicator is only clinically useful if it reflects the underlying biological processes associated with HIV disease progression; however, its ability to do so has not yet been evaluated. Therefore, we aimed to assess its utility by examining the relationship between the provider-recorded summary of clinical status and other indicators of HIV progression available in the medical records. To ensure that these recorded characteristics were associated with HIV progression, we also assessed their association with established biological markers of disease progression, CD4 cell count and viral load.

Methods

Participants and study setting

Located in the southeast Dominican Republic, Clínica de Familia is a Dominican, non-governmental organization funded by grants from foundations and USAID, private donations, and income-generating activities. Since 2004, the clinic has offered comprehensive primary care to families and has also developed and maintained an HIV care and treatment program that provides free services to over 1600 HIV-positive patients annually.²³ The retrospective review consisted of 201 charts randomly selected from adult (≥18 years of age), HIV-positive patients (positive test result in their medical record) who had an appointment at the Clinic between August 1, 2013 and July 31, 2014. Sample size was determined by estimating that if a parameter or characteristic appeared in 5% of the population, we could be 95% certain that the true prevalence of that parameter in our source population was between 2 and 8%points (5% ±3%) from the value obtained from our sample.²⁴ This study was approved by the Columbia University Medical Center institutional review board (IRB) and by the Dominican Bioethics Committee, Consejo Nacional de Bioética en Salud (CONABIOS), prior to data collection. We requested a waiver of informed consent as the chart review represented no more than minimal risk and did not compromise any of the patients' rights.

Data collection

The review was conducted in August 2014. A random sample of 250 medical records of HIV-positive adults was selected using an online random number generator available at: http://openepi.com/Random/Random. htm. The first author, SS, used a data abstraction guide created in Excel to collect demographic characteristics and clinical indicators. During abstraction, inclusion criteria were confirmed and if they were not met or if the chart could not be located, the researcher continued with the next identifier on the randomized list until the target sample size, 201, was reached.

Study variables

Dependent variable: Provider-reported summary of clinical status

The first variable assessed was the provider-recorded summary of clinical status recorded in medical records as ‘good,’ ‘fair,’ or ‘bad.’ It is recorded as ‘fair’ or ‘bad’ in the medical records if the patient presented with clinical symptoms during one or more of their scheduled visits. To identify patients potentially in need of further clinical management, we dichotomized this variable as ‘has symptoms’ or ‘does not have symptoms’ where ‘fair’ and ‘bad’ clinical status were combined to create the ‘has symptoms’ category.

Dependent variable: Most recent CD4 cell count

The second variable assessed was most recent CD4 cell count categorized as <200, 200–499 or > 500 cells/mm² according to international guidelines for ART initiation when CD4 cell count falls below 500 cells/mm³ and national guidelines to initiate therapy when CD4 cell count falls below 350 cells/mm³ which falls in the second category of the dependent variable.^25,26 The third category, CD4 cell count < 200 cells/mm³, is a critical cutoff used to identify high-risk patients, AIDS diagnosis, and is also as a guide for initiation of prophylaxis of opportunistic infections.^25,27

Dependent variable: Most recently recorded viral load

The third variable assessed was the most recently recorded viral load dichotomized as ≤1000 or >1000 copies/mL based on WHO guidelines defining treatment failure as having a plasma viral load of >1000 copies/mL on two consecutive measurements at least three months apart.^19,25

Independent variables

Independent variables from regression models included age (years), sex (male or female), nationality (Dominican or Haitian), marital status (married, single or domestic partnership), highest level of education completed (no formal education, some or all of primary school, some or all of secondary school and some or all of university), length of time since diagnosis (years), and length of time at the Clinic (years). Provider-recorded adherence was charted as ‘good’ (did not miss a dose in the last three days), ‘fair’ (missed one dose in the last three days), or ‘bad’ (missed more than one dose in the past three days) and was dichotomized for regression analysis as ‘good’ or ‘not good’ where an adherence status recorded as ‘fair’ or ‘bad’ were combined into the ‘not good’ adherence category to indicate patients potentially needing adherence counseling.

Statistical analyses

Three regressions were conducted. The first was a logistic regression to determine the characteristics significantly associated with the provider-recorded clinical status. To verify that patient characteristics from the medical records were associated with HIV disease progression, we conducted two additional regressions to assess their association with known predictors of disease progression, CD4 cell count and viral load.

Initially, descriptive statistics were calculated and included frequencies for categorical variables and means with standard deviations for continuous variables. Bivariate analyses were then conducted to assess associations between the three dependent variables and the independent variables defined above. Chi square or Fisher's exact tests were used for categorical variables, while continuous variables were compared with the dichotomous clinical status and viral load variables using simple logistic regression. The relationship between the continuous independent variables and the categorical dependent variable, CD4 cell count, was assessed using ordinal logistic regression, with ≥ 500 cells/mm³ CD4 cell count as the reference group. Variables associated with dependent variables with p value of ≤0.10 were included in final regression models. For the two dichotomous dependent variables, clinical status and most recent viral load, multivariable logistic regression was conducted with significantly associated variables from initial analyses of association. For the categorical CD4 dependent variable, a multi-variable ordinal logistic regression was used based on the assumption that the categories of CD4 cell count (<200, 200–499 or ≥ 500 cells/mm³) followed an ordering from higher to lower cell counts and thus satisfied the proportional odds criteria. Observations with missing values for dependent or independent variables were excluded from final regression models. All analyses were conducted with SAS version 9.3 (SAS Institute, Cary, NC).

Results

Demographics

The mean age of patients was 41.2 years (SD = ±10.9, range 19.4–76.7) (Table 1). The majority of patients were female (n = 115, 57.2%), Dominican (n = 164, 81.6%), and lived in La Romana (n = 138, 68.7%). A small number of patients were married (n = 13, 7.1%), while the rest were either single (n = 91, 49.5%) or in a domestic partnership (n = 80, 43.5%). Patients had either no formal education (n = 38, 18.9%), had completed some or all of primary school (n = 85, 42.3%), some or all of secondary school (n = 56, 27.9%), and eight (4%) had attended all or some university. The mean time participants had been living with an HIV diagnosis was 5.1 years (SD = ±3.7, range 0.3–22.2) and had been attending the Clinic for a mean of 4.5 years (SD = ±3.0, range 0.2–12.6). Of participants, 180 (89.6%) were on ART and (n = 105, 64.8%) had a viral load ≤1000 copies/mL.

Table 1.

Characteristics of HIV-positive, adult patient population.

Characteristic	N=201a
Age (years), mean ± SD	41.2 ± 10.9
Range	19.4–76.7
Sex, N (%)
Male	86 (42.8)
Female	115 (57.2)
Lives in La Romana, N (%)
Yes	138 (68.7)
No	63 (31.3)
Race, N (%)
Dominican	164 (81.6)
Haitian	34 (16.9)
Highest level of education, N (%)
No formal education	38 (18.9)
Primary school	85 (42.3)
Secondary or high school	56 (27.9)
University	8 (4.0)
Marital status, N (%)
Married	13 (7.1)
Single	91 (49.5)
Domestic partnership	80 (43.5)
Primary language, N (%)
Spanish	153 (76.1)
Creole	23 (11.1)
Both	12 (6.0)
Currently on ART, N (%)
Yes	180 (89.6)
No	21 (10.5)
CD4 cell count (cells/mm3), N (%)
<200	38 (18.9)
200–499	96 (47.8)
≥500	61 (30.5)
Viral load (copies/mL), N (%)
> 1000	57 (35.2)
≤1000	105 (64.8)
Time since diagnosis (years), mean±SD	5.1 ±3.7
Range	0.3 – 22.2
Time at the clinic (years), mean ±SD	4.5 ±3.0
Range	0.2 – 12.6

^aSample sizes vary based on missing data.

Associations between provider-recorded clinical status and patient characteristics

The final model's convergence criterion was satisfied, the max-rescaled r-square value was 0.23, and the area under the receiver operator curve (ROC) was 0.77. There were no significant associations between provider-assessed clinical status and tested characteristics after controlling for the other covariates in the model (Table 2).

Table 2.

Adjusted odds ratios from logistic regression to determine factors significantly associated with provider-recorded clinical status (N = 146).

Variable	Adjusted OR (95% CI)a
Age (years)	0.97 (0.93–1.01)
Sex
Female	Ref
Male	2.35 (0.88–6.85)
Nationality
Dominican	Ref
Haitian	0.41 (0.12–1.43)
Highest level of education, N (%)
Secondary school or university	Ref
No education or primary school	0.41 (0.13–1.15)
Provider-recorded adherence
Good	Ref
Not good	0.38 (0.14–1.03)
Not taking ART	0.48 (0.08–3.13)
Most recent CD4 cell count (cells/mm3)
>500	Ref
200–499	1.24 (0.46 – 3.29)
<200	2.01 (0.47 – 10.34)
Most recent Viral Load (Copies/mL)
≤1000	Ref
>1000	0.50 (0.18–1.42)

Note: Hosmer and Lemeshow goodness-of-fit test: p > 0.15.

^aEstimates were adjusted for all other variables in the model.

Associations between most recent CD4 cell count and patient characteristics

The final model's convergence criterion was satisfied, the max-rescaled r-square value was 0.28 and the score test for proportional odds assumption had a p value of 0.22, concluding that our assumptions held. The percent concordance was 74.7, indicating that 75% of the pairs of observations with different observed responses had a lower CD4 cell count (lower response value) with a lower predicted mean score than the observations with a higher CD4 cell count (higher response value).²⁸ Individuals whose first recorded CD4 cell count was <200 cells/mm³ had 0.15 times the odds (95% CI: 0.05–0.42) of having a most recent CD4 cell count ≥500 cells/mm³ (Table 3). Women had 2.2 times the odds (95% CI: 1.12–4.31) of having a CD4 cell count ≥500 cells/mm³ than men and those with an initial viral load >1000 copies/mL had 0.33 times the odds (95% CI: 0.15–0.72) of having a CD4 cell count ≥500 cells/mm³ than those who had an initially recorded viral load ≤1000 copies/mL after controlling for the other covariates in the model.

Table 3.

Adjusted odds ratios from regression to determine factors significantly associated with most recent CD4 cell count (N = 152).

Predictors	Adjusted OR (95% CI)a
Age (years)	1.01 (0.98–1.04)
Time with diagnosis (years)	0.92 (0.80–1.06)
Time at clinic (years)	1.19 (0.99–1.44)
Sex
Male	Ref
Female	2.2 (1.12–4.31)
Initial CD4 cell count (cells/mm3)
≥500	Ref
200–499	0.60 (0.22–1.62)
<200	0.15 (0.05–0.42)
Most recent viral load (copies/mL)
≤1000	Ref
>1000	0.33 (0.15–0.72)

Note: Score test for the proportional odds assumption: p >0.22.

^aEstimates were adjusted for all other variables in the model.

Note: Significant associations indicated in bold.

Associations between most recent viral load and patient characteristics

The final model's convergence criterion was satisfied, the max-rescaled r-square value was 0.64, and the area under the ROC was 0.93 indicating an excellent model fit (Figure 1). Factors significantly associated with likelihood to be in treatment failure were, ‘not good’ adherence to ART, most recent CD4 cell count, and first recorded viral load (Table 4). Those with ‘not good’ adherence had 4.46 times the odds of having a viral load >1000 copies/mL (95% CI: 1.11–20.29) than those who had ‘good’ adherence. Patients with a most recent CD4 cell count <200 cells/mm³ had 6.84 times the odds (95% CI: 1.47–37.23) of having a viral load >1000 copies/mL than those with a most recent CD4 cell count ≥500 cells/mm³. Those with first recorded viral load >1000 copies/mL had 34.8 times the odds (95% CI: 10.97–138.51) of having a most recent viral load >1000 copies/mL than those with a first recorded viral load ≤1000 copies/mL after controlling for the other covariates in the model.

Figure 1.

Receiver operating curve for logistic regression model to determine factors significantly associated with viral load >1000 copies/mL. The ROC curve shows the sensitivity and specificity threshold for each point on the curve. The area under the curve indicates that the model has an excellent fit.

Table 4.

Adjusted odds ratios from logistic regression to determine significant associations with most recent viral load (N=144).

Predictors	Adjusted OR (95% CI)a
Age (years)	0.96 (0.90–1.01)
Nationality
Dominican	Ref
Haitian	1.83 (0.23–16.46)
Time with diagnosis (years)	0.77 (0.53–1.12)
Time at clinic (years)	1.17 (0.75–1.77)
Provider-recorded adherence
Good	Ref
Not good	4.46 (1.11–20.29)
Not taking ART	5.03 (0.48–94.91)
Most recent CD4 cell count (cells/mm3)
≥500	Ref
200–499	2.13 (0.64–7.63)
<200	6.84 (1.47–37.23)
Initial viral load (copies/mL)
≤1000	Ref
>1000	34.78 (10.97–138.51)

^aEstimates were adjusted for all other variables in the model.

Note: Significant associations indicated in bold. Hosmer and Lemeshow goodness-of-fit test: p > 0.54.

Discussion

We used a retrospective chart review to assess whether a provider-recorded clinical status summary was associated with established biologic markers of HIV disease progression. None of the tested characteristics were significantly associated with provider-recorded clinical status as it is currently operationalized in the medical records, but there were significant associations between those characteristics and CD4 cell count and viral load. Hence, the provider-recorded clinical status may not be capturing the underlying biological processes that determine the clinical trajectory of patients. Contrary to previous research, the most recent CD4 cell count was not significantly associated with clinical status in our models nor was viral load, or reported adherence, although all have been shown to be independently associated with clinical status in the literature.^13,29–31

For the clinical-status variable to be useful and reliable for clinical decision-making, it should be standardized, providers trained to use it, and then quality checks conducted to ensure effectiveness. As this measure is currently recorded, there is no standardized way that providers assess whether a patient has a ‘good’, ‘fair’, or ‘bad’ clinical status, leading to inconsistencies in how a patient's clinical status is recorded. A standardized definition of clinical status and staff training in how to use it are essential to assure that assessments across patients are valid and reproducible. In addition, the use of this clinical assessment measure would require ongoing quality monitoring to ensure that it is accurately capturing patient's clinical status. In addition to improving and standardizing the clinical assessment of patients, clinicians must also be cognizant of the need for ongoing monitoring of CD4 cell counts and viral loads as biologic markers of disease progression.

Several routinely collected patient data elements were associated with both current CD4 cell count and most recent viral load. Sex, first recorded CD4 cell count, and most recent viral load were significantly associated with CD4 cell count. Similarly, reported adherence, most recent CD4 cell count, and first recorded viral load were associated with viral load >1000 copies/mL. We had also anticipated that first recorded CD4 cell count and viral load would be significantly associated with most recent CD4 cell count and viral load, respectively. One unique contribution of this study, however, is the independent association between CD4 cell count and viral load, as there is some controversy about the utility of CD4 cell counts to accurately and consistently predict viral load.^32,33 We add to the evidence that current CD4 cell count is significantly associated with viral load, which is particularly useful in resource poor settings where viral loads may not always be available.

Provider-recorded adherence was not significantly associated with CD4 cell count as would have been expected. An abundance of literature has shown that adherence is associated with CD4 cell count and we would have expected this to show in our models.^13,34,35 Adherence was, however, significantly associated with viral loads >1000 copies/mL. This is to be expected since adherence to ART is considered the gold standard for identifying treatment failure,^21,32 which leads us to conclude that use of the more defined guidelines for adherence which are available for provider use within the medical records may make this a more discriminating variable. However, there is no guarantee that these guidelines are strictly followed. Also, provider-recorded adherence at the Clinic is based on patients' self-report, which may negatively influence accuracy of reporting. Further training for providers on how to enquire into treatment adherence in a way that minimizes reporting bias and reiterating the importance of precisely recording a patient's adherence may increase the functionality of this measure.

There are several limitations associated with this study. The statistical power was limited by small sample size, which precluded finding potentially small associations. Missing data (between 49 and 57 observations were omitted from regressions), resulting from difficulty reading handwritten charts, exacerbated this problem. Additionally, because of our sample size, we were unable to control for additional comorbidities, for example tuberculosis, which may have affected the patient's clinical status. Furthermore, because of small cell counts for categorical variables, we collapsed two of the dependent variables into dichotomous categories, which may have masked our ability to find significant associations. Regardless of these limitations, this study is useful to inform providers and administration that the variables recorded in the chart based on provider judgment (clinical status and adherence) may not be the most reliable measures of HIV status as they fail to be significantly associated with previously determined indicators of disease progression. Therefore, when making clinical and treatment decisions, these indicators should be used with caution. Additionally, how clinicians report clinical status and adherence in the charts should be reconsidered so that what is recorded is more clinically useful. These results will be useful to other healthcare organizations in similar settings so they may improve documentation in patients' records. Future research in this area should evaluate indicators collected and used within the medical records of HIV-positive patients to ensure the information is a consistent and accurate reflection of what is biologically sound and clinically relevant to the patient.

Conclusion

The provider-assessed summary of clinical status within the medical records was not significantly associated with biologic markers previously established to be significantly associated with HIV disease progression and should be used with caution during clinical decision-making. We recommend that providers at the clinic update how this variable is operationalized in the medical records, train providers on effective use of the variable, and monitor the quality of the resultant measure. Providers must also continue to refer to laboratory measured immunologic and virologic data when making treatment decisions for HIV-positive patients as they are shown to be the most effective indicators of client status. Other healthcare sites that use provider-assigned summaries of clinical status should evaluate the performance of these assessments to ensure they are truly associated with disease progression.