Discordance between patient and clinician report of multidrug-resistant tuberculosis adverse drug reactions

Summary

Setting

An urban outpatient clinic in Durban, South Africa providing community-based treatment for drug-resistant TB.

Objective

Describe concordance between patient report and clinician documentation of adverse drug reactions (ADRs) from multidrug-resistant tuberculosis (MDR-TB) treatment.

Design

ADRs were documented by interview using an 18-item symptom checklist and medical record data abstraction during cross-sectional parent study with 121 MDR-TB patients, 75% co-infected with HIV. Concordance was analyzed using Cohen’s kappa statistic, Gwet’s AC1, and McNemar’s test.

Results

ADRs were reported much more frequently in the patient interviews (μ = 8.6) compared to medical records (μ = 1.4). Insomnia was most common (67 vs. 2%), followed by peripheral neuropathy (65 vs. 18%), and confusion (61 vs. 4%). Kappa scores were very low, with the highest degree of concordance found in hearing loss (kappa = 0.23), which was the only ADR not found to be significantly different between the two data sources (p = 0.34).

Conclusions

Our study showed a lack of concordance between patient report and clinician documentation of ADRs. These findings indicate the need for improved documentation of ADRs to better reflect the patient experience during MDR-TB treatment. These data have important implications for country-level pharmacovigilance programs that rely on clinician documentation of ADRs for MDR-TB policy formation.

Globally, treatment success for multidrug-resistant tuberculosis (MDR-TB) remains alarmingly low, approximately 50%, in many settings.¹ With an estimated 480,000 new cases of MDR-TB diagnosed each year, treatment default and failure represent a major public health crisis.¹ Tolerability of the regimen is a large part of the problem,² with patients describing the treatment as “worse than the illness itself”.³ Adverse drug reactions (ADRs) continue to be one of the greatest challenges to treating MDR-TB,⁴ especially in high HIV-burden populations found in sub-Saharan Africa.^5,6

The Stop TB Strategy outlined by the World Health Organization (WHO) to reach the 2015 Millennium Development Goals emphasized the need for patient-centered models of care.⁷ While MDR-TB care has been decentralized to community settings to help meet this goal, there has been a lack of research on patient-reported outcomes.⁸ The majority of studies on MDR-TB treatment-related ADRs have been retrospective cohorts utilizing medical record data abstraction of clinician report of ADRs.⁹ Several studies outside the MDR-TB context have shown a lack of concordance between patient report and clinician medical record documentation.^10–14

The WHO has called for greater emphasis on the detection, assessment, management, and prevention of ADRs during TB treatment, which is referred to as pharmacovigilance.^15,16 The incidence of ADRs is generally obtained from the medical records and thus relies entirely on clinician documentation. Since there is considerable under-reporting of ADRs by clinicians, this is a major drawback of pharmacovigilance, as it can lead to an inaccurate representation of actual outcomes.^17,18

Patient-reported outcomes are subjective reports of a patient’s health status obtained directly from the patient without interpretation from the clinician. Symptoms are the way patients experience ADRs; hence, patient self-report is the gold standard for recording symptoms.¹⁹ Clinicians are often unaware of patient symptoms, especially those that cannot be objectively measured, or may discount symptoms of which they are aware.^20,21 Yet, patient-reported outcomes have been found valid and reliable in comparison to objective measures,^22,23 able to detect serious ADRs earlier than clinician report,²⁴ and to be significantly associated with nonadherence.²⁵

A cross-sectional, observational study was conducted in 2014 to document patient-reported outcomes during MDR-TB treatment. This paper provides an analysis of the discordance found in that parent study between ADRs reported during patient interviews and ADRs documented by the clinicians in the medical record.

METHODS

Setting

The parent study was conducted in an urban, outpatient clinic in Durban, KwaZulu-Natal Province, South Africa providing community-based treatment for drug-resistant TB with over 250 outpatient visits twice each week. KwaZulu-Natal Province has the highest prevalence of MDR-TB in South Africa.²⁶ MDR-TB patients are seen once a month by a clinician at the clinic.

MDR-TB and HIV treatment overview

MDR-TB, defined as TB resistant to the two first-line medications — isoniazid and rifampicin — presently requires a minimum of two years to treat. This is divided into an initial intensive phase, typically 6–8 months, depending on the date of the first negative sputum culture result, and an 18-month continuation phase. The difference between the phases is the administration of a second-line injectable agent during the intensive phase.⁴ At the time of the study, MDR-TB patients in KwaZulu-Natal received a standardized regimen of kanamycin, moxifloxacin, terizidone, ethionamide, ethambutol, and pyrazinamide during the intensive phase.²⁶ Diagnosis at the study site was made through the GeneXpert MTB/RIF point-of-care nucleic amplification test, which tests for rifampicin resistance only. For this reason, MDR-TB patients were also placed on isoniazid until drug susceptibility and culture results returned confirming isoniazid resistance. For patients co-infected with HIV, the first-line antiretroviral therapy (ART) consisted of a single, fixed-dose combination pill of efavirenz, tenofovir, and emtricitabine.

Data collection

Convenience sampling was used to recruit patients waiting outside the clinic from May – July 2014. MDR-TB patients still receiving injections during the intensive phase of treatment were eligible for the study. As there were no validated scales to capture symptoms associated with MDR-TB treatment, the HIV Symptom Index checklist was modified for MDR-TB and translated to isiZulu by a professional translation service in South Africa.²¹ The principal investigator and an isiZulu-speaking research assistant conducted the interviews, each lasting approximately 20 minutes. Participants were read a list of 18 symptoms (Table 1) in either English or isiZulu and asked to recall if they experienced each symptom in the past 30 days. Symptoms present before treatment initiation either by participant self-report or medical record abstraction were excluded.

Table 1.

Definitions used to indicate presence of ADR by patient interview and medical record data abstraction

ADR	Wording used in patient interview	Clinician documentation in medical record
Insomnia	“Difficulty falling or staying asleep?”	“Insomnia” and prescription for diphenhydramine
Peripheral Neuropathy	“Pain, numbness or tingling in the hands or feet?”	“Burning feet”, “painful feet”, “pins and needles” and/or prescription for low-dose amitriptyline (25mg) without other indication
Confusion	“Trouble remembering or confusion?”	“Confusion” or “hallucinations” and/or prescription for haloperidol
Nausea or Vomiting	“Nausea or vomiting?”	“Nausea” or “vomiting” and/or prescription for metoclopramide
Depression	“Felt sad, down or depressed?”	N/A
Rash	“Skin problems such as rash, dryness or itching?”	“Itching”, “rash” or “dry skin” and/or prescription for steroid cream
Dizziness	“Feeling dizzy or lightheaded?”	“Dizziness”
Gastritis	“Bloating, pain or gas in your stomach?”	“Heartburn”, “gastric reflux”, or “abdominal pain” and/or prescription for aluminum hydroxide/magnesium oxide
Myalgia	“Muscle aches?”	“Muscle pain” or “muscle cramps”
Anorexia	“Loss of appetite/ change in taste of food?”	“Lack of appetite”
Arthralgia	“Joint pains?”	“Joint pain” or pain in “shoulder, knee or ankle”
Changes in Vision	“Changes in vision?”	“Blurry vision”, “poor vision”, or “deterioration of vision” and/or referral to eye clinic
Tinnitus	“Ringing in the ear?”	Audiology report or clinician documentation of “ringing”, “buzzing” or “noise in ear”
Anxiety	“Felt nervous or anxious?”	N/A
Fatigue	“Fatigue or loss of energy?”	“Tiredness”
Hearing Loss	“Loss of hearing?”	Audiology report of increased hearing loss compared to baseline
Headache	“Headache?”	“Headache”
Diarrhea	“Diarrhea or running stomach?”	“Diarrhea”

Once a symptom, which started after the initiation of treatment, was identified (i.e., an ADR), the participant was asked to rate the degree of bother from the symptom on a Likert scale ranging from 1 (no bother) to 4 (bothers me a lot) as a measure of perceived severity. A standardized definition of an ADR was used to allow for a more accurate comparison to ADRs recorded in the medical chart and to eliminate minor unwanted reactions; the reaction had to be appreciably unpleasant or harmful enough for the patient and/or the clinician to consider the ADR problematic.²⁷ Therefore, if a patient reported a symptom as “no bother”, this symptom was no longer considered an ADR.

A data abstraction tool was designed to collect any documentation of ADRs identified through the medical records since the start of treatment. The definitions presented in Table 1 were used to standardize data collection.

Ethics approval

The study protocol was approved in South Africa and the USA. Ethical approval was granted by the Institutional Review Board at Michigan State University, the Biomedical Research Ethics Committee at the University of KwaZulu-Natal, and the KwaZulu-Natal Department of Health. The hospital site provided a letter of support for the study.

Data analysis

All data were analyzed using Stata Statistical Software: Release 13 (StataCorp, 2013, College Station, TX: StataCorp LP). Non-parametric statistics are reported. Percent agreement, Cohen’s kappa statistic, Gwet’s AC1, and McNemar’s test were used to determine the degree of concordance between the ADRs reported by the patient and documented by the clinician. The kappa statistic is presented with a 95% confidence interval calculated from the standard error of kappa. Gwet’s AC1 is presented for comparison to the kappa value, since kappa may be less resistant to the low prevalence of some of the ADRs. The p-value from McNemar’s test is presented to indicate significant differences between the two data sources, with the Bonferroni correction used to account for multiple comparisons to test each p-value at 0.05/18 = 0.0028, denoting the 18 ADR comparisons. Although multiple clinicians worked at the study site, this analysis was still considered a classic two-type, inter-rater reliability analysis, with one rater being the patients and the second rater being the clinicians.

RESULTS

Study sample

The sample included 121 MDR-TB patients in the intensive phase of treatment, 62 (51%) females with a median age of 33 (range 17 – 63) and a median time on MDR-TB treatment of 111 days (range 46 – 340) at the time of the interview. The HIV status of all participants was documented, with 90 (75%) participants co-infected with HIV and 79 (88%) receiving ART. Although exact ART start dates were not routinely documented, approximately 31 (39%) initiated ART during their MDR-TB treatment, 28 (35%) had been on ART for greater than six months, 16 (20%) for 1–6 months, and four had no date listed. Using a Mann-Whitney test, no significant difference was found in total ADRs by HIV status in the patient interviews U = −1.09, p = 0.277 or the medical records U = −1.66, p = 0.098.

Adverse drug reactions

All but two participants (98%) interviewed reported at least one ADR as compared with 78% who had at least one ADR documented in the medical record. The most common ADRs reported by the participants were insomnia (67%), peripheral neuropathy (65%), and confusion or trouble remembering (61%) (Figure 1). The most common ADRs documented by the clinicians in the medical records were loss of hearing (26%), peripheral neuropathy (18%) and nausea/vomiting (18%). The average number of ADRs per patient reported during the interview was 8.6, which was significantly different using a paired t-test from the 1.4 documented in the medical records, t(120) = 19.06, p < 0.001.

Figure 1.

Comparison of ADR frequency by patient interview and medical chart data abstraction.

The ADRs ranked with the highest degree of bother were arthralgia (72% cases reporting level 4: “bothers me a lot”), insomnia (69%), and changes in vision (62%). Clinicians did not routinely document severity or intensity of ADRs.

Concordance

Concordance between the patient interview and the medical record is presented in Table 2. Overall percent agreement includes both positive cases (ADR present) and negative cases (ADR absent). Kappa scores ranged from −0.007 (poor) to 0.23 (fair);²⁸ AC1 scores ranged from −0.23 to 0.68; percent agreement ranged from 33% to 75%. All ADRs were found to be significantly different by data source (p < 0.001), except for hearing loss (p = 0.34).

Table 2.

Concordance of ADRs by patient interview compared to clinician documentation

ADR	ADR from interview N (%)	ADR from medical record N (%)	Kappa (95% CI)	Gwet’s AC 1	Overall Percent Agreement*	P-value from McNemar’s Test
Insomnia	83 (68.6%)	2 (1.7%)	0.02 (0.05, −0.02)	−0.23	33.1%	<0.001
PN	78 (64.5%)	22 (18.2%)	0.02 (0.13, −0.09)	−0.12	42.2%	<0.001
Confusion	74 (61.2%)	5 (4.1%)	−0.001 (0.06, −0.06)	−0.08	39.7%	<0.001
N/V	69 (57%)	22 (18.2%)	0.07 (0.20, −0.05)	0.05	49.6%	<0.001
Depression	69 (57%)	0	0	0.04
Rash	65 (53.7%)	19 (15.7%)	0.09 (0.21, −0.04)	0.12	52.1%	<0.001
Dizziness	65 (53.7%)	10 (8.3%)	0.11 (0.21, 0.02)	0.11	52.9%	<0.001
Gastritis	64 (52.9%)	8 (6.6%)	−0.01 (0.08, −0.09)	0.09	47.1%	<0.001
Myalgia	61 (50.4%)	3 (2.5%)	0.02 (0.07, −0.04)	0.19	50.4%	<0.001
Anorexia	55 (45.5%)	6 (5%)	0.08 (0.17, 0.00)	0.32	57.9%	<0.001
Arthralgia	53 (43.8%)	14 (11.6%)	0.10 (0.23, −0.02)	0.32	59.5%	<0.001
Vision	52 (43%)	13 (10.7%)	0.2 (0.32, 0.08)	0.41	64.5%	<0.001
Tinnitus	50 (41.3%)	9 (7.4%)	0.10 (0.21, −0.01)	0.42	62.8%	<0.001
Anxiety	50 (41.3%)	0	0	0.39
Fatigue	49 (40.5%)	1 (0.8%)	−0.02 (0.02, −0.06)	0.39	58.7%	<0.001
Hearing	39 (32.2%)	32 (26.4%)	0.23 (0.40, 0.05)	0.45	67.8%	0.34
Headache	34 (28.1%)	4 (3.3%)	0.05 (0.15, −0.05)	0.62	71.9%	<0.001
Diarrhea	31 (25.6%)	1 (0.8%)	0.05 (0.10, −0.01)	0.68	75.2%	<0.001

^*Overall percent agreement was calculated by Stata and defined as the number of concordant counts (ADR present in both sources or absent in both sources) divided by the total sample size and expressed as a percent

The ADRs with the greatest concordance between patient self-report and the medical chart based on kappa were hearing loss (kappa = 0.23) and change in vision (kappa = 0.20). Although diarrhea and headache had the greatest percent agreement between the two sources, the kappas were low (0.05), likely due to the lower prevalence of these ADRs, with concordant negative cases inflating the percent agreement. Gwet’s AC1 also reflected this effect with diarrhea and headache showing the highest scores (0.68 and 0.62, respectively), but followed by the same ADRs identified using kappa of hearing loss (0.45) – in addition to tinnitus (0.42) – and vision changes (0.41).

DISCUSSION

Consistent with previous findings in patient populations outside TB, the authors believe this study is the first to demonstrate the discordance between patient self-report and clinician documentation of ADRs within the MDR-TB patient population. Comparable MDR-TB studies in South Africa reported an incidence of 99% for at least one ADR through patient interviews,²⁹ and 81% using medical record data abstraction,⁵ similar to the 98% by interview and 78% by medical record reported in this study.

The findings of this study have important implications for the MDR-TB pharmacovigilance program. First, insomnia was the most common ADR and one of the most bothersome, yet sleep disturbances were rarely documented in previous studies on MDR-TB ADRs.⁹ Another noteworthy finding was that neither depression nor anxiety was ever documented in the medical records. Even though both depression and anxiety are common ADRs during MDR-TB treatment,³⁰ psychiatric symptoms are often overlooked by clinicians.¹⁰

Not surprisingly, the highest kappa scores were found among ADRs that could be measured objectively. Hearing loss had the greatest kappa score, as all patients undergo baseline and monthly audiology screening on-site at the clinic to address the high degree of ototoxicity related to aminoglycoside administration.²⁶ The higher degree of concordance for hearing loss is promising, and the authors postulate that the early identification and management of hearing loss could also account for the lower bother rating from patients. Changes in vision had the second highest degree of concordance and was taken very seriously at the study site, as it may indicate the rare, but serious condition of optic neuritis as a result of ethambutol administration, which would constitute immediate removal of the offending agent. Of note, ethionamide was the probable cause of the high percentage of vision changes reported, but not as critical from a clinician standpoint.

The South African National Department of Health guidelines for the management of MDR-TB indicate that “patients must be interviewed weekly about adverse reactions to the drugs” and documented in the medical record.²⁶ This has been a challenge for healthcare workers for many reasons. The community-based patients in this study only return to the MDR-TB clinic once a month for a brief visit with a clinician. Patients often receive their daily kanamycin injections from nurses in the community who have limited MDR-TB training. A less than optimal relationship between patients and clinicians has also been cited as a problem in TB care.³¹ Previous studies have evaluated methods for improving documentation of patient-reported outcomes, such as the use of a nurse case manager fluent in the language of the patients³² or using a checklist system to capture ADRs versus the current method of clinicians using an open-ended question to ask patients if they are having any problems.¹² This study supports the use of a checklist system for identifying ADRs.

Of interest is the high number of patients co-infected with HIV who were started on ART concomitantly with MDR-TB treatment. This may have resulted in more frequent documentation of ADRs related to ART rather than from MDR-TB therapy. Yet, insomnia, a well-known ADR during HIV treatment, was not found to have a significant difference by HIV status, which was similar to the findings of two other studies conducted in South Africa documenting insomnia as an MDR-TB ADR without finding a statistically significant difference by HIV status.^5,29 This relationship warrants further exploration, as this study was not powered to determine this difference by HIV status.

This study was limited by the fact that medical records were handwritten, making them sometimes difficult to read. Clinic nurses provided assistance with interpreting clinician handwriting when needed. Another limitation was the potential for recall bias. With any study relying on self-report, there is a risk of participants over- or under- reporting symptoms.³³ To minimize recall bias, patients were asked to recall ADRs in the last 30 days, as memory has been shown to be most reliable up to 30 days in a similar TB population.³⁴ Another difficulty in recording ADRs is determining if the effect is due to the TB medication or the TB disease itself, which is why any symptoms present before treatment were not included. Another limitation was that data on which clinician documented each ADR was not collected, so it was not possible to control for variation among clinicians.

It is clear that patients’ ADRs are underreported in the medical record. In 2014, the South African National Department of Health introduced a checklist system to improve ADR reporting during MDR-TB treatment. The checklist represents all the symptoms listed in this study, in addition to symptoms more likely with HIV treatment, and would provide an efficient method for documentation of ADRs during the monthly visit, as long as policy and staffing levels adequately reflect this addition to the workload.³⁵

CONCLUSIONS

The findings presented here are some of the first to document a quantitative analysis of patient-reported outcomes during MDR-TB treatment. Some discordance is expected, as symptoms may not be recorded due to a heavy workload and an understanding by the clinicians that triaging must take place, since some ADRs must be considered acceptable to get patients through this life-saving treatment. Yet, even considering the need for clinician-triaging and potential measurement bias of patient over-reporting during an interview, the significant difference in reporting for every ADR besides hearing loss was worrisome. The authors recommend a collaborative approach to ADR reporting, in which patients directly report their symptoms in a checklist fashion while waiting for their appointment with the clinician to better inform their treatment.²⁵ This integration of the patient’s perspective into the clinician’s reporting can only help to strengthen the overall accuracy of the active pharmacovigilance surveillance program and ultimately improve MDR-TB patient outcomes.