Abstract
SETTING:
Assessment of bedaquiline roll-out in South Africa requires accurate patient data in EDRWeb, a national case-based rifampicin-resistant TB (RR-TB) surveillance register.
OBJECTIVE:
To ensure EDRWeb data reflect programmatic DR-TB source data, we implemented a data quality improvement initiative.
DESIGN:
We conducted data quality assessments of EDRWeb data compared to paper patient folders at two South African RR-TB treatment facilities in 2015 and 2016. We assessed 80 patient records before the intervention for completeness of clinically relevant data fields, and 80 different records after the intervention for completeness and concordance. The intervention involved reviewing and updating EDRWeb along with data quality audits with direct feedback to sites.
RESULTS:
At baseline data completeness per site was lowest for variables related to electrocardiogram (ECG) data, adverse events, and concomitant medications (completeness for these fields ranged from 0% to 80%). Post-intervention data completeness and concordance were high for all fields except those related to ECG data (ECG-related field completeness ranged from 10% to 100%).
CONCLUSION:
After a data quality initiative, data completeness improved at each site with the exception of ECG data fields. Our findings suggest that data quality interventions may improve patient clinical registries, ultimately enabling better evidence-based decision making for TB programmes.
Keywords: drug-resistant tuberculosis, data quality, information systems
Abstract
CONTEXTE :
L’évaluation du lancement de la bédaquiline en Afrique du Sud requiert des données précises relatives aux patients sur EDRWeb, notamment grâce à un registre de surveillance national de cas de la TB résistante à la rifampicine (RR-TB).
OBJECTIF :
Pour s’assurer que les données d’EDRWeb reflètent les sources des données de programme de TB résistante, nous avons mis en œuvre une initiative d’amélioration de la qualité des données.
SCHÉMA :
Nous avons réalisé des évaluations de qualité des données d’EDRWeb comparés aux dossiers papier des patients dans deux structures de traitement de RR-TB en Afrique du Sud en 2015 et 2016. Nous avons évalué 80 dossiers de patients avant l’intervention pour l’exhaustivité des données des domaines pertinents et 80 autres dossiers après l’intervention à la recherche d’exhaustivité et de concordance. L’intervention a consisté à revoir et à mettre à jour EDRWeb avec des audits de qualité des données directement renvoyés aux sites.
RÉSULTATS :
Au départ, l’exhaustivité des données dans chaque site était la plus faible en ce qui concerne les variables liées à l’électrocardiogramme (ECG), aux effets secondaires et aux médicaments concomitants ; elle allait de 0% à 80%. L’exhaustivité des données post-intervention et leur concordance ont été élevées pour tous les champs excepté ceux liés aux données de l’ECG qui allaient de 10% to 100%.
CONCLUSION :
Après une initiative de qualité des données, leur exhaustivité s’est améliorée dans chaque site à l’exception des données relatives à l’ECG. Nos résultats suggèrent que les interventions de qualité des données pourraient améliorer les dossiers cliniques des patients, ce qui permettrait des meilleures prises de décision, basées sur des preuves, pour les programmes TB.
Drug-resistant TB (DR-TB) remains a high priority for national TB programmes due to its high mortality and cost of treatment. Bedaquiline (BDQ) containing treatment regimens for rifampicin-resistant TB (RR-TB) have demonstrated improved clinical outcomes.1–3 The WHO currently recommends the incorporation of BDQ in programmatic RR-TB regimens.4,5
Both drug-susceptible TB (DS-TB) and DR-TB have had a devastating public health impact in South Africa, which has among the highest estimated TB incidence rates in the world, and had over 13,000 cases of RR-TB in 2018.6,7 BDQ became available in South Africa through the Bedaquiline Clinical Access Programme in 2013 for use in patients with previously limited treatment options.8
TB programmes require accurate patient-level data to monitor and improve treatment outcomes, provide surveillance data and inform resource allocation. The WHO provided guidance for electronic recording and reporting of TB data in 2012, although the success of implementation varies, particularly in resource-constrained settings.9–12 Poor data quality in both paper records and electronic health registries remain problematic in many TB high-burden settings.13–15 The national electronic health register for DR-TB patients in South Africa is the web-based, electronic DR-TB register (EDRWeb).16
In the present study, we assessed data completeness related to the use of BDQ-containing treatment regimens in EDRWeb compared to patient folders before and after implementing a team model intervention. We also evaluated completeness of a cross-section of EDRWeb data and concordance of a subset of post-intervention data to assess data quality in a high TB burden province of South Africa.17
METHODS
Study design
We performed a retrospective data quality assessment of patient folders compared to EDRWeb data from two high RR-TB burden treatment facilities in the Eastern Cape Province of South Africa from 2015 to 2016. We used data collected before and after the implementation of a team model intervention at two TB treatment facilities. We also assessed cross-sectional EDRWeb data from all RR-TB patients treated at both facilities for post-intervention data completeness. We evaluated a subset of post-intervention data for concordance.
Routine data
Patient folders are paper charts containing a patient’s primary RR-TB medical records maintained at each facility. EDRWeb is an electronic RR-TB register used in all nine provinces in South Africa for monitoring and evaluation of key performance indicators by the South African National TB Control Programme. Data capturers routinely enter data from the patient folders into EDRWeb for all patients with RR-TB who are initiated on RR-TB treatment. Data capturers receive initial orientation at job initiation, but otherwise have limited contact with clinical staff.
Team model intervention
The intervention involved a combination of training, quality improvement and monitoring visits conducted by clinical and roving teams to strengthen data quality in patient folders and EDRWeb. The clinical team was composed of a data capturer and a professional nurse at each site, and a pharmacy assistant at Jose Pearson TB Hospital, Port Elizabeth, South Africa. The role of the professional nurse was the accurate abstraction of clinical source data from patient clinical folder notes to a data quality assessment tool developed by the team. The professional nurse also mentored the data capturer, whose role involved the accurate entry and correction of clinical field data in EDRWeb. The pharmacy assistant supported the professional nurse in data abstraction and completion of adverse drug reaction forms for submission to the National Department of Health Pharmacovigilance Centre. A roving team, composed of a data manager and professional nurse, performed data quality audits at each site after completion of data abstraction and EDRWeb data entry. The findings of both the clinical team and those of the roving team were reviewed by a clinical advisor.
The intervention was implemented in June 2018 up to December 2019. Prior to the start, a workshop for the two teams was held once at each site and involved didactic training about clinical management of RR-TB, pharmacovigilance and data management, including data review, use of the data quality assessment tool, and EDRWeb data entry and correction. The clinical team performed the intervention once for the records of each patient treated for RR-TB from 2015 to 2016. The roving team performed data quality audits once yearly at each site. The clinical advisor performed training updates on site-specific areas of need identified by clinical and roving teams once yearly at each site. For example, when a roving team data audit from one site showed low pre-intervention completeness data for adverse events, feedback was given to address the problem. Additional training updates occurred with the release of newer versions of EDRWeb.
To ensure sustainability of data quality improvement efforts, a formal workshop was held after the intervention period to train site nurses, pharmacy assistants and data capturers to use data abstraction and entry standard operating procedures developed by the intervention clinical team. After the workshop, site nurses and data capturers were mentored during monthly onsite sessions from April to September 2020.
Study population
Records of all patients who received BDQ-containing treatment regimens for RR-TB at the two sites from 2015 to 2016 were eligible. We randomly selected patients for before-and-after data completeness assessment using Stata v15.1 (StataCorp, College Station, TX, USA). Patients who received RR-TB treatment without BDQ were also included for post-intervention data review. We used all eligible RR-TB patients treated at the two TB treatment facilities for the cross-sectional post-intervention data completeness analysis, and a randomly selected subset for concordance analysis.
Study sites
Jose Pearson TB Hospital (Port Elizabeth) and Nkqubela TB Hospital (East London, South Africa) were selected because they are specialised district treatment centres for inpatients and outpatients initiating treatment for multidrug-resistant TB (MDR-TB; defined as resistance to rifampicin and isoniazid) and extensively drug-resistant TB (XDR-TB; defined as MDR-TB plus resistance to a fluoroquinolone and a second-line injectable). The proportion of patients treated with BDQ increased from 2015 to 2016 at Jose Pearson TB Hospital (32% to 61%) and Nkqubela TB Hospital (17% to 26%). Upon DR-TB treatment initiation, patients are registered in EDRWeb, with updates from clinical encounters entered until treatment completion. Patient data entry occurs at the facility level and data flows via the district to the provincial DR-TB office, and in turn, to the National DR-TB Directorate.
Previously collected data
From all patients who were treated with BDQ-containing treatment regimens, 20 pre-intervention (baseline) patient folders were selected randomly for review per year (2015 and 2016) at each facility (80 total). Following the intervention, another 20 patient folders were selected randomly for review per year from each facility (80 total), such that pre- and post-intervention folder review was not necessarily performed on the same folders. Twenty folders of patients not treated with BDQ were reviewed per year at each site after the intervention, but not before. Because of changes to standard patient charts over time, 23 data fields were reviewed in 2015 and 26 data fields were reviewed in 2016; we reported 16 overlapping data fields.
Fields from patient folders at the two facilities were abstracted using the data quality assessment tool to assess data completeness and concordance. Data fields were selected according to importance for programmatic reporting and included fields that covered 1) demographics, 2) HIV status and treatment outcomes, 3) BDQ initiation, and 4) pharmacovigilance.
Analysis
We determined differences in data completeness of patient folders and EDRWeb before and after the data quality intervention. We defined completeness as the presence of data in predetermined fields in the patient folder or EDRWeb, with 100% considered optimal completeness.18 We expressed completeness as the proportion of total fields that were completed. In a separate analysis, we assessed data completeness in a cross-section of EDRWeb data from all RR-TB patients treated at the two sites. We also reported post-intervention data concordance between patient folders and EDRWeb in a random selection of patients treated at the two sites. We defined concordance as the agreement between the data in the predetermined fields of interest in the patient folder and EDRWeb.18 We compared the proportion agreed between the patient folder and EDRWeb data after the intervention using percentage agreement and κ statistic. We performed analyses using MS Excel v16.32 (MicroSoft, Redmond, WA, USA) and Stata.
Ethics approval
The study was approved by the Health Research Ethics Committee of The University of the Witwatersrand, Johannesburg, South Africa (HREC 190414), and received provincial approval.
RESULTS
Of 2,335 patients with RR-TB treated at both sites in 2015 and 2016, 1,256 (54%) were at Nkqubela TB Hospital and 1,079 (46%) at Jose Pearson TB Hospital (Table 1). Pre-intervention patients had a median age of 34.5 years, were predominantly male (54%) and HIV-positive (65%; Table 2). Post-intervention patients were similar: median age 36 years, male predominance (61%), and 70% HIV-positive. These did not differ according to whether or not treatment included BDQ. The majority of patients treated with a BDQ-containing regimen either had pre-extensively-resistant TB (defined as MDR-TB plus resistance to either a fluoroquinolone or a second-line injectable) or XDR-TB. In contrast, the majority of those whose treatment did not include BDQ had MDR-TB without additional resistance. Favourable treatment outcomes occurred in 69% of reviewed patients who received a BDQ-containing regimen (preand post-intervention) and 46% of those who did not receive BDQ.
TABLE 1.
Total number of drug-resistant TB patients with at least rifampicin resistance (RR-TB) at both study sites, those on BDQ-containing treatment regimens and the numbers of patient folders reviewed pre- and post-intervention per study year
| Year | Total number of patients with RR-TB | Patients on BDQ | Pre-intervention Post-intervention BDQ folders reviewed | BDQ folders reviewed | Post-intervention non-BDQ folders reviewed |
|---|---|---|---|---|---|
| 2015 | 879 | 232 | 40 | 40 | 40 |
| 2016 | 1456 | 563 | 40 | 40 | 40 |
| Total | 2335 | 795 | 80 | 80 | 80 |
RR-TB = rifampicin-resistant TB; BDQ = bedaquiline.
TABLE 2.
Characteristics of study patients at both sites from 2015 and 2016 before and after data quality intervention by type of TB treatment regimen (BDQ-containing and non-BDQ-containing)
| Pre-intervention | Post-intervention | ||
|---|---|---|---|
| BDQ (n = 80) n/N (%) |
BDQ (n = 80) n/N (%) |
Non-BDQ (n = 80) n/N (%) |
|
| Age, years, median [IQR] | 34.5 [29–44.5] | 36 [29–43.5] | 35.5 [26.5–41.5] |
| Female sex | 37 (46) | 26 (33) | 36 (45) |
| Previous TB treatment history | |||
| New | 36 (45) | 25 (31) | 34 (43) |
| Previous treatment with Regimen 1* | 29 (36) | 36 (45) | 30 (38) |
| Previous treatment with Regimen 2† | 15 (19) | 19 (24) | 16 (20) |
| Drug-resistant TB type | |||
| Rifampicin-resistant TB | 1 (1) | 1 (1) | 3 (4) |
| Multidrug-resistant TB | 37 (46) | 19 (24) | 55 (69) |
| Pre-XDR-TB | 21 (26) | 26 (33) | 7 (9) |
| XDR-TB | 21 (26) | 34 (43) | 13 (16) |
| HIV-positive | 52 (65) | 56 (70) | 56 (70) |
| Antiretrovirals started before TB diagnosis | 32/52 (62) | 39/56 (70) | 41/56 (73) |
| Antiretrovirals started at or after TB diagnosis | 20/52 (38) | 17/56 (30) | 13/56 (23) |
| Antiretrovirals not started | 0 (0) | 0 (0) | 2/56 (4) |
| Six-month interim outcome‡ | |||
| Died | 1 (1) | 3 (4) | 17 (21) |
| Culture conversion (two consecutive negative cultures) | 72 (90) | 70 (88) | 46 (58) |
| Smear- and/or culture-positive | 5 (6) | 4 (5) | 5 (6) |
| Unknown | 2 (3) | 3 (4) | 12 (15) |
| Final treatment outcomes | |||
| Cured | 51 (64) | 50 (63) | 33 (41) |
| Treatment completed | 5 (6) | 4 (5) | 4 (5) |
| Died | 11 (14) | 15 (19) | 25 (31) |
| Treatment failure | 5 (6) | 3 (4) | 6 (8) |
| Lost to follow-up | 8 (10) | 8 (10) | 12 (15) |
| Treatment success: cured/treatment completed | 56 (70) | 54 (68) | 37 (46) |
* 2HRZE4HR.§
† 2HRZES+1HRZE + 5 HRE.§
‡ No individuals had 6-month interim outcomes recorded as failure, transfer out or lost to follow-up.
§ H = isoniazid; R = rifampicin; Z = pyrazinamide; E = ethambutol; S = streptomycin. Numbers before the letters indicate the duration in months of the phase of treatment.
BDQ = bedaquiline; IQR = interquartile range; XDR-TB = extensively drug-resistant TB.
Nkqubela data completeness
Data completeness at Nkqubela TB Hospital was ⩾90% for demographic and HIV-related data fields in patient folders and EDRWeb before and after the intervention, except for the South African Identification Number (complete in 70% of pre-intervention EDRWeb entries in 2015) (Table 3). Completeness of BDQ treatment fields ranged from 60% to 90% at baseline in 2015 and 70–95% at baseline in 2016, but increased to 100% post-intervention for both years. However, electrocardiogram (ECG) corrected QT interval (QTc) and ECG findings field completeness ranged from 40% to 55% at baseline in paper charts and EDRWeb in 2015 and 2016, but had lower post-intervention completeness in 2015 (15% paper charts and EDRWeb) and stable post-intervention completeness in 2016 (55% paper charts and EDRWeb). ECGs were performed at facilities where ECG machines were available and working properly. Adverse events, onset dates of adverse events, and concomitant medications data fields had 0–25% completeness at baseline EDRWeb in 2015 and 2016. Post-intervention, however, these fields were 100% complete in both years for paper charts and EDRWeb.
TABLE 3.
Baseline and post-intervention data comparing completeness of data recorded in patient clinical records to EDRWeb for patients who received BDQ at Nkqubela TB Hospital, East London, South Africa, in 2015 and 2016
| 2015 | 2016 | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | Post-intervention | Baseline | Post-intervention | |||||||||||||
| Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | |||||||||
| n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | |
| Patient name and surname | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| DR-TB registration no. | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| DR-TB type | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| ID no. | 19 | 95 | 14 | 70 | 18 | 90 | 18 | 90 | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 |
| Registration type | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 | 18 | 90 | 20 | 100 | 20 | 100 | 20 | 100 |
| HIV status | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| HIV positive, on ART* | 12/12 | 100 | 12/12 | 100 | 12/12 | 100 | 12/12 | 100 | 14/14 | 100 | 13/14 | 93 | 16/16 | 100 | 16/16 | 100 |
| BDQ treatment start date | 16 | 80 | 18 | 90 | 20 | 100 | 20 | 100 | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 |
| BDQ treatment stop date | 12 | 60 | 12 | 60 | 20 | 100 | 20 | 100 | 14 | 70 | 14 | 70 | 20 | 100 | 20 | 100 |
| Reason for initiating BDQ | 18 | 90 | 17 | 85 | 20 | 100 | 20 | 100 | 19 | 95 | 19 | 95 | 20 | 100 | 20 | 100 |
| Baseline ECG QTcF | 11 | 55 | 8 | 40 | 3 | 15 | 3 | 15 | 11 | 55 | 8 | 40 | 11 | 55 | 11 | 55 |
| ECG findings | 11 | 55 | 9 | 45 | 3 | 15 | 3 | 15 | 11 | 55 | 8 | 40 | 11 | 55 | 11 | 55 |
| AE* | 13/20 | 65 | 0/20 | 0 | 16/16 | 100 | 16/16 | 100 | 10/20 | 50 | 0/20 | 0 | 20 | 100 | 20 | 100 |
| AE onset date* | 13/20 | 65 | 0/20 | 0 | 16/16 | 100 | 16/16 | 100 | 10/20 | 50 | 0/20 | 0 | 20 | 100 | 20 | 100 |
| Concomitant medications* | 6/20 | 30 | 5/20 | 25 | 20/20 | 100 | 20/20 | 100 | 2/20 | 10 | 1/20 | 5 | 18/18 | 100 | 18/18 | 100 |
| Treatment regimen | 20 | 100 | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 | 19 | 95 | 20 | 100 | 20 | 100 |
| Treatment outcome | — | — | — | — | — | — | — | — | 7 | 35 | 5 | 25 | 20 | 100 | 20 | 100 |
* Denotes variable denominator.
DR-TB = drug-resistant TB; ID = identification; ART = antiretroviral therapy; BDQ = bedaquiline; ECG = electrocardiogram; QTcF = corrected QT interval; AE = adverse event.
Completeness of Jose Pearson TB Hospital data
Completeness of data fields in Jose Pearson TB Hospital records was similarly lowest at baseline in EDRWeb for ECG QTc, ECG findings, adverse events, adverse event onset date and concomitant medications in both years (Table 4). Adverse events, adverse event onset date and concomitant medications increased to 100% post-intervention completeness in both years in paper charts and EDRWeb, but post-intervention completeness of ECG QTc and ECG findings was 75% in 2016 for both paper charts and EDRWeb (95% to 100% in 2015 post-intervention completeness).
TABLE 4.
Baseline and post-intervention data comparing completeness of data recorded in patient clinical records to EDRWeb for patients who received BDQ at Jose Pearson TB Hospital, Port Elizabeth, South Africa, in 2015 and 2016
| 2015 | 2016 | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline | Post-intervention | Baseline | Post-intervention | |||||||||||||
| Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | Patient folder (n = 20) | EDRWeb (n = 20) | |||||||||
| n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | n/N | % | |
| Patient name and surname | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| DR-TB registration no. | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| DR-TB type | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 18 | 90 | 20 | 100 | 20 | 100 | 20 | 100 |
| ID no. | 18 | 90 | 17 | 85 | 18 | 90 | 18 | 90 | 20 | 100 | 18 | 90 | 19 | 95 | 19 | 95 |
| Registration type | 17 | 85 | 20 | 100 | 20 | 100 | 20 | 100 | 17 | 85 | 20 | 100 | 20 | 100 | 20 | 100 |
| HIV status | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 18 | 90 | 20 | 100 | 20 | 100 |
| HIV-positive, on ART* | 13/13 | 100 | 13/13 | 100 | 14/14 | 100 | 14/14 | 100 | 13/13 | 100 | 11/13 | 85 | 14/14 | 100 | 14/14 | 100 |
| BDQ treatment start date | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| Reason for initiating BDQ | 19 | 95 | 18 | 90 | 20 | 100 | 20 | 100 | 18 | 90 | 18 | 90 | 20 | 100 | 20 | 100 |
| BDQ treatment stop date | 7 | 35 | 6 | 30 | 20 | 100 | 20 | 100 | 2 | 10 | 2 | 10 | 20 | 100 | 20 | 100 |
| Baseline ECG QTcF | 16 | 80 | 8 | 40 | 20 | 100 | 20 | 100 | 17 | 85 | 9 | 45 | 15 | 75 | 15 | 75 |
| ECG findings | 16 | 80 | 8 | 40 | 19 | 95 | 19 | 95 | 16 | 80 | 9 | 45 | 15 | 75 | 15 | 75 |
| AE* | 16/20 | 80 | 3/20 | 15 | 18/18 | 100 | 18/18 | 100 | 15/20 | 75 | 0/20 | 0 | 16/16 | 100 | 16/16 | 100 |
| AE onset date* | 16/20 | 80 | 3/20 | 15 | 18/18 | 100 | 18/18 | 100 | 15/20 | 75 | 0/20 | 0 | 16/16 | 100 | 16/16 | 100 |
| Concomitant medications* | 4/20 | 20 | 1/20 | 5 | 20/20 | 100 | 20/20 | 100 | 4/20 | 20 | 0/20 | 0 | 15/15 | 100 | 15/15 | 100 |
| Treatment regimen | 20 | 100 | 19 | 95 | 19 | 95 | 19 | 95 | 20 | 100 | 20 | 100 | 20 | 100 | 20 | 100 |
| Treatment outcome | — | — | — | — | — | — | — | — | 12 | 60 | 11 | 55 | 20 | 100 | 20 | 100 |
* Denotes variable denominator.
BDQ = bedaquiline; DR-TB = drug-resistant TB; ID = identification; ART = antiretroviral therapy; ECG = electrocardiogram; QTcF = corrected QT interval; AE = adverse event.
Cross-sectional data and data concordance
Completeness of data fields in EDRWeb for all RR-TB patients treated at Nkqubela and Jose Pearson TB Hospitals was ⩾96% for most available data elements, with the exception of ECG-related fields (37%) and identification number (87%; Table 5). Paper folders and EDRWeb entries at both sites were found to have 100% agreement in the post-intervention data concordance assessment of patients treated with BDQ. Among those treated without BDQ, one patient’s chart at Jose Pearson had a discordant entry in 2016 (percentage agreement 95%; κ = 0.89).
TABLE 5.
Cross-sectional completeness data for selected data elements available in EDRWeb for patients with DR-TB treated at both study sites, 2015–2016
| EDRWeb | ||
|---|---|---|
| Patients with field complete (n = 2335)n/N | Completeness % | |
| Patient name and surname | 2335 | 100 |
| DR-TB registration number | 2335 | 100 |
| DR-TB type | 2335 | 100 |
| ID number | 2032 | 87 |
| Registration type | 2335 | 100 |
| HIV status | 2333 | 99 |
| HIV-positive, on ART* | 1496/1532 | 98 |
| BDQ treatment start date* | 794/795 | 99 |
| Reason for initiating BDQ* | 794/795 | 99 |
| BDQ treatment stop date* | 794/795 | 99 |
| Baseline ECG QTcF | 866 | 37 |
| ECG findings | 866 | 37 |
| Adverse event* | 1470/1470 | 100 |
| Adverse event onset date* | 1470/1470 | 100 |
| Concomitant medications* | 1590/1590 | 100 |
| Treatment regimen | 2316 | 99 |
| Treatment outcome | 2335 | 100 |
* Denotes variable denominator.
DR-TB = drug-resistant TB; ID = identification; BDQ = bedaquiline; ECG = electrocardiogram; QTcF = (Fridéricia) corrected QT interval.
DISCUSSION
We found substantial gaps in data completeness in a random sample of patient charts and corresponding EDRWeb entries before a data quality intervention in the Eastern Cape Province of South Africa. Data completeness improved after a data quality intervention, although some gaps persisted, and concordance between paper folders and EDRWeb data entries was generally high.
The deficiencies in baseline completeness were most notable in data fields that involved BDQ treatment, ECG data, adverse events and concomitant medications. Data completeness improved to 100% after a data quality intervention for fields that involved BDQ treatment, adverse events and concomitant medications. However, ECG data fields had low completeness even after the intervention, particularly at Nkqubela TB Hospital. ECG machines were either broken or not available during portions of the study period, and were also frequently not included in transfer records, even after the data quality intervention. This suggests that data quality may be tied to factors other than data entry, which highlights the importance of adapting resources to ensure all clinically relevant data can be recorded.
In our study, the main discrepancies in data completeness between paper folders and EDRWeb data occurred with the same groups of data fields in which both were low, particularly data fields that involved ECG data and adverse events. Discrepancies between multiple sources of TB and HIV data in South Africa have been reported previously. Jamieson et al. discovered gaps in clinically relevant data fields between EDRWeb and ART (antiretroviral therapy) registers, including underreported death and treatment status for either HIV or RR-TB.19 Other studies have reported that some TB patients are not captured in the DS-TB registry or EDRWeb.20–22 Furthermore, Podewils et al. highlighted the marked variability between paper patient folders and the South African DS-TB registry in the data fields that had discrepancies.22
The data quality intervention was implemented while BDQ use was not yet recommended as standard treatment for RR-TB patients in South Africa.23 As a consequence, clinic stationary had to be updated to accommodate fields that were specific to BDQ, including fields containing ECG data. Corresponding changes were made in the EDRWeb, and our team assisted in developing auto-calculated fields for QTc and body mass index (BMI) data to minimise data entry errors and provide consistent, clinically informative fields in the data set. The development of an adverse event report on the EDRWeb was a direct consequence of the data quality intervention.
Our study had several strengths, including the use of programmatic data and adaptable data improvement mechanisms built into the work of the intervention teams. Our study results quantified data quality from a sample of a much broader intervention effort, with an infrastructure that provided opportunities for informed post-intervention training sessions for non-study staff. Furthemore, our study highlights the potential benefit, and therefore, the importance of continued monitoring and evaluation of programmatic data quality.
Limitations
Our non-experimental study design was limited in its ability to assert causality of the team model quality improvement intervention. Our sample size of reviewed charts was small. However, our findings suggest that further study of locally adapted data quality improvement interventions could help determine feasible and scalable methods to improve patient folder and registry data for enhanced care of RR-TB patients in other areas of South Africa and TB-endemic regions. We did not capture pre-intervention concordance data between patient folders and the EDRWeb. Although we could not compare to pre-intervention data, the post-intervention data showed universally high concordance between patient folders and EDRWeb.
CONCLUSION
Ensuring high-quality TB data is critical for excellent clinical care and reliable reporting of data to TB programmes for surveillance and resource allocation.15,24 TB data quality problems can have an impact ranging from individual patient outcomes to national programme policy.25,26 Our findings suggest that additional study regarding improvement of the quality of clinically relevant data fields is warranted, particularly as new and repur-posed anti-TB drugs and combinations of these drugs are implemented.
Acknowledgments
This work was supported by Janssen Pharmaceuticals (Beerse, Belgium) and the National Department of Health, Pretoria, South Africa.
Footnotes
The funders of the study had no role in any aspect pertinent to the work, including design, survey, analysis or the writing of the article. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders.
Conflicts of interest: none declared.
References
- 1.Pym AS, et al. Bedaquiline in the treatment of multidrug- and extensively drug-resistant tuberculosis. Eur Respir J. 2016;47(2):564–574. doi: 10.1183/13993003.00724-2015. [DOI] [PubMed] [Google Scholar]
- 2.Gler MT, et al. Delamanid for multidrug-resistant pulmonary tuberculosis. N Engl J Med. 2012;366(23):2151–2160. doi: 10.1056/NEJMoa1112433. [DOI] [PubMed] [Google Scholar]
- 3.Skripconoka V, et al. Delamanid improves outcomes and reduces mortality in multidrug-resistant tuberculosis. Eur Respir J. 2013;41(6):1393–1400. doi: 10.1183/09031936.00125812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.World Health Organization WHO consolidated guidelines on drug-resistant tuberculosis treatment. WHO/CDS/TB/2019.3. Geneva, Switzerland: WHO; 2019. https://apps.who.int/iris/bitstream/handle/10665/311389/9789241550529-eng.pdf?ua=1 Accessed June 2020. [PubMed] [Google Scholar]
- 5.World Health Organization Rapid Communication: key changes to the treatment of drug-resistant tuberculosis. WHO/CDS/TB/2019.26. Geneva, Switzerland: WHO; 2019. https://www.who.int/tb/publications/2019/WHO_RapidCommunicationMDR_TB2019.pdf?ua=1 Accessed June 2020. [Google Scholar]
- 6.World Health Organization Global tuberculosis report, 2018. WHO/CDS/TB/2018.20. Geneva, Switzerland: WHO; 2018. http://apps.who.int/iris/bitstream/handle/10665/274453/9789241565646-eng.pdf?ua=1 Accessed June 2020. [Google Scholar]
- 7.World Health Organization Global tuberculosis report, 2019. WHO/CDS/TB/2019.15. Geneva, Switzerland: WHO; 2019. https://apps.who.int/iris/bitstream/handle/10665/329368/9789241565714-eng.pdf?ua=1 Accessed June 2020. [Google Scholar]
- 8.South African Department of Health Media statement: new bedaquiline data shows reduction in TB mortality cases. Pretoria, South Africa: SA DoH; 2018. http://www.tbonline.info/media/uploads/documents/new_bedaquiline_data_shows_reduction_in_tb_mortality_cases.pdf Accessed June 2020. [Google Scholar]
- 9.World Health Organization Manual on use of routine data quality assessment (RDQA) tool for TB monitoring. WHO/HTM/TB/2011.1. Geneva, Switzerland: WHO; 2011. http://apps.who.int/iris/bitstream/handle/10665/44545/9789241501248_eng.pdf?sequence=1&isAllowed=y Accessed December 2017. [Google Scholar]
- 10.World Health Organization Electronic recording and reporting for tuberculosis care and control. WHO/HTM/TB/2011.22. Geneva, Switzerland: WHO; 2012. http://apps.who.int/iris/bitstream/10665/44840/1/9789241564465_eng.pdf?ua=1&ua=1 Accessed August 2017. [Google Scholar]
- 11.World Health Organization Definitions and reporting framework for tuberculosis—2013 revision. WHO/HTM/TB/2013.2. Geneva, Switzerland: WHO; 2013. http://apps.who.int/iris/bitstream/10665/79199/1/9789241505345_eng.pdf Accessed June 2017. [Google Scholar]
- 12.World Health Organization Standards and benchmarks for tuberculosis surveillance and vital registration systems: checklist and user guide. WHO/ HTM/TB/2014.02. Geneva, Switzerland: WHO; 2014. http://apps.who.int/iris/bitstream/10665/112673/1/9789241506724_eng.pdf?ua=1 Accessed April 2017. [Google Scholar]
- 13.Chan M, et al. Meeting the demand for results and accountability: a call for action on health data from eight global health agencies. PLoS Med. 2010;7(1):e1000223. doi: 10.1371/journal.pmed.1000223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chen H, et al. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–5207. doi: 10.3390/ijerph110505170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.van der Heijden YF et al. Overcoming limitations of tuberculosis information systems: researcher and clinician perspectives. Public Health Action. 2019;9(3):120–127. doi: 10.5588/pha.19.0014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Schnippel K, et al. Predictors of mortality and treatment success during treatment for rifampicin-resistant tuberculosis within the South African National TB Programme, 2009 to 2011: a cohort analysis of the national case register. Int J Infect Dis. 2015;39:89–94. doi: 10.1016/j.ijid.2015.09.002. [DOI] [PubMed] [Google Scholar]
- 17.Askham N, et al. The six primary dimensions for data quality assessment: defining data quality dimensions. Bristol, UK: DAMA UK Working Group; 2013. [Google Scholar]
- 18.Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144–151. doi: 10.1136/amiajnl-2011-000681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jamieson L, et al. Data quality of drug-resistant tuberculosis and antiretroviral therapy electronic registers in South Africa. BMC Public Health. 2019;19(1):1638. doi: 10.1186/s12889-019-7965-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rose PC, et al. Completeness and accuracy of electronic recording of paediatric drug-resistant tuberculosis in Cape Town, South Africa. Public Health Action. 2013;3(3):214–219. doi: 10.5588/pha.13.0041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dunbar R, et al. Accuracy and completeness of recording of confirmed tuberculosis in two South African communities. Int J Tuberc Lung Dis. 2011;15(3):337–343. [PubMed] [Google Scholar]
- 22.Podewils LJ, et al. Completeness and Reliability of the Republic of South Africa National Tuberculosis (TB) Surveillance System. BMC Public Health. 2015;15:765. doi: 10.1186/s12889-015-2117-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Department of Health Republic of South Africa Management of rifampicin-resistant tuberculosis: a clinical reference guide. Pretoria, South Africa: DoH; 2019. http://www.health.gov.za/index.php/2014-03-17-09-09-38/policies-and-guidelines/category/535-2019-policies-and-guidelines# Accessed May 2020. [Google Scholar]
- 24.Arts DG, De Keizer NF, Scheffer GJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002;9(6):600–611. doi: 10.1197/jamia.M1087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Tweed C, Nunn A. Data in TB have to be both big and good. Int J Tuberc Lung Dis. 2018;22(5):476. doi: 10.5588/ijtld.18.0200. [DOI] [PubMed] [Google Scholar]
- 26.Sharma A, et al. A review of data quality of an electronic tuberculosis surveil-lance system for case-based reporting in Kenya. Eur J Public Health. 2015;25(6):1095–1097. doi: 10.1093/eurpub/ckv092. [DOI] [PMC free article] [PubMed] [Google Scholar]