Availability and quality of routine morbidity data: review of studies in South Africa

Abstract

Objectives: Routine health information systems (RHISs) provide data that are vital for planning and monitoring individual health. Data from RHISs could also be used for purposes for which they were not originally intended, provided that the data are of sufficient quality. For example, morbidity data could be used to inform burden of disease estimations, which serve as important evidence to prioritize interventions and promote health. The objective of this study was to identify and assess published quantitative assessments of data quality related to patient morbidity in RHISs in use in South Africa.

Materials and Methods: We conducted a review of literature published between 1994 and 2014 that assessed the quality of data in RHISs in South Africa. World Health Organization (WHO) data quality components were used as the assessment criteria.

Results Of 420 references identified, 11 studies met the inclusion criteria. The studies were limited to tuberculosis and HIV. No study reported more than 3 WHO data quality components or provided a quantitative assessment of quality that could be used for burden of disease estimation.

Discussion: The included studies had limited geographical focus and evaluated different source data at different levels of the information system. All studies reported poor data quality.

Conclusion: This review confirmed concerns about the quality of data in RHISs, and highlighted the need for a comprehensive evaluation of the quality of patient-level morbidity data in RHISs in South Africa.

BACKGROUND AND SIGNIFICANCE

South Africa is an upper-middle-income developing country¹ with a fragmented health care system.² The country is moving toward a national health insurance system to improve access to health care services for all South Africans.³ The national health insurance system will be dependent on effective information systems to guide its operations and expenditure.³ Improvements in the quality, coverage, and standardization of public and private health information systems are pivotal to their successful implementation.³

Enhancing the use of health information and improving data quality are essential to scale up health service delivery.^4–7 Governments and nongovernmental organizations spend significant resources on routine health information systems,⁴ and it is important to determine whether these investments result in improved data systems, enhanced program management, and better data quality.

An RHIS is defined as “a system that provides information at regular intervals of a year or less through mechanisms designed to meet predictable information needs. This includes paper-based or electronic health records, and facility- and district-level management information systems.”⁴ Data that are reliable, accurate, complete, and accessible in a timely manner make up one of the building blocks of a good RHIS.⁸

The public sector in South Africa uses several well-established RHISs. Systems such as the District Health Information System (DHIS), the Electronic TB Register (ETR.Net), the Electronic Register for Drug-Resistant TB (EDRWeb.Net), and the Electronic Register for ARV treatment (TIER.Net) have been implemented nationally.⁹ A pathology-based national cancer register was established in 1986,¹⁰ but the last published incidence report was for 2004.¹¹ Several other RHISs are used, but often only implemented in a single province, such as eKapa (ART and TB monitoring system) (Western Cape) and Nootroclin (Northern Cape).⁹

Data quality issues of the DHIS at facility, district, and provincial levels have been raised in information audits by the auditor general of South Africa.¹² The need to find quantitative studies of data quality was identified because it is often reported that “the quality of data in RHISs in South Africa is poor,”^3,13–16 but the extent of the problem cannot be quantified on the basis of qualitative data. Qualitative data can describe the nature of the problem and can also provide possible reasons why the problem exists. A quantitative assessment of the quality of data in RHISs gives an idea of the extent of the problem, and would ideally provide a numerical factor that could be used to adjust routine data to inform planning or burden of disease analysis.

In South Africa, routine mortality data from civil registration and injury surveillance are used to inform burden of disease studies^17,18 and contribute to global burden of disease studies,¹⁹ but it is unclear whether routine morbidity data could be used in a similar way. Although RHISs are designed to support national and decentralized decision-making and health service management,^4,5,20 they could also provide morbidity data to help estimate the national burden of disease, provided they are of sufficient quality.

OBJECTIVE

In the absence of comprehensive information on the quality of data reported in the RHISs in South Africa, a literature review was conducted to identify and assess published studies that report on quantitative assessment of the quality of data in RHISs, with a focus on morbidity data.

MATERIALS AND METHODS

Data quality

Multiple frameworks for analyzing RHISs have been developed.^8,21–26 Many of these frameworks overlap with regard to components that make up data quality. As the focus of this study is to review reported RHIS data quality in already published studies,^7,27 the WHO framework⁸ was found to be appropriate.

The WHO has described the components of data quality in a guide aimed at improving data quality in developing countries (Table 1).⁸ These commonly used criteria for the evaluation of data quality^28–32 have been adopted as the framework for this review. Furthermore, they are largely consistent with the components stated in the standard operating procedure for the District Health Management Information System by the National Department of Health, which include accuracy, reliability, completeness, timeliness, and accessibility.³³ This standard operating procedure also includes integrity, coherence, and compatibility. Of the 8 data quality components identified by WHO, “accuracy and validity,” “reliability,” and “completeness” were found to be the most relevant for this study.

Table 1.

Description of WHO data quality components

Components	Description
Accuracy and validity	The original data must be accurate in order to be useful. If data are not accurate, then wrong impressions and information are being conveyed to the user. Documentation should reflect the event as it actually happened.
Reliability	Data should yield the same results on repeated collection, processing, storing, and display of information.
Completeness	All required data should be present and the medical/health record should contain all pertinent documents with complete and appropriate documentation.
Legibility	All data, whether written, transcribed, or printed, should be readable.
Timeliness	Information, especially clinical information, should be documented as an event occurs, treatment is performed, or results are noted. Delaying documentation could cause omission of information or recording of errors.
Accessibility	All necessary data are available when needed for patient care and for all other official purposes. The value of accurately recorded data is lost if it is not accessible.
Meaning or usefulness/ relevance	Information is pertinent and useful.
Confidentiality and data security	Public health registries must be governed by the strictest rules of data protection and confidentiality. The data should be used only for purposes related to public health (and not for discrimination or criminalization).

Source: Adapted from WHO, 2003⁸ and WHO, 2013.³⁴

Review of publications

This study identified English-language publications evaluating South African RHISs from January 1994 to December 2014. Keywords from the systematic review of health data quality management and best practices by Ndabarora et al.³⁵ guided the basic search terms that were adapted for searching the data sources. Basic terms were (“routine health” OR “district health” OR “health management” OR “public health” OR “national health”) AND (“information system*” OR “informat*” OR “data”) AND (“data quality” OR “data accuracy” OR “data quality improvement” OR “data audit”) AND “South Africa.” Table 2 describes the search terms used for the different data sources.

Table 2.

Search terms for review of publications

Data source	Search terms
PubMed	(((((“Routine health” OR “district health” OR “health management” OR “public health” OR “national health”)) AND (“information system*” OR “informat*” OR “data”)) AND (“data quality” OR “data accuracy” OR “data quality improvement” OR “data audit”)) AND “South Africa”)
	Restricted: 1994–2014
Scopus	ALL (“routine health” OR “district health” OR “health management” OR “public health” OR “national health”) AND ALL (“information system*” OR “"informat*” OR “data”) AND ALL (“data quality” OR “data accuracy” OR “data quality improvement” OR “data audit”) AND ALL (South Africa) AND PUBYEAR > 1993 AND PUBYEAR < 2015
	Restricted: South Africa
Web of Science	TOPIC: (“Routine health” OR “district health” OR “health management” OR “public health” OR “national health”) AND TOPIC: (“information system*” OR “informat*” OR “data”) AND TOPIC: (“data quality” OR “data accuracy” OR “data quality improvement” OR “data audit”) AND TOPIC: (South Africa)
	Restricted: 1994–2014
Science Direct	> 1993 and pub-date < 2015 and (((((“routine health” OR “district health” OR “health management” OR “public health” OR “national health”)) AND (“information system*” OR “informat*” OR “data”)) AND (“data quality” OR “data accuracy” OR “data quality improvement” OR “data audit”)) AND “South Africa”)

Websites were also searched for relevant gray literature, including peer-reviewed conference proceedings. Titles and abstracts of publications were screened by the first author. At least 2 reviewers (the first author and 1 or more co-authors) reviewed each potentially eligible full-text publication for inclusion. Full-text publications that reported on RHIS in South Africa were then assessed by at least 2 reviewers (the first author and 1 or more co-authors). In addition, references of included articles were checked and experts were contacted to identify additional publications.

Inclusion and exclusion criteria

A quantitative assessment of data quality would provide a numerical factor that could be used to adjust routine data to inform planning or burden of disease analysis. Therefore, even though qualitative assessments can provide clues to the reasons for poor data quality, this was not the focus of the review, and thus studies that focused only on qualitative assessments were excluded, as were commentaries and methodologies. If a study did not include any South African RHISs or focused only on mortality data, it was also excluded.

A team of researchers, including RHIS experts, reviewed studies that addressed the identified WHO data quality components.^8,34 All included publications were analyzed to confirm whether they contained quantitative information relating to the WHO data quality components. A publication was included if it quantitatively addressed any of the following components: accuracy and validity, reliability, completeness, legibility, timeliness, accessibility, meaning or usefulness, and confidentiality and data security.^8,34 If this information was present, the data system, data flow, actual data elements, and any other related data were noted.

RESULTS

As shown in Figure 1, 464 publications were retrieved and 44 duplicates were removed, resulting in 420 publications to be assessed. After applying all the exclusion criteria, 11 publications were identified that reported on a quantitative assessment of data quality in an RHIS in South Africa. The study characteristics are summarized in Table 3 by study period, setting, population, information system(s), morbidity and related data elements, and WHO data quality component addressed.

Figure 1.

Study inclusion flow diagram.

Table 3.

Summary of included studies

Publication	Study period	Setting (province)	Study population	System	Morbidity and related data evaluated	Accuracy/ Validity	Reliability	Completeness
A. PMTCT
Mate et al. (2009)36	Jan to Dec 2007	eThekwini, Umgungun-dlovu and Ugu Districts in KwaZulu-NatalI	316 facilities (fixed clinics, mobile clinics, community health centers, hospitals)	Data were collected from facility PMTCT registers and monthly tally sheets (collated from facility registers) reported to DHIS. Monthly DHIS reports were compared to facility data. Systems: (1) facility PMTCT registers, (2) facility monthly reports, (3) DHIS	Antenatal care client tested HIV positive	X	X	X
Mphatswe et al. (2012)37	May to Nov 2008	eThekwini, Umgungun-dlovu and Ugu Districts in KwaZulu- Natal	78 facilities (58 antenatal clinics and 20 delivery wards)	Data interventions were conducted at facilities aimed at data collection, feedback, and monthly data reviews. Facility data audits were conducted before, during, and after the intervention. Facility registers were compared to DHIS reports. Systems: (1) facility PMTCT registers, (2) DHIS	Pregnant women who tested positive for HIVa	X	X	X
B. TB
Marais et al. (2006)38b	Feb 2003 to Oct 2004	Cape Town, Western Cape	5 primary health care clinics reporting to Tygerberg Children’s Hospital All cases of children (< 13 years of age) receiving anti-tuberculosis treatment at a referral hospital (n = 443)	A study surveillance system was established to detect all cases of children (< 13 years of age) receiving anti-tuberculosis treatment in 5 clinics and a hospital, and these records were compared to the TB treatment registers at the clinics. Systems: (1) study surveillance system, (2) clinic TB treatment register	TB status HIV status Age, sex	X		X
Heunis et al. (2011)39c	Nov to Dec 2007	All 5 districts in the Free State	20 facilities (mobile clinic, fixed clinic, community health center, or large clinic and district hospital)	Data were recorded from facility-based records (patient files or electronic register) and compared with data for the same patients at the provincial level. Systems: (1) TB patient files (or electronic registers where available), (2) provincial-level patient data	Date TB treatment started (for patients diagnosed with both TB and HIV) TB patient treatment status	X
Du Preez et al. (2011)40	Jul 2007 to Jun 2009	Tygerberg Children’s Hospital, Cape Town, Western Cape	A retrospective cohort of children (≤ 13 years) with culture-confirmed TBd in Tygerberg Children’s Hospital (n = 291)	Cases from hospital data sources were compared to the provincial ETR.Net. Systems: (1) electronic laboratory-based hospital surveillance database, (2) administrative department data, (3) hospital folders, (4) notification records, (5) TB meningitis home-based care program records, (6) discharge summaries, (7) ETR.Net	Culture- confirmed TB status Type of TB HIV status Age, sex			X
Dunbar et al. (2011)41	Oct 1, 2006 to Mar 31, 2008	Cape Town, Western Cape	All cases of bacteriologically confirmed TB in 2 communities (2 community clinics and 1 hospital)	Records were linked in 3 sources (TB treatment registers, the nearest central laboratory, and the referral hospital laboratory). Data were assessed before and after record linking. Systems: (1) TB treatment registers, (2) nearest central laboratory, (3) referral hospital laboratory	Bacteriologically confirmed TB status Age, sex	X		X
Dilraj et al. (2013)42	Fourth quarter 2009	Ethekwini District in KwaZulu- Natal	46 high TB burden facilities (type of facility not stated), n = 1036 smear positive patients in ETR.net	Retrospective evaluation conducted among all acid fast bacilli smear-positive TB patients recorded in ETR.Net. Patient data in ETR.Net were collected and compared against laboratory TB sputum smear registers (LIS). Systems: (1) ETR.Net, (2) LIS (NHLS)	Smear result and smear grading for TB on each specimen Date of birth, sex			X
Rose et al. (2013)43	Jan 1 to Dec 31, 2012	Cape Town, Western Cape	2 hospitals; n = 77 children (< 15 years old) treated for DR- TB	Clinical cohort data were compared to the electronic register data. Systems: (1) pediatric DR-TB ward register; (2) inpatient folders; (3) discharge summaries; (4) NHLS LIS (where available), (5) EDRWeb.Net	Type of DR-TB DR-TB treatment start date HIV status Date of birth, age, sex	X		X
Ebonwu et al. (2013)44	2011	Sizwe Hospital,e Gauteng	All MDR-TB cases in Gauteng at government facilities with laboratory confirmed results in period	A list of all new diagnoses in Gauteng from the NHLS LIS was compared to case files from the MDR-treatment register and patient MDR-TB treatment card. This study reports on diagnosis to treatment initiation. Systems: (1) NHLS LIS, (2) MDR-TB treatment register and MDR-TB patient card	Date of MDR-TB diagnosis MDR-TB treatment initiation status HIV status Age, sex			X
Botha et al. (2008)45	Apr 2004 to Mar 2005	Stellenbosch Sub-district, Western Cape	13 primary health care facilities (clinics)	Names of all individuals submitting sputum samples were noted in newly introduced sputum register. The facility TB treatment register was checked against the new sputum register to note any deficiencies. This study reports on cases before receiving treatment. Systems: (1) facility TB register, (2) sputum register	TB culture and smear results Culture-confirmed TB status Name, date of birth, sex, address			X
C. HIV TB Co- infection
Auld et al. (2013)46	Jan to Aug 2011	Bitou, Knysna, and George sub-districts in Eden District, Western Cape	8 TB clinics, 2 HIV clinics, 2 TB-HIV co- located clinics n = 206 co-infected cases from THAT’SIT database	The study evaluated HIV and TB co-infected patients in the THAT’SIT database against 2 sets of information systems (TB sources and HIV sources). TB sources: TB Blue Card, TB Register, ETR.Net. HIV sources: ART file and eRegister. Systems: (1) THAT’SIT, (2) TB Blue Card, (3) clinic TB register, (4) ETR.Net., (5) ART file, (6) ART register	TB site of disease HIV status Current TB (at ART start) Date of birth, sex		X	X

^aMphatswe et al. (2012) is a smaller, follow-up study of the Mate et al. (2009) study. The same data elements were examined in both studies, although they are named differently. Mate et al. (2009) conducted the study in 3 districts in KwaZulu-Natal that are not stated in the article. They are most likely the same districts used in the Mphatswe et al. (2012) study.

^bThe title refers to accuracy. However, the study reports on accuracy of diagnosis and not of data.

^cThis study does not give a measure of whether cases were missing at the provincial level.

^dTB cases were classified as pulmonary TB (PTB), including hilar and mediastinal lymphadenopathy; extrapulmonary TB (EPTB); or both PTB and EPTB.

^eDesignated MDR-TB treatment hospital in Gauteng Province.

ART: antiretroviral treatment; DHIS: district health information system; DR-TB: drug-resistant tuberculosis; EDRweb.Net: Electronic Register for Drug-Resistant Tuberculosis; ETR.Net: Electronic Register for Tuberculosis; MDR-TB: multidrug-resistant tuberculosis; NHLS: National Health Laboratory Service; TB: tuberculosis; THAT’SIT: HIV/AIDS Treatment Support and Integrated Therapy information system; PMTCT: prevention of mother-to-child transmission of HIV.

None of the included publications reported on more than 3 of the WHO data quality components.⁸ Definitions of the data quality component used in the different studies were rarely given, and few data quality definitions were referenced from another source, indicating a lack of standardized definitions to assess data quality. In the absence of specific definitions, the WHO data quality components’ definitions were assumed.⁸ Most studies had a relatively limited focus (eg, facility/community level [n = 4], district level [n = 5], provincial level [n = 2]), and thus were not representative of the whole country. The 11 studies included focused on prevention of mother-to-child transmission of HIV (PMTCT) (n = 2), tuberculosis (TB) (n = 8), and HIV and TB co-infection (n = 1). Although several studies evaluated TB data, they involved different sources at different levels of the health system (Figure 2). Drug-resistant TB studies were not included in Figure 2.

Figure 2.

TB source data flows indicating level of care evaluated.

The findings of the data quality evaluations are summarized in Table 4. For studies reporting on recording of HIV in pregnant women, there was a problem with transfer of aggregated information from facility registers to DHIS.^36,37 These studies also highlighted challenges with completeness between clinic registers and DHIS, and challenges with reliability between clinic registers and monthly summary sheets. Furthermore, the data item “antenatal clients tested HIV positive” was reported as highly inaccurate by Mate et al.³⁶ in their study setting, when compared to the facility (fixed clinic, mobile clinic, community health center, hospital) registers. Mphatswe et al.³⁷ reported on an intervention to improve the quality of PMTCT data between facility registers and the DHIS, and significant improvements were reported as a result of the intervention.

Table 4.

Main findings of quality evaluations

Publication	RHIS	Morbidity and related data evaluated	Findings
A. PMTCT
Mate et al. (2009)36	Facility PMTCT registers, facility monthly reports, DHIS	Antenatal care client tested HIV positive	Data elements reported 50.3% of the time Data elements accurate 12.8% of the time No information on number of cases
Mphatswe et al. (2012)37	Facility PMTCT registers, DHIS	Pregnant women who tested positive for HIV	After the intervention, the level of data completeness increased from 26% to 64% Accuracy increased from 37% to 65% Correlation between registers and DHIS increased from 0.54 to 0.92 No information on number of cases
B. TB
Marais et al. (2006)38	Study surveillance system, clinic TB treatment register	TB status HIV status Age, sex	n = 443 from study surveillance system 389 (88%) recorded in TB register Major proportion of gap due to children not treated at clinic level (ie, severe cases diagnosed at referral hospital)
Heunis et al. (2011)39	TB patient files, electronic registers, provincial-level patient data	Date TB treatment started (for patients diagnosed with both TB and HIV) TB patient treatment status	n = 400 (random sample; 20 patients from each of 20 facilities) 21% inconsistency overall between 7 items of data for each patient recorded at facility and provincial levels 44% discrepancy in TB treatment start date between facility- and provincial-level records (highest recorded for the study)
Du Preez et al. (2011)40	Electronic laboratory-based hospital surveillance database, administrative department data, hospital folders, notification records, TB meningitis home-based care program records, discharge summaries, ETR.Net	Culture-confirmed TB status Type of TB HIV status Age, sex	n = 267 culture-confirmed cases 166 (62.2%) registered in ETR.Net
Dunbar et al. (2011)41	TB treatment registers, laboratory information system (LIS) (ie, NHLS), nearest central hospital, referral hospital	Bacteriologically confirmed TB status Age, sex	204 cases in TB register recorded as bacteriologically confirmed 306 bacteriologically confirmed cases of TB identified in laboratory records 12.7% of cases in the register but misclassified 33% of confirmed cases not recorded in the register
Dilraj et al. (2013)42	ETR.Net, LIS (NHLS)	Smear result and smear grading for TB on each specimen Date of birth, sex	1036 smear positive in ETR.Net 66% (683) laboratory records confirmed smear positive 3.1% misclassified in ETR.Net End treatment (n = 683) 364 smear results 219 (32%) lost to follow-up 331 patients had end of treatment results in ETR.net: 302 reported cured but only 105 (34%) confirmed in laboratory
Rose et al. (2013)43	Pediatric DR-TB register, EDRWeb.net, inpatient folders, discharge summaries, LIS (NHLS)	Type of DR-TB DR-TB treatment start date HIV status Date of birth, age, sex	77 children identified and treated 49 (64%) recorded in EDRWeb.net
Ebonwu et al. (2013)44	LIS (NHLS), MDR TB treatment register, MDR-TB patient card	Date of MDR-TB diagnosis MDR-TB treatment initiation status HIV status Age, sex	942 newly diagnosed MDR-TB cases 593 (63%) initiated treatment
Botha et al. (2008)45	Facility TB register, sputum register	TB culture and smear results Culture-confirmed TB status Name, date of birth, sex, address	2037 suspects 367 (18%) had 2 positive smears 303 (87%) of cases recorded in TB treatment register (ie, started treatment), 17% initial defaulters 153 suspects had 1 positive smear 99 (65%) started treatment 520 cases confirmed (367 + 153) 23% did not start treatment Estimated 162 cases untraced
C. HIV/TB co-infection
Auld et al. (2013)46	THAT’SIT, TB blue card, clinic TB register, ETR.Net, ART file, (ART register), eRegister	TB site of disease HIV status Current TB (at ART start) Date of birth, sex	94% of 206 cases in THAT’SIT database eligible for study (n = 193) 85% of 193 study cases had records in all 6 systems TB site of disease Complete in 92–100% of TB sources Complete in 49% of ART files 100% complete in THAT’SIT

Some of the studies indicated gaps in the completeness of TB data across multiple information systems, i.e., TB cases that were not being recorded.^38,40,43,45 These studies showed that between 12% and 38% of cases were not being recorded in the primary data source (either patient records or laboratory records) for each study.^35,40–43

Auld et al.⁴⁶ focused on TB and HIV co-infection, specifically the HIV/AIDS Treatment Support and Integrated Therapy (THAT’SIT) information system, which serves as the primary source for integrated TB and HIV patient-level information in the study facilities. Data in the standard RHISs for TB and ART programs were compared to the THAT’SIT data for each case: a paper-based TB patient file (TB Blue Card), a paper-based TB register, an electronic TB register (ETR.Net), a paper-based ART patient file, and an electronic ART register (eRegister) all feed into THAT’SIT. Reliability and completeness across these 6 RHISs were assessed. Auld et al.⁴⁶ found that 85% of the cases in THAT’SIT had records in all 6 systems. TB systems were almost complete for reporting on HIV co-infection (≥97%), while only 78% of the cases had TB status reported in the ART file. TB status was recorded in 97% of cases in the ART eRegister. A kappa score > 0.9 was reported for age and sex variables across systems. For clinical indicators, agreement on TB site of disease was high across the 3 TB information sources (k = 0.90) but low across all the systems (k = 0.53).

DISCUSSION

Although many studies reported poor data quality, this differed by data item, setting, and system.^36–46 Assessments were performed at different points in the RHIS system^38–42,45 and there was a lack of standard criteria of data quality, making it impossible to pool the findings from different studies. The WHO definitions of components for data quality provided a practical framework for our review.

The results of this analysis of 11 diverse studies give weight to the comments of multiple authors that the quality of data in RHISs in South Africa is a cause for concern.^36–46 The studies cover several major programs: PMTCT, the national TB program, and the combined HIV/TB treatment program. They reflect problems at primary, secondary, and tertiary care levels of the health care system and among multiple RHISs, most notably registers of patients receiving treatment, laboratory records, and aggregated reporting systems. Our findings are echoed by a recently published comprehensive evaluation of TB surveillance data in 3 provinces by Podewils et al.⁴⁷ showing that completeness and reliability were inconsistent across data sources.

Studies focusing on PMTCT highlighted problematic data transfer accuracy, which may be related to human and organizational factors rather than technical issues. Factors such as a lack of core competencies for data collection, staff attitudes toward RHIS tasks, multiplicity of registers to monitor programs, and the absence of processes for feedback and data quality checks and analysis at the facility level were highlighted by Nicol et al.^16,48 Heunis et al. reported that gaps in the completeness of HIV-TB systems also relate to human and organizational factors such as staff shortages, lack of training, multiplicity of forms, and difficulty keeping track of patients.³⁹ System issues may also hinder data quality. For example, systems like THAT’SIT (only rolled out in some provinces), which have been developed to track treatment of patients with HIV-TB co-infection, depend on data from the current TB and HIV information systems and thus inherit all of their challenges.⁴⁶ The same authors noted that where mechanisms had been put in place to improve data quality, the impact was not reported.⁴⁶

Data quality can also be affected by policy. One possible cause identified for problems with the level of completeness reported in TB studies was that the South African National Tuberculosis Programme approaches TB through a decentralized model and states that TB patients must be registered and recorded at the primary health care level regardless of where they are diagnosed, whether at hospitals or elsewhere.³⁵ It seems that a formal process is not in place to ensure that cases diagnosed at hospitals are captured in the system. For example, some children, especially those diagnosed with severe TB, are diagnosed and treated only at referral hospitals and thus were not counted in the routine TB surveillance systems.^40,43

While the studies cannot be assumed to be representative of the whole South African health care environment, and some were conducted more than a decade ago, they do reflect problems in a wide range of environments and give an indication as to how to address them. An underlying problem (identified in multiple studies, including those in this analysis) is that the RHISs in use in South Africa are not generally regarded as important tools to be used correctly and effectively in informing decision making from the individual patient care level to national-level treatment and/or prevention programs.

Furthermore, the results of this review indicate that the National Department of Health should be strengthening the current RHISs using the infrastructure that is already available and identifying the gaps that need to be addressed, particularly data quality. Improved data quality in RHISs has the potential to improve the accuracy of routine reporting of the morbidity profile of patients attending health facilities, which is required to support decision-making on resource allocation. Although the RHISs are in place, this study confirms the assessment by the Health Data Advisory and Co-ordination Committee that there is room for improvement.¹² In addition to the RHISs discussed in the reviewed studies, other existing RHISs, such as the national cancer register,^10,11 reporting on notifiable conditions,⁴⁹ and injury morbidity surveillance,⁵⁰ need attention. Unless data from these RHISs can be improved, it will not be possible to use them to inform burden of disease studies in the future. The National Health Normative Standards Framework was established to direct efforts to improve poor data quality (as a result of RHIS fragmentation and inoperable systems) by setting the foundations for interoperability as stated in eHealth Strategy South Africa 2012–2016.⁹

One local study by Mphatswe et al.³⁷ and 2 studies from similar low-resource settings^51,52 report on interventions to improve data quality, which have resulted in sustained improvement. The potential benefits of wide implementation of good practices in record-keeping, thus making the best possible use of existing RHISs, include improved patient care (since all patients requiring care would be identified and therefore monitored), improved program reporting from clinic to district and provincial levels (essential for program monitoring and resourcing), and accurate and consistent reporting at the national and international levels. A national effort by all stakeholders is required to reach this goal, which should be achievable with existing resources for health services in South Africa. Effective RHISs could have the additional benefit of providing data to inform burden of disease assessments, which are essential to integrated longer-term planning for health in South Africa.

CONCLUSION

This review has highlighted the need for a comprehensive evaluation of the quality of patient-level morbidity data in RHISs in South Africa. Even though RHISs are primarily established to produce data for public health decision making⁵² these data could also be extremely valuable for routine morbidity surveillance and empirical estimation of the burden of disease at the national, provincial, and district levels, if data quality could be relied upon.

The piecemeal nature of the identified work emphasizes that different levels within routine health information systems require quality assessment. A situational analysis of the availability and quality of patient-level morbidity data at public hospitals is being planned by the Burden of Disease Research Unit of the South African Medical Research Council. In the longer term, it will be important to institute a standardized data quality assurance system to ensure sustainable improvement of RHIS data quality.

FUNDING

This work was supported by the South African Medical Research Council in terms of the MRC’s Flagship Awards Project, grant number SAMRC-RFA-IFSP-01-2013/ SA CRA 2.

COMPETING INTERESTS

The authors declare that there is no conflict of interest.

CONTRIBUTORS

All the authors contributed to the study design, selection, and analysis of the reviews as well as revisions and approval of the article. R.R. conceived the study, conceptualized the article, and was responsible for searching, sourcing, and reviewing the literature and data and the study analysis. She led the interpretation of the results, wrote the first draft of the article, and revised the drafts according to inputs from the co-authors. V.PvW contributed to the conceptualization of the article, the selection and critical review of articles, the reporting structure of the review, and the write-up of all sections of the article. O.A. worked on data extraction and analysis of the article. J.J. critically reviewed the article drafts and contributed conceptually to technical discussions around terminology and definitions of quality criteria. E.N. worked on the data quality definitions framework, data extraction, and analysis of the article. D.B. contributed to the conceptualization of the article and the reporting structure of the review, and critically reviewed the drafts. L.H. contributed to the conceptualization of the article, the data quality definitions framework, the selection of articles, and the write-up of all sections of the article. All authors approved the final draft.