Skip to main content
BMJ Open logoLink to BMJ Open
. 2023 May 19;13(5):e071414. doi: 10.1136/bmjopen-2022-071414

Effects of primary healthcare reform on routine health information systems (RHISs): a mixed-methods study in Lesotho

Ermyas Birru 1,2,, Melino Ndayizigiye 2, Ryan McBain 3,4, Mathabang Mokoena 2, Masebeo Koto 5, Orvalho Augusto 1,6, Edinah Casmir 1,7, Nancy Puttkammer 1, Joia Mukherjee 3,8
PMCID: PMC10201246  PMID: 37208141

Abstract

Background and objective

The Ministry of Health of Lesotho and Partners In Health piloted the Lesotho National Primary Health Care Reform (LPHCR) from July 2014 to June 2017 to improve quality and quantity of service delivery and enhance health system management. This initiative included improvement of routine health information systems (RHISs) to map disease burden and reinforce data utilisation for clinical quality improvement.

Methods and analysis

The WHO Data Quality Assurance framework’s core indicators were used to compare the completeness of health data before versus after the LPHCR in 60 health centres and 6 hospitals across four districts. To examine change in data completeness, we conducted an interrupted time series analysis using multivariable logistic mixed-effects regression. Additionally, we conducted 25 key informant interviews with healthcare workers (HCWs) at the different levels of Lesotho’s health system, following a purposive sampling approach. Interviews were analysed using deductive coding based on the Performance of Routine Information System Management framework, which inspected organisational, technical and behavioural factors influencing RHIS processes and outputs associated with the LPHCR.

Results

In multivariable analyses, trends in monthly data completion rate were higher after versus before the LPHCR for: documenting first antenatal care visit (adjusted OR (AOR): 1.24, 95% CI: 1.14 to 1.36) and institutional delivery (AOR: 1.19, 95% CI: 1.07 to 1.32). When discussing processes, HCWs highlighted the value of establishing clear roles and responsibilities in reporting under a new organisational structure, improved community programmes among district health management teams, and enhanced data sharing and monitoring by districts.

Conclusion

The Ministry of Health had a strong data completion rate pre-LPHCR, which was sustained throughout the LPHCR despite increased service utilisation. The data completion rate was optimised through improved behavioural, technical and organisational factors introduced as part of the LPHCR.

Keywords: health services administration & management, international health services, organisation of health services, primary care


STRENGTHS AND LIMITATIONS OF THIS STUDY.

  • A strength of this study was the use of quantitative and qualitative methods to generate evidence on data quality change and why the change occurred.

  • Another strength of the study was the use of an interrupted time series framework to assess data completeness, a rigorous quasi-experimental method for inference about the real-world effect of Lesotho’s healthcare reform.

  • A limitation of the study was not having data from comparison districts or facilities throughout the intervention period, which would have strengthened causal inference.

Introduction

Routine health information systems (RHISs) are designed to collect, process and use health information collected at a regular interval of a year to inform clinical service delivery as well as administrative and management systems.1 RHISs aim to improve patient/population health outcomes and strengthen health systems,2 3 and they play a central role in data collection, collation, validation, analysis, dissemination and decision-making.4 Within primary healthcare (PHC), RHISs seldom receive adequate resources to operate successfully in sub-Saharan African (SSA) countries.5

Historically, RHISs in low/middle-income countries (LMICs) have been paper based. However, over the last 25 years, electronic systems have been introduced, often by non-governmental organisations supporting vertical, selective and disease-oriented programmes.6 In certain instances, targeted interventions have given rise to fragmented and parallel systems that necessitate their own unique data collection requirements, forms, and indicators.7–11 To integrate disease-based programmes into a comprehensive PHC package, it is necessary for RHISs to be strengthened and coordinated to improve administration of the health system and the delivery and assessment of care.12–14

Lesotho was behind on key millennium development goals aimed at improvement of maternal and child health outcomes.15 To address gaps and to strengthen primary care, the Government of Lesotho, in partnership with Partners In Health, launched the first phase of the Lesotho National Primary Health Care Reform (LPHCR)—a pilot—in April 2014, in four districts: Berea, Leribe, Butha Buthe and Mohale’s Hoek.15 The three main objectives of the LPHCR were to improve service delivery, strengthen health system management and empower professional village health workers. As a central input to chart progress, the Ministry of Health (MoH) enhanced the RHISs to map disease burden and to define and monitor progress against the United Nations Sustainable Development Goals of 2030—including achieving universal health coverage.16

Health information is one of the six core building blocks of health systems.17 As such, the LPHCR included inputs and activities relevant to RHIS data collection, processing and utilisation. These inputs and activities were: the introduction of new data officers in the four district health management teams’ (DHMTs) Strategic Information Division; compilation of PHC indicators to facilitate quarterly DHMT performance meetings; the use of RHIS data for regular health facility mentorship and supervision; and the use of data for quarterly district-based and national health performance meetings.

Our study aims to evaluate the effect of LPHCR on RHIS, following a methodological framework previously applied in other low-resource settings.18 19 This study hypothesised that completeness of a core set of preselected RHIS data would improve with the implementation of the LPHCR in Berea, Leribe, Thaba Tseka and Mohale’s Hoek districts. To further support this hypothesis, key informant interviews (KIIs) were completed with healthcare workers (HCWs) to explore their experience of the impact of RHISs within the LPHCR.

Methods

Study setting and population

Lesotho is an LMIC with a population of slightly over 2 million in 2017 and geographically surrounded by South Africa (figure 1).20 21 There are 10 districts, and this pilot study included Berea, Butha Buthe, Leribe and Mohale’s Hoek. These four districts account for 40% of the total population in Lesotho. Berea and Leribe are semiurban districts located close to the capital, representing over 65% of the four districts’ population.20

Figure 1.

Figure 1

Map of districts in Lesotho. Included are LPHCR pilot districts highlighted in colour and those not LPHCR in black ink. Three of the districts are conterminous, while Mohale’s Hoek is located in the southeast by itself. LPHCR, Lesotho National Primary Health Care Reform.

The Kingdom of Lesotho was one of the original member countries that ratified the Alma Ata Declaration in 1978 and implemented PHC as the key to the attainment of the goal of health for all.22 23 Yet, the Basotho population has been profoundly affected by the HIV pandemic, which still has the second highest HIV prevalence globally (22.7%).24 The WHO lists Lesotho as 1 of the 30 countries with high HIV and tuberculosis (TB) incidence rates.25 Because international funding was largely tied to the treatment of these two conditions in Lesotho,26 PHC became fragmented. By 2015, Lesotho had backslid on health indicators for maternal and child health.21 27 These districts, chosen by the MoH, contained 66 health facilities including six hospitals, covering an estimated catchment population of 880 000 individuals.20

Procedures

Study design

We employed a mixed-methods study with a convergent parallel design to assess completeness of a core set of preselected RHIS data and factors influencing data completeness.28 Quantitative and qualitative data were collected in parallel, analysed separately and then merged for interpretation. This study considered full sampling for quantitative and purposeful sampling for qualitative interviews.29

Quantitative

The WHO Data Quality Assurance (DQA) framework was used to select key PHC indicators for data completeness assessment.30 Per this framework, completeness is defined as monthly health facility register data completion for selected PHC indicators (online supplemental file 1). For this study, data completeness was only selected due to limited information on the other dimensions of data quality.31 The WHO DQA framework provides guidance on which indicators and how to measure them using data collected from RHISs. This study compared trends of DQA indicators data completion in three time periods: pre-LPHCR, LPHCR, and LPHCR with Test and Treat (TT). TT was a national health policy that universally rolled out antiretroviral therapy (ART) for persons living with HIV, regardless of their CD4 count.32 This national policy also included RHIS inputs for HIV and TB programmes, which may have also affected other PHC programmes.

Supplementary data

bmjopen-2022-071414supp001.pdf (46.6KB, pdf)

Qualitative

We used the Performance of Routine Information System Management (PRISM) framework33 to guide interpretation of KIIs focused on RHIS determinants, processes and outputs. Interpretation was structured using the PRISM framework by looking at organisational, technical and behavioural determinants, and RHIS processes and outputs (data utilisation). Organisational determinants analysed were: information culture, structure, resources, and roles and responsibilities at the different levels of the health system. Technical determinants assessed use of data collection forms, and the processes, systems and methods by which they are used. The behavioural determinants reviewed included the knowledge, skills, attitudes, values and motivations of people involved in data collection and utilisation.34 Lastly, RHIS processes—including data collection, transmission, processing, analysis, display, quality assurance and feedback—were analysed (figure 2).

Figure 2.

Figure 2

The Performance of Routine Information System Management framework was developed by MEASURE Evaluation to assess quality and utilisation of RHISs while examining possible gaps in the process related to behavioural, technical and organisational determinants. IT, information technology; RHISs, routine health information systems.

Data collection

Quantitative

Quantitative data were extracted from health facility (n=66) paper-based registers in the four LPHCR pilot districts. To collect data for the study, paper-based data collection forms (online supplemental file 2) were digitised in District Health Information Software-2 (DHIS2).35 The study identified eight key primary health indicators from maternal and child health, HIV, TB and outpatient programmes, excluding malaria.30 36 Data were de-identified in aggregate each month at the facility level, collected from July 2013 through June 2017. Pre-LPHCR period was from July 2013 to June 2014, LPHCR period was from July 2014 to June 2016 and LPHCR/TT period was from July 2016 to June 2017. Tablets for DHIS2 data collection were encrypted and password protected, and devices were kept in a locked file cabinet. Data were collected from September 2017 through February 2018. Site-based data quality audits were conducted at the end of each facility-based data collection visit.

Supplementary data

bmjopen-2022-071414supp002.pdf (188.1KB, pdf)

Qualitative

The qualitative team conducted KIIs following a semistructured format (online supplemental file 3) with HCWs in the following categories: former minister of health, MoH programme directors, district health managers, district public health nurses, health facility managers, community health worker coordinators and community health workers. Interviews were conducted in English or Sesotho, depending on the interviewee’s preference. Transcripts were stored in English in Dedoose,37 which also served as the basis for analysis. Audio-recorded interviews were transcribed in English in Dedoose by qualitative officers who conducted interviews, consisting of two individuals who were a Basotho and non-Basotho. Participant confidentiality was maintained by assigning unique identifiers to each interview.

Supplementary data

bmjopen-2022-071414supp003.pdf (218.6KB, pdf)

Data analysis

Quantitative

Analysis began by descriptively assessing the average monthly data completion rate comparing the three study periods (pre-LPHCR, LPHCR and LPHCR/TT), for each indicator, district and facility type. Findings from this descriptive assessment determined independent variables for our regression analysis. For data quality assessments, the outcome of interest was monthly health facility data completeness for the eight WHO DQA indicators. A binary variable was created in our dataset indicating if any of the eight monthly PHC indicators chosen for this study were available or missing (0=indicators are missing, 1=indicators were available). The study included health facilities (n=66) that reported any numerical value as available, and ‘0’ for any missing data for each indicator. Interrupted time series (ITS) analysis was performed, entailing logistic mixed-effects regression with health facility random intercepts to assess the trend change over the three periods.38 This study modelled overall time effects for each indicator. Then, in the multivariable model, the analysis adjusted for independent variables selected following descriptive analysis. Sensitivity analysis was conducted by including random-effects slope and seasonality in the unadjusted and multivariable model. Effects of seasonality in the models were tested by applying the sin/cos approach.39 Also, we tested the model fit using analysis of variance (ANOVA). Regression analysis using ITS only considered changes in slope with no immediate level change due to the expected gradual change in the outcome related to various national health system factors. The proposed ITS analysis included only WHO DQA indicators with an average data completion rate change above 10% between pre-LPHCR and LPHCR/TT periods. This 10% threshold was designated arbitrarily. Additionally, excluded were indicators with consistently over 90% completion rate during the three periods. The lmerTest package in R was used to model mixed-effects regressions.40 All quantitative analyses were conducted in R V.4.1.2.41 Equation for our model is included in the supplemental documents (online supplemental file 4).

Supplementary data

bmjopen-2022-071414supp004.pdf (81KB, pdf)

Qualitative

Following the PRISM framework, emerging themes were organised under inputs to RHIS performance (organisational, technical and behavioural factors), processes affecting RHIS performance and RHIS output via data utilisation, using codes extracted from the transcripts. Double-blind coding was conducted to each transcript where differences were addressed by a consensus in a regular meeting. Transcripts in Dedoose were analysed using a thematic analysis approach.42 In the coding process, EB and MM read completed transcripts and applied child codes for level of health system (community, health facility, district or central MoH), data reporting process, data for quality assurance and improvements, workload, training and partners collaboration. Excerpts were summarised from these child codes to the parent codes referring to the themes using PRISM framework. In other terms, we grouped leading labels according to thematic groups selected from the PRISM framework. Coded excerpts were exported to Microsoft Excel43 to clean, organise and report main findings. The final findings are synthesised and included in the Results section.

Patient and public involvement

The public were not involved in the design, conduct, reporting or dissemination plan of our research.

Results

There were 12 facilities each in Butha Buthe and Mohale’s Hoek districts, 17 facilities in Berea and 25 facilities in Leribe participating in the evaluation. We completed 25 KIIs: 6 interviews each in Berea and Butha Buthe, 5 interviews each in Leribe and Mohale’s Hoek and 3 interviews at the central MoH office (table 1).

Table 1.

Background primary healthcare-related information in the four pilot districts and national

Background indicators Berea Butha Buthe Leribe Mohale’s Hoek National
Population* 262 616 118 242 337 521 165 590 2 159 067
Public health facilities 17 12 25 12 285
Hospital 2 2 1 1 20
Health centre 15 10 24 11 265
Number of key informant interviews (KIIs) 6 6 5 5 3†

*Bureau of Statistics (Lesotho).

†KIIs from the central MoH office based in the capital city, Maseru.

MoH, Ministry of Health.

Quantitative results

Descriptive results

Based on a descriptive analysis, the average data completion rate progressively increased from 85.8% to 97.3% through the three study periods among hospitals. Health centres reported a slight reduction in the completion rate from 90.1% in pre-LPHCR to 88.8% during LPHCR. However, this rate progressively increased to 94.2% during LPHCR/TT period (online supplemental file 5).

Supplementary data

bmjopen-2022-071414supp005.pdf (405.3KB, pdf)

When assessed by DQA indicator, data completion rates increased over 10% during LPHCR period for first antenatal care (ANC), fourth ANC, institutional delivery and postnatal care (PNC). Immunisation indicator had a 99% data completion rate during pre-LPHCR, which was sustained in the following two study periods. Similarly, ART initiation and TB treatment initiation had over 90% completion rate through the three study periods (figure 3). Monthly changes in combined (all eight indicators) average data completion rate by district started above 70%, with gradual improvement over time (online supplemental file 6).

Figure 3.

Figure 3

The WHO Data Quality Assurance framework was used to select key primary healthcare indicators for data completeness assessment. Here we used this framework to assess the average completion rate during the three study periods. We included two time segments referring to LPHCR and Test and Treat (TT) implementation. ANC, antenatal care; ART, antiretroviral therapy; LPHCR, Lesotho National Primary Health Care Reform; TB Tx, tuberculosis treatment.

Supplementary data

bmjopen-2022-071414supp006.pdf (108.7KB, pdf)

Regression results

Following descriptive assessment of data completion rate by indicator and following our criteria of indicator inclusion, of the eight indicators, the ITS analysis included first ANC, fourth ANC, institutional delivery, PNC and outpatient indicators. In the multivariate analysis adjusting for health facility type and districts, trends in monthly data completion rate were higher after versus before the LPHCR for: documenting first ANC visit (adjusted OR (AOR): 1.24, 95% CI: 1.14 to 1.36) and institutional delivery (AOR: 1.19, 95% CI: 1.07 to 1.32). This association was opposite for outpatient indicator (AOR 0.88; 95% CI: 0.82 to 0.95). We did not find statistically significant association during these comparison periods for fourth ANC and outpatient indicators. Trends in monthly data completion rate were higher after versus before the LPHCR/TT for documenting PNC (AOR 1.08; 95% CI: 1.05 to 1.10) and lower for first ANC (AOR 0.91; 95% CI : 0.87 to 0.95) and institutional delivery (AOR 0.93; 95% CI: 0.88 to 0.99). Figure 4 depicts the above results visually, and tabular outputs are included (figure 4 and online supplemental file 7). Similar outcome magnitude and direction were observed when we included seasonality and random-effects slope in our models.

Figure 4.

Figure 4

Forest plot visualisation of unadjusted and adjusted regression results for the key WHO Data Quality Assurance indicators. ANC, antenatal care; LPHCR/TT, Lesotho National Primary Health Care Reform with Test and Treat; PNC, postnatal care.

Supplementary data

bmjopen-2022-071414supp007.pdf (75.7KB, pdf)

Qualitative findings

Below are qualitative findings based on KIIs with individuals representing the different levels of the Lesotho health system. Leading labels are grouped under the core components of the PRISM framework.

The newly implemented organisational structure established clear roles and responsibilities for data completeness and reporting (PRISM organisational factor).

HCWs mentioned that the newly implemented organisation structure (online supplemental file 8) introduced clear roles and responsibilities to different HCWs who were also tasked with using routine data for decision-making. They stated that each level of the health system was tasked with reviewing performance using RHIS, before the LPHCR data at the district were compiled and sent to the ministry programme staff for review without analysis by district-based health teams.

Supplementary data

bmjopen-2022-071414supp008.pdf (63.3KB, pdf)

So that [DHMT reorganization] was the first step, to create accountability or clear margins. This reorganization guided who is supposed to do and, who has to report to who and when…then it was clear on the method of reporting and the accountability and so-forth. (DHMT)

HCWs received training on RHIS data utilisation as part of comprehensive health system performance monitoring and reporting (PRISM organisational factor).

Respondents indicated that training was conducted at the different levels of the health system to prepare HCWs for the newly introduced organisational structure in the districts. Included in this training was RHIS data quality assurance and utilisation for performance management.

They (District Medical officers) were trained on LPHCR activities, and that kind of training program has given them the confidence to do things. (central MoH)

Motivation to collect, verify and report routine data improved following the bidirectional data-sharing mechanisms created by the LPHCR (PRISM behavioural factor).

There were a consensus among respondents on the improved motivation to collect and report routine data associated with the bidirectional information-sharing mechanism introduced by the LPHCR. It was mentioned that the bidirectional data-sharing mechanism gave a platform for implementers to communicate their successes and challenges with decision-makers at the district and central MoH levels. Respondents stated that facilities interacted with districts quarterly using their routine data to discuss progress and shortcomings. As a result, respondents agreed that it improved their awareness and motivation in using RHIS data.

There was a report produced from the district to be submitted to the higher level every month. So that made a point that I deal with my data, I compile a report, then I submit it. So, I think it was a whole process that was helping us. (DHMT)

Routine data reporting tools were provided to community and facility teams, along with support on how to complete them (PRISM technical factor).

Respondents mentioned availability of reporting forms for community health workers to report indicators related to maternal and child health, HIV and TB programmes conducted in the community level. They also indicated that these reports were reviewed monthly by supervisors, health facility nurse in charge and public health nurse. As for health facilities, routine reports electronically with the support of data clerk and Health Management Information System officer supervising the submission.

….there are forms to fill in that information. And they [districts] are also given reporting tools that, as they are doing their work….they know that they need to document this and that and report on it. (central MoH)

Health facility managers were provided with computers to submit monthly reports (PRISM technical factor).

Respondents mentioned that each health facility nurse in charge was provided computers to submit their monthly report. There was a consensus whereby via this computer, facilities were submitting reports without leaving their workplace unlike in the pre-LPHCR period.

We use a computer to send them we no longer have to wait for someone from the DHMT to come here (health facility)… (health facility)

Quarterly PHC meetings facilitated the use of data to monitor and evaluate progress, which indirectly improved routine data completeness (PRISM RHIS outputs).

Respondents indicated that the reform enhanced quarterly PHC meetings in districts and nationally, by emphasising the use of data to assess health facility and DHMT performance. They believed this quarterly meeting promulgated data quality and data utilisation. They mentioned that each district had their own internal quarterly PHC review meetings. Then, the central MoH team conducted all district PHC meetings quarterly on different dates from district internal PHC meetings.

It is a three day workshop [quarterly district health meeting] all the partners are there all the health centers that are in our district, the staff from DHMT each facility will be present according to the indicators that according to all the services that we offered at the clinic we are presenting on them. After every two presenters we discuss and find solutions. (health facility)

Consistent transportation infrastructure strengthened mentorship and supervision visits to health facilities by the DHMT (PRISM RHIS outputs).

There was a consensus among respondents that the LPHCR project provided consistent transportation for DHMT staff to visit and provide mentorship to health facilities in the pilot districts. Respondents mentioned that this consistent availability of transportation strengthened routine mentorship and supervision—using data generated by health facilities—to provide targeted supportive supervision. Respondents mentioned that prior to the LPHCR, even with availability of transportation, supervisions were conducted sporadically.

Because now [LPHCR], even before visiting, the team visiting a facility…discusses the facility [performance], looking at data, looking at everything. So, kind of prioritizing certain questions before going to the health center – using what data is available…. then determine what we need to follow up on more directly. (DHMT)

Discussion

Our assessment employed a mixed-methods study design to evaluate completion of routine health data reported by health facilities as well as factors influencing data completion and data utilisation associated with LPHCR.

Descriptive analysis indicated that the MoH already had a strong data completion rate pre-LPHCR, which was sustained with LPHCR despite an increase in service utilisation. With the reform, the data quality improvement efforts were optimised through improved behavioural, technical and organisational factors introduced by the LPHCR. Based on our quantitative analysis, we observed noticeable improvement in data completion for maternal and child health indicators: first ANC, completion of four ANC visits, institutional delivery and PNC. Vertical programmes such as HIV, TB and immunisation had a high completion rate before LPHCR and sustained with LPHCR.

Qualitative results using the PRISM framework offered complementary information, with interviewees expressing satisfaction with improved data quality. Based on these interviews, respondents (HCWs) underscored improved organisational structure, staffing, training, data reporting and analysis processes at all levels. Additionally, they mentioned quarterly PHC review meetings as an important part of the LPHCR that facilitated improved data quality and data utilisation practices.

During the LPHCR, DHMTs were equipped to support health facilities due to increased data generation and a data utilisation culture. Likewise, central MoH programme leaders encouraged and supported district teams following their quarterly PHC meetings. The reform strengthened data reporting processes across different levels of the health system, where health facilities had the ability to know what was happening in their catchment communities; DHMTs were routinely reviewing facility performance to make sure facilities falling behind received all the support they needed; and central MoH teams extended their support to district teams after reviewing monthly and quarterly performance data reported by the DHMTs. Though most findings were positive, KIIs also mentioned that facility-level HCWs were demotivated when data clerks from Berea district health facilities were phased out a year after the LPHCR roll-out. DHMTs and facility-level staff faced major challenges coordinating donors who had different competing monthly report priorities despite the organised structure introduced by the LPHCR. Explanation for the decrease in the trends of monthly data completion rate for some indicators during versus before LPHCR period was not raised during KIIs.

Results from this study are particularly important for healthcare stakeholders in SSA looking to improve RHISs through a comprehensive investment in PHC. Though the LPHCR targeted some of the health system building blocks, it was a holistic, systematic intervention that touched various levels of the health system building blocks. Having a clear organisational reporting process and data-sharing platform was important in creating a bidirectional mechanism in information sharing, which optimises health service delivery and patient outcomes.44 Organisational factors are often bypassed due to precedence given to technical and clinical characteristics in RHIS evaluations, though it is an important characteristic influencing RHIS performance.45 46 Findings from this study highlighted the importance of incorporating organisational, behavioural and technical determinants of RHIS process and outputs to better understand factors influencing RHIS change.47 A strength of the study was the use the use of a mixed-methods design to evaluate changes in RHISs as part of the LPHCR.

In a recent LPHCR study conducted by Ndayizigiye et al, the authors found that the quarterly average number of women who attended their first ANC visit rose from 1877 (SD ±263) pre-LPHCR to 2729 (SD ±287) in the last year of the intervention (LPHCR/TT).15 Observed data completion improvement during the LPHCR/TT period and during the LPHCR highlights the synergy between different national-level initiatives aimed at improving population health outcomes. Mixed-effects regression analysis showed that the association was not confounded by health facility type and differences by district. Per our model diagnostic using ANOVA, we did not find a significant difference in our result when we adjusted for seasonality or random slope. Following a lagged effect assessment using autocorrelation function (ACF) and partial ACF, a lag 1 autoregressive correlation (AR-1) was detected in our time series; however, we did not find a statistically significant effect of AR-1 on our model. Due to word limits in this journal, we chose not to report alternative models used for sensitivity analysis.

A limitation of the study was not setting comparison districts or facilities during the intervention for better causal inference. However, to account for this key limitation, the study reverted to using an ITS framework to evaluate before and after LPHCR trends in data completion in the same sites. Second, monthly data extraction from facilities was cumbersome and required extensive human and financial resources. Fortunately, facility and district administrators were instrumental in co-identifying and addressing gaps with the study team. Third, facility registers were prone to displacement, damage and inconsistent storing procedures. To ensure quality of data reports, the data collection team conducted a thorough audit and follow-up with relevant health facilities and district team personnel before releasing data for analysis. As for qualitative data, though we were able to get important responses from all the KIIs, a disproportional number of feedback about RHISs were collected from district and facility staff compared with central and community programme staff. Though it would have been interesting to further examine the unique impact of TT on RHIS, this study chose not to do this analysis due to the task being out of our scope.

Further studies assessing the cost of conducing similar research will be beneficial for policymakers to scale up this intervention to other sites in Lesotho or elsewhere. The study design used in this study could be improved by adding more KIIs from central MoH staff and community teams, given availability of resources and approval from ethics committees.

In conclusion, Lesotho’s RHISs appear to have improved in pilot districts once the LPHCR was introduced. These results were sustained despite the substantial increase in PHC service utilisation in the four districts. Findings underscore the role of strengthening comprehensive PHC RHISs as a core input in shaping the success of health system reform.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

We would like to extend our utmost gratitude to Lesotho's Ministry of Health for facilitating and supporting this study. We thank the Skoll and Wagner Foundations for their support. This project was made possible by the great work conducted by the LPHCR evaluation team (Abera Leta, Afom Andom, Joel Curtain, Annie Michaelis, Collin Whelley, Merida Carmona, Relebohile Mjadu, Bolaoane Ntlape, Matsisli Tseka, Jonase Nthunya, Patrick Smith, Tholoana Marole, Relebohile Lebatla, and Thiane Mohlouoa). Thanks to everyone at University of Washington Department of Global Health for advising on study design, analysis and dissemination. We also thank everyone at PIH (Likhapha Ntlamelle, Sophie Motsamai, Joalane Mabathoana, Palesa Chetane, Seyfu Abebe, Thaothe Bosiu, Tholoana Mohapi, Mojela Masupha, Renang Letima, Mpho Mosisili, Bokang Ramothamo, Anna Maphela, Qenhelo Matjama, Moqhaola Liphoto, Malebo Matobo, Kevin Dolan, Hind Satti, Archie Ayeh, David Omotayo, Mpho Marole, Masebina Mpae, Kabi Setsi, Emily Dally, Emily Gingras, Jean Claude Mugunga, Hari Iyer and Lisa Hirschhorn). We also extend our immense appreciation to key informant interview participants who took time from their busy schedule to participate in this study. Finally, we thank everyone at Ministry of Health and Lesotho partners who worked tirelessly to implement the LPHCR.

Footnotes

Twitter: @ermula517, @Melino_MD

Contributors: EB assumes the role of the content guarantor, taking full responsibility for the completed work and/or the study's execution. The guarantor had access to the data and made the decision regarding publication.EB, MN, RM and JM conceived the study design. MN, MM, MK, RM and EB facilitated and supervised data collection. EB, OA and RM conducted quantitative analysis. EB, EC, MM and NP prepared and analysed qualitative analysis. EB led the write-up. All authors provided thorough and critical feedback to finalise this paper.

Funding: Funding for this study came from the Skoll Foundation (grant number 19-44478) (JM) and the Wagner Foundation (project code AVA0436).

Disclaimer: The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Map disclaimer: The inclusion of any map (including the depiction of any boundaries therein), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study involves human participants and received ethical approval from the Lesotho National Health Research Ethics Committee (ID 117-2017). Harvard Human Research Protection Program granted ethical approval for this study (IRB17-19888) for both qualitative and quantitative study tools.

References

  • 1.Hotchkiss DR, Diana ML, Foreit KGF. How can routine health information systems improve health systems functioning in Low- and middle-income countries? assessing the evidence base. Adv Health Care Manag 2012;12:25–58. 10.1108/s1474-8231(2012)0000012006 [DOI] [PubMed] [Google Scholar]
  • 2.Hoxha K, Hung YW, Irwin BR, et al. Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: a systematic review. Health Inf Manag 2022;51:135–48. 10.1177/1833358320928729 [DOI] [PubMed] [Google Scholar]
  • 3.Dagnew E, Woreta SA, Shiferaw AM. Routine health information utilization and associated factors among health care professionals working at public health institution in North Gondar, Northwest Ethiopia. BMC Health Serv Res 2018;18:685. 10.1186/s12913-018-3498-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Wagenaar BH, Hirschhorn LR, Henley C, et al. Data-driven quality improvement in low-and middle-income country health systems: Lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Serv Res 2017;17:830. 10.1186/s12913-017-2661-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Odekunle FF, Odekunle RO, Shankar S. Why sub-Saharan Africa lags in electronic health record adoption and possible strategies to increase its adoption in this region. Int J Health Sci 2017;11:59–64. [PMC free article] [PubMed] [Google Scholar]
  • 6.Druetz T. Integrated primary health care in Low- and middle-income countries: A double challenge. BMC Med Ethics 2018;19:48. 10.1186/s12910-018-0288-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Madon S, Krishna S, Michael E. Health information systems, Decentralisation and Democratic accountability. Public Admin Dev 2010;30:247–60. 10.1002/pad.571 [DOI] [Google Scholar]
  • 8.Bernardi R, Bernardi R. Health information systems and accountability in Kenya: a Structuration theory perspective. J Assoc Inf Syst 2017;18:931–58. 10.17705/1jais.00475 [DOI] [Google Scholar]
  • 9.Sahay S, Sundararaman T, Braa J. Public health informatics: designing for change-a developing country perspective. Oxford University Press, 2017. [Google Scholar]
  • 10.Dehnavieh R, Haghdoost A, Khosravi A, et al. The district health information system (Dhis2): A literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Inf Manag 2019;48:62–75. 10.1177/1833358318777713 [DOI] [PubMed] [Google Scholar]
  • 11.Mutemwa RI. HMIS and decision-making in Zambia: Re-thinking information solutions for district health management in decentralized health systems. Health Policy Plan 2006;21:40–52. 10.1093/heapol/czj003 [DOI] [PubMed] [Google Scholar]
  • 12.Wagenaar BH, Sherr K, Fernandes Q, et al. Using routine health information systems for well-designed health evaluations in Low- and middle-income countries. Health Policy Plan 2016;31:129–35. 10.1093/heapol/czv029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Arsenault C, Yakob B, Tilahun T, et al. Patient volume and quality of primary care in Ethiopia: findings from the routine health information system and the 2014 service provision assessment survey. BMC health Serv Res 2021;21:1–10. 10.1186/s12913-021-06524-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Ashton RA, Bennett A, Al-Mafazy A-W, et al. Use of routine health information system data to evaluate impact of malaria control interventions in Zanzibar, Tanzania from 2000 to 2015. EClinicalMedicine 2019;12:11–19. 10.1016/j.eclinm.2019.05.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Ndayizigiye M, Allan-Blitz L-T, Dally E, et al. Improving access to health services through health reform in Lesotho: Progress made towards achieving universal health coverage. PLOS Glob Public Health 2022;2:e0000985. 10.1371/journal.pgph.0000985 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Department of Economic and Social Affairs . Transforming our world: The 2030 agenda for sustainable development. n.d. Available: https://sdgs.un.org/2030agenda
  • 17.World Health Organization . Everybody’s business—strengthening health systems to improve health outcomes: WHO’s framework for action 2007.
  • 18.Nicol E, Bradshaw D, Uwimana-Nicol J, et al. Perceptions about data-informed decisions: an assessment of information-use in high HIV-prevalence settings in South Africa. BMC Health Serv Res 2017;17:25–38. 10.1186/s12913-017-2641-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Nisingizwe MP, Iyer HS, Gashayija M, et al. Toward utilization of data for program management and evaluation: Quality assessment of five years of health management information system data in Rwanda. Glob Health Action 2014;7:25829. 10.3402/gha.v7.25829 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Bureau of Statistics (Lesotho) . Lesotho population and housing census. 2016. [Google Scholar]
  • 21.Satti H, Motsamai S, Chetane P, et al. Comprehensive approach to improving maternal health and achieving MDG 5: Report from the mountains of Lesotho. PLoS One 2012;7:e42700. 10.1371/journal.pone.0042700 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Declaration of Alma-ATA. n.d. Available: https://www.who.int/teams/socialdeterminants-of-health/declaration-of-alma-ata
  • 23.Regional Office for Africa, WHO . Lesotho reviews primary health care strategy, community health worker policy and Ministry of health supervision policy. n.d. Available: https://www.afro.who.int/news/lesotho-reviews-primary-health-care-strategy-community-health-workerpolicy-and-ministry
  • 24.ICAP at Columbia Lesotho summary sheet 2020. PHIA project. n.d. Available: https://phia.icap.columbia.edu/esotho-summary-sheet-2/
  • 25.World Health Organization . WHO global lists of high burden countries for tuberculosis (TB), TB/HIV and multidrug/rifampicin-resistant TB (MDR/RR-TB), 2021–2025: background document, 2021. [Google Scholar]
  • 26.Mukherjee J, Lindeborg MM, Wijayaratne S, et al. Global cash flows for sustainable development: a case study of accountability and health systems strengthening in Lesotho. J Health Care Poor Underserved 2020;31:56–74. 10.1353/hpu.2020.0008 [DOI] [PubMed] [Google Scholar]
  • 27.Hogan MC, Foreman KJ, Naghavi M, et al. Maternal mortality for 181 countries, 1980–2008: A systematic analysis of progress towards millennium development goal 5. Lancet 2010;375:1609–23. 10.1016/S0140-6736(10)60518-1 [DOI] [PubMed] [Google Scholar]
  • 28.Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd edn. Los Angeles: Sage Publications, 2011. [Google Scholar]
  • 29.Ames H, Glenton C, Lewin S. Purposive sampling in a qualitative evidence synthesis: a worked example from a synthesis on parental perceptions of vaccination communication. BMC Med Res Methodol 2019;19:1–9. 10.1186/s12874-019-0665-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.World Health Organization . Data quality review: module 1: framework and metrics. Geneva, Switzerland: World Health Organization, 2017. Available: https://apps.who.int/iris/handle/10665/259224 [Google Scholar]
  • 31.Pipino LL, Lee YW, Wang RY. Data quality assessment. Commun ACM 2002;45:211–8. 10.1145/505248.506010 [DOI] [Google Scholar]
  • 32.World Health Organization . Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach. Geneva, Switzerland: World Health Organization, 2016. Available: http://apps.who.int/iris/bitstream/10665/208825/1/9789241549684_eng.pdf?ua=1pdficonexternal [PubMed] [Google Scholar]
  • 33.Prism: Performance of routine information system management series — measure evaluation. n.d. Available: https://www.measureevaluation.org/prism.html
  • 34.Aqil A, Lippeveld T, Hozumi D. PRISM framework: A paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan 2009;24:217–28. 10.1093/heapol/czp010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Braa J, Hanseth O, Heywood A, et al. Developing health information systems in developing countries: The flexible standards strategy. MIS Quarterly 2007;31:381. 10.2307/25148796 [DOI] [Google Scholar]
  • 36.World Health Organization . Countries and territories certified malaria-free by WHO, 2020. [Google Scholar]
  • 37.SocioCultural Research Consultants . Dedoose version 9.0.17, web application for managing, analyzing, and presenting qualitative and mixed method research data (2021). Los Angeles, CA. [Google Scholar]
  • 38.Lopez Bernal J, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2017;46:dyw098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Stolwijk AM, Straatman H, Zielhuis GA. Studying Seasonality by using sine and Cosine functions in regression analysis. J Epidemiol Community Health 1999;53:235–8. 10.1136/jech.53.4.235 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Kuznetsova A, Brockhoff PB, Christensen RHB. lmerTest package: tests in linear mixed effects models. J Stat Softw 2017;82:1–26. 10.18637/jss.v082.i13 [DOI] [Google Scholar]
  • 41.R Core Team . R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2021. Available: https://www.R-project.org [Google Scholar]
  • 42.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
  • 43.Microsoft Corporation . Microsoft excel. 2018. Available: https://office.microsoft.com/excel
  • 44.McQueen DV, Puska P, eds. Global behavioral risk factor surveillance. New York; London: Kluwer Academic/Plenum Publishers, 2003. [Google Scholar]
  • 45.Cresswell KM, Robertson A, Sheikh A. Lessons learned from England’s National electronic health record implementation: Implications for the international community. In proceedings of the 2ND ACM SIGHIT International health Informatics symposium 2012:685–90.
  • 46.Sligo J, Gauld R, Roberts V, et al. A literature review for large-scale health information system project planning, implementation and evaluation. Int J Med Inform 2017;97:86–97. 10.1016/j.ijmedinf.2016.09.007 [DOI] [PubMed] [Google Scholar]
  • 47.Eslami Andargoli A, Scheepers H, Rajendran D, et al. Health information systems evaluation Frameworks: A systematic review. Int J Med Inform 2017;97:195–209. 10.1016/j.ijmedinf.2016.10.008 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2022-071414supp001.pdf (46.6KB, pdf)

Supplementary data

bmjopen-2022-071414supp002.pdf (188.1KB, pdf)

Supplementary data

bmjopen-2022-071414supp003.pdf (218.6KB, pdf)

Supplementary data

bmjopen-2022-071414supp004.pdf (81KB, pdf)

Supplementary data

bmjopen-2022-071414supp005.pdf (405.3KB, pdf)

Supplementary data

bmjopen-2022-071414supp006.pdf (108.7KB, pdf)

Supplementary data

bmjopen-2022-071414supp007.pdf (75.7KB, pdf)

Supplementary data

bmjopen-2022-071414supp008.pdf (63.3KB, pdf)

Reviewer comments
Author's manuscript

Data Availability Statement

Data are available upon reasonable request.


Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES