Impact and cost-effectiveness of Neotree, a digital data capture and decision support tool designed to improve neonatal survival in Zimbabwe: an interrupted time series analysis and economic evaluation

Abstract

Introduction

Many neonatal deaths are avoidable using existing low-cost evidence-based interventions. This study evaluated the effectiveness and cost-effectiveness of Neotree, a digital quality improvement tool combining data capture with education and clinical decision support, implemented in a Zimbabwean hospital.

Methods

Neotree was implemented in Chinhoyi Provincial Hospital (CPH) in December 2020. Using data collected for all neonates admitted to CPH from March 2020 to October 2023, a single group interrupted time series analysis was conducted to estimate the impact of Neotree implementation. Subgroup analyses explored the impact in low birth weight (1.5–2.5 kg) neonates, a key group targeted by the intervention.

Activity-based costing and expenditure approaches estimated costs of developing and implementing Neotree in CPH from a provider perspective. Both total within-study costs and total costs at scale were estimated and used to derive cost per life saved, cost per life year saved and cost per healthy life year (HLY) gained.

Results

Analysis suggests reduced overall mortality in the post-implementation period, though this difference was not statistically significant (RR: 0.877, 95% CI 0.541 to 1.423, p=0.596). This was primarily driven by reduced mortality among the low birth weight subgroup (RR: 0.356, 95% CI 0.127 to 1.002, p=0.051). Cost-effectiveness analysis based on an assumed mortality impact in this subgroup suggests a within-study cost of around $28.44 per HLY gained, reducing to $6.35 per HLY gained at scale, substantially below the range of potential cost-effectiveness thresholds considered for Zimbabwe (US $17– US $855).

Conclusion

Neotree is a potentially low-cost and highly cost-effective digital quality improvement tool to improve newborn care, morbidity and survival, while also providing quality data. This study contributes to limited economic evidence of mHealth tools in low-income and middle-income settings.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• We searched PubMed for recent economic evaluations of mobile health clinical decision support tools that aim to improve neonatal outcomes. We searched key Boolean terms for “neonatal” OR “infant” OR “newborn” OR “antenatal” and “cost” OR “economic” AND “mobile health” OR “mHealth” OR “clinical decision”. The search was restricted to primary studies in low- and middle-income countries that were published between 1 January 2000 and 1 May 2024. Additional screening of bibliographies was also conducted. Six relevant studies were identified: two each in India and Ghana, and one each in Nigeria and Tanzania. Despite differing methodologies, all of these studies broadly conclude that the intervention was cost-effective. However, none of these studies took place in a hospital setting, and all took place within primary healthcare facilities or community settings.

WHAT THIS STUDY ADDS

• Neotree is an open-source digital quality improvement tool combining data capture with education and clinical decision support. Our study estimated the effectiveness and cost-effectiveness of the Neotree intervention in Chinhoyi Provincial Hospital in Zimbabwe. The outcome of the effectiveness analysis was neonatal mortality. Costs were estimated from the provider perspective. The Neotree intervention is potentially highly cost-effective, with a cost of around US $6.35 per HLY gained.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• Digital health tools have the potential to be highly cost-effective, including in hospital settings. Although our study relies on observational data, we do not account for wider benefits beyond reduced mortality, including the benefits of capturing data which could be used to improve neonatal care in future.

Introduction

Two-thirds of the 2.4 million newborn deaths that occur within the first 28 days of life could potentially be avoided by ensuring existing low-cost evidence-based interventions are implemented for all sick and small newborns.^{1 2} However, health systems in many low- and middle-income countries (LMICs) are constrained by factors such as lack of essential equipment and drugs,^{3 4} lack of training⁵ and poor access or adherence to clinical guidelines.^{6 7}

Clinical decision support systems, which directly assist decision-making or facilitate decisions through using patient data, offer a potential solution to improve the quality of neonatal care. Evidence largely from high-income settings suggests such systems can improve adherence to clinical guidelines and clinical outcomes.⁸ Evidence on the effectiveness, cost and cost-effectiveness of mobile health (mHealth) decision-support tools in LMICs is more limited. A randomised controlled trial of an mHealth clinical decision-making support intervention within hospitals in Ghana found increased risk of neonatal mortality in the intervention arm, though the authors note potential issues with death registration affecting the analysis.⁹ Other studies suggest that such tools have the potential to be cost-effective in the context of maternal and child health.^10–13 However, these analyses were based on small-scale and short-term implementation of interventions, and none were delivered in hospital settings, where the majority of small and sick newborns are cared for.

Neotree is an open-source digital quality improvement tool that combines data capture with education and clinical decision support.¹⁴ Neotree is a tablet-based application that works both offline and online and guides clinicians from admission to discharge for all babies in a newborn care unit.¹⁵ The draft logic model has been published elsewhere.¹⁴ The intervention was conceptualised as a digital learning health system and is based on the premise that small or vulnerable babies in low resource settings do not consistently receive high quality, evidence-based care. It has been developed with and for all healthcare professionals, but predominantly those least experienced in newborn care. On admission of all babies, clinical data are entered and trigger educational messages (eg, how to check a baby’s temperature) and prompts (e.g. check the baby’s oxygen levels) at the bedside. These data then feed into evidence-based algorithms embedded within the application that guide the user through a set of diagnoses and management guidelines, based on national guidelines for caring for small and sick newborns.^{16 17} A shorter data capture procedure occurs on discharge and on reporting of blood cultures. Pseudonymised data is then exported to a local database, visualised on data dashboards and made available for audit and quality improvement. Fully identifiable patient records can be linked directly to national data systems (e.g. DHIS2).

To date, Neotree has been used to support the care of over 40 000 babies by more than 1400 healthcare providers across seven hospitals and two primary health centres in Malawi and Zimbabwe. In its initial development, it was conceptualised as an intervention to improve quality of care in low birth weight babies. However, in response to feedback from newborn healthcare providers in Bangladesh, it rapidly pivoted to an intervention for all babies admitted for postnatal care.¹⁴ The objectives of this study were to model the potential impact of Neotree implementation on neonatal case-fatality rate in one study hospital, model the potential cost-effectiveness of Neotree implementation and estimate costs of implementation at scale, relative to standard care.

Methods

Setting and outcome data collection

Pilot implementation of the Neotree intervention took place in two hospitals in Zimbabwe: Sally Mugabe Central Hospital (SMCH) in 2018 and Chinhoyi Provincial Hospital (CPH) in 2020. SMCH is the largest of three tertiary neonatal units in Zimbabwe and delivers around 12 000 babies annually.¹⁸ CPH is a provincial hospital which delivers around 4500 newborns annually, with audit data showing case fatality rates of 180 per 1000 admitted babies.¹⁸ The hospital functions as a level 2 newborn care unit and is similar to most district (and some central) hospitals in providing non-invasive ventilation as the maximum level of support. Findings from CPH are therefore generalisable to similar levels of newborn care units, including district level hospitals.

Cost data from both hospitals are used in this analysis (see below). However, individual-level outcome data are available for CPH only, including all neonates admitted to CPH between March 2020 and October 2023. Clinical and mortality data for this hospital are available both pre-intervention and post-intervention implementation, and include variables such as demographics, clinical observations and outcomes. Health workers used the Neotree database to capture outcome data by completing digital forms during admission and discharge.¹⁵

Statistical analyses

All analyses were conducted using R v4.2.2.¹⁹ Descriptive analysis of neonatal admissions and deaths was conducted by summarising monthly values both numerically and graphically. Additional descriptive analyses are also presented for birth weight, as a key risk factor of interest.

The interrupted time series analysis²⁰ used weekly data windows and included all neonates admitted to CPH during the observation period. Intervention implementation began on 1 December 2020. The primary outcome for this analysis was the count of mortality, defined as an outcome of ‘neonatal death’ listed on the outcome form. All other neonates were assumed to be alive (ie, including those discharged or transferred). Poisson regression models were fitted, contingent on no observed overdispersion defined as a dispersion parameter defined as residual deviance divided by df in excess of 1.1. The log of the weekly number of admissions was used as an offset term to model event rates. For the primary outcome, we hypothesised that the intervention impacted both the level and the slope of the pre-existing trend. Thus, all models included both a binary intervention indicator and an interaction term between intervention and time. The assumption of independent observations was verified using residual autocorrelation and partial autocorrelation plots, in addition to the Breusch-Godfrey test.

The model was adjusted for the national COVID-19 lockdown in Zimbabwe, which took place between 30th March and 11th June 2020. We also considered a national nurses’ strike that took place in Zimbabwe from 17th June to 9th September 2020 over pay and availability of personal protective equipment during the pandemic.²¹ Adjustments for seasonality using harmonic terms consisting of pairs of sine and cosine functions were also considered. Adjusting for seasonality and nurses’ strikes did not improve model fit, and therefore our primary results are unadjusted for both. Final model choice was based on comparisons of goodness-of-fit using likelihood ratio tests for nested models and minimising Akaike’s information criterion (online supplemental materials 1).

Subgroup analyses were also conducted by stratifying analyses based on a key risk group of interest. Low-birth weight (1.5 kg-2.5 kg) admissions may be particularly important to analyse, as they are the clinical target group. Neotree was originally designed to impact and was most amenable to simple, low-cost interventions to reduce mortality, such as keeping them warm. It is also likely that this group of babies would always have been admitted (i.e. before and after Neotree implementation), while it is conceivable that Neotree implementation influenced the decision to admit heavier (and healthier) babies. The same model used for the interrupted time series analysis of overall mortality was used for this subgroup. Data on birth weight was missing for a very small number of admissions, which were excluded from the subgroup analysis.

Costs

We previously conducted a cost analysis of pilot implementation of Neotree in CPH and SMCH, as described in detail elsewhere.^{18 22} We used a combination of activity-based costing and expenditure approaches to estimate the provider perspective cost of developing and pilot implementation of Neotree over a time horizon of 12 months. Data were collected through expenditure reports, monthly staff time-use surveys and interviews with project staff. In our previous analysis, the average cost per child admitted, including intervention development costs, was US$58 in CPH and US $15 in SMCH (2021 values). However, under routine (non-research) conditions and at scale, total costs were estimated to reduce substantially, down to a cost per admitted child of US $14 in CPH and US $4 in SMCH. The cost of US $58 is used for a within-CPH cost-effectiveness analysis, while analysis of cost at scale is based on an average of costs in the two intervention facilities in the base case analysis (ie, US $9 per admission).

Cost-effectiveness analysis

First, we report within-study incremental cost-effectiveness ratio (ICERs) using primary data from CPH. ICERs were calculated in terms of incremental cost (in 2021 USD) per healthy life year (HLY) gained from Neotree implementation relative to standard care. The number of HLYs gained was estimated based on the estimated intervention impact on mortality in the low birth weight subgroup (1.5–2.5 kg). We conservatively assume that the intervention does not impact mortality of babies outside of this range. They assumed that weight distribution and associated mortality risks are based on pre-intervention data for CPH. HLYs are equivalent to disability adjusted life years (DALYs) but are expressed in terms of gains, which may be more intuitive for decision makers.²³ Base case analyses are based on estimated healthy life expectancy of 53.56 years for Zimbabwe.²⁴

The time horizon for the costing was 12 months. The cohort size used in the analysis is therefore the annual number of neonatal admissions. A lifetime horizon was used for outcomes. Discounting was not applied to costs as no future costs are incurred. A 0% discount rate was also used for outcomes, in line with Global Burden of Disease methodology and with recent WHO analysis of interventions to improve newborn health.²³

Second, we report ICERs at scale, whereby the effect of the intervention on neonatal mortality in CPH was extrapolated to neonatal admissions nationally in Zimbabwe. While the methodology is largely the same as that outlined for the within-study analysis above, several further assumptions are required for key parameters. Detailed assumptions for this analysis are presented in online supplemental materials 2.

Additionally, the impact of uncertainty in these assumptions on results of the analysis was explored in a one-way sensitivity analysis. We varied intervention effectiveness, costs, health-adjusted life expectancy, baseline mortality probability and discount rate parameters between plausible ranges and assessed the impact on estimated ICERs (see online supplemental materials 2 for details). Given that the estimated impact of the intervention is of particular importance and subject to uncertainty, multiple scenario analyses were conducted for this parameter to inform likely cost-effectiveness of the intervention. This includes a scenario where the intervention may reduce mortality in all birth weight categories, as well as a scenario where the intervention has no impact on mortality.

A probabilistic sensitivity analysis, using Monte Carlo simulation with 1000 replicates, was also conducted for the cost-effectiveness analysis at scale, to simultaneously model variation in all parameters based on assumed distributions (see online supplemental materials 2 for models’ specification). A cost-effectiveness acceptability curve was also created based on the probabilistic sensitivity analysis results, which shows the probability of the Neotree intervention being cost-effective under different assumptions of cost-effectiveness thresholds. A likely range of cost-effectiveness thresholds of between 1% and 51% of GDP per capita²⁵ was considered, amounting to a threshold of between US $17 and US $855 for Zimbabwe based on 2022 estimates.²⁶

Budget impact analysis

The total cost of intervention scale-up is also reported to facilitate budget impact analysis. This requires an estimate for the total annual number of neonatal admissions to public facilities in Zimbabwe. In the absence of data to inform this, the following formula was used:

$${\displaystyle {\displaystyle Pop\times BR\times PFDR\times AR,}}$$

where $Pop$ is the total population size of Zimbabwe, $BR$ is the crude birth rate, $PFDR$ is the public facility delivery rate of 65% of all births²⁷ and $AR$ is the admission rate for public facility deliveries, which, in the absence of any data to inform this, was assumed to be one in seven. The impact of uncertainty in the number of admissions on total costs was explored in scenario analyses, via a range of assumptions regarding admission rates. Total cost of intervention scale-up was compared with GDP and public health expenditure in Zimbabwe²⁶ to contextualise the magnitude of the intervention cost in this setting.

Results

Descriptive analysis of monthly admissions and deaths

In total, there were 2879 neonatal admissions to CPH and 425 neonatal fatalities between March 2020 and October 2023. Figure 1 shows the number of monthly admissions and deaths at CPH over the observation period. The number of monthly admissions is increasing over time, while the number of those admissions resulting in death remains relatively stable.

Figure 1.

Monthly number of admissions and deaths in Chinhoyi Provincial Hospital (March 2020-October 2023).

Table 1 summarises the number of admissions and deaths among neonates by birth weight category, with separate results for the pre- and post-intervention periods. The overall mortality rate pre-intervention was 18.39%, compared with 14.12% post-intervention. In the key risk group of low birth weight neonates, mortality rates at CPH pre-intervention and post-intervention were 18.25% and 7.60% respectively.

Table 1.

Admissions and mortality by birth weight in Chinhoyi Provincial Hospital

	<1.5 kg	1.5–2.5 kg	>2.5 kg	Missing	Total
Admissions by birth weight category n (% of total admissions)
Pre-intervention	78 (17.93%)	126 (28.97%)	231 (53.10%)	0 (0.00%)	435 (100.0%)
Post-intervention	390 (15.96%)	737 (30.16%)	1310 (53.60%)	7 (0.29%)	2444 (100.0%)
Mortality by birth weight category n (% mortality)
Pre-intervention	33 (42.31%)	23 (18.25%)	24 (10.39%)	0 (NA)	80 (18.39%)
Post-intervention	172 (44.10%)	56 (7.60%)	116 (8.85%)	1 (14.29%)	345 (14.12%)

Interrupted time series analyses

Figure 2 presents results of the interrupted time series analyses on overall mortality in CPH. The Poisson regression model suggests a decrease in both the level (RR: 0.877, 95% CI 0.541 to 1.423, p=0.596) and slope (RR: 0.997, 95% CI 0.977 to 1.018, p=0.781) of mortality following Neotree implementation, though neither was statistically significant.

Figure 2.

Interrupted time series for overall mortality (Chinhoyi Provincial Hospital (CPH)).

Figure 3 presents results of the interrupted time series analysis of mortality in CPH when considering only the low-birth weight subgroup (ie, 1.5–2.5 kg). The Poisson regression model suggests a decrease in the level (RR: 0.356, 95% CI 0.127 to 1.002, p=0.051) but not the slope (RR: 1.004, 95% CI 0.965 to 1.046, p=0.833) of mortality following Neotree implementation. Notably, 58% of pre-intervention weekly windows saw zero deaths in this subgroup compared with 66% of post-intervention weekly windows, though this difference was not statistically significant when compared using Fisher’s exact test (p=0.438).

Figure 3.

Interrupted time series for mortality among 1.5–2.5 kg birth weight subgroup (Chinhoyi Provincial Hospital (CPH)).

Cost-effectiveness analysis

Base case analysis assuming a mortality impact among low birth weight admissions only suggests a cost per HLY gained of US $28.44 for CPH, or around 1.7% of current GDP per capita for Zimbabwe (table 2). Given potential cost savings at scale, this ICER is estimated to reduce to US $6.35 per HLY gained if the intervention were scaled up nationally, equating to around 0.38% of annual GDP per capita for Zimbabwe. The total annual cost of the intervention at scale is estimated at US $422,820 in the base case analysis.

Table 2.

Base case cost-effectiveness analysis

	CPH	At scale
Cost per neonatal admission	14	9
Total incremental cost (US $)	38 512	422 820
Total deaths averted	25	1243
Total HLYs gained	1354	66 597
ICER (US $ per HLY gained)	28.44	6.35
Zimbabwe 2022 GDP per capita ($ US)	1677	1677
Cost per HLY gained as % of GDP per capita	1.70%	0.38%

Note: Base case analysis assumes that the intervention only impacts mortality in the low birth weight (1.5kg-2.5kg) subgroup. Incremental outcomes are therefore based on this subgroup, while incremental costs are based on the total number of admissions across all birth weight categories.

CPH, Chinhoyi Provincial Hospital ; HLY, healthy life year; ICER, incremental cost-effectiveness ratio .

Sensitivity and scenario analyses

The impact of one-way variation in key parameters on the results of the cost-effectiveness estimates at scale is shown in online supplemental materials 2. In all cases, the ICER remains below likely cost-effectiveness thresholds for Zimbabwe, with the exception of using the upper 95% CI of 1.002 for effectiveness, which results in Neotree being dominated by standard care. The impact of assumed effectiveness on results is explored further in online supplemental materials 2 (table 2). For example, conservatively assuming a 5% mortality reduction in the low-birth weight subgroup only results in an ICER of US $81.77 per HLY gained, which is still at the lower end of possible cost-effectiveness thresholds for Zimbabwe, amounting to approximately 5% of GDP per capita.

In the probabilistic sensitivity analysis, incremental HLYs gained were positive in 98.6% of simulations (online supplemental figure 1). Online supplemental figure 2 shows the cost-effectiveness acceptability curve. At a cost-effectiveness threshold of 1% of GDP per capita (US $17), the Neotree intervention has a 91% probability of being cost-effective, while at a threshold of 51% of GDP per capita (US $855), the probability increases to 98.6%.

Budget impact analysis

Online supplemental materials 3 table 1 shows the impact of different assumptions regarding total number of admissions on total cost at scale. In the base case analysis, the cost of implementing the Neotree intervention in public facilities nationwide is estimated to amount to around 0.23% of total annual government health expenditure (an estimated US $183 million in 2020). Uncertainty in this figure is explored via uncertainty in the assumed admission rate, which suggests a plausible range of between 0.08% and 0.32% of annual government health expenditure. Uncertainty in total admissions numbers also influences the estimated number of deaths averted, as highlighted in the final column of the table.

Discussion

This study estimated the potential effectiveness and cost-effectiveness of implementing Neotree in Zimbabwe. The results of the interrupted time series suggest a reduction in overall mortality among neonatal admissions, primarily driven by reduced mortality among the subgroup of neonates weighing 1.5–2.5 kg. Based on effectiveness estimates in this subgroup, Neotree is likely to be highly cost-effective in Zimbabwe, with an estimated cost at scale of US $6.35 per HLY gained, far below likely cost-effectiveness thresholds.²⁵

Economic evidence on directly comparable interventions is limited, all from community or primary healthcare settings and, similar to our study, largely relies on observational data. However, these studies have found that similar interventions can reduce neonatal mortality and have the potential to be cost effective. A study in Ghana evaluated an mHealth intervention combining both demand- (i.e. education messages for mothers) and supply-side components (i.e. digitised clinical care information to better track and deliver care in primary care facilities).²⁸ The intervention was found to reduce neonatal mortality and cost US $174 per DALY averted in the first year, falling to US $6.54 per DALY averted by the tenth year. In Nigeria, an mHealth intervention providing case management and decision support was found to cost US $13 739 per life saved and US $594 per DALY averted.²⁹ In India, another mHealth app used by community health volunteers was found to cost US $205 per DALY averted.³⁰ In Tanzania, a study in six rural health centres found that an electronic clinical decision support system cost US $338 per 1% change in process quality for childbirth care, though health outcomes were not captured.¹⁰ In Ghana, the ICER for a similar intervention when compared with a paper-based system was estimated at US $1142 per pregnancy complication detected.¹³

To our knowledge, only one randomised controlled trial has been conducted for a similar intervention. In India, an mHealth tool designed to assist workers in primary care and the community reduced neonatal mortality and was found to cost US $74 per life-year saved, with the authors noting the potential for cost savings at scale. ¹¹ This compares favourably with our ‘within-study’ estimate of around US $25 per life year gained.

Although we conservatively assume that the intervention only has mortality benefits (i.e. does not impact morbidity), and only for the subgroup of babies weighing 1.5–2.5 kg, there remains substantial uncertainty in the effectiveness estimate used in our base case analysis. Despite this, even non-inferiority is an important finding, given important benefits of electronic data capture as part of wider quality improvement initiatives. In theory, the benefits of data capture could be expressed in monetary form within an economic evaluation, as could other less direct impacts, such as user experience.³¹ However, this is methodologically challenging and only a recently emerging area of research, which may inform future economic evaluations of digital health interventions.

Despite our probabilistic sensitivity analysis demonstrating better outcomes in 98.6% of simulations, this reflects only uncertainty in estimated parameters. This impact evaluation was based on a relatively small sample and was likely underpowered to detect changes in mortality. Additionally, the parameter used for intervention effectiveness is based on data for CPH only, and therefore may lack external validity when applied nationally, as many public facility deliveries likely take place outside of hospitals. This also impacts the budget impact analysis, where there also is a lack of available data to inform the number of admissions. However, it is worth noting that costs would likely also reduce in a national scale-up including different facility types, given lower staff costs in primary healthcare settings.

Additionally, in a single group interrupted time series analysis, it cannot be ruled out that the observed effects were caused by other events occurring at a similar time to the intervention. Finally, due to limited data availability, our costing was from the provider perspective, which is inevitably less comprehensive than a societal perspective. It is unlikely that the choice of perspective would change our conclusions. However, it is worth noting that bottlenecks such as frequent medicine stockouts and shortages, and inappropriate equipment, are prevalent in the facilities where Neotree has been implemented. Ideally, future budget impact and cost-effectiveness analyses should incorporate the costs and potential impact of addressing these bottlenecks, as well as other essential infrastructure improvements necessary for ensuring quality maternal and child healthcare. Addressing these issues may also improve the effectiveness of the Neotree intervention. We mitigate against these issues to some extent by exploring a range of possible effectiveness estimates in scenario analyses.

Neotree is a potentially low-cost and highly cost-effective digital quality improvement tool to improve newborn care, morbidity and survival. This study contributes to limited economic evidence of mHealth clinical decision support tools in low- and middle-income settings and provides evidence to support policymakers in Zimbabwe. Further evidence is needed on the effectiveness of Neotree to help build on this study. A large-scale evaluation of the effectiveness and cost-effectiveness of Neotree in hospitals and primary healthcare facilities in Zimbabwe and Malawi is planned, subject to funding availability.

Data availability statement

Data are available on reasonable request. Data collected for the study cannot yet be made publicly available because secondary analyses are ongoing. We are aiming to establish an open-source anonymised research database of Neotree data to maximise the reach and utility for researchers aiming to improve outcomes for neonates in low-income settings. This database is under development and subject to negotiation with relevant Ministries of Health.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and was approved by Ethical approvals were obtained from University College London (17123/001, 6681/001, 5019/004), the Medical Research Council of Zimbabwe (MRCZ/A/2570), the Biomedical Research and Training Institute and Joint Research Ethics Committee for the University of Zimbabwe institutional review boards (AP155/2020; JREC/327/19) and the Sally Mugabe Hospital Ethics Committee (071119/64; 250418/48). Participants gave informed consent to participate in the study before taking part.