Skip to main content
PMC home page
PLOS Global Public Health logoLink to PLOS Global Public Health
. 2022 Jul 28;2(7):e0000757. doi: 10.1371/journal.pgph.0000757

Cost-effectiveness of pulse oximetry and integrated management of childhood illness for diagnosing severe pneumonia

Solomon H Tesfaye 1,2,3,*, Eskindir Loha 1,2, Kjell Arne Johansson 2, Bernt Lindtjørn 1,2
Editor: Melissa Morgan Medvedev4
PMCID: PMC10021260  PMID: 36962478

Abstract

Pneumonia is a major killer of children younger than five years old. In resource constrained health facilities, the capacity to diagnose severe pneumonia is low. Therefore, it is important to identify technologies that improve the diagnosis of severe pneumonia at the lowest incremental cost. The objective of this study was to conduct a health economic evaluation of standard integrated management of childhood illnesses (IMCI) guideline alone and combined use of standard IMCI guideline and pulse oximetry in diagnosing childhood pneumonia. This is a cluster-randomized controlled trial conducted in health centres in southern Ethiopia. Two methods of diagnosing pneumonia in children younger than five years old at 24 health centres are analysed. In the intervention arm, combined use of the pulse oximetry and standard IMCI guideline was used. In the control arm, the standard IMCI guideline alone was used. The primary outcome was cases of diagnosed severe pneumonia. Provider and patient costs were collected. A probabilistic decision tree was used in analysis of primary trial data to get incremental cost per case of diagnosed severe pneumonia. The proportion of children diagnosed with severe pneumonia was 148/928 (16.0%) in the intervention arm and 34/876 (4.0%) in the control arm. The average cost per diagnosed severe pneumonia case was USD 25.74 for combined use of pulse oximetry and standard IMCI guideline and USD 17.98 for standard IMCI guideline alone. The incremental cost of combined use of IMCI and pulse oximetry was USD 29 per extra diagnosed severe pneumonia case compared to standard IMCI guideline alone. Adding pulse oximetry to the diagnostic toolkit in the standard IMCI guideline could detect and treat one more child with severe pneumonia for an additional investment of USD 29. Better diagnostic tools for lower respiratory infections are important in resource-constrained settings, especially now during the COVID-19 pandemic.

Introduction

Pneumonia is a leading cause of death among children younger than five years old [1]. Childhood pneumonia is associated with chronic obstructive pulmonary disease, reduced lung function, and chronic bronchitis during adulthood [2]. The World Health Organization’s integrated management of childhood illness (IMCI) guideline has had a huge impact on clinical management of severe pneumonia in children aged 2 months to 59 months in resource constrained settings [3]. However, the diagnostic recommendations in this guideline are not sensitive enough to detect or specific enough to rule out severe pneumonia [4, 5]. Thus, even though the standard IMCI guideline improves the identification and treatment of pneumonia [6], severe cases with high risk of mortality still may be missed. Combining pulse oximetry with the standard IMCI guideline may improve diagnostic precision and thus prevent childhood deaths due to severe pneumonia [7].

Implementation of the standard IMCI guideline requires training staff, provision of essential drugs and supplies, and consistent supervision [8]. These requirements can be challenging in low-income countries [9, 10]. Implementation of the standard IMCI guideline in Ethiopia can be difficult due to a lack of trained staff, a lack of essential drugs and supplies, and inconsistent supervision [9, 10]. For example, Ethiopia officially adopted the IMCI guideline in 1997 [8], but around 30,700 children under five years old continue to die due to pneumonia annually in the country [11]. More evidence on how to improve the effectiveness of standard interventions and on the cost of adding advanced diagnostics tools to existing pneumonia policies is needed in Ethiopia and other low-income countries.

Administering pulse oximetry and oxygen therapy improves hospital management of pneumonia [12, 13]. In Papua New Guinea, improving oxygen system and pulse oximetry has improved the quality of care and reduced mortality from pneumonia by 35% among children age younger than 5 years [14]. Incorporation of pulse oximetry into the usual clinical management of pneumonia among young children in developing countries thus seems promising [15–17]. From our previous cluster-randomized controlled trial, we found that combining pulse oximetry with the standard IMCI guideline improves the capacity of health workers: diagnoses of severe childhood pneumonia increased from 4% using the standard IMCI guideline alone to 16% using the combination of pulse oximetry with the standard IMCI guideline [18].

Several studies have assessed the effect of adding pulse oximetry in settings where the standard IMCI guideline already is used to manage childhood pneumonia [17, 19–21]. However, none of those studies estimated the opportunity cost of adding pulse oximetry to standard IMCI guideline. Policy makers use cost-effectiveness analysis to identify economical and effective purchases, and this analysis is especially important in health care settings [22]. Health economic evaluations based on randomized controlled trials in policy decision making may prevent countries with scarce resources not to waste health investments [23]. This study therefore aimed to compare the cost-effectiveness of combined use of pulse oximetry with standard IMCI guideline compared with the standard IMCI guidelines alone to improve diagnostic precision of severe childhood pneumonia in rural Ethiopia.

Materials and methods

Ethics statement

The study was approved by the institutional review board of the College of Medicine and Health Sciences at Hawassa University (ref: IRB/009//2017) and the Regional Committees for Medical Research Ethics, South East Norway (ref: 2017/2473/REK sør-øst). Children were included in the study after obtaining written informed consent from parents.

Study design and settings

This cost-effectiveness study was conducted alongside a cluster-randomized controlled trial whose main objective was to improve diagnosis of severe childhood pneumonia by adding pulse oximetry to the standard IMCI guideline. A detailed description of the trial methodology and setting is provided elsewhere [18]. The trial is registered with trial registration number: PACTR, PACTR201807164196402 in 14/06/2018 and available at https://pactr.samrc.ac.za/TrialDisplay.aspx?TrialID=3466. Briefly, 24 government health centres were studied. Each health centre was defined as a cluster, and clusters with at least one case of pneumonia per day were included in the study. Children aged two months to 59 months who sought care at a health centre for cough or difficulties breathing lasting fewer than 14 days were eligible for inclusion in the study. Recruitment took place between September 2018 and April 2019.

The study was conducted in Gedeo district located in Southern part of Ethiopia. Out of a total population of more than one million, about 173,000 are children under five years of age. The district has 38 health centres, three of which were recently upgraded to primary hospitals. It has 146 health posts and one teaching and referral hospital. IMCI is implemented in health centres for the management of common childhood illnesses, including pneumonia. Radiology or laboratory diagnostic tools are neither available nor required to diagnose childhood pneumonia in such settings. Health centres are expected to refer severe pneumonia cases to a primary hospital after offering the recommended pre-referral drugs. None of the health centres had pulse oximetry for measuring oxygen saturation. Oxygen therapy for severely ill children is available at one hospital, but the supply is unreliable. In 2018, 106,583 outpatient visits were reported in the 24 health centres selected for the trial, representing 85% of all outpatient visits in the study area. Of those, 22,542 (21%) visits were made by children younger than five, and 6,677 (30%) of those children were diagnosed with pneumonia.

Description of the interventions compared

The detailed description of the interventions and the sample size of the population are provided in the published trial article [18]. Twelve clusters were randomly selected into each of the two arms. Health workers in the intervention arm used the standard IMCI guideline [24], and a paediatric fingertip pulse oximetry (ADC Adimals 2150) to diagnose pneumonia. The health workers measured each child’s oxygen saturation by taking two pulse oximetry measurements at five minutes apart. One-day training was given to health workers on the use of pulse oximetry. Health workers in the control arm used the same IMCI guideline alone to manage children with suspected signs and symptoms of pneumonia. All health workers were trained on the standard IMCI guideline.

Measurement of health effects

The health effect for this trial based cost-effectiveness analysis was based on cluster randomised trial [18]. In the intervention group, the measure of effectiveness was severe pneumonia cases detected using the standard IMCI guideline [24] with or without hypoxemia (oxygen saturation < 90%), as measured by a paediatric fingertip pulse oximetry (ADC® Adimals 2150). In the control group, severe pneumonia was detected using the same IMCI guideline alone. Because of the lack of reference standard diagnostic method in the study settings we didn’t estimate the improved health outcome and the averted cost related to improved diagnosis at the study settings. We use detected severe pneumonia cases using pulse oximetry and IMCI guidelines as health outcome for the cost-effectiveness analysis. Using such intermediate outcome is also recommended by Drummond et al. [23].

Intervention costs measurement

Intervention costs of diagnosing severe pneumonia were assessed from both the provider and patient perspectives [23], using 2018 US dollars. We use the word “provider” to refer to the health systems/institutions. All costs were converted to US dollars using the official National Bank of Ethiopia average exchange rate for 2018 (US dollar 1 = Ethiopian Birr 27.4220). Data were collected prospectively, starting from the beginning of the trial.

We used a structured questionnaire to obtain cost information. The type, quantity, and price of each resource used in the trial were recorded via interviews with caregivers and by assessing facility records. Patient costs included patient’s direct out-of-pocket expenses for consultations, transportation, drugs, supplies (Intravenous fluids and Intravenous cannula 24 gages), and hospital admission. Patients pay out-of-pocket for drugs and supplies from government health facilities, and the government applies a 25% surcharge to these items. Costs for drugs and supplies represent both the health system costs and patient costs. Therefore, we include cost for drugs and supplies on patient side to avoid double counting. For this cost-effectiveness analysis we considered drugs and supplies, hospital stays and transportation costs as patient opportunity cost. Drug costs include cost for single dose of antibiotics given at health centres level before referral to hospital. Cost for drugs and supplies, hospital stays, oxygen therapy and IV fluids were included at hospitals for severe pneumonia cases admitted to hospital. The unit costs for all resources used for diagnosing pneumonia, oxygen treatment, training, and patient expenses are presented in Table 1.

Table 1. Unit cost of items for pneumonia diagnosis and treatment, 2018 USD.

Items Unit costs
Diagnosis Intervention arm Control arm
    Pulse oximetry 149.81 Not applicable
    Batteries 1.01 Not applicable
    Personnel 0.59 0.34
Training 13.96 7.40
Oxygen treatment (per cubic meter) 0.77 0.77
Patient expense
    Drugs 1.01 1.01
    Intravenous fluids 0.88 0.88
    Intravenous cannula 24 gages 0.34 0.34
    Hospital stay 1.18 1.18
    Consultation 0.40 0.40
    Transportation per kilometre 0.03 0.03
Open in a new tab

Provider cost items were divided into capital and recurrent costs. Capital costs were defined as costs for items expected to last longer than a year [23], such as a pulse oximetry. The capital cost of the pulse oximetry was recorded from invoices and annuitized based on an expected product life of two years [25], initial costs, and interest rate of 7% [23]. Recurrent costs were defined as costs for products used regularly that have duration of less than a year. We included costs of personnel (including health workers’ time spent diagnosing childhood pneumonia), oxygen therapy (provided by the government), training of health workers (including materials and estimated based on the per diem used by the district health office), and pulse oximetry alkaline batteries. To estimate the average cost of health workers’ time spent on diagnosis of pneumonia, a monetary value was assigned by allocating a corresponding percentage of the health worker’s salary and duty fees (this is payment for extra working hours). The salary per year in the study area for mid-level health worker is USD 1, 474, and a duty fee is USD 2,580. All the mid-level health workers in the study area are Bachelor of Science degree holder with four years medical education background and their salary is the same.

The cost of drugs, intravenous fluids, and intravenous cannula 24 gages used were quantified and multiplied by their respective unit costs to estimate the total cost of each items in both arms. The total cost of hospital stays was also estimated by multiplying the cost per day by the duration of hospital stays. Similarly, we estimated the total cost of oxygen therapy by multiplying the amount of oxygen consumed in cubic meter by the unit cost. The total distance travelled by the patient from home to health facilities was multiplied by the unit cost per kilometre to estimate the total transportation costs. All costs for each items was added up to get total costs for each arms. Total and average costs per diagnosed severe and non-severe pneumonia cases were estimated. To estimate the average cost per diagnosed severe pneumonia case, the total cost was divided by the number of children diagnosed with severe pneumonia. Likewise the total cost was divided by the number of children diagnosed for non-severe pneumonia.

Cost-effectiveness model

A decision tree model built using TreeAge Pro Suit 2021 (© 2021 TreeAge software, Inc.) was used for the analysis [26]. The model follows a sequence of steps to construct a tree structure under uncertainty for alternative interventions and select the least expected cost per benefit as the best alternative. A cost-effectiveness ratio was estimated for each of diagnostic methods per diagnosed severe pneumonia case. The cost-effectiveness model is presented in Fig 1. The model compares the opportunity cost and the proportion of severe pneumonia cases detected using combined use of pulse oximetry and the standard IMCI guideline versus the standard IMCI guideline in an Ethiopian rural setting. Probabilities in the decision tree represent possible events in the process of diagnosis after a child presents with symptoms of lower respiratory infection. The pathways are mutually exclusive. We conducted the study in rural part of Ethiopia where non-clinicians are responsible for child health care. Once a child is identified with severe pneumonia the health workers should refer the child to hospital where clinicians are responsible for child care. As a result we didn’t include the final health outcome at hospital level in the cost-effectiveness model. Therefore, in this study we didn’t analyse improved health outcomes and averted costs to the health system and patient, averted medical costs, averted health system expenditures from improved diagnosis at health centre levels. To use mortality averted or life years gained due to interventions as final end point is not feasible, due to the fact that we conducted the study for short period of time (8 moths). During this period there were only four deaths.

Fig 1. Model structure of the cost-effectiveness study.

Fig 1

Open in a new tab

Table 2 shows the input parameters, and specifies sources of those data. To estimate the proportion of diagnosed severe pneumonia cases for the two diagnostic modalities, we divided the number of severe pneumonia cases identified in the study period by the total number of children that attended the health centers. We make an assumption about the ability of both diagnostic modalities to accurately classify a child as severe pneumonia (sensitivity) and non-severe pneumonia (specificity). We used the sensitivity and specificity for both diagnostic modalities from literatures. Although data were available on the sensitivity and specificity of the standard IMCI guideline, sufficient data were not available to inform the specificity of standard IMCI guideline when combined with pulse oximetry. Therefore, we assumed the specificity of pulse oximetry combined with standard IMCI guideline to be similar with specificity of the standard IMCI guideline.

Table 2. Input parameters for cost-effectiveness model.

Input parameters Base value Minimum value Maximum value SD Distribution Data sources
Cost of diagnosed severe pneumonia with pulse oximetry and IMCI combined 25.74 14.87 34.51 3.62 Gamma Trial data [18]
Cost of diagnosed non severe pneumonia with pulse oximetry and IMCI combined 3.58 2.84 4.36 0.34 Gamma Trial data [18]
Cost of diagnosed severe pneumonia with IMCI 17.98 14.23 24.55 2.97 Gamma Trial data [18]
Cost of diagnosed non-severe pneumonia with IMCI 2.14 1.46 2.97 0.33 Gamma Trial data [18]
Proportion of severe pneumonia with pulse oximetry and IMCI combined 0.16 0.05 0.27 0.03 Beta Trial data [18]
Proportion of severe pneumonia with IMCI 0.04 0.01 0.07 0.01 Beta Trial data [18]
Sensitivity of pulse oximetry and IMCI combination 0.85 0.72 0.98 0.07 Beta [27]
Specificity of pulse oximetry and IMCI combination 0.87 0.73 1.00 0.07 Beta Assumed to be similar to IMCI
Sensitivity of IMCI 0.56 0.39 0.73 0.09 Beta [28]
Specificity of IMCI 0.87 0.73 1.00 0.07 Beta [29]
Open in a new tab

Cost-effectiveness analysis

Incremental cost-effectiveness ratio (ICER) was used to summarize and present the cost-effectiveness result computed for the number of severe pneumonia cases identified as a result of the intervention.

Uncertainty and sensitivity analysis

Two types of sensitivity analyses were performed to deal with uncertainties. First, a one-way sensitivity analysis was done using a tornado diagram for maximum and minimum values of costs, proportion of diagnosed severe pneumonia cases, and sensitivity and specificity of the interventions from the base case. The minimum and maximum values of the 95% confidence interval were used for both outcome and cost (Table 2). Second, probabilistic sensitivity analysis (PSA) was conducted to distribute the parameters used for one-way sensitivity analysis. PSA was performed with Monte Carlo simulations with 10,000 iterations. We assumed cost parameters to follow gamma distributions and proportions of outcome and sensitivity and specificity of interventions to follow beta distributions [30]. In the analysis, we replaced the variables in the model with distributions. The results are presented as cost-effectiveness acceptability curves and scatter plots. The time horizon for cost-evaluation was 8 months, as this is the data collection period and average cost was calculated by dividing total cost by all diagnosed cases over the study period. We used discount rate of 0% both for cost and outcome as recommended for economic evaluation with short time horizon (< 1 year) [31].

Results

Baseline characteristics

The flow chart and baseline characteristics of the study participants are presented elsewhere [18]. We included 1,804 participants (928 in the intervention group and 876 in the control arm) from 24 health centres.

Effectiveness

The proportion of children diagnosed with severe pneumonia was 148/928 (16.0%, 95% CI 4.7–27.2) in the intervention arm and 34/876 (4.0%, 95% CI 1.2–6.6) in the control arm.

Cost of interventions

The total cost was USD 3,809.24 for severe pneumonia and USD 2,794.45 for non-severe pneumonia in the intervention arm. The total cost was USD 611.30 for severe pneumonia and USD 1,800.17 for non-severe pneumonia in the control arm. The cost per diagnosed severe pneumonia is USD 25.74 for intervention and USD 17.98 for control arms (Table 3). Of the total costs, the cost for drugs and supplies, transportation, and hospital stay account for most of the costs in both the intervention and control arms (Table 4).

Table 3. Total and average cost for diagnosed pneumonia cases, 2018 USD.

Diagnostic alternatives Severe pneumonia non-severe pneumonia
Number diagnosed Total cost (USD) Average cost (USD) Number diagnosed Total cost (USD) Average cost (USD)
Standard IMCI alone 34 611.30 17.98 842 1800.17 2.14
Standard IMCI and pulse oximetry combined 148 3809.24 25.74 780 2794.45 3.58
Open in a new tab

Table 4. Itemized cost to diagnose severe pneumonia, 2018 USD.

Costs Pulse oximetry with integrated management of childhood illnesses
(% share)		 Integrated management of childhood illnesses alone (% share)
Provider costs  
Pulse oximetry 191 (5) Not applicable
Batteries 1 (0.01) Not applicable
Training 27 (1) 3 (1)
Personnel 135 (4) 15 (2)
Oxygen therapy 743 (20) 171 (28)
Patient costs
Consultation 59 (2) 14 (2)
Drugs and supplies 1221 (32) 181 (30)
Transportation 559 (15) 139 (23)
Hospital stay 873 (23) 89 (14)
Total cost 3809 (100) 611 (100)
Open in a new tab

Cost-effectiveness

The ICER from base case analysis was USD 29 for standard IMCI guideline and pulse oximetry combined for diagnosing one additional severe pneumonia case as compared to standard IMCI guideline alone.

Sensitivity analysis

One-way sensitivity analysis with minimum and maximum values of the selected variables is presented in Fig 2. The tornado diagram indicates that the cost of diagnosing severe pneumonia using standard IMCI guideline alone and sensitivity of combined use of standard IMCI guideline and pulse oximetry had the highest impact on the incremental cost-effectiveness ratio. The ICER ranged from USD 26.24 to USD 140.35 when the cost of diagnosing severe pneumonia using standard IMCI guideline alone varied from USD 14.23 to USD 24.55. The ICER was less sensitive to change in most of the other variables.

Fig 2. Tornado diagram-sensitivity of ICER variations.

Fig 2

Open in a new tab

Figs 3 and 4 show the probabilistic sensitivity analysis results using the cost-effectiveness analysis scatterplot and cost-effectiveness acceptability curve. Fig 3 indicates that there was less variability both in cost and effectiveness of the standard IMCI guideline alone, but variability in standard IMCI and pulse oximetry was high.

Fig 3. Scatterplot of the costs and health effects of interventions from the Monte Carlo simulation.

Fig 3

Open in a new tab

Fig 4. Cost-effectiveness acceptability curve.

Fig 4

Open in a new tab

In a probabilistic sensitivity analysis, the ICER ranges from USD 7.16 to USD 89.29 per diagnosed severe pneumonia case. The acceptability curve in Fig 4 plots the proportion of iterations in which each alternative had the greater ICER for different Willingness-to-pay thresholds. The probability of combined use of standard IMCI guideline with pulse oximetry being cost-effective was 33% at a willingness-to-pay threshold of USD 26 per diagnosed severe pneumonia case. While at willingness-to-pay threshold of USD 39 per severe pneumonia case diagnosed, the rank order swops and the probability of combined use of standard IMCI with pulse oximetry appears with the highest probability of being cost-effective.

Discussion

We found that pulse oximetry in combination with the standard IMCI guideline approach increased the detection of severe pneumonia in children by 12%. The ICER was 29 USD per severe case of pneumonia detected, compared with using the standard IMCI guideline alone. The results show that one more child with severe pneumonia could be detected and treated for an additional investment of USD 29. The combination of the standard IMCI guideline with pulse oximetry is more likely to be cost-effective, compared to IMCI guideline alone, at willingness-to-pay threshold of USD 39 per severe pneumonia case diagnosed. These findings should be useful for policy makers in defining the benefit package for management of lower respiratory infections in resource constrained settings.

A direct comparison of our findings with other cost-effectiveness analyses is difficult due to differences in study design, costing perspectives, and effectiveness of measurement. Nevertheless, some findings from other studies are comparable to ours. For example, providing oxygen system and pulse oximetry in Papua New Guinea, found that the ICER was USD 51 per treated severe pneumonia case [14]. A modelling study of 15 countries with high pneumonia mortality rates showed that in Ethiopia, combined use of pulse oximetry and the standard IMCI had an ICER of USD 6 per disability-adjusted life-years averted [32]. However, this modelling study collected cost data only from the provider perspective and excluded costs related to oxygen therapy, parenteral antibiotics, and hospital stays.

Data from 74 countries, including Ethiopia, were used to model the cost-effectiveness of the IMCI guideline [33]. In their study, the IMCI guideline has a median cost-effectiveness ratio of USD 26.6 (interquartile range: 17.7–45.9) per disability-adjusted life-years averted. Another study from Zambia revealed that the cost per outpatient visit for pneumonia management using the IMCI guideline is USD 48 per out-patient visit [34]. We found that the average cost for diagnosis of pneumonia using the IMCI guideline alone was lower than that found in both of these studies. This difference could be due to the inclusion of health facility building maintenance cost included in these two studies.

Our trial revealed that direct out-of-pocket payments for drugs, supplies, transportation, and hospital admission in both the intervention and control arms are the main cost components. Private expenditures pose a substantial financial risk to households and could be an important barrier for seeking health care [35, 36]. A study from Ethiopia showed that among total household out-of-pocket expenditures, costs of medication, hospital stays, and diagnostic investigations are the most important private health expenditures [36]. Moreover, 7% of households in Ethiopia with severe pneumonia fall below the extreme poverty line due to out-of-pocket payments to health care [36].

Universal health coverage is a sustainable development goal to be achieved by 2030 [37]. To achieve this goal, countries must develop health financing systems so that people can access services without incurring financial hardship [38]. Accordingly, the Ethiopian Federal Ministry of Health has endorsed a health care financing strategy [39]. A fee-wavier system for the poor is one of the reforms included in this strategy. However, as shown in our trial and others [36], the largest out-of-pocket payments are for drugs, supplies, transportation, and hospital stays. Ethiopia therefore may be far from achieving universal health coverage.

We found that the main drivers of increased costs in the intervention arm were the capital cost of the pulse oximetry unit and the recurrent costs after diagnosis of pneumonia, such as medication, transportation, and hospital stays. However, the societal gain in terms of health benefit from using pulse oximetry is huge. Comparing this health benefit from using pulse oximetry against the consequences of false positive severe pneumonia identified using pulse oximetry should be the focus of future research area.

The ICER from the probabilistic sensitivity analysis doesn’t change the conclusion from the deterministic analysis result, which implies low uncertainty.

The strength of this study is that data were collected prospectively in a cluster-randomized controlled trial. Resource identification and costing information were compiled based on routine care for childhood pneumonia at health centres in Ethiopia. The information was collected from typical rural health facilities in Ethiopia. Cost data were collected prospectively to reduce recall bias. Randomization was performed at the cluster level to avoid contamination. Broad-range cost categories were included, and the analysis accounted for the hierarchical nature of the data.

This cost-effectiveness analysis is not without limitations. First, we did not use a final endpoint, such as life years gained, as a health outcome. Instead, we used severe pneumonia diagnosis, which is an intermediate outcome. In economic evaluations of clinical trials, using intermediate outcome is misleading unless there is an established link between intermediate and final outcomes [23]. However, it is well-established that severe pneumonia is a leading cause of death in children under five years old [15, 40]. Additionally, it is reasonable to use diagnosed severe pneumonia as a health outcome if there is a clinical and cost-effective therapy to treat detected cases [23]. Second, our estimation did not include caregivers’ lost productivity due to time spent seeking care and caring for a sick child. This omission might have led to underestimating the cost-effectiveness ratio. Third, the capital building costs was not included in the cost estimates. However, we expect that improved diagnostics with pulse oximetry may not have a huge impact on capital facility costs compared to the standard intervention assessed in the trial. Therefore it is unlikely, the exclusion of building cost to have substantial impact on ICER.

The study area is typical of the rural population of Ethiopia, where outpatient visits due to pneumonia are high and pulse oximetry is not available. Ethiopia’s Federal Ministry of Health plans to ensure regular availability and functionality of oxygen therapy and pulse oximetry [41], although this has not yet been achieved. Therefore, we believe that our findings can be applied in rural health centres where diagnostic capacity is low. They also can help the on-going efforts by the Federal Ministry of Health of Ethiopia to improve health care access in rural areas.

Conclusions

Based on our trial findings, supplementing pulse oximetry with the standard IMCI guideline resulted in higher detection of severe childhood pneumonia than using the standard IMCI guideline alone. Therefore, using a combination of pulse oximetry and the standard IMCI guideline has both economic and public health importance.

Acknowledgments

We sincerely acknowledge the contributions of the Gedeo Zone Health Department and district health offices in helping to successfully launch the implementation of this study. We are grateful for the health workers and health facilities where the study was conducted. We also sincerely thank the study participants.

Data Availability

All relevant data are within the paper.

Funding Statement

The authors received no specific funding for this work.

References

  • 1.World Health Organization. Levels and Trends in Child Mortality 2014. Available http://www.who.int/maternal_child_adolescent/documents/levels_trends_child_mortality_2014/en/ Accessed 06 December 2016. [Google Scholar]
  • 2.Hayden LP, Hobbs BD, Cohen RT, Wise RA, Checkley W, Crapo JD, et al. Childhood pneumonia increases risk for chronic obstructive pulmonary disease: the COPDGene study. Respiratory research 2015;16:115. doi: 10.1186/s12931-015-0273-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Gove S. for WHO Working Group on Guidelines for Integrated Management of the Sick Child. Integrated management of childhood illness by outpatient health workers: technical basis and overview. Bulletin of the World Health Organization 1997;75(Suppl 1):7–24. [PMC free article] [PubMed] [Google Scholar]
  • 4.Duke T, Blaschke AJ, Sialis S, Bonkowsky JL. Hypoxaemia in acute respiratory and non-respiratory illnesses in neonates and children in a developing country. Archives of disease in childhood 2002;86(2):108–12. doi: 10.1136/adc.86.2.108 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Gerensea H, Kebede A, Baraki Z, Berihu H, Zeru T, Birhane E, et al. Consistency of Integrated Management of Newborn and Childhood Illness (IMNCI) in Shire Governmental Health Institution in 2017. BMC research notes 2018;11(1):476. doi: 10.1186/s13104-018-3588-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Patwari AK, Raina N. Integrated Management of Childhood Illness (IMCI): a robust strategy. Indian journal of pediatrics 2002;69(1):41–8. doi: 10.1007/BF02723776 [DOI] [PubMed] [Google Scholar]
  • 7.UNICEF (2016): One is too many: ending child deaths from pneumonia and diarrhoea. http://data.unicef.org/wp-content/uploads/2016/11/UNICEF-Pneumonia-Diarrhoea-report2016-web-version.pdf. Accessed 06 March 2019. [Google Scholar]
  • 8.Lulseged S. Integrated management of childhood illness: a review of the Ethiopian experience and prospects for child health. Ethiopian medical journal 2002;40(2):187–201. [PubMed] [Google Scholar]
  • 9.Seid SS, Sendo EG. A survey on Integrated Management of Neonatal and Childhood Illness implementation by nurses in four districts of West Arsi zone of Ethiopia. Pediatric health, medicine and therapeutics 2018;9:1–7. doi: 10.2147/PHMT.S144098 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Hailemariam S, Gebeyehu Y, Loha E, Johansson KA, Lindtjorn B. Inadequate management of pneumonia among children in South Ethiopia: findings from descriptive study. BMC health services research 2019;19(1):426. doi: 10.1186/s12913-019-4242-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Global Health Estimates 2016: Deaths by Cause, Age, Sex, by Country and by Region, 2000–2016. Geneva, World Health Organization; 2018. [Google Scholar]
  • 12.Wandi F, Peel D, Duke T. Hypoxaemia among children in rural hospitals in Papua New Guinea: epidemiology and resource availability—a study to support a national oxygen programme. Annals of tropical paediatrics 2006;26(4):277–84. doi: 10.1179/146532806X152791 [DOI] [PubMed] [Google Scholar]
  • 13.Djelantik IG, Gessner BD, Sutanto A, Steinhoff M, Linehan M, Moulton LH, et al. Case fatality proportions and predictive factors for mortality among children hospitalized with severe pneumonia in a rural developing country setting. Journal of tropical pediatrics 2003;49(6):327–32. doi: 10.1093/tropej/49.6.327 [DOI] [PubMed] [Google Scholar]
  • 14.Duke T, Wandi F, Jonathan M, Matai S, Kaupa M, Saavu M, et al. Improved oxygen systems for childhood pneumonia: a multihospital effectiveness study in Papua New Guinea. Lancet (London, England) 2008;372(9646):1328–33. doi: 10.1016/S0140-6736(08)61164-2 [DOI] [PubMed] [Google Scholar]
  • 15.Onyango FE, Steinhoff MC, Wafula EM, Wariua S, Musia J, Kitonyi J. Hypoxaemia in young Kenyan children with acute lower respiratory infection. Bmj 1993;306(6878):612–5. doi: 10.1136/bmj.306.6878.612 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Blacklock C, Mayon-White R, Coad N, Thompson M. Which symptoms and clinical features correctly identify serious respiratory infection in children attending a paediatric assessment unit? Archives of disease in childhood 2011;96(8):708–14. doi: 10.1136/adc.2010.206243 [DOI] [PubMed] [Google Scholar]
  • 17.Emdin CA, Mir F, Sultana S, Kazi AM, Zaidi AKM, Dimitris MC, et al. Utility and feasibility of integrating pulse oximetry into the routine assessment of young infants at primary care clinics in Karachi, Pakistan: a cross-sectional study. BMC pediatrics 2015;15(1):141. doi: 10.1186/s12887-015-0463-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Tesfaye SH, Gebeyehu Y, Loha E, Johansson KA, Lindtjørn B. Pulse oximeter with integrated management of childhood illness for diagnosis of severe childhood pneumonia at rural health institutions in Southern Ethiopia: results from a cluster-randomised controlled trial. BMJ open 2020;10(6):e036814. doi: 10.1136/bmjopen-2020-036814 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Alwadhi V, Dewan P, Malhotra RK, Shah D, Gupta P. Tachypnea and Other Danger Signs vs Pulse Oximetry for Prediction of Hypoxia in Severe Pneumonia/Very Severe Disease. Indian pediatrics 2017;54(9):729–34. doi: 10.1007/s13312-017-1163-6 [DOI] [PubMed] [Google Scholar]
  • 20.Fu LY, Ruthazer R, Wilson I, Patel A, Fox LM, Tuan TA, et al. Brief hospitalization and pulse oximetry for predicting amoxicillin treatment failure in children with severe pneumonia. Pediatrics 2006;118(6):e1822–30. doi: 10.1542/peds.2005-2673 [DOI] [PubMed] [Google Scholar]
  • 21.McCollum ED, King C, Deula R, Zadutsa B, Mankhambo L, Nambiar B, et al. Pulse oximetry for children with pneumonia treated as outpatients in rural Malawi. Bulletin of the World Health Organization 2016;94(12):893. doi: 10.2471/BLT.16.173401 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Johansson KA, Tolla MT, Memirie ST, Miljeteig I, Habtemariam MK, Woldemariam AT, et al. Country contextualisation of cost-effectiveness studies: lessons from Ethiopia. BMJ global health 2019;4(6):e001320. doi: 10.1136/bmjgh-2018-001320 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, GW T. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. 198 Madison Avenue (NY): Oxford,:UK: Oxford University Press; 2015. 379 p. [Google Scholar]
  • 24.World Health Organization (2014). Integrated Management of Childhood Illness, chart booklet. Available http://apps.who.int/iris/bitstream/10665/104772/16/9789241506823_Chartbook_eng.pdf. Accessed 02 ‎February ‎2017. [Google Scholar]
  • 25.UNICEF. Supply Catalogue https://supply.unicef.org/ (UNICEF, 2015).
  • 26.TreeAge Pro 2021, R1. TreeAge Software, Williamstown, MA; software available at http://www.treeage.com
  • 27.Madico G, Gilman RH, Jabra A, Rojas L, Hernandez H, Fukuda J, et al. The role of pulse oximetry. Its use as an indicator of severe respiratory disease in Peruvian children living at sea level. Respiratory Group in Peru. Archives of pediatrics & adolescent medicine 1995;149(11):1259–63. doi: 10.1001/archpedi.1995.02170240077012 [DOI] [PubMed] [Google Scholar]
  • 28.Kelly JM, Osamba B, Garg RM, Hamel MJ, Lewis JJ, Rowe SY, et al. Community health worker performance in the management of multiple childhood illnesses: Siaya District, Kenya, 1997–2001. Am J Public Health 2001;91(10):1617–24. doi: 10.2105/ajph.91.10.1617 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Simoes EA, Desta T, Tessema T, Gerbresellassie T, Dagnew M, Gove S. Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia. Bull World Health Organ 1997;75 Suppl 1:43–53. [PMC free article] [PubMed] [Google Scholar]
  • 30.Gray A, Clarke PM, Wolstenholme JL, Wordsworth S. Applied Methods of Cost-effectiveness Analysis in Healthcare: Oxford University Press; 2010. [Google Scholar]
  • 31.Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS). Value in health: the journal of the International Society for Pharmacoeconomics and Outcomes Research 2013;16(2):231–50. doi: 10.1016/j.jval.2013.02.002 [DOI] [PubMed] [Google Scholar]
  • 32.Jessica F, Lindsey W, Deborah Hay B, Rasa I, David M, Azra CG. Evaluating the impact of pulse oximetry on childhood pneumonia mortality in resource-poor settings. Nature 2015;528(7580):S53. doi: 10.1038/nature16043 [DOI] [PubMed] [Google Scholar]
  • 33.Zhang S, Incardona B, Qazi SA, Stenberg K, Campbell H, Nair H. Cost-effectiveness analysis of revised WHO guidelines for management of childhood pneumonia in 74 Countdown countries. Journal of global health 2017;7(1):010409. doi: 10.7189/jogh.07.010409 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Chola L, Robberstad B. Estimating average inpatient and outpatient costs and childhood pneumonia and diarrhoea treatment costs in an urban health centre in Zambia. Cost Effectiveness and Resource Allocation 2009;7(1):16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.World Health Organization (2005). Distribution of health payments and catastrophic expenditures methodology. Available http://www.who.int/health_financing/documents/dp_e_05_2-distribution_of_health_payments.pdf. Accessed 17 May 2020.
  • 36.Memirie ST, Metaferia ZS, Norheim OF, Levin CE, Verguet S, Johansson KA. Household expenditures on pneumonia and diarrhoea treatment in Ethiopia: a facility-based study. BMJ global health 2017;2(1):e000166. doi: 10.1136/bmjgh-2016-000166 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Sustainable development; Open Working Group Proposal for Sustainable Development Goals. Available http://sustainabledevelopment.un.org/content/documents/1579SDGs%20Proposal.pdf. Accessed 17 May 2020.
  • 38.World Health Organization (2010). Health systems financing, the path to universal coverage; The World Health Report. Available http://whqlibdoc.who.int/whr/2010/9789241564021_eng.pdf. Accessed 17 May 2020. doi: 10.2471/BLT.10.078741 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Federal Ministry of Health of Ethiopia. Implementation manual for health care financing reforms. Addis Ababa, Ethiopia: Federal Ministry of Health of Ethiopia, 2005. Available https://www.hfgproject.org/wp-content/uploads/2015/02/Ethiopia_Health_Care_Reform_Brief1.pdf. Accessed 17 May 2020. [Google Scholar]
  • 40.McAllister DA, Liu L, Shi T, Chu Y, Reed C, Burrows J, et al. Global, regional, and national estimates of pneumonia morbidity and mortality in children younger than 5 years between 2000 and 2015: a systematic analysis. The Lancet Global health 2019;7(1):e47–e57. doi: 10.1016/S2214-109X(18)30408-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Federal Ministry of Health, Ethiopia. Medical oxygen and pulse oximetry scale up road map in Ethiopia (2016–2021). Addid Ababa: Federal Ministry of Health, 2016. [Google Scholar]
PLOS Glob Public Health. doi: 10.1371/journal.pgph.0000757.r001

Decision Letter 0

Melissa Morgan Medvedev

29 Mar 2022

PGPH-D-22-00195

Cost-effectiveness of Pulse oximetry and integrated management of childhood illness for diagnosing severe Pneumonia

PLOS Global Public Health

Dear Dr. Tesfaye,

Thank you for submitting your manuscript to PLOS Global Public Health. After careful consideration, we feel that it has merit but does not fully meet PLOS Global Public Health’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================

We invite you to submit a revised version of the above manuscript, taking into consideration the comments received from the two reviewers. In particular, please address the second reviewer's detailed comments regarding the methods and consider adding a table to show how the input costs were derived. In addition, please be sure to address the first reviewer's concern about facility space, i.e., by either incorporating this cost category in the cost estimates or clearly explaining the rationale for and likely impact of omitting this cost category. Finally, please carefully check grammar and ensure all issues have been corrected. For example, common nouns (e.g., pulse oximetry, pneumonia) should not be capitalized unless they come at the beginning of a sentence.

==============================

Please submit your revised manuscript by May 13 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at globalpubhealth@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pgph/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Melissa Morgan Medvedev, M.D., Ph.D.

Academic Editor

PLOS Global Public Health

Journal Requirements:

1. Please ensure you have included the registration number for the clinical trial referenced in the manuscript.

2. Please amend your Financial Disclosure statement. If you did not receive any funding for this study, please simply state: “The authors received no specific funding for this work.”

3. Please provide separate figure files in .tif or .eps format only and ensure that all files are under our size limit of 20MB.

For more information about how to convert your figure files please see our guidelines: https://journals.plos.org/globalpublichealth/s/figures

Additional Editor Comments (if provided):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors provide a very well written economic evaluation of a randomized controlled trial examining pulse oximetry and integrated managemetn of childhood illness for diagnosing severe pneumonia. While there are a few places the manuscript could be strengthened, this is as close as I’ve come in recent memory to suggesting a paper for publication without any revisions. I have only minor suggestions to improve the manuscript that should be addressed before acceptance. They are as follows:

- Could use some copy editing to tighten up the grammar and English in a few places, but overall very clearly and thoughtfully written manuscript.

- A clarification is necessary on the final row of Table 1. Is transportation per kilometer correct, if applied as a unit cost? Does this assume that the average unit cost of transport per patient is only for 1 km?

- Page 7 line 141. The authors mention an interest rate on the pulse oximeter. Do they mean a discount rate? If so, should provide justification for the interest rate of 7% when 3% is the industry standard. Otherwise, please explain the need for an interest rate… this is unclear.

- Page 7 line 149. Are all relevant health workers for this kind of intervention mid-level health workers? Is there a possible upper range of salary reimbursement for some medical professionals that is perhaps not captured in the analysis? Would be good to clarify in the text.

- Figures need to be properly labeled. Not all figures have a full description in the text (see Fig2, especially -not all readers will be familiar with a Tornado diagram).

- Lines 329 - 333 (p16-7) It does seem to be a problem that the facility space itself wasn’t formally added to the cost estimate (even if the facilities already exist, some rental costs, utilities, or other amortized capital costs for the space should be provided to the overall estimate - its not correct to say that just because a new building doesn’t need to be built that the use of the space doesn’t incurr cost). In this case, omitting a cost category of this nature would likely lead to an overestimate of the cost effectivness of either program, but have little to no change in the ICER. That said, this seems like a minimal concern because the facilities already exist, and the program is imbedded in large multiuse facilities. For this reason, space is unlikely to be even a moderate cost driver. Tacit recognition of the absence of these costs is sufficient, but the justification and likely impact on the cost effectivness estimates needs to be reworded.

A few additional unnecessary comments to address, but ones you may find worthwhile to consider:

- I understand the conversation around not including ‘final’ health measures. There could be a back-of-the-envelope calcualtion you could give to estimate the number of cases of severe pneumonia that lead to child mortality, though this is unnecessary. In either case, I don’t think this is enough to overtake the underestimate in cost effectivnesss you face by having excluded opportunity costs of care from the patient perspective however, so I dont see this as a substantial limitation of the paper.

- Given the paucity of good primary cost data on these kinds of studies, unless you are planning a separate costing exercise, you should consider a table breakdown of the average cost at each of the facilities in the study, and provide some characteristics of these facilities to help explain how costs differ by study setting. This would be a major value add to the field and would only strengthen the overall publication.

- Providing any additional detail of the sample of participants out of pocket costs would also be helpful in framing the relevant population under study. This data can probably be cross referenced from the RCT, but a recitation of those characteristics here would add value.

Reviewer #2: This paper has the potential to make an important contribution to the literature. The authors do define a specific objective to compare the cost-effectiveness of combined use of pulse oximetry with standard IMCI guideline compared with the standard IMCI guidelines alone to improve diagnostic precision of severe childhood pneumonia in rural Ethiopia. In the methods section, they narrow the scope a bit by focusing on opportunity costs of combined use of Pulse oximetry and the standard IMCI guideline versus the standard IMCI guideline in an Ethiopian rural setting. This paper would benefit from a careful explanation of the methods using best practice and guidance from the literature on CEA of diagnostic approaches. As written, I can’t really assess whether what the authors have done is a cost-consequence analysis or truly a cost-effectiveness analysis. It may be useful to be more explicit and systematic about the methods throughout the paper. For example, justify your choice of effectiveness, excluding all benefits of improved dx in intro, be clear in methods why some treatment costs included, but averted treatment costs not included, and add any caveats to this approach in discussion). It may be the authors want to focus on cost-effectiveness of an approach to detect pneumonia, rather than cost effectiveness of diagnosis, which has implications for the analysis. For instance, CEA of diagnostic tests/approaches are stronger when looking at cases ‘accurately’ diagnosed. This information is not available. In addition, the authors do not explicitly look at a health outcome per se, which is also a hallmark of cost effectiveness analyses, if not looking at diagnostic accuracy. I understand the authors saw this as a caveat, but it is worth including these issues in the methods, rather than a caveat.

Introduction

1. Line 55-57: Feasibility of adding a new technology that requires training in a situation where even a more basic established protocol (IMCI guidelines) can’t be scaled….Pulse oximeter is a medical supply, requires training, maintenance…Implementation of standard IMCI is difficult due to lack of trained staff, lack of drugs and supplies and inconsistent supervision. Can the authors explain how adding additional diagnostic tools and supplies (for oxygen therapy) at added cost addresses/overcomes the health system constraints described and the consequent IMCI ‘know-do’ gap? The justification for this could be stronger in the intro (and the authors may want to revisit this in the discussion).

Methods

1. Line 115, Page 5- the health effect is the number of severe pneumonia cases detected—do the authors have any information cases were correctly detected? (from looking at the RCT results, I believe not). Or, if the cases detected were treated with pre-referral drugs, or referred to a primary hospital for treatment? From table 1, it appears patients in both settings did receive oxygen treatment, drugs and IV fluids. Would it be possible to look at improvements in dx and treatment at the health center level?

2. Intervention costs from provider and patient perspective—

a. The authors capture treatment costs; however, they do this by capturing the patient out-of-pocket expenses, which include treatment costs (drugs and supplies). Do the authors consider this the full opportunity costs to the patient? Or do these costs also represent the health system costs of treatment at the health center level, since it captures drugs, supplies and a government surcharge of 25% surcharge, which presumably captures personnel and overhead costs. Suggest the authors be more explicit about this in methods, and revise, line 133 to read, “We therefore included treatment costs as part of patient costs (Table 10).”

i. If these treatment costs represent both patient cost and health system costs, you may want to justify by indicating you captured treatment costs on the patient side to avoid double counting.

ii. Are these intended to represent opportunity costs to the patient and health system, or just to patient? If just patient opportunity costs, please indicate more clearly this is your measure of patient opportunity costs.

b. Are there any averted costs associated with untreated severe pneumonia, such as hospitalization or longer hospitalization stays? If yes, and these are not captured in this analysis, suggest being explicit that this analysis does not include averted health system and patient costs from improved dx.

c. Related to averted costs due to improved dx, it does appear that this analysis does not look at treatment outcomes and net costs (intervention costs plus incurred health system treatment costs minus health system and patient medical costs averted), however, (1) it is a little confusing why the authors are capturing the treatment (partial? Pre-referral?) costs to the patient at the health center level; and (2) why they don’t include full range of benefits from improved dx. It would be useful for the authors to clarify this in intro, methods, and/or explicitly note this in the discussion.

d. Page 7. Line 151. First indicate how total cost was calculated. This isn’t clear. You estimate costs for each arm (intervention and control) using input costs shown in table 1, but how do you get a total cost for each arm, and then describe the three three average costs (all pneumonia, severe, non-severe)?

e. Page 9, table 2—how did the authors pick the min and max estimates for costs?

3. Cost-effectiveness model

a. The authors develop a model that compares the opportunity cost of combined use of Pulse oximetry and the standard IMCI guideline versus the standard IMCI guideline in an Ethiopian rural setting. I find the wording of this strange, because they are comparing approaches on an intermediate outcome, i.e. the measure of effectiveness is the proportion of children diagnosed with severe pneumonia in the intervention and control. Perhaps just refine the statement on what they are comparing and be more explicit up front about comparing opportunity cost and intermediate outcomes.

b. A cost effectiveness analysis ought to evaluate what occurs after the diagnosis, and fully evaluate the health and cost outcomes for each diagnosis. This could be an improvement in diagnostic accuracy, or changes in health outcomes, resulting from diagnosis and treatment.

i. The authors should explain why additional benefits (i.e. improved health outcomes and averted costs to the health system and patient—averted medical costs, averted health system expenditures), from improved dx, were not included.

ii. Can the authors justify why they chose the measure of effectiveness they did?

c. To the points above, I recommend that the authors more clearly describe their cost-effectiveness model and better define the incremental cost effectiveness ratio up front, and not leave it to the discussion section and the caveat. This is an important issue. I don’t disagree with what the authors say, but one may also wonder, if pneumonia is a well-established leading cause of death and they have information on numbers treated and their costs, then why not model that out? Explain up front, why this approach was not done, or feasible. And to support your approach, reference Drummond et al in the methods section.

i. Perhaps the authors can review this useful check list to justify their approach. Kip, Michelle MA, et al. "Toward alignment in the reporting of economic evaluations of diagnostic tests and biomarkers: the AGREEDT checklist." Medical decision making 38.7 (2018): 778-788.

d. Table 2. Input parameters. Do the costs of diagnosed severe and non-severe pneumonia include the cost of treatment at the health center? If so, shouldn’t the input parameter read, cost of dx and pre-referral treatment of severe pneumonia…’ if and when that is the case?

e. How are Table 2 and 3 related? Is Table 3 JUST diagnostic costs, without treatment? If treatment costs aren’t used in the ICER, I’m not sure why they are included in the cost analysis.

f. Pg. 10, Line 198-199- What was the time horizon for the actual RCT study? The total and average costs have been calculated from a sample over a given period (i.e. one year or two years?)- I believe this is the correct time horizon, if they are calculating total costs and dividing by all diagnosed cases in intervention and control.

Results and Discussion

1. Since effectiveness estimates are the result of another study (referenced), perhaps just refer back to Table 2, and do not make this a result for this study. It is an input into your model.

2. Could you add a new table to show total costs, numbers of cases and average costs for (1) Intervention dx severe (2) intervention non-severe; (3) Control dx severe, and (4) control non-severe –so four columns, and this would correspond to the input costs shown in Table 2, but the reader can understand better how they were derived. You could include information from previous RCT study shown in lines 205 and 206, p. 10 in this table, as it is a given input into your analysis, not a result.

3. I don’t understand the relationship between Table 1 and 3. If in methods you describe how you calculate total cost, this may become clearer.

a. I would think Table 3 would present a summary and disaggregated cost categories for total costs, corresponding to lines 212 and 213, but these look like average unit costs again. It makes more sense to me to look at cost shares of the total costs disaggregated by cost categories.

4. Table 4 and CE results seem more like a cost analysis to me. In the discussion, the authors note that comparisons are difficult to make, in part b/c other similar studies do extend the analysis out to cases treated or DALYs averted. I think this paper could be reframed as a cost consequence analysis and would be an excellent complement to the published RCT. Especially since much of the discussion focuses on costs.

Minor

1. Line 141, what are ‘initial costs’ for pulse oximeter?

2 Add year of currency to Table 1

3. Add currency and year to Table 3 and indicate these are costs for diagnosing severe pneumonia.

4 Since the approach is incremental to the existing IMCI health program, I think the authors can omit lines 330 to 333—unless the improvements used more space/bldg., this would net out to be zero.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLOS Glob Public Health. doi: 10.1371/journal.pgph.0000757.r003

Decision Letter 1

Melissa Morgan Medvedev

7 Jul 2022

Cost-effectiveness of pulse oximetry and integrated management of childhood illness for diagnosing severe pneumonia

PGPH-D-22-00195R1

Dear Mr Tesfaye,

We are pleased to inform you that your manuscript 'Cost-effectiveness of pulse oximetry and integrated management of childhood illness for diagnosing severe pneumonia' has been provisionally accepted for publication in PLOS Global Public Health.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact globalpubhealth@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Global Public Health.

Best regards,

Melissa Morgan Medvedev, M.D., Ph.D.

Academic Editor

PLOS Global Public Health

***********************************************************

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Does this manuscript meet PLOS Global Public Health’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Global Public Health does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: No additional comments.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

**********

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (69.8KB, docx)

    Data Availability Statement

    All relevant data are within the paper.

    Articles from PLOS Global Public Health are provided here courtesy of PLOS

    RESOURCES