Information and Communication Technology to Enhance the Implementation of the Integrated Management of Childhood Illness: A Systematic Review and Meta-Analysis

Abstract

Objective

To evaluate the impact of Information and Communication Technology (ICT) on the implementation of Integrated Management of Childhood Illness (IMCI) and integrated Community Case Management (iCCM) through a systematic review and meta-analysis (PROSPERO registration number: CRD42024517375).

Methods

We searched MEDLINE, EMBASE, Cochrane Library, and gray literature from January 2010 to February 2024, focusing on IMCI/iCCM-related terms (Integrated Management of Childhood Illness, IMCI, integrated Community Case Management, iCCM) and excluding non-ICT interventions. A meta-analysis synthesized the effect of ICT on clinical assessment, disease classification, therapy, and antibiotic prescription through odds ratio (OR; 95% CI) employing a random effects model for significant heterogeneity (I²>50%) and conducting subgroup analyses.

Results

Of 1005 initial studies, 44 were included, covering 8 interventions for IMCI, 7 for iCCM, and 2 for training. All digital interventions except 1 outperformed traditional paper-based methods. Pooling effect sizes from 16 studies found 5.7 OR for more complete clinical assessments (95% CI, 1.7-19.1; I², 95%); 2.0 for improved disease classification accuracy (95% CI, 0.9-4.4; I², 93%); 1.4 for more appropriate therapy (95% CI, 0.8-2.2; I², 93%); and 0.2 for reduced antibiotic use (95% CI, 0.06-0.55; I² 99%).

Conclusion

This review is the first to comprehensively quantify the effect of ICT on the implementation of IMCI/iCCM programs, confirming both the benefits and limitations of these technologies. The customization of digital tools for IMCI/iCCM can serve as a model for other health programs. As ICT increasingly supports the achievement of sustainable development goals, the effective digital interventions identified in this review can pave the way for future innovations.

From 1990 to 2021, the global under-five mortality rate decreased by 59%.¹ However, child survival continues to be a critical issue, with ∼13,800 under-five deaths daily in 2021, predominantly from preventable diseases in Low-income and Middle-Income Countries (LMIC). To address this, World Health Organization and United Nations International Children's Emergency Fund introduced the Integrated Management of Childhood Illness (IMCI) strategy in 1996.² The IMCI represented an important innovation, adopting a holistic strategy for treating children who were sick at first-level health care facilities. It encompasses a comprehensive assessment of a child’s overall health, including their nutritional and immunization status.³ From a clinical perspective, this strategy involves a set of evidence-based guidelines that propose effective interventions for health workers (HW), even those with minimal medical backgrounds and working with limited or non-existent diagnostic support. The IMCI allows HWs to diagnose and manage the top 5 main diseases—malaria, measles, malnutrition, diarrhea, and pneumonia—that account for 70% of the mortality among children under 5 years old.³

The IMCI has regularly updated its protocols to align with specific countries’ health profiles⁴ and to include new diseases.⁵ Its adoption varies from full national implementation in some countries to regional or facility-based adoption in others and many incorporating only its training without broader health system reforms.^6,7 The IMCI comprehensive implementation has been linked to a 15%-50% reduction in child mortality^8,9 and contributes to the sustainable development goal of fewer than 25 deaths per 1000 live births by 2030 by improving the quality of care alongside relevant cost savings.¹⁰ In addition, the IMCI promotes responsible antimicrobial use,¹¹ which is crucial amid the increasing antibiotic resistance threat.¹² On the basis of these promising results, the IMCI was subsequently streamlined into integrated Community Case Management (iCCM), empowering community health workers (CHW) to treat pneumonia, diarrhea, and malaria, provide nutritional assessments, and offer health advice in underserved areas.¹³

Despite their potential, the IMCI and iCCM have seen suboptimal utilization because of common constraints in LMICs, such as health care system fragmentation, inadequate training, poor supervision, shortages of essential drugs, and high staff turnover.^7,14 In Western Kenya, for instance, the implementation reached just 14%, well below the optimal 68% threshold.¹⁵ HWs often view the IMCI consultations as protracted and burdensome, with an increase in workload and patient waiting time.^7,16,17 Also, the IMCI training, comprising 8 days of theory and 3 days of clinical practice, has been criticized for being lengthy, costly, and inadequately resourced.^16,18 A South African study revealed a significant decline in adherence to IMCI guidelines over time, with less than 2% of HWs applying the protocol 32 months after training.¹⁹ The effectiveness and efficiency of health care service delivery depend on the commitment and motivation of the HWs to adhere to guidelines.²⁰ Inadequate staff commitment can considerably contribute to poor quality services.²¹ Surveys performed in several countries have indicated that the lack of motivation is the second most critical issue within the health care workforce, preceded only by staff shortages.²⁰

In the past decade, Information and Communication Technology (ICT) has enhanced health care by providing personalized, efficient care and quick, and accurate diagnostics.^22,23 In LMIC, ICT extends its reach to remote and underserved areas.²⁴ Digitized protocols can increase guideline adherence through guided support in diagnosis, treatment, and follow-up, and integrates data analysis.²⁵ We hypothesize that ICT can improve adherence to medical protocols and enhance the impact of IMCI/iCCM programs. This systematic review evaluates the integration of ICT into these programs, focusing on how ICT improves care delivery, training, and child health outcomes. This includes aspects such as medical examination, diagnosis, therapy, and the rational use of antibiotics.

Methods

This systematic review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines²⁶ and is registered in PROSPERO (CRD42024517375).

Inclusion Criteria

We considered studies focusing on ICT interventions integrated into IMCI/iCCM programs and used by HWs or caregivers. These interventions included Electronic Health Records (EHRs), Clinical Decision Support Systems (CDSS), mHealth, Short Message Service (SMS), software applications, health information exchange and communication platforms, telehealth, and any other technology finalized to improve the delivery of IMCI/iCCM programs. We evaluated both quantitative and qualitative outcomes. Our research encompassed various study designs, including randomized controlled trials (RCT), quasi-experimental, observational, mixed-methods, and qualitative research. Studies comparing the different digital interventions were excluded.

Search Strategy and Selection

Databases searched included MEDLINE, EMBASE, and Cochrane library. We focused solely on terms related to IMCI or iCCM (Integrated Management of Childhood Illness, IMCI, integrated Community Case Management, iCCM) and excluded all the studies not involving ICT. Grey literature, including unpublished research and conference abstracts, were sourced from Google Scholar, OpenAlex, GreyNet, and IEEE Xplore using the additional keywords “Digital Health” and “eHealth/mHealth”. We included studies in English, French, and Spanish from January 2010, to February 2024, marking the publication of principles for digital development in 2010.²⁷ Details of our search strategy and Population, Intervention, Comparator, Outcome, and Study Design framework are in the supplementary files (Supplementary File 1, available online at https://www.mcpdigitalhealth.org/).

Data Extraction

Two authors (A.B. and C.S.Y.) independently screened titles and abstracts, resolving disagreements with a third reviewer (M.L.). A weighted κ coefficient of Cohen was calculated to assess the degree of agreement between the 2 reviewers. On retrieving the full texts, one author (A.B.) extracted the data, which was then double-checked by another one (C.S.Y.). The extracted data included study design, implementation country, sample size, digital intervention type, and results and conclusions of the study. References of these articles were also reviewed for additional relevant ones.

Bias Assessment

The quality of the studies was evaluated by 2 authors (A.B. and C.S.Y.) using the Joanna Briggs Institute critical appraisal tools and the mixed methods appraisal tool specifically for mixed-method or convergent design studies. The risk of bias was categorized as high, medium, or low. Given the inherent characteristics of involving digital devices and their comparison to those without ICT interventions, blinding of interventions was deemed inapplicable. Also, in this case, we assessed the degree of agreement between the raters through a weighted κ coefficient of Cohen.

Meta-Analysis

The meta-analysis evaluated the effect of ICT on assessment completeness for children who were sick, disease classification accuracy, therapy appropriateness, and antibiotic reduction. These outcomes were measured in comparison with the standards set by the IMCI protocols. We calculated pooled effects using odds ratios (OR) and 95% CI, focusing on intention-to-treat analysis. We reported the degree of inter-study heterogeneity using the Cochrane’s Q test. Where a significant heterogeneity was present (P≤10 or I²>50%) we applied a random effects model, with further subgroup analysis for specific effects. The decision to include or exclude studies in the meta-analysis was made jointly by 2 authors (A.B. and C.S.Y.), with a third author consulted in case of disagreement (M.L.). The R, version 4.3.1,²⁸ and the “meta” package²⁹ were used for this analysis.

Included studies are presented based on their integration with either the IMCI or the iCCM programs. We define e-IMCI (electronic IMCI) and e-iCCM (electronic iCCM) interventions as those where the program has been enhanced by an ICT solution. In comparison, we define p-IMCI (paper IMCI) and p-iCCM (paper iCCM) as the standard implementations based on paper.

Results

Our search yielded 1005 results, from which we identified 20 duplicates. After excluding 922 studies for lacking relevance to ICT based on their titles and abstracts, we retrieved 64 full texts for a detailed review.

By applying our inclusion or exclusion criteria, out of the 64 retrieved articles, 20 were excluded for reasons presented in the PRISMA flow diagram (Figure 1) and in supplementary files (Supplementary File 2, available online at https://www.mcpdigitalhealth.org/), resulting in 44 studies included in our analysis and 16 in the meta-analysis.

Figure 1.

PRISMA flow diagram of the studies selection process.

The interrater reliability for selecting the included studies, measured using Cohen’s κ, was 0.72, suggesting good agreement between the reviewers.³⁰

Characteristics of Included Studies

Among the 44 included studies (in details in supplementary files, Supplementary File 3, available online at https://www.mcpdigitalhealth.org/), 27 focused on IMCI,^25,31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56 14 on iCCM,57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70 and 3 on distance learning.71, 72, 73 These studies collectively represent 8 ICT interventions for IMCI, 7 for iCCM, and 2 for IMCI training.

Most studies were RCTs (n=11: including 8 cluster RCTs,^{31,46,56,60,62,68,69,73} 2 stepped-wedge design studies^40,67 and 1 RCT),⁵³ mixed-methods research (n=9)^{38,41,44,47,49,59,65,70,71} and qualitative assessments (n=8).^{32,37,50,52,55,63,64,72} The other studies were quasi-experimental (n=4),^36,39,48,51 diagnostic accuracy studies (n=2),^34,57 observational studies (n=2),^43,58 1 cost-analysis study,⁶¹ and 1 stakeholder analysis.⁶⁶ We also retrieved 6 reviews that were important for contextualizing the development and use of these digital tools: 2 focused on ALgorithms for the MANagement of Acute CHildhood illnesses (ALMANACH)^33,54 development, one on ePOCT,⁴⁵ 2 on ICT integration in IMCI^25,35 and 1 on integrated e-diagnostic approach (IeDA) adoption in Burkina Faso.⁴²

The research spanned 16 LMICs, with a major focus on Africa (n=36 studies) and Asia (n=3). One study was multicentre (Bangladesh, Burkina Faso, and Ecuador).⁵⁷

A total of 77,291 children were enrolled in the quantitative analysis, reporting a wide range of outcomes, with several investigations examining more than one at the same time. The most explored outcomes were therapy appropriateness (n=11),^{34,36,39,40,46,51,59,60,62,65,68} user satisfaction (n=11),^{37,38,44,47,49,50,52,59,64,65,72} completeness of assessment (n=10),^{31,34,36,39,40,46,48,49,51,59} antibiotic prescription (n=7),^{31,36,39,43,47,48,53} and disease classification (n=6).^{31,34,40,46,57,65}

All the interventions were CDSS based on predictive decision tree algorithms. CDSS align HWs’ inputs with a computerized database to offer precise point-of-care recommendations to guide in clinical assessment, physical examination, treatment, and follow-up planning or referral. However, the CDSS approach varied. In Ghana, an interactive voice response-system aids caregivers in symptom severity assessment.³⁴ mPneumonia merges software-based breath counting and pulse oximetry for enhanced diagnosis for pneumonia.⁵⁵ In India a digital IMCI also includes algorithms for neonatal care.⁴⁹ The MEDSINC, a web-based platform, diverges from standard decision tree solutions by using bayesian pattern recognition to prevent misdiagnoses from singular data point overemphasis.⁵⁷ Furthermore, MEDSINC analyses clinical history, symptoms, and vital signs employing cluster-pattern data for a comprehensive clinical risk assessment.⁷⁰

Among all the ICT inventions retrieved, only the electronic-Integrated Management of Childhood Illness (eIMCI)⁴⁴ in South Africa and the interactive voice response system³⁴ were not mobile device-based.

Table^2,7,8,31, 32, 33, 34^,36, 37, 38, 39, 40, 41^,43, 44, 45, 46, 47^,49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63^,65, 66, 67, 68, 69, 70^,72,73 summarizes the findings for the 17 ICT interventions. Although the majority (n=13, 76.5%) focused on their effectiveness, only a few published their algorithms (n=6, 33.3%) or provided a cost-analysis (n=3, 17.6%).

Table.

Evidence Gathered for Each Intervention Included in This Review

Interventions	No.	System	Countries	Algorithm published	Efficacy	Effectiveness	Qualitative assessment	Cost-analysis
Distance learning
DIMCI	2		Tanzania	N/P	No	Yes63	Yes72	Yes72
Low-cost tablets	1		Uganda	N/P	No	Yes73	No	No
Integrated community case management
CommCare mHealth application	1	CommCare	Niger	No	No	Yes69	No	No
DHIS 2 & J2ME	2	DHIS 2 & J2ME	Zambia	No	No	Yes62	No	Yes61
inSCALE	2	CommCare	Mozanbique and Uganda	No	No	Yes60,68	No	No
MEDSINC	2	Web-based platform	Bangladesh, Burkina Faso, Ecuador, and Nigeria	Yes57	No	Yes57,70	Yes57,70	No
Mobile phone use	1	N/A	Uganda	No	No	Yes59	Yes59	No
ODK-Liberia	1	ODK	Liberia	No	No	No	Yes58	No
SL e-CCM application	5	Honeycomb	Malawi	No	No	Yes65,67	Yes65,66	No
Integrated management of childhood illness
ALMANACH	9	CommCare	Tanzania, Afghanistan, Nigeria, Somalia, and Lybia	Yes54	Yes33,53	Yes31	Yes36,39,43,52,56	No
eCare	1	Mangologic	DRC, CAR, Mali, and Niger	No	Yes47	No	No	No
eIMCI SA	4		South Africa	No	No	Yes38,41,46	Yes38,44	No
eIMCI TZ	2		Tanzania	No	No	Yes51	Yes50	No
ePOCT	1		Tanzania	Yes45	Yesa	Yesa	Yesa	No
IeDA	4	CommCare	Burkina Faso, Niger, Mali, and India	No	No	Yes40	Yes37	Yes40,49
IVR-system	1	IVR	Ghana	Yes34	Yes34	No	No	No
mPneumonia	2	ODK	Ghana	Yes55	No	No	Yes32	No

Abbreviations: ALMANACH, ALgorithms for the MANagement of Acute CHildhood illnesses; DHIS, district health information software; DIMCI, distance learning IMCI; eIMCI SA, electronic-IMCI South Africa; eIMCI TZ, electronic-IMCI Tanzania; IeDA, integrated e-diagnostic approach; IMCI, integrated management of childhood illness; inSCALE, Innovations at Scale for Community Access and Lasting Effects; IVR, interactive voice response; J2ME, Java 2 Micro Edition; N/P, not pertinent; ODK, Open Data Kit; SA, South Africa; SL e-CCM, supporting LIFE electronic CCM; TZ, Tanzania.

^aExcluded from this review as it was not compared against standard care.

Risk of Bias Assessment

Excluding cost and stakeholders’ studies, out of 36 studies, 21 (58.4%) presented a low risk of bias, 12 (33.3%) a medium risk, and 3 (8.3%) a high risk. Major concerns in RCTs included nonblinding of outcome assessors (91.9%). For 50% of the mixed methods with a quantitative, nonrandomized component, it was not clear if participants were representative of the target population. For a quasi-experimental study, concerns existed about the independence of the control group (100%) and for the qualitative ones about the cultural orientation of the researcher and the representativeness of the participants’ voices (87.5%). The weighted κ coefficient of Cohen among the raters was very good (0.83).³⁰ Details of the bias assessment are in the supplementary files (Supplementary File 4, available online at https://www.mcpdigitalhealth.org/).

Digital Integrated Community Case Management tools

We found information on digital iCCM across Bangladesh, Burkina Faso, Ecuador, Liberia, Malawi, Mozambique, Niger, Nigeria, Uganda, and Zambia.

Supporting LIFE electronic CCM Application

Supporting LIFE electronic CCM application (SL e-CCM) gathers patient details, symptoms, and vital signs like breathing rate through a tap screen.⁶⁴ SL e-CCM found a higher accuracy in diagnosing (81% vs 58%, P<.01) without therapeutic improvement (P=.27).⁶⁵ It increased urgent referrals and reduced repeated consultations (P<.01), but hospital admissions remained unchanged (P=.3).⁶⁷ Qualitative assessment highlighted that HWs praised its ease of use and efficiency, despite technical and network challenges resulting in poor integration into their workflow. However, scalability appears jeopardized by inability to access past records and discrepancies between digitalized and paper guidelines.⁶⁴

Innovations at Scale for Community Access and Lasting Effects

Innovations at Scale for Community Access and Lasting Effects (InSCALE), developed by the Malaria Consortium with input from the Ministry of Health and CHWs of Uganda and Mozambique, aim to improve the iCCM performance, morale, and satisfaction. It supports diagnosing (featuring a respiratory rate counter), treating, and referring children, and identifying danger signs in pregnancy. A supplementary application for supervisors includes data management for analysis, facilitating timely reporting.⁶⁰ Piloted in Uganda and Mozambique, the application considerably increased treatment appropriateness by 26% (Mozambique)⁶⁸ and 11% (Uganda)⁶⁰ and by 15% considering the pooled effect (P<.01). However, no significant improvements were noted in community CHWs utilization (P=.06), motivation (P=.4), and knowledge (P=.5).⁶⁰

MEDSINC

MEDSINC helps CHWs in clinical severity assessment, triage, treatment, and follow-up. It found a specificity correlation of 84%-99% with health care professionals’ suggestions.⁵⁷ Users praised MEDSINC for its user-friendliness and effectiveness in job performance^57,70 and an unpublished Nigerian study indicated a 41% rise in iCCM adherence with MEDSINC.⁷⁰ A recent update featuring a machine learning-based malaria algorithm further improved its malaria detection sensitivity (from 43% to 60%) and specificity (from 64% to 79%).⁷⁴

Other e-iCCM ICT Solutions

In Zambia, a mobile platform using District Health Information Software (DHIS 2) and Java 2 Micro Edition (J2ME, a Java emulator for the Android system) slightly improved overall treatment rates (66% vs 63%), increased correct pneumonia treatment by 21%, and enhanced supportive supervision by 18%. However, these improvements were not statistically significant (P>.05). Only CHWs logistic supply significantly improved (P<.05).⁶²

In Uganda, CHWs used mobile phones for data entry and immediate server uploads, ensuring record accuracy and prompt medicine resupplies through SMS. Correct child management was at 93%, comparable to 94% in usual care (P>.05). Additionally, this approach improved treatment planning, supply management, and logistics, eliminating stockouts (P>.05).⁵⁹ In Niger, a CommCare mHealth application led to considerable improvement in quality of care regarding child assessments and referral decisions, without notably enhancing treatment.⁶⁹ In Liberia, a tool based on open data kit has been evaluated after 4 years of piloting for its connectivity before scaling up, but further information about its effectiveness are unavailable.⁵⁸

Digital Integrated Management of Childhood Illness tools

Evidence supporting the enhancement of care quality through IMCI digitalization initially emerged from a pilot in Tanzania in 2008.⁷⁵ Although initially based on a modest sample of 23 consultations, subsequent confirmation in a larger study underscored significant improvements in clinical assessments (71% vs to 21%), diagnoses (91% vs to 83%),^50,51 and communication between HWs and caretaker⁵⁶ (P<.05).

At the time of this review, digital IMCI was in use in Tanzania,⁵¹ Malawi, Adamawa State (Nigeria),³⁹ India,⁴⁹, Niger,⁴² Guinea, Burkina Faso,^40,42 Mali,⁵⁴ South Africa,⁴¹ Kenya,⁷⁶ Somalia,^25,48 Zambia⁷⁶ and in use to Médecins sans Frontières’missions.⁴⁷ It was not possible to obtain evidence for all these digital tools. Some digital adaptations of the protocols, such as ALMANACH,⁵⁴ ePOCT,⁴⁵ mPneumonia,⁵⁵ and eCARE,⁴⁷ deviate from traditional IMCI by introducing new point-of-care tests and diseases.

Integrated e-Diagnostic Approach

Integrated e-Diagnostic Approach (IeDA) encompassing e-coaching, EHR, e-learning, and DHIS 2 integration, launched in Burkina Faso in 2010 by the Ministry of Health and Terre des Hommes, was in 2023 used in 85% of primary health care centers, covering 92% of consultations with 90% guideline adherence. It serves over 350,000 children monthly, registering over 8 million.^42,77 IeDA improved diagnosis accuracy (79% vs 54% in control) and therapy, with a 15% reduction in antibiotic⁴⁰ (P<.05) and 1.6 USD million annual savings.⁴² It is also used for epidemic surveillance.⁷⁸ Accepted by 91% of HWs for its user-friendliness, it boosts caregiver satisfaction, though tablet slowness and limited common illness guidance are drawbacks.³⁷

IeDA has been piloted and adapted also to Mali (2017),⁴² Niger (2019),⁴² India (2020, state of Jharkhand),⁴⁹ and Guinea (2022),⁴² and soon in Bangladesh.⁷⁹ IeDA's latest version includes also algorithms for antenatal care⁸⁰ and newborn⁴⁹.

ALgorithms for the MANagement of Acute CHildhood illnesses

The ALgorithms for the MANagement of Acute CHildhood illnesses (ALMANACH) was developed in Tanzania in 2011 by the Swiss TPH and then implemented by the International Committee of the Red Ccross in conflict zones such as Afghanistan (2015),^33,36 Nigeria (2016),^36,43 Somalia (2020),⁴⁸ and Libya (2022).²⁵ Because it operates in unsafe or war zones, for security reasons it does not store any individual data and uploads aggregated data to DHIS 2. ALMANACH significantly improved clinical assessment in Tanzania (71% vs 21%),³¹ Afghanistan (84% vs 24%),³⁶ and in Nigeria (58% vs 46%), obtaining a better diagnosis (91% vs 83%, Tanzania),³¹ and treatment (85% vs 35%, Afghanistan;³⁶ 48% vs 30%, Nigeria)³⁹ (P<.05). These results have been achieved by reducing antibiotic prescriptions by 80%^31,36,48,52 and, at follow-up, child recovery was higher in comparison with usual care^43,53 (P<.05). ALMANACH was also, in general, well accepted by the HWs.⁵² In Nigeria, after scaling up, the tool was handed over to the local health authority of Adamawa by the end of 2021. At the state level, the antibiotic prescription rate has been reported to decrease from 78% in 2018 to 19%-21% in 2023.⁸¹ On the basis of the experience gained with ALMANACH, the Swiss TPH developed ePOCT, which combines advanced algorithms with an oximeter and includes a C-reactive protein point-of-care test that helped safely reduce antibiotic prescriptions and improve confidence in management.⁴⁵

eCare

eCare aims to support Médecins sans Frontières’s HWs. It has shown promise in reducing antibiotic prescriptions to 25% and covering 90% of clinical situations.⁴⁷ However, its effectiveness is not documented in peer-reviewed journals, with current findings primarily presented at conferences.

electronic-Integrated Management of Childhood Illness

In South Africa, the electronic-Integrated Management of Childhood Illnes (eIMCI) was launched in KwaZulu-Natal in 2018.¹⁹ Its adoption found promise and was well-received,³⁸ but the COVID-19 pandemic hindered its full implementation, with usage varying from 0 to 66.5%.⁴¹ Excluding screenings for anemia, tuberculosis, HIV, and malnutrition, conventional care outperformed e-IMCI in symptom classification, treatment effectiveness (91% vs 82%), and reduced unnecessary antibiotic use (5% vs 10.5%)⁴⁶ (P<.01).

mPneumonia

In Ghana, PATH piloted mPneumonia, a tool combining IMCI algorithms with a breath counter and oximeter, to improve respiratory infection diagnostics.⁵⁵ Health administrators found it implementable, whereas HWs deemed it user-friendly and helpful for accurate patient care, enhancing confidence in diagnosis and treatment. The main challenges were device charging and the time required for usage.³²

Meta-Analysis

The meta-analysis evaluated the effectiveness of ICT in 4 areas, as follows: (1) completeness of clinical assessment (5 studies; 1,828 intervention children vs 2,988 control children); (2) disease classification accuracy (5 studies; 1,933 vs 2,433); (3) therapy appropriateness (10 studies; 10,766 vs 13,685); and (4) antibiotic reduction (7 studies; 3,236 vs 3,100).

Significant results favored the intervention in 100% of the studies (5/5) for clinical assessment completeness, 80% (4/5) for disease classification accuracy, 40% (4/10) for therapy appropriateness, and 83.3% (6/7) for antibiotic reduction. Conversely, 2 studies related to therapy appropriateness significantly (P<.05) favored the control arm.

Using a random effects model, we calculated the pooled effect sizes as follows: an OR of 5.7 (95% CI, 1.7-19.1; I², 95%) for the completeness of clinical assessment; an OR of 2.0 (95% CI, 0.9-4.4; I², 93%) for disease classification accuracy; an OR of 1.4 (95% CI, 0.8-2.2; I², 93%) for the appropriateness of therapy; and an OR of 0.2 (95% CI, 0.06-0.55; I², 99%) for the reduction in antibiotic use (Figure 2).

Figure 2.

Forest plot for completeness of clinical assessment, disease classification accuracy, therapy appropriateness, and antibiotic reduction of the ICT interventions reviewed. ICT, information and communication technology; OR, odds ratio.

In subgroup analysis of clinical assessments, variations in effect size were linked to the intervention (IeDA and eIMCI in Tanzania showing more improvements), with heterogeneity dropping to 51% in samples with fewer than 600 children. For diagnostic accuracy, better results were associated with the iCCM program and solutions like ALMANACH, IeDA and SL e-CCM application. Therapy was significantly more often inappropriate for observational studies, whereas antibiotic prescription reduction correlated with study quality (low), design (RCT and cRCT), and intervention (ALMANACH and eCare). Details of the subgroup analysis are present in Supplementary File 5, available online at https://www.mcpdigitalhealth.org/.

ICT solutions for IMCI training

ICT has been used to enhance IMCI training, with a Tanzanian study showing distance learning is as effective as standard courses but 70% cheaper. This course lasts 10-12 weeks, mixing face-to-face sessions, self-study, and clinical practice, supported by SMS communication with facilitators.^71,72 In Uganda, a study compared traditional training with tablet-based videos for pneumonia diagnosis, showing improvements in the intervention arm, though not statistically significant (P>.05).⁷³

Discussion

This review reported that ICT considerably facilitated IMCI and iCCM implementation, mainly by improving protocol adherence. The digitalization of IMCI/iCCM algorithms through a CDSS technical approach ensured crucial steps, such as checking for stiff necks in febrile children, and emphasized often overlooked tasks like measuring temperature and counting breaths. According to the meta-analysis, children evaluated using digital tools were 6 times more likely to receive a through clinical examination and twice as likely to be accurately diagnosed. Moreover, these children had a 40% higher chance of receiving suitable treatment with an 80% reduction in unnecessary antibiotic prescriptions. The integration of ICT into IMCI/iCCM programs brings additional advantages, such as improved cure rates,^43,53 more streamlined patient referrals,^67,69 and better communication between HWs and caregivers.^43,56 The HWs generally recognized the important contributions of ICT: digital tools were lauded for their user-friendliness, enhancing accuracy in care, and bolstering diagnostic confidence.32, 37, 52, 64 Regarding the IMCI training, it benefited from ICT by shortening course durations and reducing costs.⁷¹ Furthermore, many digital tools incorporate educational elements like videos, images,³⁴ and even gamification elements,⁵⁷ offering effective hands-on training that enhances learning through practical application. In addition, digital systems facilitate rapid updates and adaptations to evolving guidelines or specific local requirements without the need for further extensive retraining.³⁶

However, although there were many positive outcomes, our analysis also identified some negative aspects. The meta-analysis found that disease classification accuracy was somewhat subdued, and the appropriateness of therapy exhibited considerably varied results. For both of these outcomes, the CI crosses one, which implies that there is no statistical evidence that the interventions have a beneficial effect. This may suggest that digital tools can enhance adherence to guidelines, compelling HCWs to follow protocols completely and providing diagnostic suggestions. However, the final decision remains with the human agent using the tool. Even when protocols are correctly followed, it is the HCW who determines the disease classification and prescribes therapy. These decisions can be influenced by various factors outside the guidelines, such as personal experience, disease prevalence in the area, pressure from caregivers, and drug availability. Furthermore, it is important to note that adherence to guidelines is often measured by the HCW’s response to all prompts and the entry of all requested data, as many applications do not allow skipping essential checks. However, this does not guarantee that all entered data are accurate. Some data may be inputted without asking the caregiver, to save time, or may be incorrect because of limited clinical skills. Other data may be entered approximately to expedite the consultation, such as breath count or temperature. We cannot even exclude that, in the case of digital tools based on decision tree algorithms, the HCWs may eventually start manipulating the answers by inputting data to bypass additional prompts or to match a diagnosis they already have in mind. It is challenging to disentangle what may truly enhance these outcomes. Broadly, it would be possible to identify a range of interventions focusing on people and processes, such as sample verification of diagnoses through ex-post verification visits and follow-ups, more accurate and continuous supervision, on-the-job coaching, staff performance appraisals, and improved governance. However, these interventions are often expensive and may not be feasible in resource-constrained environments.

Other specific challenges were explicitly highlighted by the authors in the reviewed studies. The SL e-CCM application struggled with integration, visibility, and funding⁶⁶ and required continuous support from the Ministry of Health and humanitarian organizations for long-term success.⁶³ For IeDA, staff turnover in remote areas of Burkina Faso and sufficient supervision by district teams were problematic, constrained by budget limitations, vehicle accessibility, and time availability.³⁷ Maintaining user motivation without appropriate incentives emerged as a widespread long-term challenge across various interventions. A commonly reported complaint was increased consultation time with digital tools,^36,46,52 from 11 minutes in early interventions⁵⁰ to 28 minutes,⁴⁶ likely due to the use of more advanced and comprehensive tools. None of these obstacles, however, have prevented a smooth implementation of digital tools. Only in South Africa the attempt to digitalize the IMCI protocol encountered a setback due to limited computer literacy, equipment and staff shortages, and lack of adequate supervision and evaluations. The eIMCI's recording demands also conflicted with existing clinic programs, heightening administrative workload³⁸ and further problems like chart mismatches, medication omissions, and printout errors led the HWs to favor p-IMCI.^44,82 The South African experience prompts us to reflect more broadly on how the local political and administrative context can influence the implementation of digital health. By 2016, there were nearly 150 mHealth projects supporting HWs in LMICs, yet few achieved substantial scale.⁸³ Key issues remain and include fragmented donor and implementer platforms, unstable political support, failed integration, unsustainable business models, and inadequate regulatory guidelines.⁸⁴ Thus, establishing country-level digital health leadership is essential for upholding standards, fostering investment in neglected areas, and crafting adaptable solutions for enduring e-health systems.⁸⁵ At a technical level, sustainability could benefit from integration with health information systems and EHRs, ensuring interoperability. The EHRs enable patient tracking, summary assessments, and data consolidation for routine reports, while health information systems monitors IMCI/iCCM programs through indicators like illness incidence and treatment outcomes.^86,87 Although many e-IMCI/iCCM in this review upload data to DHIS 2, only IeDa and e-IMCI in South Africa were designed for automatic synchronization with the central server.^38,78 Furthermore, this review found scant evidence of the impact of these interventions on health managers and policymakers. Ideally, stakeholders should easily access a large volume of clinical data, but most research focus on care delivery by HWs rather than data utilization. Moreover, the reviewed literature rarely assessed if the software was a digital public goods, open and free for public benefit. This omission is partly because of the studies’ age and the recent emergence of the digital public goods concept.⁸⁸ In addition, most interventions lack a cost-effectiveness analysis, which is crucial for investment prioritization and policy guidance in resource-constrained systems. Likely, these gaps could be filled soon as the World Health Organization is pushing to facilitate digital transition in LMICs and Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable guidelines have been recently published to facilitate digitalization of evidence-based guidelines.⁸⁹ The antenatal care version of IeDA is already a product of this innovative approach.⁸⁰

We finally identified 2 more challenges for implementing digitalized tools from the experiences we examined, as follows: (1) balancing the completeness and accuracy of the tool with practical constraints, such as maintaining reasonable consultation times, a manageable number of diseases to treat, limited availability of diagnostic tools and tests, and varying levels of clinical skills; (2) finding a compromise between adapting to local epidemiological profiles (which enhances diagnostic accuracy but complicates standardization of procedures and data collection) and maintaining standardization (which simplifies processes, ensures consistent data collection, and facilitates quick deployment, but may reduce relevance and diagnostic accuracy).

Our systematic review has also several limitations. First, it is limited by strict adherence to exclusion criteria, which led to the omission of a few newer studies comparing different digital interventions with each other. Second, while using the Population, Intervention, Comparator, Outcome, and Study Design framework, there is a potential for subjective bias in selection, data extraction, and interpretation, which we tried to mitigate through a third-party consultation. Third, the meta-analysis presented high heterogeneity; subgroup analyses were conducted to explore this, but meta-analysis might not be the optimal method to pool the effect size in this context. Also, the funnel plots (see supplementary files, Supplementary File 6, available online at https://www.mcpdigitalhealth.org/) exhibit clear asymmetry. Fourth, our evidence search might have missed some gray literature, particularly in local languages. However, to optimize our search results and account for potential shifts in ICT terminology over recent decades, we concentrated exclusively on terms associated with IMCI or iCCM, and we selectively excluded sources that did not incorporate ICT approaches. Fifth, for many studies, limited follow-up and small sample sizes preclude reliable predictions regarding their sustainability. Follow-up studies should be conducted to confirm these results over the long-term. Consequently, the results cannot be fully generalized and should be considered from a critical perspective. Sixth, we cannot exclude the high risk of bias associated with non-blinded studies, as the intervention, such as a tablet in the hands of the HCW, is clearly visible. The use of new technology may influence the judgment of caregivers and healthcare workers, leading them to perceive the tablet or smartphone as a superior advancement in itself.

Conclusion

This systematic review and meta-analysis is the first comprehensive evaluation of ICT's impact on IMCI/iCCM programs and reports improvements through enhanced guideline adherence and health system strengthening. The decade-long evolution of digitized IMCI/iCCM programs anticipated the Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable guidelines’ approach and offers a blueprint for digital public health interventions in LMICs, paving the way for future innovations.

This review underscores the potential of ICT in enhancing health program implementation in LMICs. ICT can considerably improve delivery, efficiency, and effectiveness. Through dedicated investment in infrastructure, training, and development, alongside robust policy frameworks, LMICs can leverage technology to advance their health agendas, ultimately contributing to global health equity and the attainment of sustainable development goals.

Potential Competing Interests

The authors report no competing interests.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work the author Dr Andrea Bernasconi used ChatGPT3.5 to improve readability. After using this tool/service, the author reviewed and edited the content as needed and take full responsibility for the content of the publication.