Tracking health commodity inventory and notifying stock levels via mobile devices

Abstract

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

Primary

• To assess the effects of strategies for notifying stock levels and digital tracking of healthcare‐related commodities and inventory via mobile devices.

Secondary

• To describe what mobile strategies are currently being used to improve reporting and digital tracking of health commodities;

• to identify factors influencing the implementation of mobile interventions targeted at reducing stock‐outs of health commodities.

Background

Access to medicines and other health commodities remains one of the most serious global public health problems and results in critical gaps in delivery of healthcare services.

Description of the condition

Reliable availability of health commodities is fundamental to diagnosing and treating illnesses in primary healthcare settings. Health commodities include health products, health and medical supplies, and other items that may be needed for the provision of health services, including medicines, vaccines, medical supplies such as contraceptives dressings, needles and syringes, and laboratory/diagnostic consumables (Family Health International 2001). The World Health Organization (WHO) Global Strategy for Women’s and Children’s Health highlights the importance of equitable access to life‐saving medicines and other health commodities (WHO 2010). A hallmark of functioning health systems is the availability of essential medicines in adequate amounts, appropriate dosage forms, assured quality, and at a price that is affordable for the local community (WHO 2002). However, stock‐outs of critical medical commodities, such as medicines, are widespread especially in low and middle income countries (LMICs).

At least one third of the world’s population does not have regular access to medicines, which makes health care highly inequitable (WHO 2011). A survey of the national AIDS programmes in 12 countries by the Pan American Health Organization (PAHO) found that between January 2011 and April 2012, over 67% of countries reported experiencing stock‐outs of at least one drug, lasting an average of 40 days each (Sued 2011). Another study in Kenya reported that over 75% of health facilities had shortages of one component of the combination of drugs used to treat malaria, while one in four reported a lack of all related drugs (Kangwana 2009). This lack of access to critical drugs caused by a stock‐out has profound effects on the ongoing treatment of diseases. A study in Côte d’Ivoire reported that patients who experienced interruptions in HIV treatment caused by drug shortages were twice as likely to permanently discontinue treatment or die (Pasquet 2010). Drug stock‐outs have been linked to increases in morbidity and mortality across a number of disease states, including malaria (Chuma 2010), HIV (Pasquet 2010), and the prevention or treatment of pregnancy complications in low‐resource settings (Hill 2006).

Lack of access to medicines and other health commodities is often symptomatic of broader systemic problems. For example, access to medicines is determined by rational use of medicines, affordable pricing, sustainable financing, and reliable health and supply systems (WHO 2004). A reliable medicine supply system must comprise of appropriate procurement and distribution. A good distribution system ensures timely availability of medicines across all levels of the healthcare system and prevention of stock‐outs.

Description of the intervention

The rapid global expansion of mobile technology has provided a potential low cost solution to the challenge of drug distribution and stock‐outs. Plummeting costs of mobile handsets and services have made mobile phone technology accessible to people living in rural and underserved areas. Mobile interventions may address stock‐outs of medicines and health commodities primary through two strategies: supply chain management, and assessment and reporting of essential commodities (Mehl 2017 [pers comm]).

Supply chain management involves approaches for monitoring and reporting stock levels, consumption and distribution of medical commodities, as well as approaches to analyse and project usage of medical commodities. This can include the use of communication systems such as short message service (SMS) and data dashboards, to manage and report on supply levels of medical commodities. Some specific examples where mobile tools may be used to improve supply chain management include tracking inventory of health commodities, notifying stock levels of health commodities, monitoring cold‐chain sensitive commodities, and managing distribution of health commodities.

Digital approaches for assessment and reporting of essential commodities is often used for reporting and tracking the authenticity and quality of medical commodities. This can include using mobile functionalities, such as barcode readers and SMS communication to validate an authentication code on the drug packaging (Frøen 2016), as well as to report on adverse drug effects. Specific examples where mobile devices may be used for assessment and reporting of commodities include reporting counterfeit or substandard drugs, reporting adverse drug interactions, and registering licensed drugs and health commodities.

How the intervention might work

Mobile devices are being used for the purpose of supply chain management to improve data visibility, improve decision‐making, and help to address availability of commodities. There is substantial amount of variation in how such systems might operate. At the most basic level, interventions may involve citizen reporting of counterfeit medicines using SMS sent to a toll free phone number. Such interventions may use a mobile product authentication (MPA) application or a barcode that allows consumers to text a set of unique numbers to a toll‐free phone number to verify if a medicine is authentic. In response, the consumers may receive a SMS back that indicates the legitimacy of the medicine.

Other interventions may involve frontline healthcare workers or healthcare administrators in primary healthcare settings using mobile devices to collect data on stock levels so that data can be instantly digitized and be used to predict and prevent stock‐outs and respond to drug shortages. More comprehensive interventions may aim to develop a technology‐based system for reporting of drug stock levels, and also change the culture around the use of data (on stock levels) and accountability for responding to projected shortages. For example, cStock is an open‐source internet‐accessible logistics management information system that targets availability of health commodities at the community level in Malawi (Shieshia 2014). Health Surveillance Assistants (HSAs), who typically deliver primary healthcare services in the community, send information about the amount of medicine stocks they have on hand via a text message to a toll‐free number. The logistics management system automatically collates this data from multiple HSAs, calculates total quantities of commodities needed, and sends a text message to HSAs when the medicines are available at the nearest health centre. These data are also available on a internet‐accessible dashboard with simple, easy‐to‐use reports showing stock levels, HSA reporting rates, and alerts from central and district level health managers. Making real‐time data available with regular reports of stock levels allows managers and other stakeholders to coordinate, plan, and identify solutions to better meet community needs in a timely manner.

Why it is important to do this review

There is rapid progress in the use of mobile devices to address systemic challenges in the delivery of healthcare services. Despite the exponential growth of mobile device‐based interventions and their potential, there remain several unanswered questions about the effectiveness of such interventions. The reliable availability of essential medicines and health commodities is foundational to a responsive health system, and an area that is of much interest to governments, especially in LMICs. However, the evidence on the use of mobile devices to address drug and commodity shortages is sparse, and offers no clear way forward. To respond to this need, the World Health Organization (WHO) is in the process of establishing guidelines that aim to inform investments of digital health applications for strengthening health systems. This Cochrane Review is one of a suite of reviews that will contribute to these guidelines. We aim to assess the effectiveness of using mobile devices to address stock‐outs of drugs and essential health commodities, as well as the acceptability, resource use, and unintended consequences of such interventions.

Objectives

Primary

• To assess the effects of strategies for notifying stock levels and digital tracking of healthcare‐related commodities and inventory via mobile devices.

Secondary

• To describe what mobile strategies are currently being used to improve reporting and digital tracking of health commodities;

• to identify factors influencing the implementation of mobile interventions targeted at reducing stock‐outs of health commodities.

Methods

Criteria for considering studies for this review

Types of studies

For the primary study objectives, we will include the following study designs:

• randomised trials;

• non‐randomised trials, where the assignment is not random;

• controlled before‐after studies provided they have at least two intervention sites and two control sites;

• interrupted time series studies if there is a clearly defined point in time when the intervention occurred and at least three data points before and three after the intervention.

We will include published studies, conference abstracts, and unpublished data. We will include studies irrespective of their publication status and language of publication.

To address the secondary study objectives, we will include all the studies included in the primary analyses, as well as any studies that use descriptive, qualitative, or quantitative methods to describe interventions that are aimed at improving stock‐outs of health commodities.

Types of participants

• All cadres of healthcare providers (i.e. professionals, para‐professionals, and lay health workers) or others involved in the distribution of health commodities located at any level of the health system (e.g. administrative staff, managerial and supervisory staff in purchasing or distribution, or dispensary staff);

• other individuals or groups involved in stock notification, monitoring, and tracking commodity inventories. These individuals or groups may be based in a primary healthcare facility or in the community and must be involved in supporting the delivery of primary healthcare services.

• Clients or recipients of health services

Types of interventions

We will include interventions that are aimed at improving the availability of health commodities including medicines and other medical supplies, using mobile devices, for the delivery of primary healthcare services in healthcare facilities or in the community, if they involve one or both of the following:

• strategies for tracking health commodity inventory using mobile devices. Tracking health commodity inventory may involve the use of databases and dashboards to manage the availability of health commodities and project availability of medical supplies. While some aspects of commodity tracking might involve mobile devices, the data may be linked to a logistics management information system (LMIS) or supply chain management system where inventory levels and historic data are maintained on desktops;

• notification of stock levels conducted via mobile devices. This may involve the transmission of information on stock levels by health workers within healthcare facilities or by members of the community to alert higher‐level facilities about potential stock shortages. For example, health workers at facilities/dispensaries may use text messaging, SMS, or Unstructured Supplementary Service Data (USSD)‐based systems to notify district or central authorities about stock levels. In some interventions of interest, notification of stock levels using mobile phones may be a component of a broader strategy for tracking health commodities.

By “mobile devices” we mean mobile phones of any kind (but not analogue landline telephones), as well as tablets, personal digital assistants, and smartphones.

By “primary healthcare services” we mean a combination of the following:

• the first contact point of healthcare (Awofeso 2004), including care delivered at an individual or community level or both (Muldoon 2006), by individual healthcare providers or teams of providers, and intended to bring care to where people work and live (Muldoon 2006), or coordinate or provide continuity of care (WHO 2008), or both;

• any rehabilitative, therapeutic, preventive, and promotive healthcare (Global Health Watch 2011).

The comparisons for this review will be: tracking commodity inventory and notifying stock levels conducted via mobile devices compared with standard practice (i.e. non‐digital strategies or no intervention).

We will exclude:

• studies that focus on cold chain management only and do not report on stock levels of the vaccine;

• studies where commodity tracking and notification of commodities is conducted on stationary computers or laptops only.

Where tracking or notification via mobile device, or both, is delivered as part of a wider package, we will only include the study if we have judged the mobile component to be the major component of the intervention.

Types of outcome measures

• Availability of commodities, measured, for instance, as decreased stock‐outs, lead time for drug supply, availability at point of care;

• quality of data about stock management (accuracy of data, completeness of data);

• timeliness of stock level reporting, and time between receipt/reporting of data regarding commodity status and appropriate action;

• provider acceptability or satisfaction with the intervention, measured with a validated scale if available;

• resource use (e.g. human resources/time, including additional time spent by providers when managing/transitioning dual paper and digital reporting systems; training, supplies, and equipment);

• unintended consequences that result in adverse effect of the intervention (these could include: misreading or misinterpretation of the data; transmission of inaccurate data, for instance through software formatting errors; interrupted workflow due to infrastructural constraints for battery recharge and network coverage; decreased motivation/trust by health workers in the system if stock replenishment is not reliable; loss/misuse of device).

Search methods for identification of studies

We will restrict the search from 2000 to the date of search. This is based on the increased availability and penetration of mobile devices in LMICs starting in 2000 (ITU 2015).

Electronic searches

An independent Information Specialist (JE) will develop the search strategies in consultation with the review authors.

We will search the following databases for primary studies, from 2000 to the date of search:

• Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Library;

• MEDLINE, Ovid;

• Embase, Ovid;

• POPLINE, K4Health;

• Global Health Library, WHO.

Appendix 1 lists the search strategy for MEDLINE Ovid. Search strategies are comprised of keywords and controlled vocabulary terms. We will not apply any limits on language.

Searching other resources

Trial registries

We will also search for ongoing trials in the following trial registries and contact authors for further information and data, if available:

• WHO International Clinical Trials Registry Platform (ICTRP) (www.who.int/ictrp);

• US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov (www.clinicaltrials.gov).

We will search Epistemonikos (www.epistemonikos.org) for relevant systematic reviews and potentially eligible primary studies. Additionally, the WHO will issue a call for papers through popular digital health communities of practice such as the Global Digital Health Network, to identify additional primary studies as well as grey literature.

Grey literature

In addition to the above databases, we will search www.mhealthevidence.org for grey literature. The search portal for mhealthevidence.org is more limited; we will therefore review title and abstracts of all contributed literature that is not referenced in MEDLINE Ovid.

We will review reference lists of all included studies and relevant systematic reviews for additional potentially eligible primary studies. We will also conduct a citation search for studies that have cited any included studies. We will contact authors of included studies/reviews to clarify reported published information and to seek unpublished results/data.

Data collection and analysis

Selection of studies

A core team of two authors (NH and HB), with assistance where necessary from one additional review author (SA), will be responsible for the selection of studies. We will download all titles and abstracts retrieved by electronic searching to a reference management database and remove duplicates. Two review authors (NH and HB) will independently screen titles and abstracts for inclusion. We will retrieve the full‐text study reports/publication and two review authors (NH and HB) will independently screen the full‐text and identify studies for inclusion and identify and record reasons for exclusion of the ineligible studies. We will resolve any disagreement through discussion or, if required, we will consult a third review author (SA).

We will list studies that initially appeared to meet the inclusion criteria but that we later excluded in the 'Characteristics of excluded studies' table. We will collate multiple reports of the same study so that each study, rather than each report, is the unit of interest in the review. We will also provide any information we can obtain about ongoing studies. We will record the selection process in sufficient detail to complete a PRISMA flow diagram (Liberati 2009).

Data extraction and management

We will modify the Cochrane Effective Practice and Organisation of Care (EPOC) standard data collection form and adapt it for study characteristics and outcome data (EPOC 2017a). We will identify key characteristics of the intervention for abstraction based on the mHealth Evidence Review and Assessment (mERA) guidelines (Agarwal 2016). We will pilot the form on at least one study in the review. Two review authors (NH and HB) will independently extract the following study characteristics from the included studies for the primary analyses:

• general information: title, reference details, author contact details, publication type, funding source, conflicts of interest of study authors;

• population and setting: country, geographical location (rural, urban, peri‐urban), healthcare setting (e.g. facility‐based, community‐based);

• methods: function of the intervention, study design, unit of allocation, study duration;

• participant characteristics: type of user (role, if in the health system; length of training, if any), description of any other participants in the intervention, withdrawals;

• interventions: intervention purpose, components, infrastructure to support the technology, type of technology (software platform), type of mobile device(s) used (smartphone, tablets with a screen size larger than 7 inch, feature phones that can run java applications, basic phone with SMS and call functions, laptops), mode of delivery, content of the intervention, participant/provider training, interoperability, compliance with national guidelines, data security, comparison, fidelity assessment, duration of intervention;

• outcomes: primary and other outcomes specified and collected, time points reported, adverse events, results of any subgroup analyses.

Two review authors (NH and HB) will independently extract outcome data from included studies. We will note in the 'Characteristics of included studies' table if outcome data were reported in an unusable way. We will resolve disagreements by consensus or by involving a third review author (SA).

For the secondary analyses, we will extract all the information listed above, if available, to describe the intervention. To understand factors affecting the implementation of relevant interventions, we will be guided by the Supporting the Use of Research Evidence (SURE) framework (Glenton 2017). This framework provides a possible list of factors that may influence the implementation of health systems intervention. We will review the included studies and extract any data that describe enabling or restricting factors related to the implementation of the intervention based on themes identified in the framework.

Assessment of risk of bias in included studies

Two review authors (NH and HB) will independently assess the risk of bias for each study included in the primary analyses using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions Section 8.5 (Higgins 2011), and guidance from the Cochrane EPOC group (EPOC 2017b). Any disagreement will be resolved by discussion or by involving a third review author (SA). We will assess risk of bias for randomised trial/non‐randomised trials and controlled before‐after studies using the following criteria‐ random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, baseline outcomes measurement, similarity of baseline characteristics, and other bias. For interrupted time series studies, we will assess the risk of bias following these seven standard criteria (EPOC 2017b): independence of the intervention from other changes, pre‐specified shape of the intervention effect, independence of the intervention and data collection procedures, objectivity of the outcomes, bias resulting from missing outcome measures, selective outcome reporting, and other biases such as seasonality.

We will judge each potential source of bias as either high, low, or unclear and provide a quote from the study report together with a justification for our judgment in the 'Risk of bias' table. We will summarize the 'Risk of bias' judgements across different studies for each of the domains listed. We will consider blinding separately for different key outcomes where necessary (e.g. for unblinded outcome assessment, risk of bias for all‐cause mortality may be very different than for a patient‐reported pain scale). Where information on risk of bias relates to unpublished data or correspondence with a trialist, we will note this in the 'Risk of bias' table. We will not exclude studies on the grounds of their risk of bias, but will clearly report the risk of bias when presenting the results of the studies. When considering treatment effects, we will take into account the risk of bias for the studies that contribute to that outcome. We will further perform assessment of quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach (Guyatt 2008). Findings will be summarized in the ‘Summary of findings’ tables.

We will conduct the review according to this published protocol and report any deviations from it in the 'Differences between protocol and review' section of the systematic review.

Measures of treatment effect

We will report pre‐intervention and post‐intervention means and proportions for the intervention and comparison groups. Where possible, we will estimate the effect of the intervention using risk ratios or risk differences for dichotomous data, together with the appropriate associated 95% confidence interval (CI) and mean difference or standardized mean difference for continuous data, together with the 95% appropriate associated CI. We will ensure that an increase in scores for continuous outcomes can be interpreted in the same way for each outcome, explain the direction to the reader, and report where the directions were reversed if this was necessary. For interrupted time series studies, we will report the pre‐ and post‐intervention slopes of analysis, the difference of the slopes and the differences of intercepts at the first intervention point and the predicted intercept by the intervention. If interrupted time series data has been analysed incorrectly, we will reanalyse the data where possible (EPOC 2017c). The data will be analysed in RevMan 5 (Review Manager 2014).

Unit of analysis issues

If cluster‐randomised trials or controlled before‐after studies are included in the review, we will report cluster adjusted risk ratios or differences and their 95% CIs. If the analysis was not adjusted for clustering, we will use the intracluster correlation coefficient (ICC), if available, to adjust the CI. If the ICC value is not available, we will present the results without a measure of variance or precision of effect for outcomes for which there is a unit of analysis error (EPOC 2017d)

Dealing with missing data

We will contact investigators in order to verify key study characteristics and obtain missing outcome data where possible (e.g. when a study is identified as an abstract only). If this is not possible, we will report the data as missing, note this in the 'Risk of bias' tables, and not attempt to impute the missing values. For all outcomes, we will carry out the analyses, to the extent possible, on an intention‐to‐treat basis based on the available cases. However, in assessing adverse events, we will relate the results to the actual treatment received i.e. we will base the analyses on the participants who actually received the intervention and the number of adverse events reported in the studies.

Assessment of heterogeneity

If we find a sufficient number of studies that evaluate similar interventions and report similar outcomes, we will conduct a meta‐analysis. We will examine heterogeneity by visual inspection of forest plots as well as using the I² statistic to measure heterogeneity among the trials in each analysis (Higgins 2011). If we identify heterogeneity we will explore it by pre‐specified subgroup analysis.

Assessment of reporting biases

We will attempt to contact study authors and ask them to provide missing outcome data. Where this is not possible, and the missing data are thought to introduce serious bias, we will explore the impact of including such studies in the overall assessment of results. If we are able to pool more than 10 trials, we will create funnel plots to explore possible publication biases or other causes for asymmetry (Sterne 2011). We will interpret the results of the funnel plot with caution as funnel plot calculations for dichotomous outcomes measured as risk ratios are not well developed, and statistical funnel plot results may not be representative if there are small‐study effects.

Data synthesis

For the primary analyses, when studies evaluate interventions that are similar, we will group these together and summarize the key characteristics of each study in tables, to facilitate comparison across studies. We will undertake meta‐analyses using a random‐effects model only where this is meaningful i.e. if the intervention, context, and outcomes are similar enough for pooling to make sense. A common way that trialists indicate when they have skewed data is by reporting medians and interquartile ranges. When we encounter this we will note that the data are skewed and consider the implication of this. Where multiple trial arms are reported in a single trial, we will include only the relevant arms. If two comparisons (e.g. intervention A versus usual care and intervention B versus usual care) must be entered into the same meta‐analysis, we will halve the control group to avoid double‐counting.

For the secondary analyses, we will identify common themes across all the included studies to qualitatively describe the interventions. If any data on the effectiveness of these interventions is available and not included in the primary analyses results, it will be presented with an assessment of the risk of bias. For the secondary objective on factors affecting the implementation of relevant interventions, we will identify common themes under each Supporting the Use of Research Evidence (SURE) framework category and summarized.

'Summary of findings' table and GRADE

We will create 'Summary of findings' tables for the main intervention comparison(s) and include the most important outcomes in order to draw conclusions about the certainty of the evidence within the text of the review:

• availability of commodities (e.g. proportion of healthworkers/facilities reporting drug stock‐outs, time between stock‐out and availability of commodities);

• quality of data about stock management (e.g. accuracy of data, completeness of data);

• timeliness of stock‐level reporting;

• provider acceptability/satisfaction with the intervention.

If, during the review process, we become aware of an important outcome that we failed to list in our planned 'Summary of findings' tables, we will include the relevant outcome and explain the reasons for this in the 'Differences between protocol and review' section.

Two review authors will independently assess the certainty of the evidence (high, moderate, low, or very low) using the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness, and publication bias) (Guyatt 2008). We will use methods and recommendations described in Section 8.5 and Chapter 12 of the Cochrane Handbook for Systematic Reviews of interventions (Higgins 2011), and the Cochrane EPOC worksheets (EPOC 2017e), and using GRADEpro software (GRADEpro GDT 2015). We will resolve disagreements on certainty ratings by discussion and provide justification for decisions to downgrade or upgrade the ratings using footnotes in the table and make comments to aid readers' understanding of the review where necessary. We will use plain language statements to report these findings in the review (EPOC 2017f)

We will consider whether there is any additional outcome information that was not able to be incorporated into meta‐analyses and note this in the comments and state if it supports or contradicts the information from the meta‐analyses. If it is not possible to meta‐analyse the data we will summarize the results in the text.

Subgroup analysis and investigation of heterogeneity

We will perform subgroup analysis to assess variations in the delivery of interventions across different geographical settings and intervention characteristics. The subgroup analysis will be conducted only if sufficient number of trials are available to make statistically significant comparisons between groups.

• Type of geographic setting: (for example, urban, rural, peri‐urban; LMICs), as we anticipate that the intervention may have different effects due to social and economic differences between settings;

• type of intervention characteristics: interventions aimed at reducing stock‐outs and improving the supply of commodities vary largely in their characteristics. We will describe each intervention by intervention characteristics described under the data extraction section of this protocol. If sufficient number of studies have similar characteristics, we will conduct subgroup analyses led by meaningful practice questions. For example, analysing studies on stock‐out notification system alone, versus notification system that is integrated with a commodity management system.

Sensitivity analysis

We will perform sensitivity analyses defined a priori to assess the robustness of our conclusions and explore its impact on effect sizes. This will involve restricting the analysis to published studies, removing studies that have a high risk of bias based on the 'Risk of bias' assessment from any meta‐analyses.

Appendices

Appendix 1. MEDLINE search strategy

Ovid MEDLINE®<1946 to present>

1 Community Mental Health Services/ (18445)

2 Family Practice/ (64554)

3 Home Care Services/ (31605)

4 Physicians, Family/ (16018)

5 Physicians, Primary Care/ (2390)

6 Community Health Services/ or Community Pharmacy Services/ or Health Facilities/ or Health Facility Administration/ (48722)

7 Community Health Nursing/ or Home Health Nursing/ or Family Nursing/ (20786)

8 Community Health Workers/ (4341)

9 Preventive Health Services/ (12573)

10 Primary Health Care/ (66970)

11 Primary Prevention/ (17001)

12 Public Health/ (73592)

13 Rural Health Services/ (11374)

14 (primary care or general practi* or primary health or community mental health* or family practice or family medicine or family doctor or family physician* or home care or home based or home health* or community health* or community nurs* or health visit* or community pharmac* or preventive care or prevention program* or preventive service* or preventive health or primary prevention or public health or rural health or health promotion or health facilit*).ti,ab,kw. (501148)

15 ((guideline* or protocol*) adj4 (adher* or comply or complian* or observ*)).ti,ab,kw. (17656)

16 ((therap* or prescrib* or prescript* or diagnos*) adj2 (computer* or digital or electronic)).ti,ab,kw. (7316)

17 or/1‐16 (694330)

18 Cell Phones/ (7155)

19 Smartphone/ (1316)

20 MP3‐Player/ (167)

21 Computers, Handheld/ (3113)

22 ((cell* or mobile*) adj1 (phone* or telephone* or technolog* or device*)).ti,ab,kw. (13227)

23 (handheld or hand‐held).ti,ab,kw. (10039)

24 (smartphone* or smart‐phone* or cellphone* or mobiles).ti,ab,kw. (5650)

25 ((personal adj1 digital) or (PDA adj3 (device* or assistant*)) or MP3 player* or MP4 player*).ti,ab,kw. (1298)

26 (samsung or nokia).ti,ab,kw. (820)

27 (windows adj3 (mobile* or phone*)).ti,ab,kw. (43)

28 android.ti,ab,kw. (1548)

29 (ipad* or i‐pad* or ipod* or i‐pod* or iphone* or i‐phone*).ti,ab,kw. (1988)

30 (tablet* adj3 (device* or computer*)).ti,ab,kw. (1005)

31 Telemedicine/ or Telecommunications/ (21237)

32 Webcasts as topic/ (285)

33 Text Messaging/ (1703)

34 Telenursing/ (176)

35 (mhealth or m‐health or "mobile health" or ehealth or e‐health or "electronic health" or "digital health" or uhealth or u‐health).ti,ab,kw. (15996)

36 (telemedicine or tele‐medicine or telehealth or tele‐health or telecare or tele‐care or telenursing or tele‐nursing or telepsychiatry or tele‐psychiatry or telemonitor* or tele‐monitor* or teleconsult* or tele‐consult* or telecounsel* or tele‐counsel* or telecoach* or tele‐coach*).ti,ab,kw. (13983)

37 (webcast* or web‐cast*).ti,ab,kw. (215)

38 (((text* or short or voice or multimedia or multi‐media or electronic or instant) adj1 messag*) or instant messenger).ti,ab,kw. (3479)

39 (texting or texted or texter* or ((sms or mms) adj (service* or messag*)) or interactive voice response* or IVR or voice call* or callback* or voice over internet or VOIP).ti,ab,kw. (2574)

40 (Facebook or Twitter or Whatsapp* or Skyp* or YouTube or "You Tube" or Google Hangout*).ti,ab,kw. (4127)

41 Mobile Applications/ (2289)

42 "mobile app*".ti,ab,kw. (1789)

43 Reminder Systems/ (3080)

44 (remind* adj3 (text* or system* or messag*)).ti,ab,kw. (1417)

45 Medical informatics/ or Medical informatics applications/ (12966)

46 Nursing informatics/ or Public health informatics/ (2477)

47 ((medical or clinical or health or healthcare or nurs*) adj3 informatics).ti,ab,kw. (5039)

48 Computer‐Assisted Instruction/ (11295)

49 ((interactive or computer‐assisted) adj1 (tutor* or technolog* or learn* or instruct* or software or communication)).ti,ab,kw. (2224)

50 or/18‐49 (108765)

51 sd.fs. (62984)

52 exp Vaccines/ec, st, sd [Economics, Standards, Supply & Distribution] (10442)

53 exp "Equipment and Supplies"/ec, sn, sd, td, ut [Economics, Statistics & Numerical Data, Supply & Distribution, Trends, Utilization] (50120)

54 (exp Therapeutic Uses/ec, st, sd or exp Pharmaceutical Preparations/ec, st, sd, ut) not Veterinary Drugs/ (47575)

55 pharmaceutical services/ or community pharmacy services/ or drug information services/ or pharmaceutical services, online/ or pharmacy service, hospital/ (25231)

56 hospital distribution systems/ or materials management, hospital/ or inventories, hospital/ or medication systems, hospital/ or product line management/ (9601)

57 ((commodit* or consumable* or stock or stocks or supply or supplies) adj3 (inventor* or level* or notif* or track* or count* or report* or chain or out or outs or manag* or order* or logistic* or system or systems or shortage* or manag* or monitor* or maintain* or maintenance or audit or auditing)).ti,ab,kw. (10270)

58 ((health or medical or medicines or vaccine* or drug or drugs or laborator* or diagnos*) adj3 (product* or supply or supplies or consumable* or commodit* or stock or stocks or stockout* or "stock out*" or shortage*)).ti,ab,kw. (34068)

59 health resources/ or clinical laboratory information systems/ or clinical pharmacy information systems/ or database management systems/ or hospital information systems/ or ambulatory care information systems/ or pharmacy administration/ or drug utilization/ or "drug utilization review"/ or drug storage/mt (57147)

60 or/51‐59 (272715)

61 17 and 50 and 60 (981)

62 limit 61 to yr="2000 ‐Current" (872)

Contributions of authors

SA, TT, MSF, GLM, CG, and SL conceived the protocol. SA, TT, MSF, NH, HB, and GLM, CG, and SL designed the protocol. SA, TT, CG, and SL co‐ordinated the protocol. SA, TT, and CG wrote the protocol. TT, GLM, CG, and SL provided general advice on the protocol. GLM secured funding for the protocol.

Sources of support

Internal sources

• No sources of support supplied

External sources

• UNDP‐UNFPA‐UNICEF‐WHO‐World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), a cosponsored program executed by the World Health Organization (WHO), Switzerland.

  Provided funding for the review

Declarations of interest

Smisha Agarwal: The author was commissioned by the WHO to conduct this review. Tigest Tamrat: none known. Marita S Fønhus: none known. Nicholas Henschke: Since June 2016 I have been employed by Cochrane Response, an evidence services unit operated by the Cochrane Collaboration. Cochrane Response was contracted by the WHO to produce this review. Hanna Bergman: none known. Garrett L Mehl: owns stock in Apple Computer. Claire Glenton: none known. Simon Lewin: I am the Joint Co‐ordinating Editor for the Cochrane Effective Practice and Organisation of Care Review Group.

Notes

This protocol is based on standard text and guidance provided by Cochrane Effective Practice and Organisation of Care (EPOC).

New