Abstract.
Diarrhea remains a leading cause of morbidity and mortality in patients worldwide. The objective of this study was to determine the relative inter-rater reliability and usability of standard and Mobile health (mHealth)-supported World Health Organization (WHO) algorithms for dehydration assessment in patients with acute diarrhea in a rural, low-income country hospital. Two nurses blinded to each other’s examinations assessed dehydration status on patients soon after hospital arrival using either the standard WHO algorithm printed on a laminated card or an mHealth-supported WHO algorithm downloaded onto a smartphone. The assignment of assessment tool was based on odd or even enrollment date. The inter-rater reliability for dehydration assessment between the two nurses was calculated using Cohen’s K statistic for each study group. A total of 496 patients (< 5 years N = 349, > 5 years N = 147) were enrolled in the study; 132 (27%) had some or severe dehydration, and 364 (73%) had no dehydration on arrival. Cohen’s K statistic demonstrated greater reliability for the mHealth-supported dehydration assessment (0.59) compared with the standard assessment (0.50) in the overall population (P < 0.0001), as well as in the pediatric (0.43 versus 0.37, P < 0.0001) and adult (0.79 versus 0.57, P < 0.0001) populations individually. This is the first study to show that mHealth can improve the reliability of nursing dehydration assessment in patients with acute diarrhea and the first to report on the reliability of the WHO algorithm in adult patients specifically. Future studies should focus on the impact of mHealth-supported dehydration assessment on patient-centered outcomes and examine its reliability in different settings worldwide.
INTRODUCTION
Diarrhea remains a significant cause of morbidity and mortality in adult and pediatric populations worldwide.1 The estimated total number of acute cases of diarrheal disease in 2015 was 2.39 billion, an increase of 7% from 2005.2 Despite recent reductions in the deaths of children younger than 5 years globally, diarrheal disease still accounts for 9.2% of deaths among children younger than 5 years, with most of these deaths occurring in the first 2 years of life.3 Among adults, diarrheal disease causes 780,000 deaths each year and remains one of the leading causes of years of life lost worldwide.4 Rotavirus, Shigella, enterotoxigenic Escherichia coli, Cryptosporidium, Norovirus, and Vibrio cholerae are all major causes of diarrheal disease in pediatric and adult populations.5–7 In developing countries, Rotavirus accounts for 124 million outpatient visits and nine million hospital admissions in children younger than 5 years, whereas Norovirus accounts for $60.3 billion in health system costs and productivity losses across all age groups.8,9 The burden of diarrheal disease disproportionately affects low- and middle-income countries and vulnerable patients, such as the very young and very old.1,10,11 Furthermore, recent cholera outbreaks, such as those in Somalia, Ethiopia, South Sudan, and Yemen, have devastated communities already suffering from drought, conflict, and famine, highlighting the necessity of a robust and swift response to diarrheal disease in the context of humanitarian emergencies.12,13
The most important step in diarrhea management for both children and adults remains appropriate rehydration based on the severity of dehydration; yet rapid and accurate assessments of the degree of dehydration in patients with diarrhea remain challenging. Multiple studies have found that clinical impression, laboratory studies, and individual physical examination signs are not accurate or reliable indicators of dehydration in patients.14,15 A meta-analysis of 13 studies in children, for instance, found that individual clinical signs lack adequate sensitivity and specificity for prediction of dehydration.15 In adult populations, recent review articles have found the diagnostic utility of individual clinical signs and symptoms to be similarly limited.16,17 To address this diagnostic challenge, World Health Organization (WHO) guidelines recommend using a combination of physical examination findings to classify children and adults with diarrhea as having “no dehydration,” “some dehydration,” or “severe dehydration” (Figure 1).18 Previous research, however, found that the WHO algorithm has variable reliability in different pediatric populations, and it has never been assessed in adult populations.19,20 Mobile health (mHealth) technology, which has shown promise in standardizing and improving other types of clinical care in resource-limited settings, may be helpful in improving the reliability of the WHO algorithm for the assessment of dehydration in both children and adults with diarrhea.21
Figure 1.
(A) Pediatric World Health Organization (WHO) algorithm. (B) Adult WHO algorithm.
The number of mobile phone subscribers in developing countries increased from 1.2 billion in 2005 to 5.8 billion in 2016.22 The ubiquitous global penetration of mobile phones provides an ideal platform through which health care delivery in resource-limited settings can be improved. Interventions using mHealth technology have been used for appointment reminders, health promotion, data collection, decision support, and clinical management in low-income countries.21 A pilot study in rural Bangladesh demonstrated that a smartphone adaptation of the WHO diarrheal disease management guidelines improved adherence to guidelines.23 In addition, results of a study conducted in Chakaria, Bangladesh, demonstrated that community members, community leaders, and health care providers were similarly enthusiastic about the use of mHealth services and expressed willingness to use mHealth in the future.24 However, the use of mHealth technology to improve the clinical assessment of dehydration in adult and pediatric patients with diarrhea has not been examined previously.
The primary objective of our study is to investigate the reliability and usability of both a standard article-based and an mHealth-supported WHO algorithm for the assessment of dehydration in patients with acute diarrhea in a rural, low-income country setting. We are interested in determining whether the use of mHealth can improve both the reliability and usability of this important diagnostic tool. A secondary aim of the study is to investigate the reliability of the individual clinical signs of dehydration used in the WHO algorithm among children and adults with acute diarrhea.15,20
METHODS STUDY DESIGN
This was a prospective diagnostic study conducted among a random sample of children and adults with acute diarrhea presenting to the Matlab Hospital of the International Center for Diarrheal Disease Research, Bangladesh (icddr,b), between April 11 and June 24, 2017. All study interventions were performed for diagnostic purposes only and were not used to influence the clinical care provided to patients in the hospital. Ethical approval for the study was obtained from the Lifespan (Rhode Island Hospital) Institutional Review Board, the icddr,b Ethical Review Committee, and the icddr,b Research Review Committee.
Study setting and patient enrollment.
The icddr,b Matlab Hospital, located in the Matlab subdistrict of the Chandpur district of Bangladesh, served as the clinical enrollment site. The Matlab Hospital provides free clinical care to a largely rural population of 2.5 million individuals, with a strong commitment to advancing care through research. Each year, about 30,000 adults and children with diarrhea receive care in the hospital’s specially designed rehydration ward for the management of diarrheal disease.25 All adults and children with acute diarrhea presenting to the Matlab Hospital during study hours were eligible for enrollment. Children and adults were selected for screening on arrival to the hospital 8 hours per day (09:00–17:00) and 7 days per week by pulling yellow (selected) or white (not selected) marbles from a blind pouch. Patients were selected in this manner to avoid overwhelming study staff with a large number of enrollments in a short time period, which might degrade data quality. Once selected, patients were assessed for predefined exclusion criteria: less than three loose stools in the past 24 hours; chronic diarrhea of 14 days or more; presence of a clear alternative explanation for their illness on arrival other than gastroenteritis; or previous enrollment in the same study. For non-excluded patients, research staff approached the patient’s parent/guardian if the patient was less than 18 years, explained the risks and benefits of study participation, and obtained written consent in the local language of Bengali. For patients aged 18 years and older, staff explained the risks and benefits of the study directly to the patient (or their family member if they were incapacitated) in the local language of Bengali and obtained written consent. If a patient, parent, or guardian was unable to provide written consent because of limited literacy, verbal consent was obtained with a thumbprint used to mark the consent form.
Staff training and supervision.
Before beginning patient enrollment, study staff were trained in all study procedures, including obtaining informed consent, use of all study equipment, and data entry and storage. Once study enrollment commenced, two research physicians supervised all study procedures to ensure the quality of data collection and protocol adherence. All clinical nurses performing examinations were familiar with the WHO algorithm for the assessment of dehydration because it is taught as part of the national nursing curriculum in Bangladesh. Before the start of the study, we conducted a brief, 1-hour training for all 14 nursing staff at the Matlab Hospital, which reviewed the application of the WHO algorithm for assessing dehydration in both children and adults.
Software development.
The WHO decision-support algorithm was adapted to a smartphone interface in collaboration with an independent software development firm Beehyv Software Solutions Pvt. Ltd. (Telangana, India) (http://www.beehyv.com/). The design of the user interface was based on previous experience, end user input, and study requirements.23 With respect to the end user, the design requirements were accuracy, simplicity, and a pleasurable user experience that expedited assessment. The interface is coded to provide color-based nudges to notify the user of fields that were not entered. The software does not capture personal health information, but to monitor performance, it does record and encrypt basic information (dehydration assessment variables, date, time, and location). The software is Android dependent, functions both online and offline, and is freely available on the Google Play Store with a keyword search of “Dehydration Assessment.” Although the software may be used on most Android devices, Samsung Note 3 (Samsung Electronics, Ridgefield Park, NJ) devices (200–400 USD) were used in this study because of their screen size (5 in), responsive display, durability, and long battery life (Figure 2).
Figure 2.
Screenshots of World Health Organization (WHO) algorithm mobile application. This figure appears in color at www.ajtmh.org.
Methods of assessment.
Reliability assessment.
After providing informed consent, patients were assessed clinically using one of the two methods: standard assessment of dehydration using the WHO algorithm printed on a laminated card or mHealth-supported dehydration assessment using the mobile application described previously. Patients who presented to the Matlab Hospital on odd days were assigned to the mHealth dehydration assessment group, whereas patients presenting on even days were assigned to the standard dehydration assessment group. The patients were assessed clinically in rapid succession by two different nurses for their degree of dehydration. The two nurses used the same dehydration diagnostic tool but were blinded to each other’s examinations and findings. Study staff recorded the time required for the nurse to complete their dehydration assessment using a stopwatch, starting with the beginning of their clinical examination and ending with their final determination of dehydration status (none, some, or severe).
Usability assessment.
After conducting dehydration assessments using the assigned clinical tool, study staff asked the clinical nurse to answer a series of questions regarding the usability of the tool. We measured usability using three different 10-point Likert scales that measured 1) the nurse’s perception of the difficulty in performing the dehydration assessment, 2) the nurse’s confidence in their dehydration assessment, and 3) whether the nurse would want to use the same tool again in actual clinical practice if given the option.
Demographic and anthropometric assessments.
Finally, research staff obtained basic historical and demographic data for each study subject including their age, sex, duration of diarrhea, frequency, and quality of stools. Malnutrition was assessed for all children and adults enrolled by measuring the mid-upper arm circumference (MUAC) to the nearest millimeter using a standard MUAC tape. In children younger than 5 years, an MUAC of < 125 mm was considered indicative of acute malnutrition.26 In patients older than 5 years, an MUAC of < 220 or < 230 mm was considered indicative of acute malnutrition in women and men, respectively.27
After study assessments were completed, all patients were managed clinically based on the discretion of the treating physician at the Matlab Hospital. The dehydration assessments performed as part of this study were not used to influence patient care in any way. For patients with severe illness, all study assessments were conducted concurrent with their initial management, so as not to delay any immediately necessary care. Figure 3 provides a flow diagram for all study procedures.
Figure 3.
Flowchart of study procedures.
Sample size calculation.
Using the algorithms developed by Walter et al.,28 we calculated that we would need at least 117 patients in each study arm to ensure 95% confidence intervals of less than 0.1 around our estimates of inter-rater reliability (as measured by Cohen’s K statistic), which we felt would represent a clinically meaningful difference between the groups. To allow for comparisons between groups (standard versus mHealth and patients younger than 5 years versus patients older than 5 years), we required at least 117 patients in each of the four groups or 468 total patients. We planned to enroll an additional 5% (23 patients) to allow for missing data.
Data analysis.
Baseline historical, demographic, and anthropometric data were summarized for all children and adults included in our study using frequencies with percentages for categorical variables and medians with interquartile ranges (IQRs) for continuous variables. Differences between variables were determined using χ2 tests for categorical variables and independent sample t-tests for continuous variables; Mann–Whitney U tests were used for non-normally distributed samples. A standard P value cutoff of less than 0.05 was used for determining statistical significance in all analyses.
We calculated the inter-rater reliability of the WHO algorithm by comparing the two nursing assessments of dehydration in our overall study population and for each of our subpopulations using overall percent agreement and the Cohen’s K statistic. Because of the small number of patients presenting with severe dehydration in our study, the categories of some and severe dehydration were combined together into one dehydration category for all reliability analyses. We used bootstrapping (selection with replacement) with 1,000 replicates to calculate a P value for comparison of the Cohen’s K statistic between the standard and mHealth groups.
We also analyzed the usability of the standard and mHealth versions of the WHO algorithms; this was reported using medians with associated IQRs. Differences in nurse-reported usability between the standard and mHealth versions of the WHO algorithm were determined using the Mann–Whitney U test. All analyses were conducted using Stata version 14 (StataCorp, College Station, TX) or R version 3.3.3 (R Development Core Team, Vienna, Austria).
RESULTS
Baseline data.
Overall, 824 patients presenting during study hours were randomly selected for screening, of whom 531 were eligible for enrollment and 496 provided informed consent (Figure 4). A total of 253 patients (51%) were assigned to the mHealth and 243 patients (49%) were assigned to the standard study arms. The median age for enrolled patients was 1.25 years (range 3 months–89 years), and 349 (70%) were younger than 5 years. Among all enrolled patients, 294 were males (49%) and 201 were females (41%). Ten patients (2%) reported bloody diarrhea, 149 (30%) rice water diarrhea (indicative of cholera), and 337 (68%) watery (but not rice water) diarrhea. Patients had a median of 3 days (IQR: 2–4) of diarrhea before presentation and eight episodes (IQR: 6–10) of diarrhea in the 24 hours before presentation. Based on their MUAC, 58 (18%) patients younger than 5 years and 55 (37%) patients older than 5 years had acute malnutrition on presentation. Among all enrolled patients, 132 (27%) had some or severe dehydration and 364 (73%) had no dehydration based on the first clinical examination (Table 1).
Figure 4.
Study flow diagram of patient enrollment.
Table 1.
Baseline characteristics of enrolled patients
| Overall | Younger than five | Older than five | |
|---|---|---|---|
| (N = 496) | (N = 349) | (N = 147) | |
| Study arm | |||
| Standard, no. (%) | 243 (49) | 170 (49) | 73 (50) |
| Mobile health, no. (%) | 253 (51) | 179 (51) | 74 (50) |
| Age, median (IQR) | – | 11 months (8–17) | 35 years (25–55) |
| Sex | |||
| Female, no. (%) | 201 (41) | 120 (34) | 81 (55) |
| Male, no. (%) | 294 (59) | 229 (66) | 65 (45) |
| Diarrhea type | |||
| Watery, no. (%) | 337 (68) | 236 (68) | 101 (69) |
| Rice water, no. (%) | 149 (30) | 106 (30) | 43 (29) |
| Bloody, no. (%) | 10 (2) | 7 (2) | 3 (2) |
| Days of diarrhea, median (IQR) | 3 (2–4) | 3 (2–5) | 2 (1–3) |
| Episodes of diarrhea, median (IQR) | 8 (6–10) | 8 (5–10) | 9 (7–13) |
| Nutritional status by age group | |||
| MUAC, median (IQR) | – | 132 (125–140) | 235 (211–260) |
| MUAC category by age group | |||
| No acute malnutrition, no. (%) | – | 258 (82) | 92 (63) |
| Acute malnutrition, no. (%) | – | 58 (18) | 55 (37) |
| Dehydration status | |||
| No dehydration, no. (%) | 364 (73) | 274 (79) | 90 (61) |
| Some dehydration, no. (%) | 105 (21) | 68 (19) | 37 (25) |
| Severe dehydration, no. (%) | 27 (5) | 7 (2) | 20 (14) |
IQR = interquartile range; MUAC = mid-upper arm circumference.
Of the 349 patients younger than 5 years, 170 (49%) and 179 (51%) were assigned to the standard and mHealth study arms, respectively. Of the 147 patients older than 5 years, 73 (50%) and 74 (50%) were assigned to the standard and mHealth study arms, respectively (Table 1). Analysis of baseline characteristics by the assigned study arm showed no significant differences in age, sex, malnutrition status, diarrhea type, days of diarrhea, episodes of diarrhea, or degree of dehydration between groups (Table 2).
Table 2.
Assessment of baseline characteristics and study procedures by study arm
| Mobile health, no. (%) | Standard, no. (%) | P value | |
|---|---|---|---|
| Age group | 0.847 | ||
| Younger than 5 years | 179 (51) | 170 (49) | |
| Older than 5 years | 74 (50) | 73 (50) | |
| Sex | 0.435 | ||
| Female | 107 (53) | 94 (47) | |
| Male | 146 (50) | 148 (50) | |
| Diarrhea type | 0.978 | ||
| Watery | 173 (51) | 164 (49) | |
| Rice water | 75 (50) | 74 (50) | |
| Bloody | 5 (50) | 5 (50) | |
| Days of diarrhea | 253 (51) | 243 (49) | 0.810 |
| Episodes of diarrhea | 253 (51) | 243 (49) | 0.925 |
| Time between examinations | 253 (51) | 243 (49) | 0.776 |
| MUAC category | 0.235 | ||
| No acute malnutrition | 129 (50) | 129 (50) | |
| Acute malnutrition | 24 (41) | 34 (59) | |
| Dehydration status | 0.795 | ||
| No dehydration | 189 (52) | 175 (48) | |
| Some dehydration | 51 (49) | 54 (51) | |
| Severe dehydration | 13 (48) | 14 (52) |
MUAC = mid-upper arm circumference.
Reliability.
All patients had two clinical assessments for dehydration performed immediately after enrollment by clinical nurses who were blinded to each other’s examinations. Years of experience for nurses participating in this study ranged from 1 to 34 years, with half of the nurses having less than 5 years of experience. The median time between the two nursing examinations was 6 minutes (IQR: 3–10) overall, and there were no significant differences by study arm (P = 0.776) (Table 2). The inter-rater reliability of the two nursing dehydration assessments, as measured by Cohen’s K statistic, was significantly higher in the mHealth study arm as compared with the standard study arm (0.59 versus 0.50, P < 0.0001). Both the overall inter-rater reliability of the clinical assessments and the absolute difference in inter-rater reliability between study arms were greater for adults than for children (Table 3). Among patients older than 5 years, the inter-rater reliability was 0.79 for the mHealth study arm and 0.57 for the standard study arm (P < 0.0001). Among patients younger than 5 years, the inter-rater reliability was 0.43 for the mHealth study arm and 0.37 for the standard study arm, although even this smaller difference was still statistically significant (P < 0.0001).
Table 3.
Reliability of clinical assessment of dehydration status by study arm
| Observed agreement | Cohen’s K (SE) | P value | |
|---|---|---|---|
| Study arm (total) | < 0.001 | ||
| Standard | 82% | 0.50 (0.06) | |
| Mobile health | 87% | 0.59 (0.06) | |
| Study arm (younger than five) | < 0.001 | ||
| Standard | 84% | 0.37 (0.06) | |
| Mobile health | 85% | 0.43 (0.07) | |
| Study arm (older than five) | < 0.001 | ||
| Standard | 79% | 0.57 (0.12) | |
| Mobile health | 91% | 0.79 (0.11) |
Table 4 demonstrates the inter-rater reliability for the four individual components of the WHO algorithm both for study patients overall and by age group. Among patients younger than 5 years, general appearance and skin pinch were the most reliable findings, whereas sunken eyes and thirst were the least reliable. Among patients older than 5 years, all signs had greater reliability, although sunken eyes remained relatively less reliable than other signs.
Table 4.
Reliability of individual clinical signs of dehydration
| Overall | Younger than 5 years | Older than 5 years | |
|---|---|---|---|
| General appearance | |||
| Observed agreement | 96% | 98% | 93% |
| Cohen’s K (SE) | 0.58 (0.04) | 0.43 (0.05) | 0.64 (0.08) |
| Skin pinch | |||
| Observed agreement | 98% | 96% | 95% |
| Cohen’s K (SE) | 0.67 (0.04) | 0.53 (0.05) | 0.68 (0.08) |
| Sunken eyes | |||
| Observed agreement | 74% | 75% | 72% |
| Cohen’s K (SE) | 0.43 (0.04) | 0.35 (0.04) | 0.47 (0.08) |
| Thirst | |||
| Observed agreement | 90% | 88% | 95% |
| Cohen’s K (SE) | 0.40 (0.04) | 0.27 (0.05) | 0.70 (0.08) |
Usability.
Table 5 demonstrates measures of usability by study arm. Overall, there were no significant differences in time to complete the dehydration assessment, self-reported difficulty in performing the assessment, self-reported confidence in dehydration assessment, or self-reported interest in using the same dehydration assessment tool again by study arm. The median time to complete the assessment was about 1 minute regardless of the study arm. The median difficulty assessment was two on a scale of 1–10 (with 10 being very difficult); the median confidence in assessment was nine on a scale of 1–10 (with 10 being very confident); and the median interest in using the same tool again was nine on a scale of 1–10 (with 10 being very interested).
Table 5.
Usability of study tool by study arm
| Standard | Mobile health | P value | |
|---|---|---|---|
| Dehydration assessment time in minutes, median (IQR) | |||
| Overall | 1 (1–1) | 1 (1–2) | 0.751 |
| Younger than five | 1 (1–1) | 1 (1–1) | 0.854 |
| Older than five | 1 (1–2) | 1 (1–2) | 0.763 |
| Difficulty of use, median (IQR)* | |||
| Overall | 2 (1–2) | 2 (1–2) | 0.372 |
| Younger than five | 2 (1–2) | 2 (1–2) | 0.180 |
| Older than five | 1 (1–1) | 1 (1–1) | 0.878 |
| Confidence in assessment, median (IQR)* | |||
| Overall | 9 (9–10) | 9 (9–10) | 0.999 |
| Younger than five | 9 (8–10) | 9 (8–10) | 0.766 |
| Older than five | 10 (9–10) | 10 (10–10) | 0.996 |
| Would use again, median (IQR)* | |||
| Overall | 9 (9–10) | 9 (9–10) | 0.736 |
| Younger than five | 9 (8–10) | 9 (8–10) | 0.987 |
| Older than five | 10 (9–10) | 10 (9–10) | 0.867 |
IQR = interquartile range.
As self-reported by nurses on a Likert scale of 1–10, with 10 being the greatest on each measure.
DISCUSSION
To our knowledge, this is the first study to examine the inter-rater reliability and usability of the standard article-based WHO algorithm as compared with an mHealth-supported WHO algorithm for dehydration assessment among patients with acute diarrhea. It is also the first study to measure the inter-rater reliability of the WHO algorithm among adults with acute diarrhea. Initial accurate assessment of dehydration in children and adults suffering from diarrhea remains a diagnostic challenge. Prompt recognition of the severity of dehydration determines resuscitation efforts, as well as patient disposition and the level of health care resource utilization. Those with severe dehydration will benefit from early initiation of intravenous fluids, whereas those with mild to moderate dehydration can be successfully treated with oral rehydration therapy and those without any dehydration require no medical intervention at all, leading to shorter hospital stays and outpatient treatment.29,30 Thus, underestimating or overestimating the severity of dehydration can lead to adverse outcomes for patients either in the form of inadequate resuscitation or unnecessarily invasive and costly interventions, respectively.
Increased prevalence of mobile technologies creates an ideal opportunity to improve multiple aspects of health care in resource-limited settings from patient outreach to disease management. Recent studies using mobile technology have documented robust adherence to appropriate treatment of childhood cases of diarrhea, as well as medication and IV fluid recommendations in management of diarrhea in rural settings.23,31 Our study investigates whether an mHealth application can improve the inter-rater reliability and usability of a clinical dehydration diagnostic tool among adults and pediatric patients with acute diarrhea.
We chose the WHO algorithm because it continues to serve as the standard of care for dehydration assessment in most low- and middle-income countries, despite recent research showing its variable or limited accuracy and reliability in some pediatric populations.19,20,32 Overall, the mHealth version of the WHO algorithm had better inter-rater reliability for the assessment of dehydration than the standard article-based version when used by nurses to evaluate both children and adults with acute diarrhea in a rural, low-income country setting. Our study was not designed to determine why the mHealth-supported assessment had greater inter-rater reliability, although there are several possibilities based on previous research.20 The WHO algorithm has some confusing components, such as the presence of sunken eyes in both the some dehydration and severe dehydration categories. In addition, if patients have one severe dehydration sign and one some dehydration sign, the provider might be confused as to which category to assign them. The mHealth application removes this confusion by simply asking clinicians to click their findings for each of the four examination signs and then determines the dehydration category for them based on the WHO algorithm.
Greater reliability of dehydration assessment may lead to more appropriate and consistent dehydration management in practice, which could both reduce health care costs while also reducing adverse events in patients presenting with acute diarrhea, one of the most common and deadly conditions in children and adults worldwide. Our results also found that the inter-rater reliability for the WHO algorithm seems to be higher overall in adults than in children, regardless of the study arm. This is not unexpected, given the greater challenge of assessing young children, who may not be willing or able to cooperate fully with the clinical examination or answer directed questions. Overall, use of the mHealth technology did not increase the time required for nurses to perform their clinical assessment of dehydration, which was relatively rapid in the vast majority of cases, regardless of the study arm. Despite this being a population of nurses without previous exposure to mHealth in a clinical setting, the nurses did not perceive the mHealth tool to be more difficult to use than the standard article-based WHO algorithm and were as likely to be interested in using it again in clinical practice. Although the nurses’ previous exposure to the WHO algorithm and experience with assessment of dehydration could impact our assessment by demonstrating higher inter-rater reliability than that which would have been otherwise achieved, the years of experience of our nurses varied from 1 to 34 years. Half of our of nurses were recent graduates with less than 5 years of clinical experience, and thus, we are confident that the mHealth-supported WHO algorithm would still perform better and achieve higher inter-rater reliability than the standard article-based WHO algorithm in settings with less-experienced nurses.
With respect to the reliability of individual clinical dehydration signs, general appearance and skin pinch had the greatest inter-rater reliability for those younger than 5 years, similar to previous studies in pediatric populations.15,20 Thirst had the poorest reliability of any dehydration measures for patients younger than 5 years, as has been reported in previous research.20,33 Of note, however, it was a much more reliable sign of dehydration among patients older than 5 years in our study. As previous research has articulated, thirst can be an especially subjective measure of dehydration in children because it is difficult to tell the difference between children who are drinking eagerly or drinking normally and between children who are not drinking because they are very dehydrated or not drinking because they have no dehydration and are not thirsty.20,33 In adults, it may be easier for clinicians to tease out their level of thirst, leading to greater inter-rater reliability of this examination measure. Both general appearance and skin pinch had good inter-rater reliability for patients older than 5 years, indicating their clinical usefulness as part of the initial adult dehydration assessment.
Limitations.
The findings of this study must be viewed in the context of the limitations of the study design and available data. Just under one-third of the overall study population had rice water diarrhea, suggesting that they may have had cholera, whose pathogenesis is characterized by a rapid onset of diarrhea. As this is similar to the proportion of diarrhea due to cholera in many resource-limited settings around the globe, we do not expect it to jeopardize the generalizability of our results. The study was conducted at a single rural hospital in Bangladesh, with all clinical dehydration examiantions performed by nurses working at this site. Thus, mHealth may not perform as well compared with standard care when used in other settings, and additional studies using diverse study sites are needed. Our study assumes access to mHealth technology (such as a smartphone), although the tool itself does not rely on Wi-Fi or an internet connection for use. Our study used local, general practice nurses without substantial previous training in mHealth technology to perform all assessments but enough medical English knowledge to recognize the clinical words on the mobile phone application. Further research is necessary to assess the reliability of mHealth-supported dehydration assessment as compared with standard dehydration assessment by community health workers and other provider types. The scales used to assess usability of the dehydration assessment tools were not previously validated and may not be sensitive enough to detect small differences between groups.
Overall, less than 5% of patients enrolled in our study had severe dehydration, including about 14% of patients older than 5 years and 2% of patients younger than 5 years This relatively small percentage of patients with severe dehydration is typical of patients with acute diarrhea worldwide because previous research has found that only about 2–3% of diarrhea cases in children younger than 5 years progress to severe disease.34 However, the small numbers of patients with severe diarrhea necessitated combining the some and severe dehydration categories for all of our inter-rater reliability analyses. Overall, we enrolled more children than adults, although again this age distribution among patients presenting with diarrheal disease is common globally.1 Finally, this study focused only on dehydration assessment; whether the improved reliability of dehydration assessment using mHealth technology translates to a clinically meaningful improvement in patient outcomes should be examined in future studies.
CONCLUSION
This is the first study to affirmatively demonstrate that the use of mHealth technology improves the reliability of dehydration assessments conducted by nurses on patients presenting with acute diarrhea. Future studies should focus on the impact of the use of mHealth technology for assessing dehydration in patients with acute diarrhea by various types of providers on patient-centered outcomes and overall health care costs in a variety of clinical settings worldwide. Specifically, further research will be necessary to assess both the accuracy and reliability of the mHealth dehydration assessment tool among less-experienced nurses. In addition, future research studies should be geared toward examining the accuracy and reliability of the mHealth tool in a larger cohort of patients with severe dehydration, with subgroup analyses based on diarrhea etiology. If proved to be accurate and reliable in these settings, mHealth may help further standardize clinical care for patients in resource-limited settings while potentially reducing both overutilization and underutilization of scarce health care resources, such as intravenous fluids and hospital admissions.
Disclaimer: There is no intellectual property associated with the dehydration assessment application collaboratively developed for this study.
REFERENCES
- 1.GBD 2015 Mortality and Causes of Death Collaborators , 2016. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 388: 1459–1544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.GBD 2015 Disease and Injury Incidence and Prevalence Collaborators , 2016. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 388: 1545–1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Liu LOS, Hogan D, Perin J, Rudan I, Lawn JE, Cousens S, Mathers C, Black RE, 2015. Global, regional, and national causes of child mortality in 2000–13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet 385: 430–440. [DOI] [PubMed] [Google Scholar]
- 4.GBD 2013 Mortality and Causes of Death Collaborators , 2015. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet 385: 117–171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Fischer Walker CL, Sack D, Black RE, 2010. Etiology of diarrhea in older children, adolescents and adults: a systematic review. PLoS Negl Trop Dis 4: e768. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lamberti LM, Bourgeois AL, Fischer Walker CL, Black RE, Sack D, 2014. Estimating diarrheal illness and deaths attributable to Shigellae and enterotoxigenic Escherichia coli among older children, adolescents, and adults in south Asia and Africa. PLoS Negl Trop Dis 8: e2705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kotloff KLN, et al. 2013. Burden and aetiology of diarrhoeal disease in infants and young children in developing countries (the Global Enteric Multicenter Study, GEMS): a prospective, case-control study. Lancet 382: 209–222. [DOI] [PubMed] [Google Scholar]
- 8.Parashar UD, Hummelman EG, Bresee JS, Miller MA, Glass RI, 2003. Global illness and deaths caused by rotavirus disease in children. Emerg Infect Dis 9: 565–572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Bartsch SM, Lopman BA, Ozawa S, Hall AJ, Lee BY, 2016. Global economic burden of norovirus gastroenteritis. PLoS One 11: e0151219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Truninger K, 2014. Diarrhea in the elderly. Ther Umsch 71: 545–550. [DOI] [PubMed] [Google Scholar]
- 11.Fischer Walker CL, Perin J, Aryee MJ, Boschi-Pinto C, Black RE, 2012. Diarrhea incidence in low- and middle-income countries in 1990 and 2010: a systematic review. BMC Public Health 12: 220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Green A, 2017. Cholera outbreak in the horn of Africa. Lancet 389: 2179. [DOI] [PubMed] [Google Scholar]
- 13.Parker LA, et al. 2017. Adapting to the global shortage of cholera vaccines: targeted single dose cholera vaccine in response to an outbreak in South Sudan. Lancet Infect Dis 17: e123–e127. [DOI] [PubMed] [Google Scholar]
- 14.Freedman SB, Vandermeer B, Milne A, Hartling L; Pediatric Emergency Research Canada Gastroenteritis Study Group , 2015. Diagnosing clinically significant dehydration in children with acute gastroenteritis using noninvasive methods: a meta-analysis. J Pediatr 166: 908–916.e1-6. [DOI] [PubMed] [Google Scholar]
- 15.Steiner MJ, DeWalt DA, Byerley JS, 2004. Is this child dehydrated? JAMA 291: 2746–2754. [DOI] [PubMed] [Google Scholar]
- 16.Hooper L, et al. 2015. Clinical symptoms, signs and tests for identification of impending and current water-loss dehydration in older people. Cochrane Database Syst Rev 4: CD009647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Armstrong LE, Kavouras SA, Walsh NP, Roberts WO, 2016. Diagnosing dehydration? Blend evidence with clinical observations. Curr Opin Clin Nutr Metab Care 19: 434–438. [DOI] [PubMed] [Google Scholar]
- 18.World Health Organization , 2005. The Treatment of Diarrhoea: A Manual for Physicians and Other Senior Health Workers. Geneva, Switzerland: World Health Organization.
- 19.Pringle K, Shah SP, Umulisa I, Mark Munyaneza RB, Dushimiyimana JM, Stegmann K, Musavuli J, Ngabitsinze P, Stulac S, Levine AC, 2011. Comparing the accuracy of the three popular clinical dehydration scales in children with diarrhea. Int J Emerg Med 4: 58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Levine AC, Glavis-Bloom J, Modi P, Nasrin S, Atika B, Rege S, Robertson S, Schmid CH, Alam NH, 2016. External validation of the DHAKA score and comparison with the current IMCI algorithm for the assessment of dehydration in children with diarrhoea: a prospective cohort study. Lancet Glob Health 4: e744–e751. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Colaci D, Chaudhri S, Vasan A, 2016. mHealth interventions in low-income countries to address maternal health: a systematic review. Ann Glob Health 82: 922–935. [DOI] [PubMed] [Google Scholar]
- 22.ICT Data and Statistics Division, International Telecommunication Union, 2016. ICT Facts and Figures 2016. Available at: http://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx. Accessed June 8, 2017.
- 23.Haque F, et al. 2017. Evaluation of a smartphone decision-support tool for diarrheal disease management in a resource-limited setting. PLoS Negl Trop Dis 11: e0005290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Khatun F, Heywood AE, Ray PK, Bhuiya A, Liaw ST, 2016. Community readiness for adopting mHealth in rural Bangladesh: a qualitative exploration. Int J Med Inform 93: 49–56. [DOI] [PubMed] [Google Scholar]
- 25.ICDDR B , 2016. Health and Demographic Surveillance System-Matlab. Registration of Health and Demographic Events 2014. Scientific Report No. 133 Dhaka, 49.
- 26.Modi P, Nasrin S, Hawes M, Glavis-Bloom J, Alam NH, Hossain MI, Levine AC, 2015. Midupper arm circumference outperforms weight-based measures of nutritional status in children with diarrhea. J Nutr 145: 1582–1587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.James WP, Mascie-Taylor GC, Norgan NG, Bistrian BR, Shetty PS, Ferro-Luzzi A, 1994. The value of arm circumference measurements in assessing chronic energy deficiency in Third World adults. Eur J Clin Nutr 48: 883–894. [PubMed] [Google Scholar]
- 28.Walter SD, Eliasziw M, Donner A, 1998. Sample size and optimal designs for reliability studies. Stat Med 17: 101–110. [DOI] [PubMed] [Google Scholar]
- 29.Diggins KC, 2008. Treatment of mild to moderate dehydration in children with oral rehydration therapy. J Am Acad Nurse Pract 20: 402–406. [DOI] [PubMed] [Google Scholar]
- 30.Fonseca BK, Holdgate A, Craig JC, 2004. Enteral vs intravenous rehydration therapy for children with gastroenteritis: a meta-analysis of randomized controlled trials. Arch Pediatr Adolesc Med 158: 483–490. [DOI] [PubMed] [Google Scholar]
- 31.Kabakyenga J, et al. 2016. A demonstration of mobile phone deployment to support the treatment of acutely ill children under five in Bushenyi district, Uganda. Afr Health Sci 16: 89–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Jauregui J, Nelson D, Choo E, Stearns B, Levine AC, Liebmann O, Shah SP, 2014. External validation and comparison of three pediatric clinical dehydration scales. PLoS One 9: e95739. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Levine AC, Glavis-Bloom J, Modi P, Nasrin S, Rege S, Chu C, Schmid CH, Alam NH, 2015. Empirically derived dehydration scoring and decision tree models for children with diarrhea: assessment and internal validation in a prospective cohort study in Dhaka, Bangladesh. Glob Health Sci Pract 3: 405–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Walker CL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, O’Brien KL, Campbell H, Black RE, 2013. Global burden of childhood pneumonia and diarrhoea. Lancet 381: 1405–1416. [DOI] [PMC free article] [PubMed] [Google Scholar]