Abstract
Background
Over two-thirds of the five million annual deaths in children under five occur in infants, mostly in developing countries and many after hospital discharge. However, there is a lack of understanding of which children are at higher risk based on early clinical predictors. Early identification of vulnerable infants at high-risk for death post-discharge is important in order to craft interventional programs.
Objectives
To determine potential predictor variables for post-discharge mortality in infants less than one year of age who are likely to die after discharge from health facilities in the developing world.
Methods
A two-round modified Delphi process was conducted, wherein a panel of experts evaluated variables selected from a systematic literature review. Variables were evaluated based on (1) predictive value, (2) measurement reliability, (3) availability, and (4) applicability in low-resource settings.
Results
In the first round, 18 experts evaluated 37 candidate variables and suggested 26 additional variables. Twenty-seven variables derived from those suggested in the first round were evaluated by 17 experts during the second round. A final total of 55 candidate variables were retained.
Conclusion
A systematic approach yielded 55 candidate predictor variables to use in devising predictive models for post-discharge mortality in infants in a low-resource setting.
Keywords: Candidate predictor variables, pediatrics, neonatal, infants, prediction, post-discharge mortality, sepsis
Introduction
Two-thirds of the five million children under five years old who die every year are under the age of one.1 Most of these deaths occur in low and middle-income countries resulting from preventable infectious causes.2,3 Thus, reducing the under-five mortality rate to less than 25 per 1000 live births by the year 2030 as targeted by the United Nations Sustainable Development Goals (SDG) relies on addressing this issue.4
Mortality rates in the months following discharge are often equal to, or greater than during hospitalization.5 Despite this burden, few studies have explored health seeking behavior and mortality or evaluated interventions in children following hospital discharge; none have implemented interventions to improve post-discharge outcomes. One proposed solution to improve post-discharge outcomes is through a precision health approach, whereby vulnerable children in resource-poor countries are identified prior to discharge, ideally at time of admission, through the use of prediction models.6 Using this approach, health systems are better able to deploy scarce resources and life-saving interventions to those most likely to benefit.
Optimal predictive models to inform health systems should be based on candidate predictor variables most likely to be associated with the outcome of interest.7,9. The Delphi process is a well-recognized process that involves the solicitation of a panel of experts through two or more rounds of structured questionnaires, combined with review and modification by the research team.8 The goal is to acquire expert opinions from those outside the primary research team to optimize input and identify context appropriate variables for evaluation in predictor models.
A similar modified Delphi approach was used previously to determine candidate predictors for post-discharge mortality in the 6 months to 5 years age group, which led to a model derivation study and the subsequent development of the Smart Discharge intervention.9–11 However, models currently in use may not be applicable nor optimal for infants less than one year of age due to differences in disease etiologies, physiology, vital signs, and presenting signs and symptoms. Using a similar approach to the prior variable selection process, the purpose of this project was to generate a comprehensive list of candidate predictor variables for infants less than one year of age. The variables identified will be used to derive prediction models for post-discharge mortality in newborns and young infants and will ultimately be included in an expanded Smart Discharges program for Uganda.
Methods
Design
A modified, two-round, Delphi process was performed to determine a potential set of candidate predictor variables for post-discharge mortality in infants less than one year of age. The inability to modify individualized responses based on the aggregate response was the modification to the standard Delphi process. Research Ethics Board approval was obtained from the University of British Columbia.
Participants
Participants were selected based on internal discussions by the primary research team. The desired expertise of participants included pediatrics, sepsis/infectious diseases, microbiology/laboratory medicine, global health, epidemiology, social sciences, neonatology and obstetrics. The target sample size was 25 individuals covering all areas of expertise, and to include multiple participants from the proposed research country, Uganda. The Delphi process does not include a required sample size or formal sample size calculation. However, it has been suggested that between 10 and 50 participants may be ideal, although no consensus has been reached between authors and studies utilizing the methodology.12 At the beginning of each round of the survey, experts were asked to self-identify their area of expertise, role, and affiliation (Table 1).
Table 1.
Respondent demographics*
| Primary affiliations | Number of respondents (18) |
| Teaching hospital | 11 |
| General/community hospital | 2 |
| Pediatric hospital | 4 |
| Outpatient clinic | 1 |
| University | 9 |
| Other | 2 (research institute) |
| Roles at the institutions | |
| Physician | 12 |
| Nurse | 2 |
| Other clinician | 0 |
| Clinical scientist | 6 |
| Social scientist | 0 |
| Hospital administration | 1 |
| Epidemiologist | 5 |
| Other | |
| Other areas of expertise | |
| Pediatrics | 14 |
| Infectious disease | 6 |
| Microbiology/laboratory medicine | 0 |
| Global health | 9 |
| Epidemiology | 4 |
| Social sciences | 0 |
| Neonatology | 7 |
| Obstetrics | 2 |
| Other areas identified: public health, pediatric pulmonology, critical care, internal medicine and surgery, informatics, implementation science, tropical oncology, immunology, emergency | |
Results based on respondent self-identification. Respondents were able to select all categories that applied; therefore, many identified more than one area.
Process
The two-round modified Delphi process was conducted between August and November 2017 through the use of emailed surveys using Research Electronic Data Capture (REDCap).13 The participants were given 14 days to respond for each round of the process. After each round, the primary research team determined whether or not existing or suggested variables should be added, modified or removed, based on survey responses and the research team's knowledge of the study setting, Uganda. A final list of candidate variables was compiled by the research team following the results of the second survey. Since there was no direct interaction between participants, this was considered a modified Delphi process. However, during the second round of the survey, participants reviewed and critiqued the variables proposed in the first round. The first round further included general questions on expert-perceived rates and importance of post-discharge mortality as a public health issue in resource-poor countries (Table 2).
Table 2.
Participant perception of post-discharge mortality rates*
| Post-discharge mortality rates | ||||
| >10% | 5–10% | 2–5% | <2% | |
| Children age 0–1 months | 39% | 50% | 11% | 0 |
| Children age 1–12 months | 17% | 44% | 39% | 0 |
within the first six months following discharge in children admitted with an infectious illness in resource-limited countries. Responses gathered pre-survey.
Round 1
An initial list of 37 candidate variables was generated by the research team through a systematic review of existing literature as well as the clinical experiences of the co-investigators (Table 3). This list included multi-part variables (e.g. anthropometric variables included mean upper-arm circumference (MUAC), weight and height, and associated z-scores). Experts were requested to evaluate the initial list of candidate variables based on (1) predictive value (2) measurement reliability (3) availability (4) applicability in low-resource settings. Those variables rated during the first round and selected as suitable for predicting risk in the specified population and setting (Uganda) were incorporated into the final list of candidate variables (Table 5).
Table 3.
Round 1 surveyed variables (N=37)
| Clinical* | Birth | Laboratory* | Social/Demographic |
| 1. Temperature 2. Respiratory rate 3. Oxygen saturation (SpO2) 4. Age 5. Dehydration (using WHO dehydration scale) 6. Central cyanosis 7. Anthropometrics (weight, height, MUAC) 8. Chest indrawing 9. Bulging fontanel 10. Feeding status 11. Jaundice 12. Multiple associated infectious symptoms (e.g. pneumonia + diarrhea/sepsis/UTI) 13. Grunting 14. Diarrhea 15. Convulsions 16. Abdominal distension 17. Hepatomegaly 18. Capillary refill 19. Coma score (i.e. Blantyre Coma Scale) 20. Other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.) |
1. Birth weight 2. Location of birth (home vs facility) 3. Use of maternal antenatal care 4. Number of weeks gestation at birth 5. |
1. Blood glucose 2. Blood culture 3. Blood lactate level 4. HIV status |
1. Sex 2. Number of siblings 3. Exclusively breastfed for first 5 months 4. Mother's education (# of years) 5. Immunization status 6. Number of previous hospitalizations 7. Distance from child's home to nearest health facility 8. Mother's age |
At time of admission
Table 5.
Final list of candidate predictor variables (N=55)
| Clinical* | Birth | Laboratory* | Social/Demographic |
| 1. Temperature 2. Respiratory rate 3. Oxygen saturation (SpO2) 4. Age 5. Dehydration (using WHO dehydration scale) 6. Anthropometrics (weight, height, MUAC) 7. Chest indrawing 8. Bulging fontanel 9. Feeding status 10. Jaundice 11. Multiple associated infectious symptoms (e.g. pneumonia + diarrhea/sepsis/UTI) 12. Grunting 13. Diarrhea 14. Convulsions 15. Abdominal distension 16. Capillary refill 17. Coma score (i.e. Blantyre Coma Scale) 18. Other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.) 19. History of weight gain/weight loss 20. Pallor 21. Oral/motor coordination impairment 22. Hypotonia 23. Spasticity 24. Blood in stool (dysentery) 25. History of cough for two or more weeks 26. Duration of present illness |
1. Birth weight 2. Location of birth (home vs facility) 3. Use of maternal antenatal care 4. Number of weeks gestation at birth 5. Mother's age at time of birth of first child 6. History of birth asphyxia 7. Mode of delivery (vaginal vs caesarean section) 8. History of resuscitation after delivery 9. Details of umbilical cord care at/after birth (ex: cutting, cleaning practices, cultural practices, etc.) |
1. Blood glucose 2. Blood lactate level 3. HIV status 4. Hemoglobin 5. Sickle cell/thalassemia status |
1. Sex 2. Number of siblings 3. Exclusively breastfed for first 5 months 4. Mother's education (# of years) 5. Immunization status 6. Number of previous hospitalizations 7. Distance from child's home to nearest health facility 8. Mother's age 9. Mother is acutely ill (at time of admission) 10. Mother is chronically ill (HIV, TB, mental illness, etc.) 11. Mother has died 12. Primary caregiver (at home) other than mother (e.g. father, grandmother, aunt) 13. Primary caregiver (during admission) other than mother (e.g. father, grandmother, aunt) 14. Mode of transport to health facility (e.g. by foot, public transport, private car) 15. Parental substance use (e.g. smoking, alcohol consumption) |
In addition to rating the candidate variables, participants were encouraged to comment on and suggest additional variables for inclusion in the second round. The research team considered each proposed variable and eliminated those considered redundant (e.g. malnutrition can be determined from MUAC or weight for age z-score). The revised list of potential candidate variables suggested through the first survey became the basis of the second round.
Round 2
The second round evaluated 27 new variables using the same criteria utilized in the first round. Participants were again encouraged to comment on each of the variables under evaluation. However, the second round disallowed for suggestion of further variables. The primary research team again utilized the results from the second round to retain, modify or eliminate the additional candidate predictor variables, and incorporated the selected variables into the final list of candidate variables.
Analysis
Each candidate variable was scored by participants as having (1) high, (2) moderate, (3) unlikely, or (4) no applicability for each of the four criteria previous described. Responses were tabulated and reported using descriptive statistics (Microsoft Excel, Seattle, WA). The proportion of respondents who scored a variable as highly applicable was of primary interest to the research team. The proportion of respondents indicating unlikely or no applicability was also of note.
Results
This modified Delphi process included 18 participants from low, middle and high-income countries (including Kenya, South Africa, Bangladesh, Malawi, Uganda, Canada, United States) (Table 1).
These participants identified roles as physicians (12), nurses (2), clinical scientists (6), hospital administrators (1), and epidemiologists (5), with expertise in areas including pediatrics, infectious disease, global health, epidemiology, neonatology, and obstetrics. During the survey period, participants evaluated 37 candidate variables during round 1 (Table 3), and resulted in an additional 27 candidate variables during round 2 (Table 4), resulting in a final list of 55 candidate variables to be used on subsequent post-discharge mortality prediction modeling research (Table 5).
Table 4.
Proposed additional variables and subsequently surveyed variables in round 2
| Category | Proposed Additional Variables (N=26) |
Round 2 Surveyed Variable (N=27) |
| Clinical Variables | Weight gain/history of weight loss | History of weight gain/weight loss |
| Pallor/anemia (marker of malnutrition/chronic illness) |
Pallor at time of admission | |
| Oral/motor coordination/impairment (predisposing to malnutrition/dehydration/aspiration) |
Oral/motor coordination impairment | |
| Malnutrition |
Not included as a new variable to survey, as it is evaluated under previous variables “anthropometrics, feeding status, and abdominal distention” |
|
| Specific comorbidities |
Not included as a new variable to survey, as it is evaluated under previous variable “other comorbidities” |
|
| Hypotonia/spasticity (restrictive lung disease/aspiration/poor nutrition) |
Hypotonia at time of admission | |
| Spasticity at time of admission | ||
| Blood in stool (dysentery) | Blood in stool (dysentery) | |
| Cough of two or more weeks | History of cough for two or more weeks | |
| Length of illness prior to admission/long duration of illness |
Duration of present illness at time of admission | |
| Birth Variables | Perinatal infection | History of perinatal infection (except HIV) |
| Birth asphyxia | History of birth asphyxia | |
| Mode of delivery and color of baby at birth |
Mode of delivery (vaginal vs. cesarean section) | |
| Skin color at birth to detect hypoxemia, anemia, infection, stress etc. | ||
| Any resuscitation/treatment needed immediately after delivery |
History of resuscitation after delivery | |
| Umbilical cord practices (risk of tetanus and sepsis) |
Details of umbilical cord care at/after birth (e.g. cutting, cleaning practices, cultural practices, etc.) |
|
|
Laboratory Variables |
Hemoglobin | Hemoglobin at time of admission |
| Platelets | Platelet count at time of admission | |
| Urea/creatinine | Urea/creatinine at time of admission | |
| White cell count | White blood cell count at time of admission | |
| Sickle cell/thalassemia status | Sickle cell/thalassemia status | |
|
Social/demographic Variables |
Deceased or sick mother | Mother is acutely ill (at time of admission) |
| Mother is chronically ill (HIV, TB, mental illness, etc.) | ||
| Mother has died | ||
| Resources/salaries/health insurance | Family wealth index | |
| Child is covered under a health insurance plan | ||
| Maternal immunization status (such as tetanus) |
Already captured by maternal chronic and acute illness, and child's comorbidities |
|
| Exclusive BF vs BF + supplementation vs supplementation as well as type of supplementation (cow milk vs formula) |
Not included as a new variable to survey, as it is evaluated under previous variable “exclusively breasted for first 5 months” |
|
| Caregiver other than mother (e.g. father, grandmother, aunt) |
Primary caregiver (at home) other than mother (e.g. father, grandmother, aunt) |
|
| Primary caregiver (during admission) other than mother (e.g. father, grandmother, aunt) | ||
| Mode of transport | Mode of transport to health facility (e.g. by foot, public transport, private car) |
|
| Smoking/drinking (behavioral factors) | Parental substance use (e.g. smoking, alcohol consumption) |
Eighty-nine percent and 83% of participants rated post-discharge mortality as ‘very important’ for children 0–1 months in age children 1–12 months, respectively (Table 2). Ninety-four percent of experts surveyed thought that, in comparison to other public health issues in resource poor countries, the current allocation of resources for post-discharge care of children under the age of 1 was ‘very inadequate’. Responses varied when asked about what they believed the post-discharge mortality rate was in the first 6 months following discharge for children admitted with an infectious illness in resource-limited countries. For children age 0–1 month of age, respondents generally believed mortality rates to be between 5% and greater than 10%. For children age 1 to 12 months, the majority chose between 2–10%.
Round 1
A total of 18 participants completed round 1 (Table 1). Each survey question received between 16 and 18 responses, out of a possible total of 18, since not every question received a response by each participant.
Predictive value
Responses varied from “no applicability” to “high predictive value”; however, only two variables (temperature and abdominal distension at admission) received a response indicating no applicability. The majority of responses ranged from “unlikely” to “high predictive value”. Those receiving the most responses (>70% of respondents) for high predictive value were: anthropometrics, coma score at admission, comorbidities (e.g. congenital defect, sickle cell anemia, tuberculosis) birth weight, number of weeks gestation at birth, HIV status (94%), immunization status, and number of previous hospitalizations. Gender and the number of siblings received less than 10% high predictive responses as well as the highest amount of responses for unlikely predictability. Those variables deemed to be the least predictive for mortality (>30% unlikely responses) included temperature at admission, jaundice at admission, and abdominal distention at admission. Comments made by participants indicated that predictive value for many variables may itself vary by the age of the child, with some being more pertinent for the younger infant and vice versa. For example, birth weight was indicated in a comment to potentially be more predictive in the younger neonate than in those approaching one year of age.
Measurement reliability
None of the variables received responses indicating no applicability for measurement reliability (inter- and intra-rater reliability). High measurement reliability was indicated by at least 70% of expert respondents for the variables gender, HIV status, blood lactate level at admission, location of birth (home vs facility), birth weight, age, and oxygen saturation (SpO2) at time of admission. Furthermore, the variables rated as less reliable (i.e. fewer than 20% rated as highly reliable)included dehydration (using WHO dehydration scale), jaundice, multiple associated infectious symptoms and abdominal distension at admission, other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.), and number of weeks gestation at birth. Most variables received ratings of moderate measurement reliability. The variables receiving more than 30% responses for unlikely reliability included central cyanosis and multiple associated infectious symptoms at time of admission (e.g. pneumonia and diarrhea). Experts repeatedly commented that reliability would diminish when intense staff training is required for variables, or when recall of memory is needed in the face of unreliable or unavailable medical records.
Availability
No respondents rated any variable as being unavailable; responses varied between unlikely to high availability within low-resource settings. Those deemed highly unlikely to be available were blood culture at admission (94%), blood lactate level at admission (88%), and oxygen saturation (SpO2) at time of admission (56%). Most available variables (as indicated by >75% responses as highly available) included signs and symptoms such as bulging fontanel, grunting, diarrhea and convulsions at admission, and location of birth (home vs facility), gender, number of siblings and mothers age. Excluding HIV testing, no respondents rated the proposed laboratory variables as highly available and instead overwhelmingly rated them as unlikely. Availability was considered as something that was mainly dependent upon consistency of supplies and training of personnel. Those variables requiring least supplies and training were regarded as highly available whereas those needing specialized equipment or supplies generally deemed as largely unavailable and increasing staff training were scored lower.
Time and material resources required
Variable ratings varied from unlikely (high amounts of resources required) to highly applicable (fewer resources required); no variables were deemed to be absolutely unavailable in terms of time and material requirements. Those indicators receiving the highest applicability (fewest resources required) included mother's age (94%), age of the child (89%), sex (88%), convulsions at admission (89%), and chest indrawing at admission (89%). Overwhelmingly, laboratory variables (excluding HIV status) were those deemed to require the most time and material resources. Variables were overwhelmingly rated as requiring few resources; however, comments suggested that even when supplies themselves may have been deemed largely available (e.g. weight scale, thermometer), the need for maintenance and calibration precluded them from being rated as highly available in terms of time and material resources.
Proposed new variables
Twenty-six new variables were proposed, which when separated into individual parts, and overlap and similarities removed, yielded the 27 variables included in the second round (Table 4).
Round 2
Participants comprised 17 of the 18 who completed round one, with the same self-described areas of expertise and roles (Table 1). Response rates for survey questions varied between 13 and 17 responses, out of a possible total of 17.
Predictive value
The 27 variables evaluated during the second round had varying perceived predictive value, ranging from noapplicability to high applicability. Most responses indicated moderate predictability. The variables receiving the most scores of “high applicability” included the child's mother having died (88%), mother being chronically ill (e.g. HIV, TB, mental illness) (69%), and history of birth asphyxia (67%). Those variables receiving the lowest ratings for predictability with high response rates of “unlikely predictability” included mode of delivery, skin color at birth, and platelet count at admission. Health insurance coverage for the child was the only variable to receive modest ratings of no applicability by experts (27%). Expert comments suggested that predictive value for indicators may be influenced in some case by objectivity or subjectivity of the measure (e.g. pallor) as well as the specificity with which the predictor indicates mortality (e.g. the implications of the presence of hypotonia varies from insignificant to grave consequences).
Measurement reliability
The variables rated as having high measurement reliability by over 75% of respondents included mode of delivery, hemoglobin, and a deceased mother. Blood in stool (dysentery), duration of present illness, mode of delivery, history of resuscitation after delivery, lab values (including hemoglobin, platelet count, urea/creatinine, white blood cell count) at time of admission, mother deceased, acutely ill, or chronically ill (at time of admission), child covered under a health insurance plan, primary caregiver (at home or admission) other than mother, and mode of transport to health facility all had 80% of respondents rate it as having moderate to high measurement reliability. Oral/motor coordination impairment (53%), skin color at birth (67%), and parental substance use (44%) were largely rated as unlikely to be measured reliably. One variable (pallor at admission) was rated as having no predictive value by one expert only. Experts once again continued to comment on the important role of training requirements for staff (e.g. oral/motor coordination impairment) on measurement reliability and the potential impact of memory on variables (e.g. history of resuscitation) in settings where no medical record is available.
Availability
More than 75% of respondents rated primary caregiver (during admission) other than mother, deceased mother as highly available. Laboratory values, including platelet count, urea/creatinine, and sickle cell/thalassemia status was not rated as highly available by any expert. Those receiving at least 85% of responses indicating moderate to high availability included the following at time of admission: pallor, hypotonia, spasticity, blood in stool (dysentery), history of cough for two or more weeks, duration of present illness, mode of delivery, mother is acutely or chronically ill, deceased mother, primary caregiver (at home or during admission) other than mother, and mode of transport to health facility. No variables received ratings indicating an absolute lack of availability. However, those consistently rated as unlikely (by greater than 60%) included urea/creatinine at time of admission, sickle cell/thalassemia status, and child is covered under a health insurance plan. Experts further commented that although variables may have predictive value, their availability limits the extent to which they are useful within low-resource settings.
Time and material resources required
Those variables rated as requiring the fewest resources to acquire (by at least 70% of respondents) included information on a deceased mother and pallor at admission. All clinical and sociodemographic variables (excluding oral/motor coordination impairment, family wealth index and health insurance coverage), history of birth asphyxia, history of resuscitation after delivery, and umbilical care were rated highly (by at least 85% of respondents), indicating low to moderate amounts of time and material requirements. Although some variables were rated as requiring moderately few resources, many (especially laboratory variables) were rated to require large amounts of resources. Experts further emphasized that laboratory tests relied on resources that were not available in most resource-limited settings. Comments also indicated that amounts of resources required depended on diagnostic approaches; for example, confirmation of perinatal infection would be resource-intensive if it relies on culture techniques rather than on clinical signs and symptoms.
Final list of candidate predictor variables
The final list of 55 candidate predictor variables accepted through the modified Delphi process (Table 5) were identified through expert opinion together with real-time considerations of budget and availability at the proposed research site. The variables identified will be included in future research aimed at establishing prediction models in Uganda.
Discussion
A modified two-round Delphi process conducted using experts from a variety of relevant backgrounds yielded a list of 55 candidate predictor variables to be utilized in the development of a predictive model for post-discharge mortality in infants in resource-limited settings. The clinical presentation of infants is likely to differ from older children due to developmental characteristics and physiology, thus assessment of risk requires an age-specific set of predictors. The differences between pertinent potential risk-factors for the infant versus the older child are further solidified based on variables selected during the Delphi process from all categories including clinical, laboratory, birth, and social/demographic, in comparison to those previously identified for the older child.9
A major strength of this study is the inclusion of experts from multiple pertinent fields. The unique and relevant knowledge participants applied to their evaluation of proposed variables provided additional candidate variables not previously considered by the research team. The breadth of experience resulting from the inclusion of participants with a broad range of expertise also helped ensure that the identified indicators were appropriately evaluated within the clinical context in which they would be utilized. The participants' experiences, knowledge, and understanding of policies, practices, procedures, and availability of personnel and resources in the proposed research country ensured that variables are indeed practical for the research and implementable within the proposed prediction models.
The selection of final candidate prediction variables was both objective and subjective, incorporating results of the Delphi survey process as well as the considerations of the primary research team in terms of relevance and feasibility. Availability at the proposed research site and resource requirements (including time, personnel, and monetary constraints) were central to the acceptance or elimination of proposed factors. For example, although 82% of respondents rated blood culture as having a moderate to high strength for predicting mortality, practical considerations at the proposed research site, as well as the unavailability of this variable as most of the targeted sites for future implementation, negated it from being an included variable. Furthermore, although a variable may have had a lower score for predictive value in the Delphi process, its easy accessibility and measurement may have enabled it to be included as a variable to be considered (e.g. mode of delivery).
The predictor variables identified through this modified Delphi process will be utilized in a model derivation study to predict death post-discharge in children less than one year of age, admitted with infectious illness. By doing so, the limited resources available may be channeled to those children at high-risk for mortality. The process outlined within this paper, coupled with the planned future research and derivation of prediction models, has been recognized as a form of precision public health.6 Precision public health has been proposed as an ideal framework to utilize in decreasing post-discharge mortality, as it improves the efficacy of public health interventions through using precise data, focusing on those who would benefit the most. Although precision public health has started to gain momentum high income countries, its effect in low-resource countries has yet to be fully explored, and yet the potential to impact child mortality could be significant. If vulnerable children can be identified during the admission through the use of risk stratification, effective interventions can be developed and implemented to target those children. Ongoing work in Uganda, described as Smart Discharges, has demonstrated that using prediction to identify high-risk children, paired with interventions including comprehensive discharge teaching and referrals for routine follow-up, could potentially reduce mortality in children (6 months to 5 years of age) during the critical post-discharge period.11 As an extension of this work, the predictor variables identified within this process will focus on predictive modeling for children under one year.
There are several limitations of this study. A primary limitation of this process is the lack of participants' ability to modify responses based on the responses of other experts. Although discussion of specific variables among the experts was not available, an opportunity to comment on each variable was provided to facilitate any questions, comments, or further clarification needed by the variable definition proposed. These comments were seen by the primary research team and discussed, thus allowing the research team to carefully weigh decisions related to inclusion and exclusion of variables. Furthermore, the participants included experts from both developing as well as developed countries as a means of eliciting a wide range of expertise and viewpoints; however, not all experts had an in-depth or practical understanding of the clinical context within the proposed country of study. Expert responses, therefore, based upon what their known context, may have resulted in variables being rated as having lower impact than they actually do, and vice versa. While this diversity may have created some heterogeneity in responses, these diverse options, however, strengthened the ability of the research team to make informed decisions regarding the final list of candidate predictor variables.
Conclusion
The modified Delphi process contributed to the evaluation and identification of potentially useful predictor variables for post-discharge mortality among infants. It helped broaden the selection of variables obtained from on a systematic review and brought objectivity and insight to aspects of predictive value, reliability, availability, and applicability in low-resource settings. The identified variables are a valuable starting point for the construction of a predictive model to identify at-risk infants, who may then be able to benefit from specific interventions aimed towards reducing mortality. Low-resource settings demand that the vulnerable be identified and resources allocated accordingly. The variables identified are an important step towards the goal of reduced childhood mortality.
Figure 1.
Flow diagram of delphi process
Funding
This study was supported by Thrasher Research Fund.
Conflict of interest
All authors declare that they have no conflicts of interest related to this work.
References
- 1.GBD 2016 Mortality Collaborators, author. Global, regional, and national under-5 mortality, adult mortality, age-specific mortality, and life expectancy, 1970–2016: PubMed a systematic analysis for the Global Burden of Disease Study 2016. The Lancet. 2017;390(10100):1084–1150. doi: 10.1016/S0140-6736(17)31833-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Zupan J. Perinatal mortality in developing countries. New England Journal of Medicine. 2005;352(20):2047–2048. doi: 10.1056/NEJMp058032. [DOI] [PubMed] [Google Scholar]
- 3.Liu L, Oza S, Hogan D, et al. Global, regional, and national causes of child mortality in 2000–13, with projections to inform post-2015 priorities: an updated systematic analysis. The Lancet. 2015;385(9966):430–440. doi: 10.1016/S0140-6736(14)61698-6. [DOI] [PubMed] [Google Scholar]
- 4.United Nations, author. Transforming our world: The 2030 agenda for sustainable development. 2015. [Google Scholar]
- 5.Wiens MO, Pawluk S, Kissoon N, et al. Pediatric post-discharge mortality in resource poor countries: A systematic review. PLOS ONE. 2013;8(6):e66698. doi: 10.1371/journal.pone.0066698. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wiens MO, Kissoon N, Kabakyenga J. Smart hospital discharges to address a neglected epidemic in sepsis in low- and middle-income countries. JAMA Pediatrics. 2018;172(3):213. doi: 10.1001/jamapediatrics.2017.4519. [DOI] [PubMed] [Google Scholar]
- 7.Hemingway H, Riley RD, Altman DG. Ten steps towards improving prognosis research. BMJ. 2009:339. doi: 10.1136/bmj.b4184. [DOI] [PubMed] [Google Scholar]
- 8.Hsu C-C, Sandford BA. The delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation. 2007;12(10):1–8. [Google Scholar]
- 9.Wiens MO, Kissoon N, Kumbakumba E, et al. Selecting candidate predictor variables for the modelling of post-discharge mortality from sepsis: a protocol development project. African Health Sciences. 2016;16(1):162–169. doi: 10.4314/ahs.v16i1.22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wiens MO, Kumbakumba E, Larson CP, et al. Postdischarge mortality in children with acute infectious diseases: derivation of postdischarge mortality prediction models. BMJ Open. 2015;5(11):e009449. doi: 10.1136/bmjopen-2015-009449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wiens MO, Kumbakumba E, Larson CP, et al. Scheduled follow-up referrals and simple prevention kits including counseling to improve post-discharge outcomes among children in Uganda: A proof-of-concept study. Global Health: Science and Practice. 2016;4(3):422–434. doi: 10.9745/GHSP-D-16-00069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Shariff NJ. Utilizing the Delphi Survey Approach: A Review. Journal of Nursing & Care. 2015;4(3) [Google Scholar]
- 13.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)— A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]