The Effect of a Mobile Health Decision Support System on Diagnosis and Management of Obesity, Tobacco Use, and Depression in Adults and Children

Mobile health (mHealth) apps are on the rise and increasingly integrated into clinical practice and education as well as used by patients and healthcare consumers either independently or in collaboration with their healthcare providers. Several recent systematic reviews and a scoping review of systematic reviews have examined the impact of handheld devices which have evolved from early personal digital assistants that were synchronized with a database through specialized cradles to today’s Smartphone apps.^1–5 One of the earliest reviews was a 2008 systematic review of 48 studies from 1996–2008 and included 10 studies focused on nurses or nursing students.¹ However, only one of the 10 studies reported interventions related to nursing care processes. The 2002 study by Ruland⁶, which demonstrated that nurses’ use of the handheld CHOICE (Creating better Health Outcomes by Improving Communication about Patients' Expectations) system resulted in nursing care that was more consistent with patient preferences and improved patients’ preference achievement, is the earliest research indexed in PubMed that includes the keywords of nurse and handheld computer.

The body of literature specifically focused on mHealth decision support systems (DSS) for healthcare professionals and students suggests that such systems improve adherence to clinical practice guidelines (CPGs),^2–4 access to medical/health information at the point-of-care,^3,4 increase screening,² improve diagnosis, decrease medical errors,¹ increase documentation,³ increase referrals,² and increase efficiency.³ The majority of studies are focused on physicians or medical students. There is little nursing research cited in these reviews. Moreover, review authors noted that most studies did not address effectiveness and that randomized controlled trials (RCTs) were infrequent in comparison to other less rigorous designs. A 2014 literature and commercial review by Martínez-Pérez and colleagues⁵ identified 192 commercial mHealth DSS apps. This suggests that regardless of the level of evidence for the effectiveness reported in the literature, the supply of such apps is increasing.

In 2002, the School of Nursing began integrating mobile devices into the Nurse Practitioner (NP) curriculum as part of a series of Health Resources and Services Administration grants focused on informatics for evidence-based NP practice including the development of informatics competencies.⁷ In the initial implementation, NP students used personal digital assistants to document their clinical encounters using an application built by our project team.⁸ Standardized terminologies and a focus on nursing process were a vital foundation of the initial NP student clinical log.⁹ Faculty stakeholders found the NP student clinical log and associated reports to be useful for a variety of purposes including monitoring of student performance, benchmarking, and quality of care assessments.¹⁰

Subsequently, the School received funding from the National Institute of Nursing Research to add decision support features to the existing NP student clinical log and to conduct an RCT of the resulting mHealth DSS. At the time, there was substantial evidence about the effectiveness of DSS in clinical information systems on physicians’ adherence to computer-based protocols,^11–14 but little was known about the impact of DSS on nurses or about DSS on handheld platforms.

The focus of the mHealth DSS was the screening and management of obesity and overweight, tobacco use, and depression in adults and children and the research team undertook a series of activities to develop the mHealth DSS. These included: 1) gaining an understanding of the guideline interpretation process of NP students and NPs;¹⁵ 2) development of a set of scenarios to inform system functionality;⁹ 3) transformation of the CPG recommendations into a format that could be processed by the computer;¹⁶ 4) representation of documentation terms using a variety of standardized terminologies;¹⁷ and 5) mapping of CPG recommendations into the five-category NP plan of care (diagnostics, procedures, prescriptions, patient education and counseling, and referrals).¹⁸ The mHealth DSS was iteratively refined during the development process and subsequently in response to evolving mobile platforms.¹⁹

The resulting mHealth DSS included a reminder to screen, standardized screening assessment appropriate to each condition, computer-generated diagnosis, ability to select patient goal (e.g., desire to quit smoking or lose weight) and ability to create a tailored plan of care that included CPG recommendations organized into the five categories.¹⁹ The control application (i.e., NP student clinical log) included the ability to document the diagnoses and plan of care items associated with the CPG recommendations, but the diagnosis was not generated based upon the standardized screening assessment, there was no opportunity to record patient goal, and the plan of care items were not organized as a tailored plan of care with DSS features.

In previous papers, we have reported screening rates in response to mHealth DSS reminders for obesity and overweight,²⁰ adult depression,²¹ and tobacco use²² and demonstrated that screening rates varied by NP specialty, patient race/ethnicity, and payer source.^21,22 The purpose of this study was to compare diagnostic rates and care planning by registered nurses in NP training randomized to mHealth DSS versus control group for obesity and overweight, tobacco use, and depression.

Methods

Human Subjects Protection

The research protocol was approved by the Columbia Health Sciences Institutional Review Board (IRB). Students were considered human subjects, but were routinely using the NP student clinical log so its use was not considered a research procedure. The IRB approved an opt-out protocol for the mHealth DSS. Students were notified that decision support features were being added to the NP student clinical log and that they would be randomized to receive mHealth DSS for one or more of the three clinical conditions. They were given the opportunity to opt out of receiving the mHealth DSS by contacting the principal investigator and several students did so. The students also had the option to choose not to screen in any eligible encounter and to simply select the reason for not screening from a list. In both instances, the NP student clinical log was available for use for documentation of the encounter. Patients were not considered human subjects since the mHealth DSS was conceptualized as an information resource to inform decision making and data such as name, address, medical record number, and other identifying information were not captured.

Design

The study design was an RCT. It was considered a pragmatic RCT because of the heterogeneity of the clinical settings and the patient groups in which it was implemented. In addition, the study design integrated capture of data to address the components of the Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to address replicability and generalizability.²³ Students (n=363) were randomized within NP specialty. Each student received mHealth DSS for one of the three conditions and the control application for one or two other conditions depending upon their cohort. Students were enrolled in 3 cohorts: 93 students (Cohort 1) were randomized within NP specialty to receive either mHealth DSS for tobacco use or obesity and overweight management. For Cohort 2, 132 students were randomized to receive decision support for one of the three conditions. The third Cohort included 138 students also randomized to receive decision support for one of the three conditions. The unit of analysis was the encounter.

Sample

The sample comprised 34,349 unique eligible encounters, i.e., encounters in which an assigned NP student entered data into either mHealth DSS and/or the NP student clinical log. Because of the study design, a single encounter could represent one instance of mHealth DSS (e.g., depression) and one or two instances of control group (e.g., obesity and overweight; smoking cessation). The allocation was not equal across CPGs because of differences in eligibility criteria and the fact that one condition (depression) was not available for Cohort 1. Encounter eligibility for inclusion in the RCT varied by CPG. Minimum patient ages for inclusion were: obesity – age 2, tobacco use – age 9, adult depression – age 18, and pediatric depression – age 8. For plan of care items, analyses related to pediatric depression were restricted to encounters entered by Pediatric NP and Family NP students due to small pediatric sample sizes in other NP specialties. Patients represented in the encounters were 57% female and primarily Latino (49%) or Black (22.5%). Medicaid, Medicare, or State Children’s Health Insurance Program (SCHIP) were the predominant payers for the encounters. Encounters were entered by a total of 363 registered nurses enrolled in one or more of the NP programs. Most were female and non-Hispanic white.

Analysis

We compared the mHealth DSS and control group encounters on the number of encounters with CPG diagnoses and the number of CPG-based plans of care items. Two null hypotheses were tested:

1. There will be no difference between mHealth DSS and control groups in the number of encounters with a CPG-related diagnosis.

2. There will be no difference between mHealth DSS and control groups in the number of plan of care items in encounters with a CPG-related diagnosis.

To test for differences in CPG-related diagnoses between mHealth DSS and control groups, we used Chi-square analysis with Yate’s correction. For this analysis, we used an intent-to-treat approach; all eligible encounters in mHealthDSS and control group were considered regardless of whether or not the students chose to screen for their assigned mHealth DSS condition.

To assess differences in the number of plan of care items between intervention and control groups for obesity and overweight, tobacco use, and adult depression, we used a zero-inflated Poisson (ZIP) model with student-level random effects. For the mHealth DSS group, the ZIP model partitions encounters without plans into two parts: 1) zeros from the Poisson distribution (probability of not receiving any plan of care items is not zero) which we call true zeros and represent encounters from students who entered care plan items in at least some of their encounters; and 2) excess zeros (probability of receiving any plan of care items is zero) which represents encounters from students who received a reminder to screen, screened, and received a screening diagnosis, but routinely failed to document plan of care items in their encounters. For the dependent variable (number of CPG-based plan of care items) used for mHealth DSS vs. control group comparisons, true zeros were included in calculation of mean score, but excess zeros were not because in those instances the students did not interact with the plan of care component of the mHealth DSS.

Results

Of 34,349 unique encounters, there were a total of 25,987 instances that met eligibility criteria for mHealth DSS in the intervention group and 46,176 instances that met eligibility criteria in the control group. Encounters for Pediatric NP students comprised 39.5 % (N=10,171) of the eligible mHealth DSS encounters. The number of assigned students that entered data in eligible encounters using mHealth DSS ranged from a low of 57.9% for Women’s Health NP students assigned to mHealth DSS-Obesity to a high of 100% for all students in Acute Care NP, Oncology NP, and Pediatric NP programs for all mHealth DSS CPGs and for Women’s Health NP students assigned to mHealth DSS-Depression. Fewer students in the Family NP and Adult NP programs used the mHealth DSS for eligible encounters ranging from a low of 58.3% for tobacco use and a high of 89.5% for depression.

The frequencies of diagnoses are displayed in Table 1. All analyses demonstrated a significant effect of mHealth DSS on diagnosis (Table 1): obesity and overweight (χ²=4743.2, p < .0001), tobacco use (χ²=943.9, p < .0001), adult depression (χ²=536.8, p < .0001), and pediatric depression (χ²=84.2, p < .0001). Thus, the null hypothesis was rejected.

Table 1.

Comparison between mHealth DSS and Control Groups on Frequency of Diagnoses

	Obesity and Overweight	Tobacco Use	Adult Depression	Pediatric Depression
Eligible encounters - Total	30,845	23,256	10,779	7,085
Eligible encounters - mHealth DSS	10,938	7,874	4,343	2,832
Eligible encounters – Control	19,803	15,720	6,436	4,217
Diagnoses – mHealth DSS (N, %)	3,707 (33.9%)	939 (11.9%)	382 (8.8%)	129 (4.6%)
Diagnoses – Control (N, %)	907 (4.8%)	355 (2.3%)	14 (.2%)	46 (1.1%)
χ2, p	χ2=4743.2 p <.0001	χ2=943.9 p <.0001	χ2=536.8 p <.0001	χ2=84.2 p <.0001

For obesity, the mHealth DSS group had a significantly higher (p = .0268) mean number of plan items (true zeros plus ≥1 items) as compared to the control group (Table 2). In encounters with a pediatric depression diagnosis, there also were significantly more plan of care items in the mHealth DSS group (3.11 vs.1.63, p = .0201). There were no significant differences in mean number of plan items for tobacco use and adult depression. Thus, the null hypothesis was rejected for only two of the four CPGs.

Table 2.

Comparison between mHealth DSS and Control Groups on Numbers of Interventions

CPG	mHealth DSS Mean (SE)	Control Group Mean (SE)	p
Obesity and Overweight	4.12 (0.411)	2.72 (0.311)	.0268
Tobacco Use	2.34 (0.289)	2.33 (0.251)	.7942
Adult Depression	1.99 (0.262)	2.77 (0.547)	.1633
Pediatric Depression	3.11 (0.286)	1.63 (0.429)	.0201

Discussion

The efficacy of mHealth DSS for increasing the number of CPG-related diagnoses was strongly supported across CPGs. This is similar to findings in the general clinical DSS literature²⁴ as well as in at least one mHealth study.²⁵ Moreover, the greater number of diagnoses is consistent with the relatively high screening rates that we have previously reported for mHealth DSS for obesity and overweight,²⁰ adult depression,²¹ and tobacco use.²²

The findings for number of care plan items varied by CPG. The RCT provided evidence that mHealth DSS was efficacious for management of obesity and overweight, and pediatric depression but not tobacco use and adult depression. The number of care plan items for tobacco use was similar for both groups. One explanation for this may be that even though the study occurred before broad adoption of quality measures for smoking cessation, the NP students may have been more familiar with the CPG-based strategies for smoking cessation than for the other conditions. For adult depression, the lack of difference may be due to small number of encounters with a depression diagnosis for the control group (n=14). In the few instances in which the student NP diagnosed the depression without the aid of mHealth DSS, the depression may have been more obvious and thus, she or he may have been more likely to document multiple interventions.

Some NP students chose to not use the mHealth DSS for any of their encounters even though they did not officially opt out of the RCT using approved study procedures. This is a reflection of the voluntariness of research process. However, because we conducted an intent-to-treat analysis, we included these students in the analyses related to diagnoses. In other instances, students used the screening and diagnosis aspects of the mHealth DSS, but not the plan of care. This resulted in two patterns, students who never interacted with the plan of care and thus were not exposed to this aspect of mHealth DSS, and students who sometimes used the plan of care. It is not clear why some students chose to never use the plan of care. All received training on use of the NP student clinical log and mHealth DSS. Consequently, there was awareness of the availability of the plan of care. One possibility is that some NP students entered data following the encounter rather than during the encounter. If they did not receive the mHealth DSS-generated diagnosis during the encounter or come up with the diagnosis on their own, it is likely that they would not deliver and consequently document relevant interventions. There are several potential explanations for this scenario. After exposure to the plan of care, students may have internalized it and no longer needed to refer to it (i.e., done, but not documented). Alternatively, they may have not perceived mHealth DSS as useful. Moreover, the number of care plan items chosen was relatively small. A recent study by Ansher et al. may offer some perspective on this.²⁶ The plan of care items in mHealth DSS were organized as a five-category documentation template in which the NP students could select from among CPG-based items in relevant categories. Ansher and colleagues compared number of orders by medical residents under two conditions. In both instances orders were organized as templates, but in one experimental condition options were visible and pre-selected, and in the other in which options were visible but unselected. In the former, the medical resident needed to unselect orders that were not desired and in the latter to select desired orders. They found fewer orders in the latter situation which is similar to the mHealth DSS plan of care documentation template in this study, i.e., NP students needed to select items. Failure to act upon decision support guidance has been widely noted in the literature, particularly in regards to medications, ^27–29 so is not unique to this study.

There are several potential limitations to this study that may limit the generalizability of study findings. Use of the NP student clinical log was integrated into all NP programs, but the number of encounters by specialty suggests that its use varied by specialty likely as a reflection of the emphasis placed on its use by the program director. Although use of mHealth DSS was voluntary and program directors did not know which students were randomized to mHealth DSS vs. NP student clinical log, it is likely that program director emphasis on the NP student clinical log also influenced mHealth DSS use. For example, the fact that almost 40% of the encounters were entered by Pediatric NP students is likely due to a combination of patient volume and program director emphasis on use of the NP student clinical log rather than difference in relevance to particular specialties. In fact, the decision to focus on obesity and overweight, tobacco use, and depression was explicitly based on the relevance of screening and management of these conditions across NP specialties.

Another potential limitation is that the encounters were entered by registered nurses in NP training. However, screening and management of the three conditions were within the purview of registered nurses and the encounters involved patients under the supervision of a preceptor. Thus, diagnoses and interventions were part of actual patient care and consequently had the potential to influence clinical decisions, patient behaviors, and patient outcomes.

The third potential limitation relates to the fact that the effect of mHealth DSS may vary across conditions and our RCT only addressed obesity and overweight, tobacco use, and depression. Our previously reported studies on screening rates demonstrated efficacy of mHealth DSS across the three conditions,^20–22 and this study provided evidence for the efficacy of mHealth DSS in CPG-related diagnoses. However, the findings related to plan of care varied among the three CPGs studied and this may also be true of others not studied.

Lastly, the study focused on two processes of care – diagnosis and management – rather than on patient outcomes such as weight loss, smoking cessation, and decrease in depressive symptomatology. This is partially due to the fact that the RCT was not longitudinal and that the encounter rather than the patient was the unit of analysis. However, diagnosis and management are prerequisites to achieving desired patient outcomes and the effect of mHealth DSS on the processes of care has not been widely studied with registered nurses, NP students, or NPs.

Implications for NP Education and Practice

There are several implications for NP education and practice. First, the positive findings related to diagnosis provide evidence for the inclusion of mHealth DSS for CPGs in both education and practice as a strategy for increasing diagnosis rates for key areas of NP practice. Second, although the results related to plan of care items varied across CPGs, the statistically significant results for both obesity and overweight, and pediatric depression support the usefulness of the mHealth DSS for care planning in NP education and care. Moreover, it can be argued that the difference of one care plan item between intervention and control group for adult depression may be clinically significant even though not statistically significant due to the low diagnosis rate in the absence of mHealth DSS. Third, given the increasing availability of mHealth apps, it is vital that NP students and NPs have the necessary competencies to integrate such tools into patient care. Fourth, mHealth apps should be carefully evaluated to ensure that they support key aspects of NP care that differentiate it from other types of primary care practice such as an emphasis on tailored education and counseling.

Conclusions

The findings of the RCT provide evidence that mHealth DSS was efficacious in increasing diagnostic rates, but had varied effects on number of care plan items. This study adds to the body of literature suggesting that mHealth DSS has a positive effect on care processes. Impacting care processes through mHealth DSS is foundational to affecting care outcomes. As mHealth apps are increasingly available, appropriate use in NP education and practice should be fostered.