Evaluation of the impact of enhanced virtual forms and gamification on intervention identification in a pharmacist-led ambulatory care clinic

Abstract

Background

Adoption of healthcare technology in the ambulatory care setting is nearly universal. Clinical decision support system (CDSS)2 technologies improve patient care through the identification of additional care opportunities. With the movement from paper-based to electronic clinical intake forms, the opportunity to improve identification of gaps in care utilizing CDSS in the ambulatory care setting exists.

Objective

To evaluate the impact of CDSS-enhanced digital intake forms, with- and without aspects of gamification, on the identification of intervention opportunities in an ambulatory care pharmacy setting.

Methods

Patients were invited to complete visit intake paperwork via virtual forms as part of a CDSS-enhanced mobile application designed to identify potential interventions based on patient age, sex, disease state(s), and user-provided information. Patients were randomized to receive optional patient-specific health questions 1) with or 2) without elements of gamification. Gamification elements included trivia questions, fun facts, and the chance to win a prize. A retrospective review was used to assess interventions identified for a random sample of patients seen within the same time frame who did not utilize the mobile application. Interventions were compared across groups utilizing ANOVA. t-tests were used for a subgroup analysis.

Results

From January to May 2019, 353 potential interventions were identified for 220 study participants. 0.44 (±0.82), 1.8 (±2.0) and 2.1 (±1.8) interventions per participant were identified for the control, virtual forms, and virtual forms + gamification groups, respectively. Significant differences in intervention identification across groups were found using a one-way ANOVA (F = 17.46, p < .001). Post hoc analysis demonstrated a significant difference in interventions identified for those completing 50–100% (n = 32) and those completing less than 50% (n = 18; p < .001) of the optional health questions in the virtual forms + gamification group.

Conclusions

Utilization of CDSS-enhanced clinical intake forms increased identification of potential interventions, though gamification did not significantly impact this identification.

1. Introduction

The American Society of Health-Systems Pharmacists (ASHP) Practice Advancement Initiative (PAI) began in 2010 with the mission to provide the tools and resources needed to support the growth of the profession of pharmacy.¹ The 2030 PAI includes recommendations for pharmacists to “leverage and expand their scope of practice…to optimize patient care,” share accountability for patient outcomes, use technology to advance patient care, and develop and apply technologies in areas such as risk assessment to identify patients needing care.² The near universal adoption of electronic medical record (EMR) systems³ has opened the door for the introduction of applications that work with the EMR to support the patient care process. These include digital clinical decision support systems (CDSS), which are designed to improve the clinical decision-making process through the integration of individualized clinical, health, and patient information. CDSS is generally employed to provide patient-specific recommendations in real time to the provider for evaluation via reminders, reports of aggregated patient data, templates for the completion of documentation, workflow support, displaying clinical guidelines, and other means.⁴ Use of CDSS has been shown to increase adherence to clinical guidelines for specific disease states.⁵

Despite the availability of evidence-based recommendations for preventative services provided and updated regularly by national health organizations, less than 10% of adults over 35 years of age received the recommended services in 2015. Five percent did not receive any of these services.⁶ Poor adherence to screening and management guidelines is further reflected by the HealthyPeople 2020 objectives which include underperforming metrics such as “increase the proportion of adults with diabetes who have a glycosylated hemoglobin measurement at least twice a year” (D-11), “increase the proportion of adults with hypertension who meet the recommended guidelines” (HDS-10), “increase the proportion of adults who get sufficient sleep” (SH-4), “increase the proportion of adults who meet current Federal physical activity guidelines for aerobic physical activity and for muscle-strengthening activity” (PA-2), and “increase the percentage of adults who are vaccinated against pneumococcal disease” (IID-13).⁷ Through CDSS utilization, the opportunity exists to identify and address non-adherence to guideline recommendations such as these.

One major barrier to successful implementation of CDSS is that needed information is often lacking in the EMR. According to Pharmacy Forecast:2020 Patient-Centered Care, it is anticipated that patient-reported data will become a valuable resource for making care decisions.⁸ Previous studies have demonstrated that applications can be utilized to collect patient-specific information necessary for improving patient care. Gray et el. developed a tool to support collection of health information and goal setting for patients with chronic disease and disability. This application allowed patients to enter information between visits, which was then used to assess their health behaviors and status. Patients reported that the application helped them feel engaged in their healthcare and improved self-care, while providers reported that the application helped to guide clinic visits.⁹ An advantage to use of these tools is enhanced care consistency. Previously, a mobile health application was demonstrated to improve the standardization of care by rural providers for a series of common disease states.¹⁰ Other work has demonstrated improved consistency in EMR documentation,⁴ and tailoring the application to the individual patient, practice, and program has been found to improve provider and patient experience.⁹

Focus on the patient experience is important for maximizing engagement with digital applications. One method for increasing engagement is gamification, using game-like features in a non-game context. Elements of gamification include use of avatars, leaderboards, completion awards, accomplishment badges, ladders (answers compared to national averages), competitions, timers, and trivia.¹¹ The impact of gamification on application utilization and its ability to impact user outcomes is well illustrated by a study of the Pokémon Go application. This study found that the more users were engaged with the application, the more it increased their physical activity.¹²

These findings suggest that implementing a CDSS application designed to identify potential patient-specific healthcare interventions may increase the identification of guideline-based interventions due to its ability to gather needed health information. Additionally, research suggests potential value of gamification as a tool to engage users of digital applications. This study sought to evaluate the impact of clinical decision support system (CDSS)-enhanced digital intake forms, with- and without aspects of gamification, on the identification of intervention opportunities in an ambulatory care pharmacy setting.

2. Methods

2.1. Location

This study took place at a university that is a self-insured employer that offers pharmacy services to employee health plan subscribers, including a dispensing pharmacy and ambulatory care clinic services. On campus at the university is a pharmacist-led clinic that provides a variety of services to university employees and their dependents. Services include immunizations, biometric screening, medication therapy management (MTM), women's health services, smoking cessation, dietetics counseling, and disease state management including hypertension, dyslipidemia, diabetes, prediabetes, asthma, and overweight and obesity. Each week, the clinic averages 150 individual appointments. The clinic staff includes clinical pharmacists, pharmacy technicians, pharmacy interns, a dietician, and dietetics students. This study was approved by the institutional review board.

2.2. Study design

All adult patients with a regularly scheduled clinic appointment were eligible to participate. Patients with an appointment for vaccine or medication administration only were excluded. Patients were invited to complete visit intake forms using a mobile application. A link to complete visit forms via the application was included in an appointment reminder email sent to patients with an upcoming clinic appointment. Patients who did not complete their intake paperwork prior to their scheduled appointment were invited to complete the forms via the application using a clinic-provided mobile device or via paper forms. Upon completion of the virtual forms or notification that the forms had been completed prior to the appointment, the clinic receptionist accessed a PDF copy of the completed forms within the patient's EMR and printed the forms for use by the clinician during the patient visit. During the second month of recruitment, technical difficulties with the application platform causing delays in PDF generation led the team to scale back recruitment to only include patients who completed the virtual forms online prior to their appointment in order to decrease interruptions in the clinic workflow. All clinic patients were given the opportunity to complete their visit forms via the application irrespective of their decision to participate in the study.

The study application was developed in collaboration with the clinic EMR provider. Over several months, a branded mobile application was developed that incorporated the clinic patient intake forms enhanced with an algorithm that individualized additional health questions for patients based on information captured by the intake forms. The algorithm was designed to identify potential interventions based on patient age, sex, disease state(s), and other user-provided information and clinical practice guidelines published by national health organizations including the U.S. Preventative Services Task Force, Centers for Disease Control, American Diabetes Association, and American Medical Association13., 14., 15., 16., 17 (Table 1). The application randomly assigned participants to receive the virtual forms and additional questions only (virtual forms group) or virtual forms and additional questions enhanced with elements of gamification (virtual forms + gamification group). Elements included trivia questions, fun facts, and the chance to win a prize (Table 2). The fun facts and trivia questions presented were related specifically to the questions asked. Participants could opt out of the additional questions at any time. Upon completion of the virtual forms and extra questions, the application generated a PDF of potential interventions within the participant's EMR. This PDF could then be accessed during the patient appointment and each intervention could be assessed for appropriateness. The control group consisted of a retrospective, random sample of patients who attended a clinic appointment within the study period but opted not to utilize the mobile application. A selection of 300 random patients seen in the clinic during the study period was reviewed. Individuals who had used the mobile application to complete their intake forms or who were seen for an ineligible appointment type were excluded. (Supplemental fig. 1).

Table 1.

Individualized questions and potential interventions.

Criteria for Viewing Question	Question(s)	Potential Interventions
All	Do you have any questions about your medications today?	Medication questions
All	Physical activity questions: -Type of activity -Frequency -Duration -Intensity	Physical activity counseling
All	Diet questions: -Fruit & vegetable servings -Are you interested in meeting with a dietician to learn more about eating healthy?	Dietary counseling
Age 30+	Have you had your thyroid checked within the last 5 years?	Physician referral: Thyroid screening
Age 45+	Have you received blood sugar testing from a doctor within the last 3 years?	A1c testing
Age 50–75	Have you been screened for colorectal cancer?	Physician referral: colorectal screening
Age 50–75	Have you been screened for Hepatitis C?	Physician referral: hepatitis C screening
Female, age 40–64	Have you had a mammogram in the last 2 years?	Physician referral: Mammogram
Female, age 65+	Have you ever had your bone health checked?	Osteoporosis screening
Patient-reported vaccination history		Immunization screening/administration
Patients reporting asthma	Asthma questions: -Rescue inhaler use -In the last month, how often has your asthma interrupted your sleep and/or daily activities? -Do you have any questions about your asthma medications today? -How would you rate your current asthma control?	MTM: asthma
Patients reporting having allergic rhinitis, gout, constipation, and/or insomnia	Do you feel like your [insert disease state] is controlled at this time?	MTM
Patients reporting having diabetes	Diabetes questions: -Do you feel that your diabetes is under control? -Do you have your diabetes checked at least 2 times per year by your doctor? - Do you do each of the following: ○ Check your blood sugar regularly? ○ Receive yearly foot, eye, and teeth exams ○ Participate in a diabetes education program?	Diabetes education/management service
Patients reporting having heart failure	Heart failure questions: -Have you noticed a recent increase in swelling in your ankles and lower legs? -Over the last couple of weeks, has it been harder to breathe when doing normal activities? -Do you have questions about your medications today? -Have you gained several pounds over the last week?	MTM: heart failure
Patients reporting having hypertension	Hypertension questions -How are you currently managing your blood pressure? -Do you have your blood pressure checked somewhere other than the doctor's office? -Is your blood pressure currently controlled? -Do you have any questions about your blood pressure medicines?	MTM: hypertension
Patients reporting high cholesterol	Cholesterol questions: -Have you had your cholesterol check in the last year? -How are you currently managing your high cholesterol? -Is your cholesterol currently controlled?	MTM: cholesterol
Patients reporting smoking		Smoking cessation counseling
Taking medications	Do you use the employee pharmacy?
Use employee pharmacy	Have you had your medication checkup this year?	Medication checkup

MTM = medication therapy management.

Table 2.

Example elements of gamification.

Element	Example
Chance to Win Prizes	Provide more information to help us serve you better and for the chance to learn more about your health and win prizes. (Y/N)
Progress Bar Fun Facts	Did you know??? According to the CDC, nearly 50% of adults have used at least one prescription drug in the last 30 days. (https://www.cdc.gov/nchs/fastats/drug-use-therapeutic.htm)
	About 1 in 10 adults have not been to the doctor in the last 2 years.
	Did you know?? 1 in 3 women over 40 years old will have a bone fracture due to osteoporosis?
	About 75 million American adults have high blood pressure. Only about half (54%) of people with high blood pressure have their condition under control.
	Did you know? The CDC estimates that only 20% of American adults get the recommended amount of physical activity.
	In Alabama in 2015, 27.1% people reported eating less than 1 serving of vegetables per day and 48.1% reported eating less than 1 serving of fruit per day.
Trivia Questions	Tue or false? Asthma is often hereditary. (T/F)
	The top 5 states with the highest cholesterol levels are: Tennessee, Alabama, Delaware, Indiana, and what other state? A. Arkansas B. Florida C. California
	One out of every how many adults has high blood pressure? A. 3 B. 5 C. 10
	About how many American adults have diabetes? A. 10 million B. 20 million C. 30 million D: 50 million
	How much exercise should an adult get each week? A. 90 min of walking B. 150 min of walking C. 75 min of running or jogging D. B or C E. A aor C

A post hoc analysis was performed to determine the relationship between the rate of completion of the additional questions and identification of intervention opportunities. Data were only available for the virtual forms + gamification group.

2.3. Statistical analyses

An a priori alpha of 0.05 was set. The number of potential interventions identified was compared across groups utilizing ANOVA. A subgroup analysis of participants in the virtual forms + gamification group was performed to determine if the rate of completion of the additional questions impacted the rate of potential intervention identification. Completion data for the virtual forms group was unavailable. t-test and Chi square analyses were used to compare rate of intervention and race/ethnicity, respectively. IBM SPSS software, version 25 (IBM, Inc., Armonk, NY) was used for all statistical analyses.

3. Results

A total of 346 individuals participated in this study, but the allocation of 126 was unknown due to an application error, resulting in their exclusion. The remaining 220 participants included in the analyses had an average age of 44.1 ± 12.4 years. The majority were female (53.6%) and Caucasian (75.9%). Demographics did not differ significantly between groups (Table 3). A total of 353 interventions were identified, with an average of 1.6 interventions per participant. Interventions were identified at a rate of 0.44 ± 0.82, 1.8 ± 2.0, and 2.1 ± 1.8 per person in the control, virtual forms, and virtual forms + gamification groups, respectively (Table 4). Though the rate of intervention identification was nearly identical in the virtual forms and virtual forms + gamification groups, there was a significant difference from the rate of intervention in the control group (p < .001) (Table 5). The most commonly identified intervention opportunities were physical activity counseling, dietary counseling, referral for thyroid screening, need for vaccination, and MTM for cholesterol management (Fig. 1). The identification of opportunities for intervention were higher in the virtual forms and virtual forms + gamification groups compared to the control group for all interventions except referral for cholesterol management and referral for hypertension management.

Table 3.

Participant demographics.

Variable	Paper Forms (n = 54)	Virtual Forms (n = 74)	Virtual Forms + Gamification (n = 92)
Mean ± SD age, yr	47.7 ± 11.0	41.8 ± 12.8	43.9 ± 12.6
Race/Ethnicity, no.(%)
Caucasian	41 (75.9)	53 (71.6)	73 (79.3)
African American	4 (7.4)	5 (6.8)	8 (8.7)
Asian	4 (7.4)	7 (9.4)	8 (8.7)
Other	5 (9.3)	9 (12.2)	3 (3.3)

SD = standard deviation; yr = year; no = number;

Table 4.

Comparison of intervention identification between groups.

Group	Paper forms (n = 54)	Virtual forms (n = 74)	Virtual forms + Gamification (n = 92)
Interventions identified	24	136	193
Mean interventions identified, no. ± SD	0.44 ± 0.82	1.8 ± 2.0	2.1 ± 1.8

SD = standard deviation.

Table 5.

Analysis of variation.

Source of variation	SS	Df	MS	F	p
Between groups	99.1	2	49.5	17.5	<0.001
Within groups	615.5	217	2.8
Total	714.6	219

Fig. 1.

Number and types of interventions identified by group.

MTM = medication therapy management; HTN = hypertension.

A post hoc analysis demonstrated a significant difference in interventions identified for those completing 50–100% (n = 32) and those completing less than 50% (n = 18; p < .001) of the optional health questions in the virtual forms + gamification group (Table 6).

Table 6.

Sub-group analysis of intervention identification in the virtual forms + gamification group based on percent of additional questions completed.

Variable	Less than 50% Completion (n = 19)	50–100% Completion (n = 32)	p
Mean ± SD age, yr	43.9 ± 11.9	46.3 ± 12.7	0.51
Race/Ethnicity, no (%)
Caucasian	12 (63.2)	32 (100)	<0.01
Other	7 (36.8)	0 (0)
Interventions identified	8	93	<0.001

4. Discussion

This study demonstrated a significant difference in the number of potential interventions identified among virtual form users compared to paper forms. However, incorporating elements of gamification into the virtual forms did not significantly impact the average number of interventions identified. The subgroup analysis demonstrated that the number of additional questions answered was directly related to the number of interventions identified. Taken together, these findings suggest that participants completed similar proportions of the additional questions in both the virtual forms and virtual forms + gamification groups. Thus, completion of the additional questions increased identification of intervention opportunities, though the elements of gamification included in this study did not increase the rate of completion.

Integration of a tool that aggregates screening recommendations into a single, user-friendly platform in an ambulatory care practice has the potential to improve patient care by enhancing identification of non-adherence to these screening recommendations. Eighty percent of the most commonly identified potential interventions in this study required completion of the additional health questions. For example, “referral for thyroid screening” was the third-most identified intervention. Patient entry of an age of 30 years or greater into the virtual form triggered display of an additional question to evaluate patient thyroid screening status (e.g. “Have you had your thyroid checked within the last 5 years?”) (Table 1). “Referral for thyroid screening” would appear on the potential intervention list for all patients opting out of the question or selecting “no.” Assessment of the need for screening could then be completed within the visit. Interestingly, two of the most commonly identified interventions were not identified in any patients utilizing paper forms (“referral for thyroid screening” and “need for vaccination”) (Fig. 1). Not only could this tool improve the rate of intervention identification but also the consistency of that identification.

Study limitations include the ability to collect percent completion data for the additional questions from only participants in the virtual forms + gamification group and the lack of patient diversity. Modification of the study protocol to recruit only patients completing the virtual forms prior to their appointment increased the risk for selection bias. The majority of clinic patients completed the virtual forms prior to their scheduled appointment, so the impact on recruitment was thought to be negligible.

Future studies should determine the impact of intervention identification and patient education on the completion of recommended interventions and should include dissemination into additional ambulatory care populations. Though this study focused on a pharmacist-led clinic, similar applications would have utility in a physician clinic due to the focus on guidelines-based health recommendations. Since this application was designed to integrate specifically with the EMR of the study clinic, a similar development process would be needed for interoperability with other EMR systems.

5. Conclusion

Use of digital forms enhanced with a CDSS for identifying intervention opportunities and gathering additional patient-specific information significantly increased the rate of identification of opportunities for intervention in an ambulatory care clinic. However, the elements of gamification utilized in this study did not significantly impact this identification. Further, rate of completion of individualized additional questions was directly related to the rate of intervention identification.

Disclosure

The authors have nothing to disclose regarding real or potential conflicts of interest.

Funding

This work was supported by the ASHP Foundation New Investigator Award, Bethesda, Maryland. The funding agency had no role in the study design, data collection, analysis, or interpretation, writing of the report, nor the decision to submit the article for publication.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary material