Development and Feasibility of a Configurable Assessment Messaging Platform for Interventions (CAMPI)

Abstract

Short message service (SMS) is a widely accepted telecommunications approach used to support health informatics, including behavioral interventions, data collection, and patient-provider communication. However, SMS delivery platforms are not standardized and platforms are typically commercial “off-the-shelf” or developed “in-house.” As a consequence of platform variability, implementing SMS-based interventions may be challenging for both providers and patients. Off-the-shelf SMS delivery platforms may require minimal development or technical resources from providers, but users are often limited in their ability to specify parameters (e.g., message delivery scheduling and rules). Conversely, platforms that are developed in-house are often specified for individual projects, requiring specialized development and technical expertise. Patients are on the receiving end of programming and technical specification challenges; message delays or lagged data affect quality of SMS communications. To-date, little work has been done to develop a generalizable SMS platform that can be scaled across health initiatives. We propose the Configurable Assessment Messaging Platform for Interventions (CAMPI) to mitigate challenges associated with SMS intervention implementation (e.g., programming, data collection, message delivery). CAMPI aims to optimize health data captured from a multitude of sources and enhance patient-provider communication through a technology that is simple and familiar to patients. Using representative examples from three behavioral intervention case studies implemented among diverse populations (pregnant women, young sexual minority men, and parents with young children), we describe CAMPI capabilities and feasibility. As a generalizable SMS platform, CAMPI can be scaled to meet the priorities of various health initiatives, while reducing unnecessary resource utilization and burden on providers and patients.

Mobile phone communication has revolutionized health care by enabling real-time data collection to inform health information dissemination and clinical decision making. One of the most versatile, highly utilized, and scalable mobile phone features, text messaging, is particularly well-suited for health information exchange between patient and provider (e.g., interventions) (Braun, Catalani, Wimbush, & Israelski, 2013; Hall, Cole-Lewis, & Bernhardt, 2015; Head, Noar, Iannarino, & Harrington, 2013; Pew Research Center, 2015). Text messaging, or short message service (SMS), is a device-agnostic and bi-directional communication system allowing users to exchange brief messages (Schilling et al., 2013). The value of SMS interventions is well-documented; they improve patient compliance (Schwebel & Larimer, 2018; Thakkar et al., 2016), facilitate behavior change (Hall et al., 2015), and support self-management of chronic health conditions (Franklin, Waller, Pagliari, & Greene, 2006; Patrick et al., 2009; Rodgers et al., 2005; Yoon & Kim, 2008).

Although the design of SMS interventions continues to evolve (Bidargaddi et al., 2018; Nahum-Shani et al., 2016), SMS intervention implementation is not codified into best practices, and few studies detail technical specifications of automated SMS interventions (Coomes et al., 2012; Hall et al., 2015; Iribarren et al., 2017). Additionally, non-standardization of SMS delivery platforms is a significant implementation barrier that limits integration with research databases and electronic health records (EHRs) (Iribarren et al., 2017). Poorly defined SMS platforms also run the risk of intervention execution errors, data loss, and poor data quality (Iribarren et al., 2015; MacLeod, Phillips, Stone, Walji, & Awoonor-Williams, 2012). Solutions for common SMS intervention issues and challenges are not widely shared, thus opportunities to improve SMS implementation processes and sustainability may be overlooked (Hall et al., 2015; Iribarren et al., 2017). A potential solution is a parameterized SMS intervention system that can be configured for a wide range of intervention designs and technical proficiency.

A standardized SMS platform solution should be configurable for non-engineers, include message-level scheduling, and integration with commonly used software, such as research data management platforms (i.e., REDCap) (Harris et al., 2009) and EHRs, to reduce duplicate data entry. However, the current standard for SMS intervention programming and delivery is software that is developed in a silo, either commercial off-the-shelf platforms or custom platforms developed in-house (Iribarren et al., 2017; Schilling et al., 2013). Off-the-shelf platforms may include a user-friendly interface and reduce programming time, however, delivery capabilities are limited because they are often not modifiable (Iribarren et al., 2017) and reoccurring service and licensing fees must be considered. Conversely, in-house platform development requires specialized technical teams, backend database development, and a local server (Oliveira, Oliveira, Garcia, & Esgalhado, 2014). Typically, in-house platforms are developed with immediate project requirements in mind, requiring ongoing modifications as project scope or clinical needs change (Oliveira et al., 2014; Schueller, Begale, Penedo, & Mohr, 2014).

Implementing SMS interventions, including identifying, selecting, and programming SMS platforms, is thus challenging (Iribarren et al., 2017). To our knowledge, there is no comprehensive SMS platform able to satisfy requirements across multiple interventions, projects, and teams. Consistent with recommendations for more robust SMS platforms (Iribarren et al., 2017) this study presents the feasibility of a Configurable Assessment Messaging Platform for Interventions (CAMPI).

CAMPI is a platform that allows research teams to exercise control over SMS intervention delivery parameters without the need for specialized programming expertise, system development, or hardware. As a generalizable SMS platform, CAMPI can be scaled to meet the priorities of various research initiatives and support varying levels of technical experience. Additionally, CAMPI re-directs provider resources away from technical challenges and toward quality patient care. We describe CAMPI features and applications using examples from three behavioral interventions. Recommendations for integrating CAMPI with other systems, such as EHRs, are also discussed.

Method

CAMPI System Features

CAMPI consists of three components: ecological momentary assessment (EMA) scheduler, ecological momentary intervention (EMI) rules engine, and SMS messages (Figure 1). The CAMPI’s rules engine pulls intervention material (e.g., EMA questions, surveys, and links to external web resources) and then generates customized text messages that are delivered to the participants. The research or clinical team create a spreadsheet (e.g., comma-delimited .csv file) (Figure 2) that contain SMS messages and delivery information. Messages can be personalized by bracketing placeholder text with [] (e.g. Hi [name], Please complete the following survey … ). This piping approach is also used to deliver surveys to participants (i.e. “Please tell us how you are doing [Perceived Stress Scale]”). Each message has a delivery timeframe representing the delivery window for when the message should be sent. Once a message is sent it is logged in order avoid duplicate messages from being sent to participants. A technical architecture of the CAMPI system and data safety and security considerations are provided as Supplemental Material.

Figure 1.

CAMPI System Overview

Figure 2.

Excerpt (txt2protect) from scheduler comma-delimited csv file

CAMPI Case Studies

Using a configurable set of features, CAMPI allows research teams to specify intervention paramaters, thereby supporting a variety of intervention designs and project methodologies. Bi-directional patient-provider communication, using technology that is convenient, can be achieved with the CAMPI system. We illustrate CAMPI’s capabilities in three case studies used to establish feasibility. Case study 1 (Wellness-4-2) was selected to illustrate a just in time adaptive intervention (Nahum-Shani et al., 2016) deployed using CAMPI (Wakschlag et al., 2020). Case study 2 (txt2protect) was used to deploy brief behavioral intervention content, collect data, and deliver appointment reminders (Gerend et al., 2020). Case Study 3 (Mental Health, Earlier) is a proposed application of CAMPI for screening for mental health risk among young children in pediatric primary care and delivery of content from an evidence-based parenting intervention. Table 1 shows how CAMPI features are utilized for each study. Preliminary usability results from Cases 1 and 2 demonstrate the compatibility of this approach with remote clinical trials and highlight the acceptability of this technology by patients and clinical staff, through their response rates and rapid response times to survey prompts. Case Study 3 provides an example for use in pediatrics.

Table 1.

Description of CAMPI Features Used in Three Case Studies

Case Study	Population / Condition Studied	SMS Design	CAMPI Features
Promoting Healthy Brain Project (PHBP)	Adult women during pregnancy / Prenatal maternal stress and infant neurodevelopment	EMA Stress management education and training	• SMS messages used to deliver multiple daily scheduled interventions. • SMS messages used to deliver weekly education and training material “piped” to surveys as well as external websites. • Activation of auditing of external website in order to capture usability and engagement. • Clinical decision support based on SMS interventions and sensor data to determine if booster intervention material should be sent.
txt2protect	Adult males / HPV vaccination	Education Skills acquisition Self-efficacy Reminders	• SMS messages used to deliver multiple education messages daily. • SMS messages used to deliver periodic intervention surveys. • SMS messages used to deliver reminder for intervention surveys. • Links to initial surveys and followup reminders to prompt completion (if necessary)

Note. EMA = ecological momentary assessment. HPV = human papillomavirus.

Case Study 1: Wellness-4-2 Intervention of the Promoting Healthy Brain Project (PHBP)

Wellness-4-2 is an adapation of the Mothers and Babies perinatal depression prevention intervention (Tandon et al., 2011, 2014) designed to reduce maternal distress during pregnancy and reduce adverse impact on infant neurodevelopment (Wakschlag et al., 2020). The 12-week intervention incorporates self-reported stress ratings collected via EMA and heart rate data measured via a novel wireless, wearable biosensor (nPoint Wearable System, MC10 Inc.) that measures electrocardiogram (ECG) data. Machine learning methods are employed to indicate when a woman meets the “stressed” threshold and prompt a just-in-time adaptive intervention containing customized messaging including skills reinforcement, self-monitoring, and homework reminders. Participants, (pregnant women; n=15) piloted the system and provided preliminary acceptability and feasibility of CAMPI-delivered EMAs as a method for remote monitoring of maternal stress response (Wakschlag et al., 2020).

CAMPI implementation.

CAMPI was programmed to send 5 daily SMS messages to capture participants’ self-reported stress. Messages contained a link to the 14-item perceived stress scale (PSS-14; Cohen, Kamarck, & Mermelstein, 1983). In conjuction with assessments, just-in-time adaptive intervention content was sent to participants whose biosensor data indicated a high degree of stress based on a priori defined stress thresholds. CAMPI delivered the intervention through SMS messages containing links to web-based resources to mitigate stress.

Feasibility.

For the 12-week pilot study, 15 participants responded to 3612 assessments (M=240) with an average response time of 53 minutes. Interestingly, the percentage of missing assessments increased consistently as the study progressed. At three weeks, 6% of data was missing. At 12-weeks, missing data increased to 35%. Participant feedback during exit interviews indicated the volume of messages should be reduced. For example, participants preferred a maximum of 3 to 4 assessments for the duration of the study. This is consistent with recommendations suggesting that 5 SMS messages would be acceptable (Burke et al., 2017).

Case Study 2: txt2protect

The CAMPI system has been used for txt2protect, an intervention aimed at increasing human papillomavirus (HPV) vaccination among young (aged 18–25) sexual minority men (Gerend et al., 2019). Participants (n=140) were randomly assigned to receive SMS messages that focused primarily on HPV vaccination (treatment condition) or a variety of sexual health practices (e.g., condom use, HIV testing) with only brief mention of HPV vaccination (control condition) (Figure 2).

CAMPI implementation.

CAMPI delivered intervention content and assessments using SMS messages. Intervention content was delivered to participants in 2 phases over 36 weeks. During Phase 1 (weeks 1–3), participants received 10–12 messages daily which were delivered morning, midday and evening. Messages were sent in batches of between 2 and 5 messages each time to simulate a more natural conversation pattern. During Phase 2 (weeks 4–36), message frequency decreased from weekly to monthly messages. Phase 1 content was presented in 3 modules that varied by condition (treatment vs. control). In total, 214 messages were sent during Phase 1. Phase 2 “booster” messages largely reinforced previous content to foster continued program engagement. Participants were asked to complete three formal assessments at baseline, after Phase 1, and after Phase 2 and three optional single-item asessments during Phase 2.

Assessments and Reminders.

Both the txt2protect survey assessments and reminders were configured through the CAMPI system for participants in both control and intervention cohorts. The suvey assessments were configured to send SMS messages with embedded links to REDCap surveys on a pre-defined schedule, while the reminders, containing the survey links were sent a pre-determined number of days after the initial survey, but only if the participant did not complete the initial survey.

Feasibility.

Intervention acceptability for the first 3 weeks of the intervention was assessed at the 3-week assessment (Phase 1). The scale included both open-ended (“What did you like about the program? What could be improved?”) and closed-ended questions (e.g., “I would recommend a program like this to my friends” 1=strongly disagree to 5=strongly agree). Closed-ended items were combined to compute an average score. An average score ≥ 4 was used to indicate an acceptable intervention. Qualitative analysis indicated the quantity and frequency of the messages were acceptable for both control and treatment groups. Results of acceptability related to the text message delivery (closed-ended questions) are presented in Table 2. Over 900 assessments were sent out to participants; 55% of the time, participants needed at least 1 reminder to prompt them to complete the survey and 40% of the time participants needed a second reminder (Table 3).

Table 2.

Case Study 2: Average Participant Intervention Acceptability Ratings (N = 140)

Item	Control (N=67)	Treatment (N=73)
t2p sent too many text messages.	2.95	3.21
t2p got in the way of my daily schedule.	1.85	2.14

Note. t2p = txt2protect. (1 = strongly disagree to 5 = strongly agree)

Table 3.

Case Study 2: Frequency of Assessment Reminders

Assessment	Frequency	First Reminder	Second Reminder	% First Reminder	% Second Reminder
Baseline	180	91	64	51	36
Phase 1	151	90	57	60	38
Treatment 1a	71	36	30	51	42
Treatment 2 a	65	37	31	57	48
Treatment 3 a	62	44	36	71	58
Control 1 a	76	41	33	54	43
Control 2 a	76	43	32	57	42
Control 3 a	76	43	30	57	39
Phase 2	148	75	52	51	35
Totals	905	500	365

^aOptional brief assessments that were sent during Phase 2 of the trial.

Case Study 3: The Proposed Mental Health, Earlier pediatrics-based screening and parenting intervention to prevent mental illness

The proposed Mental Health, Earlier project plans to use CAMPI to screen for toddler neurodevelopmental vulnerability at well-child visits in pediatric primary care and to deliver a virtual parenting intervention for toddlers identified as at risk. The planned protocol would involve the caregiver receiving a prompt via CAMPI to complete a computer adaptive test-based screener for irritability—a well-established indicator of early risk for mental illness (see Wakschlag et al. 2019) prior to scheduled well-child visits. Those toddlers with an elevated screen would be referred to the evidence-based Family Check-Up 4 Health (FCU4Health; Smith et al. 2018-a; Smith et al. 2018-b) program. The intervention would be delivered via videoconference by a trained FCU4Health coordinator over a period of 6 months following each elevated score thus affording multiple opportunities for FCU4Health during the toddler-preschool period. Families in the FCU4Health will receive SMS messages via CAMPI to enhance both uptake of intervention content as well as facilitate continued engagement in the program between virtually-delivered sessions. Intevention content will be individually tailored to each family’s identified needs in parenting practices in the areas of positive behavior support, monitoring and limit setting, and parent-child relationship quality (Smith & Dishion, 2013). SMS messages will contain brief intervention content as well as hyperlinks to lessons and additional content in the form of live-action and animated videos, infographics, storyboards, etc. The scheduler will be programmed to send SMS morning, midday, and evening, with a 75% chance of receiving a SMS on a given day and time. This probability can be individually adjusted based on opening and response rates for each family captured by CAMPI or based on feedback from families. The proposed use of CAMPI in Mental Health, Earlier provides a unique and likely efficient means of screening, delivering intervention content, and maintain engagement in a virtually-delivered parenting intervention that is highly aligned with the preventive and anticipatory guidance mission of pediatric primary care.

Discussion

CAMPI is a novel platform that supports SMS intervention implementation. To date, CAMPI has been used to implement SMS interventions with diverse research teams and unique study specifications. Collective findings from the two completed studies (Case Studies 1 and 2) reveal that CAMPI usability is acceptable to participants enrolled across a broad range of studies with varying degrees of message frequencies, intervals, requirements, and participant populations. We observed homogenous SMS configurations cannot be established; underlying the very need for a system that can easily be adapted to the particular needs of patients, researchers and clinicians. All of the study teams, to date, have had modest technical experience; CAMPI’s straightforward parameter specification approach allowed teams to successfully program interventions without obtaining specialized technical support. Research teams did not report challenges with programming intervention specifications using the CAMPI system. Researchers were then able to focus on their assessment and intervention material rather than channel additional resources toward technical implementation.

Options for EHR Integration and Other Potential Applications of CAMPI

The initial implementation of the CAMPI system used REDCap as the repository for all participant, assessment, and logging data. Repository data access interfaces can be designed so that the system can be refactored and support other repositories. This design allows the creation of adapters that communicate with EHRs or other research platforms (LimeSurvey, etc.) to insert into the scheduling and delivery CAMPI components. This framework supports complex algorithms, which often rely on clinical and possibly biosensor data to determine appropriate interventions. The CAMPI system could deliver simple text-based interventions through the scheduling component or in real-time bypassing the scheduler. For EHR integration, mechanisms for data exchange and system deployment already exist and are supported by many of the EHR vendors. For example, ‘Fast Health Interoperability Resources’ (FHIR) (HL7 FHIR Foundation, n.d.) is a specification intended to promote and increase data sharing and access of healthcare data. The FHIR specifications outline data structures and web services for data exchange. A FHIR-base CAMPI adapter would enable an EHR data repository that could be used for clinical care. That is, CAMPI could be programmed to send reminders to complete patient-reported outcomes or behavioral health screening instruments (e.g., PHQ-9) that could then be pushed back to the EHR with alerts to providers/medical staff when an elevated score requires: 1) clinical follow-up: or 2) intervention content needs to be initiated to address the elevated concern. The latter could be either done automatically or by the care team after reviewing the results. This approach to screening and measurement-based care could reduce the burden on medical staff to administer screeners in person during visits and enter scores into the EHR for providers to review. The ubiquity of mobile devices with SMS functionality facilitates such a system.

Deployment of CAMPI for clinical care can be achieved through the EHR “app store” model (e.g. EPIC App Orchard, Allscripts Application Store, Cerner App Gallery, etc.) (Allscripts Healthcare Solutions, Inc., 2019; Cerner Corporation, 2019; EPIC, 2019). EHR vendors are unlocking their systems to application software developers in order to enhance user experience and functionality for both clinicians and patients without relying on separate third-party solutions. FHIR-based CAMPI adapters or plug-ins can be developed and released in the specific EHR vendor app store, which would provide individual clinicians with greater access to the CAMPI system. The app store deployment model reduces the complexity of integrating software into EHR systems. As a result, fewer information technology would be needed because EHR vendors have certification procedures and documents in place to ensure functional stability of the third-party apps. Vendors have codified app integration through their respective app store requirements and specifications thereby simplifying EHR integration. In general, the onetime development or integration expenses are minimal, which should not present a significant barrier to most small primary care practices. However, reoccurring costs such as an annual fee charged by EHR vendors for API access may present a financial barrier. Using open source/free EHRs, that are increasingly becoming available, could be considered and would alleviate some of these financial barriers, but may introduce the need for greater IT support staff.

Limitations

Although mobile phones are ubiquitous and a proven tool for telehealth, there are important limitations to consider. The study teams had to to take into consideration that their interventions were being viewed on a mobile phone display. Interventions that can be expressed in short concise text are better suited for this platform; even when links to external web resources are embedded in the messages, the material still is viewed on a mobile phone display.

Additionally, SMS interventions that are self-contained and do not refer to previous material fit the messaging paradigm better than content that is intended for future reference, because SMS can not be easily searched, reorganized, or cataloged. Reviewing previous material can be challenging for users to search. As a result, messages should follow a simple structure that can easily be recognized at first glance (e.g. introducing each message with a capitalized topic).

Conclusion

The configurable CAMPI system mitigates challenges associated with SMS intervention implementation: namely, programming, message scheduling, and data capture. This study demonstrates the key system features and illustrates diverse representative applications of the CAMPI system. Future work is required to evaluate the scalability and performance of the CAMPI system before further dissemination. Future work is also needed to test CAMPI using multiple data repositories and EHR vendors, to ensure that the system is truly generalizable across healthcare systems and instutitions. As technology and intervention methods continue to evolve for remote clinical trials and patient monitoring, CAMPI is poised to serve as a valuable technical tool facilitating SMS intervention implementation needs and to provide a comprehensive SMS delivery platform.