Geographically-explicit Ecological Momentary Assessment (GEMA) Architecture and Components: Lessons Learned from PMOMS

Abstract

Geographically-explicit Ecological Momentary Assessment (GEMA), an extension of Ecological Momentary Assessment (EMA), allows to record time-stamped geographic location information for behavioural data in the every-day environments of study participants. Considering that GEMA studies are continually gaining the attention of researchers and currently there is no single approach in collecting GEMA data, in this paper, we propose and present a GEMA architecture which can be used to conduct any GEMA study based on our experience developing and maintaining the Postpartum Mothers Mobile Study (PMOMS). Our GEMA client-server architecture can be customized to meet the specific requirements of each GEMA study. Key features of our proposed GEMA architecture include: utilization of widely used smartphones to make GEMA studies practical; alleviation of the burden of activities on participants by: designing clients (mobile applications) that are very lightweight and servers that are heavyweight in terms of functionality; utilization of at least one positioning sensor to determine EMA contexts marked with locations; and communication through the Internet. We believe that our proposed GEMA architecture, with the illustrated foundation for GEMA studies in our exemplar study (PMOMS), will help researchers from any field conduct GEMA studies efficiently and effectively.

Introduction

Ecological Momentary Assessment (EMA) has become a widely used approach for studying hypothesized environmental effects on human behaviour(1–3). EMA can be designed to collect real-time data from participants regarding physical, social, and emotional contexts. The two distinctive features of EMA are that the data are collected in real time and regularly in the participants’ natural environments(4). These features make EMA distinct from other approaches by helping to overcome biases associated with participant recall and lend ecological validity to the study.

EMA can be designed to collect various contexts such as psychological states (5), smoking cessation (6), alcohol use (7), eating patterns (8–10), understanding pregnancy and postpartum health (11) and cardiovascular health (12) through smart devices and wearable sensors. One of the main reasons EMA has become widely used approach is that it allows for data collection through common and widely available technologies such as smartphones and wearable devices that are used by people even outside of research participation(13). When technologies, such as smartphones, are used for EMA, research study challenges such as making multiple trips to a facility or filling out time-consuming survey protocols, are reduced, and also important, individuals are not asked to recall distant past events or emotions.

The increased interest in EMA and the availability of advanced mobile devices (in particular, smartphones) have helped the development of a new generation of EMA, i.e., Geographically-explicit EMA (GEMA) in which location information is linked to the context by collecting a specific set of data at different locations and times; in other words, providing location tags to the collected EMA contexts (14,15). Location information can be provided through common positioning sensors on smartphones and other mobile devices. One predominant positioning sensor on smartphones is the Global Navigation Satellite System (GNSS), the most well-known of which is Global Positioning System (GPS).

EMA and GEMA are also known as Intensive Longitudinal Methods, since multiple measures are collected over an extended period (16). While conventional approaches only provide static snapshots of processes, EMA, and particularly GEMA, can provide detailed information, such as what behaviours and where they occur, over time and are able to take various contexts into consideration (1). Additionally, EMA and GEMA assist researchers to study contextual incidents, decrease “recall bias” related to retrospective self-reports and give a great ecological validity (4,17).

Internet-enabled devices have become commonplace for EMA researchers to administer their data collection protocols. Although, off-the-shelf and standard software, such as Qualtrics, SurveyMonkey, Google Forms or LimeSurvey, are widely used for data collection through surveys, they do not support functionalities that are appropriate for EMA protocols such as customizing for automatically sending surveys at random times, automatically sending surveys at the time specified by participants, setting a time window to take a survey, and collecting GPS locations. This means that researchers using EMA/GEMA must either customize existing software or develop their own software, both requiring a huge investment of time and resources. Furthermore, in addition to constructing survey items with EMA protocols, researchers must design, perform, and test the timing and the alternatives, e.g., when participants forget to complete surveys, which these items are presented to the participants, features that are not included in SurveyMonkey or similar tools.

With the popularity and wide accessibility of smartphones, it is becoming essential to implement EMA and GEMA studies on participant-owned devices, which presents several software engineering difficulties regarding connectivity, platform autonomy, storage, and back-end administration. In general, EMA and GEMA software can provide flexibility with respect to questionnaires considering various contexts of participants such as time, location, or even their sensed activity. Moreover, EMA/GEMA software must support features such as multimodal data and background log recordings, collection of data into a database, and deployment on devices owned by participants. While each EMA or GEMA study may feature a specific set of different requirements, they all can be based on a common architecture that allows customization to meet the requirements.

This paper describes an architecture for GEMA that can be applicable to all GEMA studies. We provide the details of the proposed architecture by discussing our experience in designing, developing, implementing, and maintaining a real-world GEMA study, called the Postpartum Mothers Mobile Study (PMOMS).

Background

In this section, we provide the background and overview of EMA and GEMA and discuss their relationships with a well-established technology known as location-based services (LBSs).

Ecological Momentary Assessment (EMA)

EMA is defined as “methods using repeated collection of real-time data on subjects’ behaviour and experience in their natural environments” (18). The term EMA first appeared in 1994 (4) and encompasses a variety of pre-existing approaches but also is based on instant data collection in participants’ natural environments (19). Since then, these approaches have evolved independently in different disciplines. As a result, it was given different names, such as experience sampling (20), daily diary assessment (21), and others. EMA entails a variety of target assessment approaches, different technologies, and different data collection schedules to achieve its goal (4). Example EMA platforms include paper and pencil or diary (22), pagers (23), palm-top computers (24), telephones (25), personal digital assistants (26), mobile phones (8), and wearable devises (12).

Smartphones are proven to be the most natural and ubiquitous technology to collect EMA data (27). as they provide new opportunities to enable automated collection of EMA data such as time, duration, and location. Perhaps the most important advantage of smartphone-based EMA is that participants do not generally need to be trained on using them since people use them for their own personal activities. As a result, there has been a growth in EMA studies that are being designed and implemented for smartphones. Currently, there are many applications of smartphone-based EMA in a variety disciplines to study a very wide range of behaviours, experiences, and conditions. Burns et al., (2011) performed an experiment to determine technical feasibility, functional reliability, and patient satisfaction while studying and predicting patients’ mood and emotions. Bossmann, et al., (2013) and Walter et al., (2013) investigated the relationship between mood, affective states and physical activity. Runyan et al., (2013) investigated self-awareness and time management of the students using smartphone-based EMA.

Geographically-explicit Ecological Momentary Assessment (GEMA)

Marking locations (raw latitude and longitude coordinates) of EMA contexts (i.e., GEMA) provides a new opportunity to study and analyse participant activities as a function of location (32). GEMA has been applied in several studies. For example, to understand the relation between tobacco retail outlets and smoking behaviour, in addition to EMA surveys and participant’s location, the location, and density of tobacco retail outlets were also used in an analysis (15,33,34). A similar study was conducted to examine the associations between contextual factors and within-person changes in snack food and sweetened beverage consumption (35) where the location and density of fast-food restaurants and convenience stores were used for the analysis. Mennis, Mason, & Ambrus, (2018) designed a GEMA study to investigate the association between urban green space and stress, where they consider the location of parks and squares as indicated by high values of normalized difference vegetation index score. GEMA data (location and ratings of mood, stress, and drug craving) were collected and used to assess mood and behaviour as a function of neighbourhood surroundings (37). In short, GEMA can be used to collect geo-social information that includes local stores, park conditions, neighbourhood crime, and other system-level economic factors that cannot be obtained by EMA alone.

EMA, GEMA, and LBSs

In this subsection we discuss and analyse the relationships among EMA, GEMA, and LBS. Table 1 includes a summary along with information about various features including methodology, study, data type, and data collection tools related to each. LBSs are services that combine mobility and location information for making real-time decisions in applications (38,39), some of which are for collecting data in real time. One important observation from this table is that the main difference between EMA and GEMA is the inclusion of location information in addition to the contexts in GEMA. This means that in EMA studies, data about exposures are collected in real time but information on location either is not provided at all or is expected to be provided by the participant; whereas in GEMA studies, the same exposures as with EMA can be marked with the location in which they occur automatically. Another observation from this table is that GEMA is a specialized LBS, in which EMA data along with their locations are collected. Table 1 also summarizes the general requirements of GEMA, which are really all the requirements of EMA data stamped with location information.

Table 1.

Relationships between EMA, GEMA, and LBS

Feature	EMA	GEMA	LBS
Methodology	Capture of experiences in real-time via self-report or passive sensing (e.g., monitors)	EMA + location	Services that integrate a mobile device’s location with other information in order to provide new information to a user
Purpose	Collect exposures in real time	Collect exposures and their locations in real time	Add location data to a variety of users’ activities to learn new information and meet specific objectives
Focus	Participant’s current State	EMA + location (location reported by participants or automatically captured)	User’s current location + activities
Study	Understand exposures in real time	Understand exposures and their locations in real time	Varies depending on application
Data type	Text, message, numeric, alphanumeric, date/time related to participants’ current state	EMA data type + location	Numeric, alphanumeric, date/time related to user’s location
Data collection tools	Written diaries, electronic diaries, physiological sensors, cell phones, smartphones	EMA tools + location	Mobile geographic location technologies that collect users’ current location
Data collection time	Varies depending on study: fixed, random, or other sampling schemes	EMA + location	Continuous, random, or through other sampling schemes based on current time, time interval, distance, direction, origin, or destination
Data collection frequency	Repeatedly over a pre-defined period of time	EMA + location	Varies depending on application
Data collection location	Participant’s natural environment	EMA + location (at predetermined locations, at locations with certain features, at a distance to certain places, randomly, continuously)	User’s natural environment and/or specific locations
Privacy	Protect participant’s privacy	EMA + location (location privacy)	Location privacy

GEMA Architecture

As a specialized LBS, a client-server architecture which is designed for a web application deployment and consists of several server components (multi-tier client-server web application architecture) is proposed for GEMA. While each GEMA study has its own specific requirements, such as EMA contexts and locations of importance, our proposed architecture can be used as the foundation for all GEMA studies and be customized to meet specific objectives of different GEMA studies.

Key features of the proposed GEMA architecture include:

• Utilization of widely used smartphones to make GEMA studies practical.

• Performance of less functions on participants’ mobile devices (lightweight clients) and performance of most functions on servers (heavyweight servers).

• Utilization of at least one positioning sensor to determine and mark EMA contexts with locations.

• Communication through the Internet, which is an integral component of smartphones and whose coverage is increasingly more pervasive and continued improvement in performance.

Our rationale for proposing the client-server architecture for GEMA studies stems from the observations on previously conducted EMA and GEMA studies by others, our own GEMA study, and other similar studies with mobile applications. Our proposed client-server architecture for GEMA studies has advantages at both the server-side and the client-side.

Advantages on the server-side include the possibility for researchers:

• To be in full control of their studies and provide as much details on surveys and other pertinent information as they deem appropriate to participants.

• To be able to modify their own original GEMA design, e.g., number, type, and schedule of surveys, during the study.

• To update and upgrade technologies during a study without participants’ intervention and with little, or no, impact on the study.

• To store all data and pertinent information in one place with appropriate public health privacy and security procedures.

Advantages on the client-side include:

• Implementation of GEMA studies on participants’ own devices (e.g., smartphones) that they are comfortable with and use for their personal activities.

• Reduced chances of missing data that may impact the overall study in case of improper functioning of participants’ devices.

Depending on the underlying study, the server-side would be responsible for various functions, the most common of which are:

• Setting up new participants - Registration of new participants through their smartphone and storage of their contact information in appropriate files/tables in a database.

• Communication with participants - Establishing a communication mechanism where each participant can be reached for survey data. This communication could be through text messaging, e-mail, etc.

• Preparing surveys - Creating a set of GEMA questions, or other survey details that researchers want to communicate with and collect from participants.

• Scheduling surveys - sets survey schedules at pre-determined or random locations and/or times.

• Sending reminders - reminds participants about regular survey items that they need to complete.

• Receiving survey data - receives each survey item from each participant.

• Storing survey data - stores the received survey data in appropriate files/tables in a database.

• Generating reports - will generates various reports on stored survey data.

In addition to the in-house servers, the study may require functionality supported through third-party servers (note that in this paper a third-party server, solution, or service is a reference to any component of the architecture which is made available from outside the home institution). The lack of expertise, cost, or both is the main reason for including third-party servers for additional functionality. Important considerations for integrating third-party servers for new functions include the interaction mechanism between the in-house servers and the third-party servers and the type and level of security measures for those interactions. Most third-party servers provide an application programming interface (API) for interfacing with their solutions. APIs allow a level of data abstraction and a low-level functionality abstraction enabling easier integration of information into GEMA studies. Such abstractions would free time for researchers and developers to focus on tasks more specific to their GEMA studies. Security for these APIs is often handled by web-based authorization methods that comply with the medical research security and privacy standards.

The client-side of the architecture is mainly responsible for all communications between the server-side and the participants. It is also responsible for acquiring location data and sending it to the server-side actively or passively and with prior participant consent. Participants can use a handheld internet-enabled device for answering survey questions at designated times. Considering the variety of mobile devices that participants may use for their own activities as well as GEMA studies, it is of practical importance to design GEMA studies on participants’ own devices (smartphones) to avoid carrying extra devices just for the purpose of research studies. The client-side would be responsible for such common functions, such as:

• Registering new participants - features a set of simple-to-use and easy-to-follow user interfaces to allow the process of registering new participants in GEMA studies.

• Receiving reminders - receives reminders about survey data sent by the server-side.

• Sending survey data - through simple-to-use and easy-to-follow user interfaces, this function allows participants to fill in and submit survey data to the server-side.

• Sending location data - allows participants to either enter data through interfaces, or the participants’ smartphones automatically obtain and send location data to the server-side.

On the location data function, important decisions regarding location data acquisition through smartphones that would impact study results include:

• Accuracy. Location accuracy may range from fine (within meter) to acceptable (a few meters) and to meet the required level of accuracy, there may be a need to integrate different positioning sensors and develop appropriate techniques. For example, while it is possible with Global Navigation Satellite System (e.g., GPS) in outdoor environments to obtain 10 m accuracy through GPS chips on smartphones, if the study requires finer accuracy, one possible approach is to integrate GPS with other sensors on smartphones such as accelerometer and gyroscope through a Kalman filter (40,41).

• Frequency. Depending on the study, the frequency at which location data needs to be collected may range from low (few locations) to high (hundreds or thousands of locations) during a day. Low frequency location data collection has an insignificant effect on the battery usage whereas high frequency location data collection can quickly drain the battery and cause interruption in the study and the regular activities of the participants.

• Active/passive. Location data collection may occur actively (participant enter it), passively (smartphones at specified locations and/or times collect it without participants’ intervention), or both (location data in some places or times is collected actively and in some other places passively).

• Privacy. Regardless of location data being collected actively or passively, participant consent must be obtained. This means that participants must agree that their locations during the study will be obtained and stored in a database for analysis.

• Environment. The environments within which GEMA studies are conducted may be outdoor only, indoor only, or both. Since GNSS either has no signal or offers very low accuracy positioning results in indoor environments and since, unlike outdoor environments where GNSS infrastructure is standard and predominant, there is no one approach or one sensor that can work in each different indoor environment, utilization of a different set of positioning sensors in each different indoor environment may be required. For example, in one building RF-based positioning (42) may be used while in another building non-RF-based positioning such as vision and accelerometer may work better (41).

To better understand the advantages and challenges of this architecture, in the remaining pages we describe the details of an architecture for a GEMA study, called PMOMS (11) (Mendez et al., 299), which we have been conducting since February 2017. The details of the PMOMS architecture will help researchers design GEMA studies that are effective, efficient, and meet the study requirements and will help developers prepare the protocols appropriately and save time and money when developing new GEMA software.

PMOMS: Case Study

The Postpartum Mothers Mobile Study (PMOMS) (11) is a longitudinal observational study with the goal of understanding contributing factors to racial/ethnic disparities in postpartum weight and cardiometabolic health in the context of childbearing. The main purpose of capturing EMA data in PMOMS is to understand women’s health and well-being in real time using smartphones and related technologies. This includes assessing experiences of stress, racism, behaviours, and contexts (e.g., behaviours, moods) in real time. The ultimate goal of PMOMS is to have a more in-depth understanding of women’s changes in health after pregnancy. PMOMS is distinct from other studies regarding pregnancy-related weight, in that it allows participants to contribute their own data and measurements while in their usual environments. For details of what PMOMS study is about and the types of EMA contexts collected in PMOMS, see (11).

The PMOMS study was designed to specifically collect location data with each survey that was administered. The location data was not automatically collected with each survey, but participants were prompted (e.g., asked for permission) for their location data. Once permission is granted, a GPS location from the participants’ device is provided to PMOMS. The location data are transferred by a secure connection to protect participants’ privacy. The University of Pittsburgh Institutional Review Board approved this study in consideration of ethical and privacy concerns. These location data allow for capturing environmental exposures of PMOMS participants. In other words, repeatedly collecting self-reported measures with the participant’s location, enables obtaining various environmental exposures over time. In addition, by using location data, mobility patterns among participants can be uncovered and geospatial analyses can be performed that can identify clusters and patterns of health risks, and behaviours.

PMOMS Architecture

In this section we present the details of the PMOMS architecture as a representative GEMA architecture. PMOMS client-server architecture has four tiers—presentation tier, logic tier, database tier, and third-party tier—and was implemented by using several different technologies. Table 2 shows the technologies used in the PMOMS software along with the purpose for each. Figure 1 shows a Unified Modeling Language (UML) diagram for PMOMS.

Table 2.

Technologies used in the PMOMS software.

Technology	Purpose
Tomcat	The servlet used to support a runtime environment for Java web applications
Java Servlet	An environment to process Java classes in Java EE that complies with the Java Servlet API
Microsoft SQL Server	A database management system to store, manipulate, retrieve, and manage data
SAS	A statistical software package to generate the required random values
G Suite Simple Mail Transfer Protocol (SMTP) Relay Service	A communication mechanism to send text messages and prompts
JavaScript, jQuery	A programming language and a library to manage the client-side interaction with participants
JavaServer Page (JSP)	A tool enabling creation of dynamic, platform-independent method for building the administrative module of PMOMS
HTML	A markup language for describing the structure of contents of the generated web pages by webserver
CSS	A stylesheet language for presentation of the HTML content
Bootstrap	A framework for presentation of the mobile web application and Responsive Web Design
OAuth1.0 and OAuth2.0	An open standard for token-based authentication and authorization on the Internet to fetch participant’s weight data on a regular schedule from the third-party database
Smart Scale (Withings)	A device to measure weight and transfer participant’s weight data to the third-party database

Figure 1.

A UML diagram for PMOMS.

Presentation Tier

The presentation tier includes interfaces to communicate both static and dynamic information, including the forms, management panel, data, and responses, with participants and staff. The device for the presentation tier, or client, in PMOMS is either iOS or Android smartphones and the tier was developed using JavaScript, jQuery, and JSP technology. PMOMS participants are prompted with four types of surveys: Beginning of Day (BOD), End of Day (EOD), Random Assessment, and non-EMA. BOD and EOD survey times can be chosen by the participants, with the only requirement being that they must be nine hours apart. The random surveys are initially set when the participant registers in PMOMS. Random prompts are generated with the constraints with respect to the mean number and maximum number per day. The non-EMA surveys are of four types: (a) two weeks after starting the study as a pregnant participant, (b) one week after delivery, (c) 3, 6, 9, and 12 months after delivery, and (d) 50 weeks after delivery to exit the survey.

The scheduler function runs continually throughout the day and reads the scheduling table in the database to know when to prompt the participants for their BOD, EOD, random, and non-EMA related surveys. When the scheduler comes upon a survey that is due to be delivered, it will send a text message to the participant with a link to follow and complete the requested survey. In addition to prompting the participant with scheduled surveys, the scheduler can put a hold on prompting participants to complete surveys for a specified duration of time. In our current version of PMOMS, a hold up to 30 days can be implemented and the participants can complete surveys for 7 days after the delivery. Figure 2 shows sample screenshots of the client-side in PMOMS.

Figure 2.

Sample screenshots of the client-side in PMOMS.

Logic Tier

The logic tier is a collection of all functions, most common of which were discussed in the last section, and its main objective is to guarantee that all activities in PMOMS are performed continually and based on the design specs. Rules and algorithms for assessing participants’ circumstances and for collecting the questionnaires are all managed in this tier. A devoted server was rented from the Computing Services and Systems Development department of the University of Pittsburgh for this tier which was developed using Java Servlet technology.

The logic tier in PMOMS performs the following functions:

• Sends messages to individual participants for instructions and reminders.

• Sends messages to all participants collectively for events like software upgrades or major changes impacting all participants.

• Produces high-level dashboard reports for quick checks on the status of the study or the software itself.

• Produces reports to check participants’ submission compliance.

• Edits account information for participants or putting a hold on sending surveys to specific participants.

• Manages the third-party solutions, especially in case a participant’s device needs to be registered with PMOMS.

• Informs the research team in case a participant experiences difficulty in using PMOMS or completing surveys.

• Requests a pause for a period of time to submit surveys by the participant.

• Alerts the correct authority (in the research team or others) in case the participant demonstrates signs that can lead to self-harm.

The logic tier performs the following administrative functions:

• Invites new participants to the PMOMS study.

• Sends messages to individual participants.

• Sends messages to all participants.

• Pauses individual participants.

• Pauses all participants.

• Registers and authorizes the smart scales.

• Alerts the correct authority when a participant feels sad/blue or there is a sign of self-harm.

Database Tier

PMOMS’s survey data (BOD, EOD, Random Assessment, non-EMA) are set to be collected at constant intervals or random times. BOD and EOD surveys are commonly taken on a daily basis, while Random Assessment and non-EMA surveys are taken on random or less frequent schedules, respectively. All survey data are collected and stored in a relational database management system (RDBMS) in appropriate schemas in the database tier. Besides being the main repository of PMOMS data, the database is intended to be used for a variety of queries about participants, generating a variety of reports, and analyzing data related to the objectives of PMOMS study. The database has a number of features to address the specific needs of PMOMS. The database easily captures data as objects and groups them as conceptual units such as a survey or a participant instead of long strings of data. Such grouping also allows for the quick querying and analyzing the data using the structured query language (SQL). A devoted server behind a firewall was rented from the Computing Services and Systems Development department of the University of Pittsburgh for this tier and the database system is MS SQL Server.

In addition to the data entered by the participants, metadata plays an important role in PMOMS. The metadata can help the PMOMS research team find important GEMA-related information such as the influence of locations on a participant’s response or if certain locations or responses are indicative of certain activities or behaviours. Examples of the information the metadata can provide include the time when a participant takes a survey, the length of time it takes for a participant to complete a survey, and the frequency at which a participant completes surveys. The metadata also can be used to influence the surveys’ configuration. Questions on the surveys also can significantly vary based on the responses to other questions. The metadata in PMOMS can also help the research team monitor different activities such as how often participants complete surveys, how long participants require to complete surveys, and how changes to the number of surveys issued or shortening the time required to complete surveys impacts how participants respond.

Third-Party Tier

The third-party tier in PMOMS includes a scale service, phone carriers, and Google G Suite. The scale service is a database, maintained by Nokia, as a depository of weight data of PMOMS participants by the smart scale. It includes a device that participants use at home to take and store readings daily. The readings are stored in Nokia’s servers and are made available to the participants through a mobile application.

In PMOMS, we download the readings through a secure web-based API, a process that runs nightly, using the OAuth2 security protocol, where data are routinely transferred to the database tier. As PMOMS has participants’ permission to access their data, token keys and secrets are created and stored in the Database tier. To accommodate participant’s own phone services, PMOMS supports AT&T, Verizon, T-Mobile, and other phone carriers. The rationale for this is that participants do not need to sign up for additional phone carriers besides the one that they have been using for their own personal use. Google G Suite is used in PMOMS to establish a communication mechanism between the PMOMS administrators and the participants to send text messages and prompts.

Communication

In PMOMS the communication between participants’ smartphones and the logic server is secured by the TLS/SSL protocol. The communication between the two in-house servers (logic tier and database tier) is secured by a firewall system carried out by the Computing Services and Systems Development department of the University of Pittsburgh. Figure 3 shows the tiers in PMOMS and the interactions among them. As surveys must be received by participants at scheduled times, data goes from the logic tier to the presentation tier (smartphone). For loading planned surveys on participants’ smartphones, a text message that has a URL of the prompts is sent to the participants. Two reminders about surveys and a URL are sent if responses are not received by the logic tier. The URL is valid for a certain time window (30 minutes for BOD and EOD surveys and 60 minutes for RAND survey) and will be expired after that time. Once the URL is expired, a text message showing that the survey is cancelled will be sent to the participant. If a participant does not start the survey while it is active, a record of “not attempted” will be inserted into the database and the survey will not be accessible for that day. However, a survey that has begun or is continuing, but did not submit in the time window, is still valid for submission for an additional hour. In addition to surveys, the invitation URL is sent from the logic tier to the presentation tier. An important feature of PMOMS, as a GEMA study, is location determination and reporting. Upon signing up, the PMOMS software requests participants’ permission to record GPS coordinates (longitude, latitude) obtained through their smartphones at certain locations and times; PMOMS participants are requested to enter a location (home, workplace, other’s home, bar/restaurant, vehicle, outside, other places) for each survey they submit. Once permission is granted, through the HTML5 Geolocation API, the participants’ device locations are provided to PMOMS. To protect participants’ privacy, location data are transferred by a secure connection.

Figure 3.

The four tiers of the PMOMS software and the interactions among them.

Data Flow

Figure 4 shows the data flow in PMOMS. The diagram has three external entities—Participants, Administrator, and Scale—and three databases—Main database server, Flat file storage, and third-party Nokia Withings’s database. There are several processes responsible for data manipulation and the three scenarios below are used to explain them.

Figure 4.

The flow of data in the PMOMS software.

Submitting a survey scenario.

This scenario has three main processes: send alarm, send survey, and submit survey. The send alarm and send survey are activated at the same time. The inputs to the send alarm are alarm time, survey type, and contact information for each participant. The output is a text message to the participants, implemented by using the G Suite SMTP relay. The inputs to the send survey are alarm time, survey type, and participant information for each participant. The output is the set of questions to be asked from the participant. Before sending a survey to a participant, this process performs an additional participant verification, which gets inputs from sending the survey process and from the participant’s smartphone. If the participant completes and submits a survey, the submitted survey process will be triggered. The inputs to the submit survey are the answers to each survey question and the location and data about the survey duration. The outputs are the participant’s answers modified to store them in the database, or alert messages that are based on how the participant answered or did not answer the survey.

Administrative data management scenario.

There are several processes that can be used by the PMOMS administrators. One process is the contact to participant process which involves receiving text data as an input and sending a text message to the participant using the same G Suite SMTP relay. In addition, participants can contact the PMOMS administrators by using the contact to administrator process that involves receiving text data and sending the data through an e-mail to the administrators. The administrators can view, pause or change participant’s information by using the manage participant data process which involves receiving updated information from the administrators and sending a query to the database as an output.

Obtaining weight data scenario.

A participant’s weight data, measured by a scale, is transferred to the third-party database using participant’s phone application connected to the scale. Once recorded, the weight data are downloaded from the Nokia Withings’s database to PMOMS’s database using the fetching weight data process.

Deployment and Recovery

Any changes made to the PMOMS software are conducted by the software developer who implemented the code. For best practices, we designated a team of two software developers, who received training on the developed code and the PMOMS requirements, to handle the changes to the PMOMS software and deploy the updated version of the software. The team coordinates with the database administrator to ensure all changes comply with the current state of the database, or the database is updated in a way to comply with any changes made to the code. The deployment of each new version of the PMOMS software is usually scheduled during a time when no participants are expected to use PMOMS. This will ensure that updating the PMOMS software will not disrupt service to the participants and risk the potential of losing survey data. The last version of the software is saved before deploying the new version. This will ensure that if any issues arise with the new deployed version, the previous working version can be used. For this, nightly automated back-ups of the PMOMS software and database are performed. By creating an automated image of the PMOMS software on a daily basis, any issue can be recovered with a minimal loss to the data collected.

Maintenance Issues, Challenges, and Lessons Learned

PMOMS was designed and implemented before the recruitment of the participants. Although PMOMS was tested internally with colleagues and others who were not participants in the study, the only way to receive real feedback on the design is when the participants use the PMOMS software to submit survey questions. Maintaining the PMOMS software is very challenging for a number of reasons, most important of which are: (a) lack of using common mobile devices by participants, as each participant may use a different smartphone with a different operating system; (b) dissimilar experience gained and type of problems encountered by each participant using PMOMS; (c) the PMOMS software reliance on the proper functioning of several different technologies (see Table 2) individually and in tandem with each other; and (d) continuous modifications of the PMOMS software as participants provide new feedbacks and observations, the research and administrative teams gain new information and updates, and the technologies are updated or upgraded (some requiring installations of new tools that may require updating the PMOMS software). We have categorized the PMOMS software issues that we have received since the study began into three groups: infrastructure (Table 3: issues related to the PMOMS architecture), application (Table 4: issues related to the PMOMS software), and third-party (Table 5: issues related to the third-party solutions).

Table 3.

Architecture related issues

PMOMS Infrastructure	Challenges Low: one day Medium: two-three days High: more than three days		Impact on study (severity) Low: few participants Medium: several participants (participants from one carrier) High: all participants	Lesson learned
	Time to find source of an issue	Time to fix an issue
Errors caused by deploying new version Final NonEMA Survey	Medium	High	High	The system requires server failure alerting, avoiding and recovering procedures
CSSD TOMCAT server failure	High	High	High

Table 4.

Software related issues.

PMOMS Software (upgrade, issue)	Challenges Low: one day Medium: two-three days High: more than three days		Impact on study (severity) Low: few participants Medium: several participants (participants from one carrier) High: all participants	Lesson learned
	Time to find source of an issue	Time to fix an issue
Issue: participants were able to submit characters instead of numbers	Low	High	Low	The system should be flexible and should process data from various phones and operation systems
Upgrades: Adding additional functions to the admin page such as pause over holidays and sending messages and contact to administrator.	Low	Low	Low	The architecture of the system should allow changes and additional function to the system
Upgrades: Upgrading main functionality of the system such as scale back, adding additional surveys and questions.	Low	Low	Low

Table 5.

Third-party related issues.

PMOMS Software (upgrade, issue)	Challenges Low: one day Medium: two-three days High: more than three days		Impact on study (severity) Low: few participants Medium: several participants (participants from one carrier) High: all participants	Lesson learned
	Time to find source of an issue	Time to fix an issue
Phone carriers: blocked messages sent from Google SMTP.	High	Low	Medium	The third-party servers can block another third-party server Google SMTP that requires actions
Web domain firewall system blocked access to the patricians with T-Mobile carrier.	High	High	High	The third-party servers can be blocked by firewall systems
SMTP relay systems has a threshold for sending certain number messages per day.	High	High	High	Check number of messages send per day and SMPT relay capabilities
Transition from NOKIA to Withings affected the process for fetching the weight data	Medium	High	N/A	The third-party servers can perform upgrades that requires update application code
Withings Oauth1 was switched to Oauth 2 Withings: upgraded the protocol to access the new API; this led to update application code	Medium	High	N/A

Table 3 shows sample infrastructure problems that were encountered in maintaining the PMOMS software and Table 4 shows sample software issues that were encountered in maintaining the PMOMS software. Each software issue is categorized as either “issue”, the software did not respond to its design specifications, or “upgrade”, the research team and/or the administrative team requested changes to the original design or adding new functions to the PMOMS software. Table 5 shows sample third-party issues that were encountered in maintaining the PMOMS software. As the first column of this table shows, these issues may arise from any or all technologies used in PMOMS including mobile phone carriers, Google, and Withings. In each table, there are two columns, one on challenges and another on impact on study. For each issue in each table two types of challenges are highlighted, one is the time it took the developer team to find the source of the issue and another the time it took to fix the issue. The severity of each challenge is marked by low (one day to find the source or fix the problem), medium (two-three days to find the source or fix the problem), or high (more than three days to find the source or fix the problem). For study impact, each issue is marked as low (few participants), medium (several participants), or high (all participants.)

For the purposes of efficiency in addressing issues, consistency in finding the source issue and fixing it, learning from previously reported and addressed issues, and providing a guideline for addressing issues by the current and future development teams, we have developed a procedure for handling issues in the PMOMS software (see Figure 5). As shown in this figure, once a new issue is reported, it is analysed to find out its source. As part of this analysis, the issue is reproduced so that it can be tracked to find its source. Once the source of the issue is found and the issue is fixed, it is tested on a local machine by the developer team. After the testing, if the developer team is confident that the issue is fixed, it is then tested on a development server and by some members of the research and administrative teams. If the result of this testing shows that the issue has been fixed, the new version of the PMOMS software is deployed on the logic server.

Figure 5.

A procedure to handle issues in maintaining the PMOMS software.

An important aspect of GEMA-based software to consider is exceptional cases. For many reasons, GEMA-based software, like PMOMS, will run into exceptional cases and it is important to be able to recover as much information as possible from the participants before they encounter an exception. In the PMOMS software, we store all inputs from the surveys in text form on the server. This captures the data before the software attempts to store it in the database. Although the PMOMS software has been extensively tested for different issues by various groups, it is still inevitable to receive errors where participants might not be able to submit their responses successfully. In these cases, to avoid missing any responses, all responses are logged into a text file before submitting them. For this, we developed a stand-alone application with the same logic layer which inserts the logged responses into the database. By using this application, we can avoid inserting manually and not having any error and inconsistency related to data entry.

Conclusion

EMA can be designed to collect various contexts such as psychological states (5), smoking cessation (6), alcohol use (7), eating patterns (8–10), understanding pregnancy and postpartum health (11) and cardiovascular health (12) through smart devices and wearable sensors. One of the main reasons EMA has become a widely used approach is that it allows for data collection through common and widely available technologies such as smartphones and wearable devices that are used by people even outside of research participation. When technologies, such as smartphones, are used for EMA, research study challenges such as making multiple trips to a facility or filling out time-consuming survey protocols, are reduced, and also important, individuals are not asked to recall distant past events or emotions.