Tracking post-trauma psychopathology using mobile applications: A usability study

Abstract

Trauma exposure markedly increases risk for psychopathology including posttraumatic stress disorder (PTSD). Understanding the course by which PTSD develops after a traumatic event is critical to enhancing early intervention. Although prior work has explored the course of PTSD symptoms in the subsequent months, relatively few studies have explored the course of symptoms in the acute post-trauma period, defined as the 30 days after a traumatic event. A key challenge to conducting such studies is the lack of efficient means to collect data that does not impose significant burden on the participant during this time. The present study evaluated the use of a mobile phone application to collect symptom data during the acute post trauma period. Data was obtained from 23 individuals who experienced a Criterion A traumatic event and were recruited from the Emergency Department of a Level 1 Trauma Center. Participants completed 44.93% of daily assessments across a 30-day period. Responses rates were uncorrelated with PTSD symptoms or depression symptoms at 1-month and 3-month posttrauma. Participants reported that the surveys were moderately helpful and posed minimal burden. These findings suggest that mobile applications can be used to learn about the course of post-trauma recovery.

Exposure to traumatic events increases risk for several mental health disorders including Posttraumatic Stress Disorder (PTSD). The prevalence of PTSD 12 months after a traumatic injury ranges from 12.3% to 22.9% [1], [2]. PTSD poses a significant public health concern in that it is associated with long-term disability, even among individuals whose symptoms resolve [3]. Because PTSD results from a known event, early intervention delivered in the acute post-trauma period, defined as the first month after the trauma, can prevent chronic cases and the associated maladaptive outcomes [4], [5]. In order to develop and improve early interventions, however, tools to monitor the onset of PTSD symptoms in the acute post-trauma period are needed.

PTSD symptoms manifest after exposure to a traumatic event that meets Criterion A of the DSM 5 diagnosis of PTSD [6]. After a traumatic injury, most individuals will receive treatment through an emergency department or hospital for physical ailments [7]. The clear point of contact (i.e., acute care center) and the presence of a known event create a seemingly ideal context for post-trauma assessment. However, a range of pragmatic issues limit efficient and effective assessments during this period [8]. Patients recently exposed to a trauma face a range of pressing concerns [9]. Injury complications, relocation from hospitals to rehabilitation centers, coordination of home care, and financial repercussions prevent trauma victims from engaging in additional activities such as time-consuming assessments. As a result, much of our knowledge about recovery from traumatic events is limited to assessments that occur months apart via telephone or face-to-face interview [10], [11]. There is a need for strategies that gather data, at minimal burden to the participant, during the acute post trauma period.

Ecological Momentary Assessment (EMA) holds considerable promise as a method to obtain accurate information about emotions, functioning, and activity throughout the acute post trauma period [12]. Although EMA originated using paper-and-pencil methods, technology has considerably expanded the data that can be captured. Mobile devices specifically offer a means to overcome the burden and methodological challenges of carrying out EMA during the acute post-trauma period [13]. Mobile device ownership is near ubiquitous among adults in the United States with 92% of American adults owning a mobile phone and nearly two-thirds of Americans owning a smartphone [14], [15]. Although smartphone ownership is greater among those with higher SES, approximately half of adults earning less than $30,000 per year own a smartphone. The proportion of smartphone ownership is expected to increase in the near future, with affordable plans making smartphone ownership feasible for low-SES populations. Measures administered via smartphone provide comparable responses to measures administered via traditional methods [16]. Recent work has suggested that EMA data, combined with predictive analytics, identified the presence of suicidal ideation in a high-risk sample [17]. If this method of data collection were used by those exposed to a trauma, a smartphone-based mobile applications could allow for a low-burden assessment of PTSD symptoms after a trauma to identify those at risk for more severe pathology.

Preliminary research on the use of mobile applications with trauma-exposed samples suggest this modality is acceptable. A survey of emergency department patients indicated that 89% owned mobile phones with 51% owning smartphones [18]. Importantly, the incidence of use of mobile applications and text messaging were consistently high across ethnic and income groups. A laboratory-based usability study with those who previously experienced a traumatic event showed that a mobile application-based symptom tracking system was highly usable and preferred method of communication [19]. Participants reported that they were willing to complete 2-3 assessments per day for 4-5 days per week during the acute post-trauma period. Taken together, these data suggest that those who have experienced a traumatic event are receptive to using mobile applications to monitor symptoms during the acute-post trauma period. However, it remains unclear if these attitudes will translate to actual use after a traumatic event. Addressing this gap in knowledge is important as positive attitudes towards the use of a technology do not necessarily translate into actual use.

Only one study to date evaluated the use of mobile phones to assess post-trauma mental illness during the acute post-trauma period [20]. Participants were recruited from a level 1 trauma center after an injury that met Criterion A for a diagnosis of PTSD. Upon discharge, participants were asked to respond to a single question about their recovery via text message for 15 days. The average response rate was 63.1% (9-10 responses out of 15), which suggests that use of this strategy was lower than what was reported with laboratory and survey studies. A single question was used to minimize burden, but limited the assessment of multiple PTSD symptoms. The use of a mobile app that can administer multiple questions, however, may result in additional burden that may affect response rates. Thus, additional feasibility studies on the use of mobile applications in post-trauma samples are warranted.

The primary aim of the current study was to evaluate the use of a mobile application to track PTSD symptoms during the acute post-trauma period. Metrics of engagement and adherence were evaluated. The relation between metrics of adherence and psychopathology at baseline, one, and three months post-trauma were evaluated to determine if symptoms influenced response rates.

Methods

Participants

Participants were recruited from a Level 1 Trauma Center Emergency Department (ED) for treatment of a traumatic injury. The inclusion criterion for the study was having directly experienced a trauma that met criterion A for a diagnosis of PTSD according to DSM 5. Such events involve actual or threatened death, physical injury, or sexual violence. Individuals who only witnessed, but did not directly experience the event, were excluded. Exclusion criteria were current suicidal or homicidal ideation, being in police custody, being a non-English speaker, active psychotic symptoms, or non-ownership of a smartphone. Patients with severe cognitive impairment (e.g., moderate or severe TBI) were also excluded because valid consent could not be obtained. Electronic medical records (EMR) for the patient’s ED visit were reviewed by trained research assistants to determine if the individual experienced a criterion A event. When a potential participant was identified, research assistants consulted with the treating provider to determine if an ED patient met criteria for inclusion and could be approached. Participants were representative of Northern New England (Table 1). All participants provided consent for inclusion in the study and an institutional review board approved all study procedures.

Table 1.

Demographics statistics for the obtained sample.

	N	%
Female	14	53.8
Race
White	21	80.8
Latino	1	3.8
Asian American	1	3.8
Bi-racial	1	3.8
Other	2	7.7
Insurance Type
No Insurance	1	3.8
Medicare/Medicaid	6	23.1
Private Insurance	13	50.0
Other	5	19.2
Annual Income
< $5,000	5	19.2
$10,000 - $15,000	2	7.7
$15,000 - $30,000	4	15.4
$30,000 - $50,000	5	19.2
> $50,000	9	34.6
Type of Smartphone
Android	16	61.5
iPhone	9	34.6
Blackberry	1	3.8
	M	SD
Age	27.56	13.16

Measures

Standardized Trauma Interview (STI, Foa et al.)

The STI is an interview assessing relevant aspects of the trauma (e.g., met Criterion A) and related demographic information. An abbreviated version of the STI was administered due to the time constraints of working with patients in the ED. The abbreviated version was used to obtain descriptive information about the traumatic event including an overview of the event, the time of day that it occurred, an approximation of the location of the event, and how much the participant slept since the event occurred.

MINI International Neuropsychiatric Interview for the DSM-IV (MINI; [21])

The MINI is a brief interview designed to assess the presence of psychopathology according to DSM-IV criteria, including mood disorders (major depressive episodes, mania, hypomania), anxiety disorders (panic disorder, agoraphobia, social anxiety, specific phobia, obsessive-compulsive disorder, generalized anxiety disorder), and substance abuse disorders (substance abuse or dependence and alcohol abuse or dependence). The MINI has shown to perform comparably to longer diagnostic instruments such as the Structured Clinical Interview for the DSM [21]. The MINI was conducted at 1- and 3-month follow-up to evaluate the diagnostic presentation of the participant.

PTSD Checklist-5 PCL; [22]

The PCL is a 20-item self-report measure that assesses PTSD symptoms experienced over the last month according to DSM 5 criteria. Items assess symptoms across 4 symptom clusters of PTSD (re-experiencing, negative mood, avoidance, and hyperarousal) on a 0-4 point Likert scale. Total scores range from 0-80.

Patient Health Questionnaire-8 PHQ-8; [23]

The PHQ-8 is an 8-item self-report measure that assesses depression symptoms experienced over the past two weeks. Ratings are made on a 0-3 point Likert scale regarding the frequency with which a symptom has been experienced. Scores range from 0-24, with higher scores indicating more severe depression. The PHQ-8 is adapted from the PHQ-9 and is identical except for the removal of an item on suicidal ideation.

Illness Intrusiveness Rating Scale (IIRS; [24])

The IIRS is a 13-item self-report measure that assesses the extent an illness interferes with important life activities. Responses are made on a 1-7 point Likert scale, with total scores ranging from 13-91. Higher scores indicate greater impairment.

Mobile Application Questions

A brief survey of PTSD symptoms was used to assess symptoms on the mobile device. Questions were originally adapted from the PCL in consultation with experts in the areas of PTSD, acute trauma care, and learning theory (Table 2). These questions were used in a prior study that used text-messaging to assess recovery after a trauma [20]. A final question asked participants to provide a free-text response regarding their most pressing concern from that day.

Table 2.

Questions included in mobile survey

Construct	Question	Response Range
Arousal	How jumpy, tense, or on edge did you feel today?	1:Not at all – 7:Extremely
Re-experiencing	How much were you bothered by thoughts about the trauma today?	1:Not at all – 7:Extremely
Sleep	How well did you sleep last night?	1:Very poorly – 7:Very well
Pain	Overall, how was your pain today?	0:No pain – 10:Extreme pain
Current Concern	What is your biggest concern at the moment?	Free text response

Satisfaction/Usability Interview

Participants rated their experience using the application to track to their symptoms based on the technology acceptance model [25]. Ratings were made as to the helpfulness and bothersome nature of using the application on a 1-7 Likert scale. Participants indicated their preference as to the number of questions asked per assessment and the frequency with which assessments occurred. Finally, an open-ended question obtained qualitative information on the use of the mobile application.

Procedure

Participants were recruited from the ED at a level 1 Trauma Center. A trained research assistant approached prospective participants’ bedsides in the hospital and administered an initial assessment battery that included a demographics form and the STI (Figure 1). All research assistants completed extensive training in the administration of all measures, diagnostic interviews, and methods to work with patients in the ED. Training was conducted by a licensed clinical psychologist, an attending ED physician, and an EMT with expertise in working in an acute care setting. Participants were contacted via telephone within M = 4.81 (SD = 2.83) days of their traumatic event. During this phone interview, participants completed the PCL, PHQ-8, IIRS, and received instructions on how to download and install the mobile application on their mobile device. Spreading the assessment across the hospital visit and an initial telephone interview reduced participant burden in the hospital. The mobile application that was used in the present study, Metricwire (Ontario, Canada), allowed the interviewer to confirm if the participant had successfully enrolled in the study and provide technical support when needed.

Figure 1.

Flow chart of participant through the study.

Participants received a local notification to complete a survey on their mobile device each day for 30 days after the initial assessment. Notifications arrived randomly between 7:00 PM and 8:00 PM. Participants had 6 hours to complete a survey regarding symptoms for that day and were allowed to skip questions.

Follow-up interviews were conducted via telephone one and three months after the time of the initial phone interview. Interviews included administration of the MINI, PCL, PHQ-8, and IIRS. The 1-month interview also included a brief satisfaction survey about using the mobile application. Interviews were administered by trained research assistants and were audio recorded for accuracy of diagnoses recorded based on the MINI. A portion (20%) of the recordings were reviewed by a licensed clinical psychologist for diagnostic accuracy. A rating of 100% agreement was obtained.

Results

Of the 33 participants recruited from the ED, 26 were reached to download the mobile application. Of the 26 participants who were given access, 23 (88.5%) successfully downloaded the mobile application. Of the three participants who did not enroll in the mobile portion, two reported that they were without a mobile device during the initial phone interview and did not remember to download the application. Of the 23 who downloaded the mobile app, M = 13.48, SD = 8.57 surveys were completed across the 30-day period (Table 3). The average adherence rate was 44.93%, with 30.4% of the sample replying to 15 or more surveys. Among surveys that were completed, missing data was minimal, with only 2 items skipped across 2,480 items. Interestingly, 280 free-text answers were recorded across the 310 (90.3%) accessed surveys. Responses were uploaded immediately upon completion of each survey. Research staff reviewed responses each day for mention of high-risk behavior in free text responses. Across the 253 obtained free-text responses, none mentioned high-risk behavior. Participants were not incentivized or given feedback to sustain responding to the mobile survey.

Table 3.

Descriptive statistics for the current sample

Measure	Week-1*	Month-1	Month-3
PCL	27.43 (14.06)	18.48 (13.80)	17.27 (8.64)
PHQ-8	8.78 (6.03)	7.48 (4.90)	6.36 (4.77)
IIRS	35.00 (10.63)	28.86 (14.56)	25.87 (12.03)

Note: PCL = PTSD Checklist. PHQ-8 = Patient Health Questionnaire. IIRS = Illness Intrusiveness Rating Scale. Values in parentheses are standard deviations.

^*= Week 1 occurred M = 4.81 (SD = 2.83) days of the traumatic event.

Of the 23 participants who were included, n=5 (21.7%) met criteria for PTSD at the 1-month follow-up according to the MINI. This prevalence is consistent with other national samples of PTSD after a traumatic injury [2]. Response rates were negatively correlated with baseline PTSD symptoms, r = −0.45, p = 0.03, but not with symptoms at 1-month (r = −0.07, p = 0.76) or 3-month (r = 0.15, p = 0.61) follow-up. Response rates were not correlated with depression symptoms or disability at any time point (p’s = 0.16 to 0.90). There was no significant difference in response rate between those who met criteria for PTSD/sub-threshold PTSD and those who did not have a PTSD diagnosis at 1-month (t (19) = −0.08, p = 0.94) or 3-month (t (14) = 1.10, p = 0.29).

Satisfaction data were obtained from N = 22 participants. Responses were largely positive, with participants reporting that the helpfulness of the surveys was M = 5.09, SD = 1.26 (out of 7). Alternatively, the surveys were not found to be bothersome or troublesome M = 1.23, SD = 0.53 (out of 7). Satisfaction scores were unrelated to PTSD symptoms, depression symptoms, or disability at any time point (p’s = 0.15 to 0.75). The majority of participants felt that 1 survey per day was right amount (N = 13, 59.1%). The majority of participants felt that the length was appropriate (N = 13, 61.9%) and a handful of participants would have preferred more questions (N = 5, 23.8%).

Participants offered qualitative feedback regarding the use of mobile applications to monitor post-trauma outcomes. A few participants (n = 7) felt that the check-ins were helpful initially, but became repetitive. It was recommended that question content vary over the course of the assessment period. They would have preferred an option to notify a provider that they no longer wanted to complete the assessments or that they had achieved a level of recovery such that continued observation was no longer warranted. Several participants were reluctant to discontinue use of the application without first notifying the research team. They also requested questions be tailored to their specific symptoms rather than a standard assessment. For example, a participant who had blurry vision would have preferred a question about their vision. A large portion of participants (n =10) requested personalized feedback on their recovery progress. The type of feedback requested varied from graphs of their progress to specific recommendations about how to improve their recovery.

Discussion

The results of the present study are among the first to demonstrate that mobile applications deployed on patient-owned mobile devices can assess PTSD symptoms during the acute post-trauma period in those who have experienced a Criterion A traumatic event. Prior investigations on the longitudinal course of PTSD symptoms have relied on assessments across several months, limiting our understanding of how symptoms may develop shortly after the trauma has occurred. The use of mobile devices to monitor symptoms presents a low-burden and low-cost method with substantial reach to learn about recovery during this critical period. The methods presented in the current study set the stage for a more comprehensive investigation of the how symptoms develop during the acute post-trauma period.

Participants in the present study completed slightly less than half of the 30 assessments, which should be considered in light of the population. All participants were recruited in the ED and began monitoring their symptoms within 5 days of their trauma. Prior work has shown that individuals face a range of competing demands during this period and asking them to complete additional tasks may prove challenging [9]. The number of mobile survey responses obtained in this study is compelling given that many of the participants identified numerous post-trauma concerns during follow-up and they were not provided incentives for their responses. Yet, this rate is lower than a single question assessment used with similar a population [20]. When asked about the rate at which they received assessments, the majority felt that one survey per day was appropriate. These data highlight a discrepancy between attitudes towards using this technology and actual use. Participants may have preferred the opportunity to respond more frequently, but did not feel an overwhelming obligation to do so. Indeed, one participant noted that even though they were unable to respond every day, they saw the notifications and made an effort to respond to at least every other alert. They reported that if fewer notifications were sent, they would have responded fewer times. Once participants accessed a survey, however, they were highly likely to complete all questions, which is attributed to the ease of use of the interface, their familiarity with their mobile device, and the brevity of the survey. Ease of use is among the strongest factors that contributed to willingness to use an application for healthcare [19], [26]. Furthermore, using an individual’s personal device significantly reduces the costs associated with conducting these studies or delivering intervention. Taken together, this suggests patient preferences for this method of communication are likely necessary but not sufficient to garner high response rates.

Qualitative feedback obtained from the participants provided methods to improve the data collection process. Participants found the inclusion of the same questions in each survey repetitive over time, which may have diminished their willingness to respond to subsequent assessments. Future iterations of the assessment tool could use alternative forms of an assessment to vary the content of the questions asked while assessing the same constructs. Relatedly, participants requested personalized questions that asked about their most prominent symptoms. This tailoring process could occur through an initial assessment with a provider or algorithmically as the computing power of mobile devices improves. The use of personalized feedback is a frequently requested component of mobile applications used to assess symptoms [19] and has been shown to be helpful in other behavior change interventions [27], [28]. A surprising finding was the high rate of free-text responses that were provided (90.3%). Participants were allowed to skip questions and thus were not required to complete the free-text question. The length of responses varied from single words of their most present concern (e.g., “headaches”) to brief paragraphs.. Such qualitative information may also lead to new insights into the acute post-trauma period and improve assessment and tools. For example, mixed-method studies have used similar responses with machine learning to derive highly accurate measures for other constructs [29].

Data collected via the application highlights the need for work on how to best analyze such information given its longitudinal nature and the amount of missingness that occurs. Sophisticated modeling methods that better reflect the course of symptoms during this period are needed, such as machine learning [30], [31]. These analytic strategies can capitalize on the large quantities of data generated via these methods to identify subtle but meaningful patterns among these symptoms. The use of such strategies may determine if the observed variability during the acute post-trauma period reflects the beginning of the process that ultimately becomes chronic PTSD [32]. Furthermore, developing methods to allow data collected via this type of application to enhance existing applications for PTSD is necessary. For example, the Veterans Administration has developed a range of mobile applications that provide information and brief interventions via smartphone [33], [34]. If such applications could take advantage of data collected from other sources, it would likely improve the quality of care offered via mobile phone and could create a robust early intervention for those at high risk for mental health problems after a trauma.

The present study had several limitations. First, because the present investigation was a usability study and so the obtained sample was small. The current study provided proof-of-principle evidence that will allow for larger, more thorough, investigations to take place. An examination of symptom trajectories, differences between those with PTSD and those without PTSD were not possible. Second, the measure to assess symptoms via the mobile application was based on a prior study, but was not subject to the psychometric validation that other measures have received. It should be noted that PTSD contains 20 symptoms and the majority of measures to assess this disorder are of considerable length [35]. Attempting to respond to such assessments via a mobile application would be highly burdensome to the patient, and might result in higher rates of noncompliance. There is a need for abbreviated scales to measure PTSD symptoms that cater to mobile administration [36], [37]. Third, a range of constructs that are understood to be important to post-trauma recovery processes, including physiological measures, hospital-based variables, and environmental assessments, were not collected [10], [38], [39]. The current study used an emergency department sample that consisted of patients who were discharged on the same day. Prior work with traumatic injury patients has relied on samples who were admitted to the hospital due to the severity of their injuries [1], [2]. As such, the presentation of the current sample may be less severe than that of individuals recruited from other locations. These findings should be replicated with a more severely injured population. Finally, we were unable to include non-English speaking participants in our study given the limited availability of assessments in other languages. Further work is needed to ensure that this strategy is viable across multiple populations. Preliminary work with other populations has suggested that non-English speaking populations are highly receptive to using such platforms to monitor their health and more work is needed on this topic [40].

Despite these limitations, the present findings support the use of mobile applications to gather data throughout the acute post-trauma period. Participants were responsive to this method of data collection and offered feedback for how to sustain engagement in the future. Mobile devices may also provide the means by which to provide highly effective treatments during this same critical period [13].