Abstract
Objectives
To review systematically the literature on patients’ experiences of, and participation in, technology-based monitoring of mental health symptoms. This practice was defined as patients monitoring their mental health symptoms, emotions or behaviours outside of routine clinical appointments by submitting symptom data using technology, with feedback arising from the data (for example, supportive messages or symptom summaries, being sent to the patient, clinician or carer).
Design
Systematic review following PRISMA guidelines of studies evaluating technology-based symptom monitoring. Tools from narrative synthesis were used to analyse quantitative findings on participation rates and qualitative findings on patient views.
Data sources
PubMed, EMBASE, PsycINFO, BNI, CINAHL, Cochrane Registers and Web of Science electronic databases were searched using a combination of ‘psychiatry’, ‘symptom monitoring’ and ‘technology’ descriptors. A secondary hand search was performed in grey literature and references.
Results
57 papers representing 42 studies met the inclusion criteria for the review. Technology-based symptom monitoring was used for a range of mental health conditions, either independently of a specific therapeutic intervention or as an integrated component of therapeutic interventions. The majority of studies reported moderate-to-strong rates of participation, though a third reported lower rates. Qualitative feedback suggests that acceptability of monitoring is related to perceived validity, ease of practice, convenient technology, appropriate frequency and helpfulness of feedback, as well as the impact of monitoring on participants’ ability to manage health and personal relationships.
Conclusions
Such symptom monitoring practices appear to be well accepted and may be a feasible complement to clinical practice. However, there is limited availability of data and heterogeneity of studies. Future research should examine robustly patients’ role in the development and evaluation of technology-based symptom monitoring in order to maximise its clinical utility.
Strengths and limitations of this study.
Can inform design and implementation of technology-based symptom monitoring, so its usefulness in clinical practice can be maximised.
Is novel in synthesising quantitative and qualitative data and provides detail on rates of participation and patient perspectives on usefulness.
Limited availability of data and heterogeneity of studies allow only for tentative conclusions.
May be conceptually limited as extant research has rarely gathered patient perspectives and findings are from a small number of studies.
Further, since a broad definition of mental health is used, this limits the comparison across studies.
Background
Symptom monitoring involves patients assessing and documenting changes in their feelings and behaviours outside of clinical appointments. This can provide mental health professionals with additional information to inform treatment, which may otherwise not be collected due to time constraints during routine psychiatric appointments.1 2 Pen and paper symptom monitoring is well established in psychiatric treatment and research.3 4 For example, in cognitive–behavioural therapy (CBT), patients ‘self-monitor’, that is, systematically record dysfunctional thoughts and associated maladaptive emotions and behaviours, enabling therapists to track progress.3 In research, a method known as ecological momentary assessment5 is used to capture the ‘ebb-and-flow’ of psychiatric symptomatology, as participants record real-time symptoms and context prompted by digital wristwatches.4 6
Increasingly, symptom monitoring is facilitated by technology such as short message service (SMS), websites and mobile devices7 which, compared with pen and paper, offer various advantages. First, completion rates may increase due to in-built reminders and greater ease of use.7 Second, the information collected may be more accurate and ecologically valid than diary data, which is often inaccurate and incomplete.8 Third, information can be accessed in real time, increasing the possibility for earlier and more immediate intervention outside of clinical encounters.2 This is achieved through the generation of feedback such as supportive messages or symptom summaries sent to patients, carers and clinicians. Consequently, patients can access personalised support when needed and clinicians can access accurate personalised information to inform and optimise treatment decisions9 and promote collaborative care.10 Technology-based symptom monitoring incorporating feedback (eg, supportive messages or symptom summaries) therefore has the potential to significantly influence routine patient treatment outside of clinical appointments, when the patient is in their own environment. Yet, at present, technology-based symptom monitoring is restricted mainly to the context of research, to identify patterns and mechanisms of psychopathology, and symptom monitoring data are not fed back to patients or clinicians in order to inform patient treatment.4 9 11 However, there are potential limitations given that other technology-based, self-directed interventions such as computerised CBT, psychoeducation and skills training websites have high drop out and poor adherence.12–14 This is associated with factors such as lack of interactive features and prompting, limited access and technological literacy, failure to provide human support, monitoring, or follow-up, and chronicity or condition severity.12–14
A limitation of the current evidence is that we know relatively little about how patients experience, engage with and appraise technology-based symptom monitoring-specifically, when employed independently, outside of clinical appointments, to generate feedback for the patient, their clinician or their carer. It is critical to establish patients’ perspectives in order to improve our understanding of how to design and implement technology-based symptom monitoring, and therefore maximise its clinical utility in community care. Accordingly, the research questions of this review were as follows:
What are the rates of participation in technology-based symptom monitoring, whereby patients report symptoms outside of routine clinical appointments, generating feedback for the patient, clinician or carer?
What are patients’ experiences of using such monitoring?
Methods
In July 2015, a systematic search of the electronic databases and grey literature was conducted following PRISMA guidelines15 and according to a protocol (available from authors on request). Search terms combined mental health, symptom monitoring and technology descriptors and are listed in online supplementary file 1.
bmjopen-2015-008362supp1.pdf (96.8KB, pdf)
Search strategy
The databases used were: PubMed, EMBASE, PsycINFO, BNI, CINAHL, Cochrane Registers and Web of Science. Databases were searched from their inception to July 2015 with no language restrictions. Publication bias was minimised by including conference papers and grey literature, with sources including Health Technology Assessment agencies identified by the Canadian Agency for Technology,16 Beacon website, a technology-based mental health intervention database,17 British Library Catalogue, Copac theses, Zetoc conferences, OpenGrey and UK Clinical Research Network Study Portfolio. Backward citation, that is, reference screening from included studies; and forward citation, that is, finding citations to included studies using Google Scholar, were also used.
Titles were inspected to remove duplicates and irrelevant papers. Abstracts were then screened for a detailed evaluation. Full texts were checked if there was insufficient information in the abstract. A second independent researcher (EG) screened a random selection of 20% of abstracts to ensure the accuracy of screening.
Selection criteria
The population of interest was defined inclusively as participants diagnosed with or experiencing subthreshold symptoms of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) axis 1 mental health conditions including adjustment disorders, anxiety disorders, eating disorders, mood disorders, and schizophrenia and other psychotic disorders. The intervention was defined as a technological tool used by patients to monitor subjectively symptoms, emotions or behaviours outside routine clinical appointments, providing feedback to the patient, clinician or another party (such as a carer). This provision of feedback helped to distinguish relevant studies from those not employing symptom monitoring as part of treatment. Symptom monitoring in clinical appointments and objective symptom monitoring (eg, physical movement) were excluded as the review focused on patients’ involvement in independently and subjectively reporting symptoms. There was no restriction on study type; however, papers were required to report on patient participation rates or experiences. All included texts were examined independently by a second reviewer (EG) to confirm inclusion.
Data extraction
Data from each study, including participant demographics, symptom monitoring characteristics and patient experience measures, were extracted into a spreadsheet. The full list of extracted data is listed in online supplementary file 1. A second reviewer (EG) extracted data from 20% of the included papers to ensure accuracy. One German paper was extracted by a native speaker.18 In four instances, authors provided clarification via email.19–22
Analysis
Tools from narrative synthesis were used including vote counting, tabulation and transforming data into a common rubric.23 Vote counting was used to provide initial descriptions of monitoring practices. Tabulation was used to present quartiles of participation rates, which were calculated by dividing actual monitoring instances by the potential number, expressed as a percentage. Where studies only reported participation rates for completers, drop outs were assumed to have completed none of the potential monitoring, resulting in conservative estimates for these studies. Transforming data into a common rubric of qualitative codes was used for the quantitative data on participants’ experiences of symptom monitoring in order to complete thematic synthesis,24 in which codes were assigned in an inductive manner and refined iteratively to identify the main themes.
Results
The search returned 11 529 records, leaving 456 after the removal of duplicates and irrelevant titles (see figure 1). A further 249 were excluded after abstracts was screened because they did not meet the criteria for symptom monitoring (n=134), were review papers (n=69), included a non-mental health population (n=37) or concerned an unrelated to topic (n=9). The full texts of all remaining 207 papers were examined, 57 of which were included. The 57 papers represent 42 unique studies as four studies were reported in multiple papers. For example, mobile-type25 26 and self-management support for bulimia nervosa were reported twice,27 28 while the ChronoRecord pilot was reported in four papers21 29–31 and its ongoing international study was reported in 11 papers.32–42
Overview of studies and symptom monitoring protocols
Online supplementary file 2 summarises the characteristics of included studies. Papers were published between 1999 and 2015, in English, except one German paper.18 Most papers were from Europe1 18 20 22 27 28 43–57 or the USA,9 21 29 31 30 58–71 with the rest from Australia,19 25 26 72 73 Japan74 and one international study reported in multiple papers.32–42
bmjopen-2015-008362supp2.pdf (314KB, pdf)
Most studies were designed to assess feasibility and were single-arm pilot studies,9 18 27 28 43 47 49 51 52,54–62 66 67 69 70–74 or observational studies.21 29–31 64 68 Some were randomised controlled trials (RCTs)19 20 22 25 26 44 45 50 59 63 65 and some were non-randomised trials46 53 that reported the efficacy of technology-based symptom monitoring in terms of reduced severity of symptoms,19 22 25 26 remission rates44 or increased adherence compared with paper monitoring.65 In other studies, monitoring was a component of an intervention, which when evaluated, reported positive impact on symptoms,20 45 59 63 mixed findings46 53 or non-significant findings.50 Five papers contained <10 participants,52 58 64 67 74 21 papers contained <50 participants,9 18 22 43 46–49 53–57 60 65 66 69–73 16 papers contained between 50 and 99 participants,1 20 21 25 26 29–31 44 45 50 51 59 61–63 and five papers contained 100 or more participants,19 27 28 68 one of which was the longitudinal ongoing study of ChronoRecord.32–42
Technology-based symptom monitoring was used in a variety of ways in patients with depression,19 22 25 26 47 53 58 63 68 73 74 anxiety,67 bipolar disorders,1 9 21 29–42,48 54 59 60 65 eating disorders18 20 27 28 43–46 49–51 55–70 and severe mental illness,69 including schizophrenia.57 62 64 In a few studies, no single diagnosis was targeted.52 61 66 71 72 Such monitoring was often an integrated component of a particular therapeutic intervention, such as CBT,58 64 67 70 psychoeducation,54 59 60 behavioural activation58 or as part of an e-therapy.19 20 27 28 43 45 46 49–51 53 55 63 66 72 Monitoring was also used as a stand-alone intervention to support ongoing treatment,1 9 21 22 25 26 29–42,47 48 52 61 62 65 68 73 74 or as a specific element of treatment, such as preventing relapse and hospitalisation,57 69 or following discharge.18 44 56 71
Commonly available technologies such as email,1 48 54 65 SMS,1 18 44 48 52 56 57 62 70 71 or websites19 20 27 28 43 45–47 49–51 53 55 59 63 72 73 74 were typically used for reporting, though purpose-built software was also used.9 21 22 25 26 29–42 58 60 61 64 66–69 Participants either reported only their symptoms,1 18 20 27 43 44 46–54 56 57 63 67 69–71 73 74 or symptoms and other relevant factors.9 19 21 22 25 26 29–42 45 46 55 58–62 64–66 68 72 The intensity of monitoring varied; patients typically reported symptoms retrospectively with respect to the preceding day,19 44–47 52 54 61 62 65 69 70 74 few days66 68 71 72 or week.1 18 20 27 28 43 48–51 53 55 56 57 63 73 Other studies were more intensive and collected multiple momentary (current) symptom reports per day.9 22 25 26 58–60 64 66 67 Patients were usually prompted to report at a fixed time,1 9 18 43 44 48–51 54 56 57 61 62 65–68 71 74 though random prompts were used.22 25 26 58 59 60 64 In other studies, participants were not prompted,21 29–42 47 53 63 69 70 73 or prompting was optional or unclear.19 20 27 28 45 46 52 55 72
Participants were usually supported in monitoring as their clinician received automated feedback generated by algorithms detecting symptom improvement or deterioration, warranting follow-up. Such feedback was in the form of messages,43 49–51 53 57 63 68 69 71 74 graphs representing symptoms over time1 25–28 45 46 48 54 57 58 63 64 67 73 74 or brief reports.20 43 53 55 61 68 69 In some studies, patients received the same feedback as clinicians, as graphs25–28 45 46 48 54 57 58 63 or messages.49–51 68 71 74 In other studies, patients received different content than clinicians, such as a supportive statement or reminder of treatment strategy.43 53 69 In some studies, only patients received feedback, either where symptom monitoring was used as part of a specific guided treatment,9 22 52 59 60 66 70 as part of an entirely unguided treatment aimed to support symptom awareness and self-management,18 21 29–42 44 47 62 65 or as part of a self-guided website.19 72
Rates of participation in technology-based symptom monitoring
Thirty-three studies provided sufficient detail for participation rates to be calculated and are presented in table 1, by quartile. The remaining studies did not provide this detail.18 27 28 32–42 44 46 48 51 56 73
Table 1.
Low (0–25%) | Lower middle (26–50%) | Upper middle (51–75%) |
High (76–100%) |
|
---|---|---|---|---|
Diagnosis | ||||
Anxiety | 67 | |||
Depression | 19, 47, 58 63 | 22, 68 | 25, 74 | |
Anxiety/depression | 72 | 52 53 | ||
Bipolar disorder | 65 | 59 | 1 9 21 54 60 | |
Eating disorder | 20, 50, 55 | 45, 49, 70 | 43 | |
Mixed | 61, 66 | 71 | ||
Schizophrenia | 62, 64 | 57 | ||
Severe mental illness | 69 | |||
Monitoring schedule | ||||
Multiple times per day | 58 | 22, 59, 64, 67 | 9, 25, 60 | |
Daily | 19, 47, 65 | 45, 52, 61, 62, 69 | 21, 54, 74 | |
Multiple times per week | 72 | 66, 68 | 71 | |
Weekly | 20, 50, 55, 63 | 49, 53, 70 | 1, 43, 57 | |
Prompting | ||||
Random prompted | 58 | 22, 59, 64 | 25, 60 | |
Fixed prompted | 50, 65 | 49, 53, 61, 62, 66, 67, 68 | 1, 9, 43, 54, 57, 71, 74 | |
Unprompted | 47, 63 | 69, 70 | 21 | |
Optional or unclear if prompted | 72 | 19, 20, 55 | 45, 52 | |
Guided | ||||
Guided | 20, 50, 55, 58, 63 | 22, 45, 49, 52, 53, 59, 61, 64, 67, 68, 69, 70 | 1, 9, 25, 43, 54, 57, 60, 71, 74 | |
Unguided | 72 | 19, 47, 65 | 62 | 21 |
Context: independent or integrated into intervention | ||||
Independent of a specific therapeutic intervention | 47, 65 | 22, 61, 62, 68, 69 | 1, 9, 21, 25 57, 60, 71, 74 | |
Integrated component of face-to-face therapy | 52, 59, 64, 67, 70 | 54 | ||
Integrated component of e-therapy | 72 | 19, 20, 50, 55, 58, 63 | 45, 49, 53 | 43 |
In the majority of studies, participation rates were in the upper range,22 45 49 52 53 59 61 62 64 67–70 with many studies reporting high rates.1 9 21 25 43 54 57 60 71 74 Approximately a third of studies reported lower rates of participation.19 20 47 50 55 58 63 65 72 When participant diagnosis and characteristics of the monitoring were considered, few patterns emerge regarding what may affect participation rates, most likely due to the variance in study characteristics. However, participation rates appear more varied and slightly lower in common mental health conditions such as anxiety and depression,19 47 58 63 72 in unguided interventions,19 47 65 72 and in e-therapy.19 20 50 55 58 63 72 Yet, such factors are not evenly represented across studies and findings are inconclusive.
Patients’ perspectives on technology-based symptom monitoring
Less than half of the studies collected data on patients' experiences of technology-based symptom monitoring. Most studies used closed-response questionnaires9 18 21 22 27 46 47 49 51 53 56 58–62 70 71 73 with some using open-ended questionnaires,9 47 53 56 semistructured interviews48 60 73 and focus groups.47 73
The overarching themes were ‘acceptability’ and ‘influence’ of monitoring, as presented in online supplementary file 3.
bmjopen-2015-008362supp3.pdf (209.5KB, pdf)
Acceptability of symptom monitoring
This refers to how satisfied participants were with aspects of the monitoring such as technology type, frequency and duration, and feedback.
General acceptability of symptom monitoring
Many studies reported moderate-to-strong acceptability in terms of perceived usefulness and satisfaction.9 27 46 53 58 62 70 73 Participants felt that monitoring was supportive, made them feel less alone, and that they would recommend or repeat it.18 However, reminders53 and fixed schedules9 became repetitive or irritating. Additionally, some participants with affective disorders reported that assessments did not always capture what patients wanted to report,9 47 48 59 for example, something was ‘missing’.47 In other cases, participants with eating disorders liked the notion of symptom monitoring, but in practice did not find it helpful49 51 and would not want to repeat it or recommend it to a friend.56
Ease of use
Participants found symptom monitoring easy as it required little effort and was simple to remember and understand.9 46–48 56 For example, ‘I like the fact that you can get it done in 40 seconds, it's nothing at all, it's very easy and good’.48 However, it was reported that losing instructions made the intervention difficult to complete.56
Technology
Participants found various types of technology acceptable including software, SMS, email and home telehealth devices.21 48 59–61 However, SMS was criticised for restricting participants' ability to report nuanced experiences: ‘Text is a constricted method of communication; [it] would have been good to have a website you could go to, email would be better—less restricted, more personal…’.56 Participants reported concern over carrying additional devices.9 60 Some technological difficulties were reported, such as failure of devices to save responses or beep loudly enough,9 or receiving incorrect or repeat messages in response to symptom reporting.56 Additionally, computer literacy was an issue for participants who found it difficult and time-consuming to log in and access monitoring.53
Frequency and duration
There were differing views on the appropriate frequency of monitoring. Some participants were satisfied with weekly reporting,18 48 56 while others suggested weekly frequency was insufficient and should be increased to improve insight into symptoms.49 There were also mixed views on daily reporting, that is, whether reporting was not frequent enough,9 or too frequent.47
Helpfulness of feedback
Feedback from technology-based symptom monitoring was reported to be useful.27 53 60 71 More specifically, access to graphs representing patients' mood disorder symptoms over time were perceived as helpful22 47 48 (or would have been if the clinician shared them73) with one participant commenting that ‘It really helps to see a picture; it really brings it to life… I was quite surprised by the changes’.47 In one eating disorder study, graphs were not perceived as helpful but no explanation was given.46 Studies in which participants received automated feedback messages varied in their acceptability from positive overall60 71 to a range of differing views.18 49 53 56 For example, some participants with eating disorders found texts encouraging, while others found them patronising.56 In two other eating disorder studies, half of participants,49 or almost all,51 did not think feedback messages were appropriate. This was echoed in participants with depression and anxiety, some of whom felt that feedback messages were repetitive and impersonal.53
Influence of symptom monitoring
This refers to the influence of monitoring on insight and health management, and relationships with others.
Insight into health and management
Participants with mood disorders reported an improved understanding of their condition and perceived themselves as better able to cope with it, or better able to play an active role in their treatment, following symptom monitoring.9 48 53 60 61 73 However, participants felt this could be further improved by allowing different symptoms to be monitored at different times, depending on mania or depression;48 or allowing the monitoring of other factors, such as stress levels and alcohol intake, to improve their understanding of the impact of these factors on symptoms.9 It was also suggested that monitoring practices should differ depending on the stage of illness, such as whether one is new to treatment or recovering from an episode of mania or depression.9 Similarly, in depression, participants mentioned symptom monitoring may be more appropriate for new patients who are less aware of mood fluctuations,47 yet some relatively young patients felt they already knew their symptoms.73 However, the opportunity for improved insight and health management was limited in two studies of eating disorders, where automated feedback was evaluated as poor.49 56
Relationships with others
Participants felt that symptom monitoring could positively impact on relationships. In studies of mood disorders, such monitoring was perceived to improve participants' interactions with clinicians9 48 and foster carers' awareness of the condition.48 However, where participants could nominate a friend or family to receive feedback, participants reported that there were issues with placing burden on that person, and participants sometimes experienced guilt as a consequence of this.47 However, where monitoring for bulimia was either completely unguided18 or accompanied by brief postal feedback,56 this lack of personal support was perceived negatively, particularly by recently discharged inpatients. In outpatients, views were more mixed, as some participants were happy to express their feelings without discussing with a stranger, while others felt unsupported and would have preferred to have received personal contact from a therapist to support them.56
Discussion
Main findings
Despite limited availability of data and heterogeneity of studies, there is some evidence that technology-based symptom monitoring is relatively well accepted, given the moderate-to-strong rates of participation, and overall positive reports of experience. Lower rates of participation were found in one-third of studies, but it was not possible to firmly establish what contributes to this. Diagnosis may play a part as the participant experience data suggest that patients with eating disorders liked monitoring in theory but not in practice; people with depression and anxiety had lower rates of participation and believed technology-based symptom monitoring in its current forms had limited utility; and people with bipolar disorder viewed such monitoring more favourably. Lack of prompting and guidance may also contribute; where participants were unguided, they reported that feedback messages were unhelpful and perceived the lack of support negatively. Implementation of technology-based symptom monitoring as part of an e-therapy may also be associated with lower participation rates; however, the experience findings on such interventions were limited.
Strengths and limitations
This main strength of this review was its systematic approach in collating the published literature to date on technology-based symptom monitoring practices, employing rigorous methodology with a wide search strategy. A further strength is that the review did not limit by diagnosis, which allowed papers that were not diagnosis-specific to be included.52 53 61 66 69 71 72 This approach also allowed for more general conclusions on design principles of mental health symptom monitoring, which may apply across diagnoses. It also reflects the reality of community mental healthcare practice where patients can have multiple diagnoses. Still, this approach has limitations. First, although it may be useful to compare across diagnoses, the heterogeneity of studies found makes it difficult to draw firm conclusions from the review. The results are narrative, rather than a meta-analysis, which was not possible due to variable quality of data among the heterogeneous studies. Further, this review does not formally appraise the quality of the studies, as doing so remains contentious in the context of descriptive accounts.75 As the evidence base becomes more established and the number of RCTs increases, it will be important to assess the quality of the studies to determine effectiveness. Although efforts were made to find grey literature, the search strategy may still have been biased towards published research. The second major limitation pertains to the limited availability of data on participation rates and patients' experiences of such monitoring, which renders it difficult to state findings conclusively. For example, in some studies, the average participation rates could not be derived. Further, the paucity of data on longitudinal engagement with monitoring makes it difficult to establish whether participants initially engage well but after a time drop out,63 or to establish the optimal length of monitoring. As only half of the included studies sought participants' views on monitoring, the resulting data are limited, especially since very brief, forced-choice questionnaires were used. As a consequence, the themes relating to acceptability emerged mainly from just five studies collecting open-ended responses or in-depth interview data, and this sample may therefore be unrepresentative. Despite these limitations, the review is a useful collation of studies that demonstrates the potential feasibility of technology-based symptom monitoring outside of appointments to inform treatment, with a range of patient groups and intervention types. The tentative conclusions may also guide future research.
Comparison with literature
The current review supports previous reports that patients engage well with technology-based symptom monitoring,4 76 while introducing a focus on routine patient treatment rather than monitoring solely for research purposes. In particular, it supports findings indicating that participation rates are moderate-to-strong in a range of diagnostic groups, including those with schizophrenia,4 76 which may be surprising given well-documented rates of treatment non-adherence in this group.77 The review also suggests a greater variation in participation for common mental health conditions, which may relate to patients with depression having impairments in focus, energy and motivation that affect willingness to engage in treatment,78 particularly technology-based interventions.12 It supports previous research suggesting that participation in guided technology-based interventions is superior to unguided ones.79 The review also provides some evidence that when monitoring is implemented as part of an e-therapy, there are lower rates of participation, as has been documented previously.12–14 However, these findings must be interpreted with caution due to sampling biases; there were fewer studies on severe mental illness, unguided monitoring and e-therapies. Further research is needed to clarify the impact of patient diagnosis and characteristics of monitoring on participation.
The findings from patients' experiences of monitoring support the existing literature, which suggests improved patient understanding of their illness and promotion of collaborative care;2 10 however, it was noted that the impact on understanding is dependent on ability to monitor the most relevant factors, at the most important times, for example, stage of illness. It is interesting that the majority of studies in this review asked for retrospective reports rather than momentary reports, which have previously been considered to be superior in providing valid data more readily for timely intervention.76 Given that the few momentary studies demonstrated good rates of participation, future research should perhaps investigate this method to capture accurately the flow of patient experience. The findings also suggest that personalised feedback should be used cautiously, as some found it patronising. One possible option for overcoming this is to involve patients in generating feedback messages, so that they receive something personal and meaningful.62 Future research should consider the role of patients in designing mental health interventions, as this can improve the acceptability of interventions.80 81
Implications for research and practice
Technology-based symptom monitoring may be a useful complement to current clinical practices, given the moderate-to-strong rates of participation and acceptability, and potential to improve insight and health management, and personal relationships. However, these findings are tentative and a number of practical factors need to be considered when designing future monitoring practices for robust evaluation. To increase the likelihood of good participation, monitoring must capture what patients want to report, be easy to complete without requiring high levels of computer literacy and use the most appropriate technology to capture the complexity of symptoms. The frequency of interventions must be carefully considered to create a balance between generating insightful data on symptoms and overburdening participants. Feedback arising from monitoring seems to be important for successful implementation, with objective representations, such as graphs, potentially more acceptable to patients than automated messages that fall short of personalisation. Future studies should consider how to facilitate personalisation, perhaps by involving patients in generating messages. To facilitate improvements in insight into health and management of conditions, flexibility is important, such that participants are allowed to monitor symptoms that are most relevant to them at particular phases of their illness. Careful consideration should be given to whether, for some conditions, monitoring is perhaps only useful in early phases of illness. Symptom monitoring is likely to be more acceptable when part of a guided intervention in which a clinician or other party is included, in order to facilitate collaboration; however, if family and friends are included, issues surrounding burdening and guilt should be considered. Future research may consider ethical aspects, which have rarely been mentioned in the included studies. Potential problems include data security and privacy when sensitive information is shared via technology82 and expectations that symptom reports are being checked around the clock.60 Nevertheless, it is held that such concerns should not limit progress in the field.83 Finally, given that there are potentially large costs in programming and developing such interventions, these must be balanced with the potential savings; cost-effectiveness needs to be established, both in feasibility studies and larger trials.82
Conclusions
The current review has identified encouraging evidence of acceptability of a range of technology-based symptom monitoring practices in routine management of mental health. Further, it highlights that a number of issues must be considered when implementing such monitoring. However, the limited quality and availability of data and heterogeneity of studies mean only tentative conclusions can be drawn. Future research into technology-based symptom monitoring should robustly examine patients' role in the development and evaluation of monitoring, to further understanding of this area and to ensure that it can be employed to good clinical effect.
Footnotes
Twitter: Follow Sophie Walsh at @Walsh1Sophie
Contributors: SW designed the study with input from SP and EG. SW conducted the systematic searches of the literature, selected the studies, interpreted the data and drafted the manuscript. EG ensured the consistency of study selection and data extraction. All authors contributed to and approved the final version of the manuscript.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: All data cited in this systematic review are available from the cited previously published studies.
References
- 1.Bopp JM, Miklowitz DJ, Goodwin GM et al. . The longitudinal course of bipolar disorder as revealed through weekly text messaging: a feasibility study. Bipolar Disord 2010;12:327–34. 10.1111/j.1399-5618.2010.00807.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Palmier-Claus JE, Ainsworth J, Machin M et al. . The feasibility and validity of ambulatory self-report of psychotic symptoms using a smartphone software application. BMC Psychiatry 2012;12:172 10.1186/1471-244X-12-172 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cohen JS, Edmunds JM, Brodman DM et al. . Using self-monitoring: implementation of collaborative empiricism in cognitive-behavioral therapy. Cogn Behav Pract 2013;20:419–28. 10.1016/j.cbpra.2012.06.002 [DOI] [Google Scholar]
- 4.Myin-Germeys I, Oorschot M, Collip D et al. . Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med 2009;39:1533–47. 10.1017/S0033291708004947 [DOI] [PubMed] [Google Scholar]
- 5.Moskowitz DS, Young SN. Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. J Psychiatry Neurosci 2006;31:13–21. [PMC free article] [PubMed] [Google Scholar]
- 6.Delespaul PAEG. Assessing schizophrenia in daily life: the experience sampling method: ISPER Foundation. 1995.
- 7.Proudfoot J. The future is in our hands: the role of mobile phones in the prevention and management of mental disorders. Aust N Z J Psychiatry 2013;47:111–13. 10.1177/0004867412471441 [DOI] [PubMed] [Google Scholar]
- 8.Kobak KA, Greist JH, Jefferson JW et al. . New technologies to improve clinical trials. J Clin Psychopharmacol 2001;21:255–6. 10.1097/00004714-200106000-00001 [DOI] [PubMed] [Google Scholar]
- 9.Wenze SJ, Armey MF, Miller IW. Feasibility and acceptability of a mobile intervention to improve treatment adherence in bipolar disorder: a pilot study. Behav Modif 2014;38:497–515. 10.1177/0145445513518421 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Palmier-Claus JE, Rogers A, Ainsworth J et al. . Integrating mobile-phone based assessment for psychosis into people's everyday lives and clinical care: a qualitative study. BMC Psychiatry 2013;13:34 10.1186/1471-244X-13-34 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Trull TJ, Ebner-Priemer U. Ambulatory assessment. Annu Rev Clin Psychol 2013;9:151–76. 10.1146/annurev-clinpsy-050212-185510 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression: systematic review. J Med Internet Res 2009;11:e13 10.2196/jmir.1194 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Eysenbach G. The law of attrition. J Med Internet Res 2005;7:e11 10.2196/jmir.7.1.e11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009;39:705–12. 10.1017/S0033291708004224 [DOI] [PubMed] [Google Scholar]
- 15.Moher D, Liberati A, Tetzlaff J et al. . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:264–9. 10.7326/0003-4819-151-4-200908180-00135 [DOI] [PubMed] [Google Scholar]
- 16.Grey Matters: a practical deep-web search tool for evidence-based medicine. http://www.cadth.ca/en/resources/finding-evidence-is/grey-matters. July 2015.
- 17.Christensen H. Beacon 2.0. https://beacon.anu.edu.au/. July 2015.
- 18.Bauer S, Hagel J, Okon E et al. . SMS in der nachstationären Betreuung von Patientinnen mit Bulimia nervosa [Experience with the application of the short message service (SMS) in the aftercare of patients with bulimia nervosa]. Psychodynamische Psychother 2006;5:127–36. [Google Scholar]
- 19.Proudfoot J, Clarke J, Birch MR et al. . Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013;13:312 10.1186/1471-244X-13-312 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jacobi C, Völker U, Trockel MT et al. . Effects of an internet-based intervention for subthreshold eating disorders: a randomized controlled trial. Behav Res Ther 2012;50:93–9. 10.1016/j.brat.2011.09.013 [DOI] [PubMed] [Google Scholar]
- 21.Bauer M, Grof P, Gyulai L et al. . Using technology to improve longitudinal studies: self-reporting with ChronoRecord in bipolar disorder. Bipolar Disord 2004;6:67–74. 10.1046/j.1399-5618.2003.00085.x [DOI] [PubMed] [Google Scholar]
- 22.Kramer I, Simons CJ, Hartmann JA et al. . A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 2014;13:68–77. 10.1002/wps.20090 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Popay J, Roberts H, Sowden A et al. . Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme. Lancaster: Institute of Health Research, 2006. [Google Scholar]
- 24.Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45 10.1186/1471-2288-8-45 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kauer SD, Reid SC, Crooke AH et al. . Self-monitoring using mobile phones in the early stages of adolescent depression: randomized controlled trial. J Med Internet Res 2012;14:e67 10.2196/jmir.1858 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Reid SC, Kauer SD, Hearps SJ et al. . A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract 2011;12:131 10.1186/1471-2296-12-131 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Carrard I, Fernandez-Aranda F, Lam T et al. . Evaluation of a guided internet self-treatment programme for bulimia nervosa in several European countries. Eur Eat Disord Rev 2011;19:138–49. 10.1002/erv.1043 [DOI] [PubMed] [Google Scholar]
- 28.Carrard I, Rouget P, Fernández-Aranda F et al. . Evaluation and deployment of evidence based patient self-management support program for bulimia nervosa. Int J Med Inf 2006;75:101–9. 10.1016/j.ijmedinf.2005.07.031 [DOI] [PubMed] [Google Scholar]
- 29.Bauer M, Rasgon N, Grof P et al. . Mood changes related to antidepressants: a longitudinal study of patients with bipolar disorder in a naturalistic setting. Psychiatry Res 2005;133:73–80. 10.1016/j.psychres.2004.08.006 [DOI] [PubMed] [Google Scholar]
- 30.Bauer M, Rasgon N, Grof P et al. . Does the use of an automated tool for self-reporting mood by patients with bipolar disorder bias the collected data? Med Gen Med 2005;7:21. [PMC free article] [PubMed] [Google Scholar]
- 31.Bauer M, Grof P, Rasgon N et al. . Mood charting and technology: new approach to monitoring patients with mood disorders. Curr Psychiatry Rev 2006;2:423–9. 10.2174/157340006778699747 [DOI] [Google Scholar]
- 32.Bauer M, Grof P, Rasgon NL et al. . Self-reported data from patients with bipolar disorder: impact on minimum episode length for hypomania. J Affect Disord 2006;96:101–5. 10.1016/j.jad.2006.05.006 [DOI] [PubMed] [Google Scholar]
- 33.Bauer M, Rasgon N, Grof P et al. . Do antidepressants influence mood patterns? A naturalistic study in bipolar disorder. Eur Psychiatry 2006;21:262–9. 10.1016/j.eurpsy.2006.04.009 [DOI] [PubMed] [Google Scholar]
- 34.Bauer M, Glenn T, Grof P et al. . Self-reported data from patients with bipolar disorder: frequency of brief depression. J Affect Disord 2007;101:227–33. 10.1016/j.jad.2006.11.021 [DOI] [PubMed] [Google Scholar]
- 35.Bauer M, Glenn T, Grof P et al. . Frequency of subsyndromal symptoms and employment status in patients with bipolar disorder. Soc Psychiatry Psychiatr Epidemiol 2009;44:515–22. 10.1007/s00127-008-0464-4 [DOI] [PubMed] [Google Scholar]
- 36.Bauer M, Glenn T, Grof P et al. . Relationship between adjunctive medications for anxiety and time spent ill in patients with bipolar disorder. Int J Psychiatry Clin Pract 2009;13:70–7. 10.1080/13651500802450514 [DOI] [PubMed] [Google Scholar]
- 37.Bauer M, Glenn T, Grof P et al. . The association between concurrent psychotropic medications and self-reported adherence with taking a mood stabilizer in bipolar disorder. Hum Psychopharmacol 2010;25:47–54. 10.1002/hup.1077 [DOI] [PubMed] [Google Scholar]
- 38.Bauer M, Glenn T, Rasgon N et al. . Association between age of onset and mood in bipolar disorder: comparison of subgroups identified by cluster analysis and clinical observation. J Psychiatr Res 2010;44:1170–5. 10.1016/j.jpsychires.2010.04.009 [DOI] [PubMed] [Google Scholar]
- 39.Bauer M, Glenn T, Rasgon N et al. . Association between median family income and self-reported mood symptoms in bipolar disorder. Compr Psychiatry 2011;52:17–25. 10.1016/j.comppsych.2010.04.005 [DOI] [PubMed] [Google Scholar]
- 40.Bauer M, Glenn T, Rasgon N et al. . Decreasing the minimum length criterion for an episode of hypomania: evaluation using self-reported data from patients with bipolar disorder. Eur Arch Psychiatry Clin Neurosci 2011;261:341–7. 10.1007/s00406-010-0187-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bauer M, Glenn T, Keil M et al. . Brief depressive symptoms in patients with bipolar disorder: analysis of long-term self-reported data. Aust N Z J Psychiatry 2012;46:1068–78. 10.1177/0004867412452017 [DOI] [PubMed] [Google Scholar]
- 42.Bauer M, Glenn T, Alda M et al. . Drug treatment patterns in bipolar disorder: analysis of long-term self-reported data. Int J Bipolar Disord 2013;1:5 10.1186/2194-7511-1-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bauer S, Moessner M, Wolf M et al. . ES S PRIT—an internet-based programme for the prevention and early intervention of eating disorders in college students. Br J Guid Couns 2009;37:327–36. 10.1080/03069880902957049 [DOI] [Google Scholar]
- 44.Bauer S, Okon E, Meermann R et al. . Technology-enhanced maintenance of treatment gains in eating disorders: efficacy of an intervention delivered via text messaging. J Consult Clin Psychol 2012;80:700–6. 10.1037/a0028030 [DOI] [PubMed] [Google Scholar]
- 45.Carrard I, Crépin C, Rouget P et al. . Randomised controlled trial of a guided self-help treatment on the Internet for binge eating disorder. Behav Res Ther 2011;49:482–91. 10.1016/j.brat.2011.05.004 [DOI] [PubMed] [Google Scholar]
- 46.Carrard I, Crépin C, Rouget P et al. . Acceptance and efficacy of a guided Internet self-help treatment program for obese patients with binge eating disorder. Clin Pract Epidemiol Ment Health 2011;7:8–18. 10.2174/1745017901107010008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Drake G, Csipke E, Wykes T. Assessing your mood online: acceptability and use of Moodscope. Psychol Med 2013;43:1455–64. 10.1017/S0033291712002280 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Foster J, Price J, Budge K et al. . Refining a mood monitoring system for bipolar disorder: user experiences of True Colours [poster abstract]. Bipolar Disord 2011;13:45. [Google Scholar]
- 49.Gulec H, Moessner M, Mezei A et al. . Internet-based maintenance treatment for patients with eating disorders. Prof Psychol Res Pr 2011;42:479–86. 10.1037/a0025806 [DOI] [Google Scholar]
- 50.Gulec H, Moessner M, Túry F et al. . A randomized controlled trial of an Internet-based posttreatment care for patients with eating disorders. Telemed J E Health 2014;20:916–22. 10.1089/tmj.2013.0353 [DOI] [PubMed] [Google Scholar]
- 51.Lindenberg K, Moessner M, Harney J et al. . E-health for individualized prevention of eating disorders. Clin Pract Epidemiol Ment Health 2011;7:74–83. 10.2174/1745017901107010074 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Matthews M, Doherty G. In the mood: engaging teenagers in psychotherapy using mobile phones. Proceedings of the Conference on Human Factors in Computing Systems; 2011:2947–56. [Google Scholar]
- 53.Meglic M, Furlan M, Kuzmanic M et al. . Feasibility of an eHealth service to support collaborative depression care: results of a pilot study. J Med Internet Res 2010;12:e63 10.2196/jmir.1510 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Miklowitz DJ, Price J, Holmes EA et al. . Facilitated integrated mood management for adults with bipolar disorder. Bipolar Disord 2012;14:185–97. 10.1111/j.1399-5618.2012.00998.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Ohlmer R, Jacobi C, Taylor CB. Preventing symptom progression in women at risk for AN: results of a pilot study. Eur Eat Disord Rev 2013;21:323–9. 10.1002/erv.2225 [DOI] [PubMed] [Google Scholar]
- 56.Robinson S, Perkins S, Bauer S et al. . Aftercare intervention through text messaging in the treatment of bulimia nervosa—feasibility pilot. Int J Eat Disord 2006;39:633–8. 10.1002/eat.20272 [DOI] [PubMed] [Google Scholar]
- 57.Spaniel F, Vohlídka P, Hrdlicka J et al. . ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr Res 2008;98:312–17. 10.1016/j.schres.2007.09.005 [DOI] [PubMed] [Google Scholar]
- 58.Burns MN, Begale M, Duffecy J et al. . Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 2011;13:e55 10.2196/jmir.1838 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Depp CA, Ceglowski J, Wang VC et al. . Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord 2015;174: 23–30. 10.1016/j.jad.2014.10.053 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Depp CA, Mausbach B, Granholm E et al. . Mobile interventions for severe mental illness: design and preliminary data from three approaches. J Nerv Ment Dis 2010;198:715–21. 10.1097/NMD.0b013e3181f49ea3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Godleski L, Cervone D, Vogel D et al. . Home telemental health implementation and outcomes using electronic messaging. J Telemed Telecare 2012;18:17–19. 10.1258/jtt.2011.100919 [DOI] [PubMed] [Google Scholar]
- 62.Granholm E, Ben-Zeev D, Link PC et al. . Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull 2012;38:414–25. 10.1093/schbul/sbr155 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Hunkeler EM, Hargreaves WA, Fireman B et al. . A web-delivered care management and patient self-management program for recurrent depression: a randomized trial. Psychiatr Serv 2012;63:1063–71. 10.1176/appi.ps.005332011 [DOI] [PubMed] [Google Scholar]
- 64.Kimhy D, Corcoran C. Use of palm computer as an adjunct to cognitive-behavioural therapy with an ultra-high-risk patient: a case report. Early Interv Psychiatry 2008;2:234–41. 10.1111/j.1751-7893.2008.00083.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Lieberman DZ, Kelly TF, Douglas L et al. . A randomized comparison of online and paper mood charts for people with bipolar disorder. J Affect Disord 2010;124:85–9. 10.1016/j.jad.2009.10.019 [DOI] [PubMed] [Google Scholar]
- 66.Macias C, Panch T, Hicks YM et al. . Using smartphone apps to promote psychiatric and physical well-being. Psychiatr Q 2015;86:505–19. 10.1007/s11126-015-9337-7 [DOI] [PubMed] [Google Scholar]
- 67.Newman MG, Consoli AJ, Taylor CB. A palmtop computer program for the treatment of generalized anxiety disorder. Behav Modif 1999;23:597–619. 10.1177/0145445599234005 [DOI] [PubMed] [Google Scholar]
- 68.Piette JD, Rosland AM, Marinec NS et al. . Engagement with automated patient monitoring and self-management support calls experience with a thousand chronically III patients. Med Care 2013;51:216–23. 10.1097/MLR.0b013e318277ebf8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Pratt SI, Naslund JA, Wolfe RS et al. . Automated telehealth for managing psychiatric instability in people with serious mental illness. J Ment Health 2015;24:261–5. 10.3109/09638237.2014.928403 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Shapiro JR, Bauer S, Andrews E et al. . Mobile therapy: use of text-messaging in the treatment of Bulimia nervosa. Int J Eat Disord 2010;43:513–19. 10.1002/eat.20744 [DOI] [PubMed] [Google Scholar]
- 71.Smith B, Harms WD, Burres S et al. . Enhancing behavioral health treatment and crisis management through mobile ecological momentary assessment and SMS messaging. Health Informatics J 2012;18:294–308. 10.1177/1460458212445349 [DOI] [PubMed] [Google Scholar]
- 72.Harrison V, Proudfoot J, Wee PP et al. . Mobile mental health: review of the emerging field and proof of concept study. J Ment Health 2011;20:509–24. 10.3109/09638237.2011.608746 [DOI] [PubMed] [Google Scholar]
- 73.Hetrick SE, Dellosa MK, Simmons MB et al. . Development and pilot testing of an online monitoring tool of depression symptoms and side effects for young people being treated for depression. Early Interv Psychiatry 2015;9:66–9. 10.1111/eip.12127 [DOI] [PubMed] [Google Scholar]
- 74.Hareva DH, Okada H, Kitawaki T. Supportive intervention using a mobile phone in behavior modification. Acta Med Okayama 2009;63:113–20. [DOI] [PubMed] [Google Scholar]
- 75.Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005;10(Suppl 1):6–20. 10.1258/1355819054308576 [DOI] [PubMed] [Google Scholar]
- 76.Ben-Zeev D, McHugo GJ, Xie H et al. . Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group. Schizophr Bull 2012;38:396–404. 10.1093/schbul/sbr171 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Staring AB, Mulder CL, van der Gaag M et al. . Understanding and improving treatment adherence in patients with psychotic disorders: a review and a proposed intervention. Curr Psychiatry Rev 2006;2:487–94. 10.2174/157340006778699738 [DOI] [Google Scholar]
- 78.DiMatteo MR, Lepper HS, Croghan TW. Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Intern Med 2000;160:2101–7. 10.1001/archinte.160.14.2101 [DOI] [PubMed] [Google Scholar]
- 79.Baumeister H, Reichler L, Munzinger M et al. . The impact of guidance on internet-based mental health interventions—a systematic review. Internet Intervent 2014;1:205–15. 10.1016/j.invent.2014.08.003 [DOI] [Google Scholar]
- 80.Ben-Zeev D, Kaiser SM, Brenner CJ et al. . Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013;36:289–97. 10.1037/prj0000019 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Doherty G, Coyle D, Matthews M. Design and evaluation guidelines for mental health technologies. Interact Comput 2010;22:243–52. 10.1016/j.intcom.2010.02.006 [DOI] [Google Scholar]
- 82.Hollis C, Morriss R, Martin J et al. . Technological innovations in mental healthcare: harnessing the digital revolution. Br J Psychiatry 2015;206:263–5. 10.1192/bjp.bp.113.142612 [DOI] [PubMed] [Google Scholar]
- 83.Arora S, Yttri J, Nilsen W. Privacy and security in mobile health (mHealth) research. Alcohol Res 2015;36:143–50. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2015-008362supp1.pdf (96.8KB, pdf)
bmjopen-2015-008362supp2.pdf (314KB, pdf)
bmjopen-2015-008362supp3.pdf (209.5KB, pdf)