Developing a mHealth Routine Outcome Monitoring and Feedback App (“SMART Track”) to Support Self-Management of Addictive Behaviours

Abstract

Background: Routine outcome monitoring (ROM) has been implemented across a range of addiction treatment services, settings and organisations. Mutual support groups are a notable exception. Innovative solutions are needed. SMART Track is a purpose built smartphone app designed to capture ROM data and provide tailored feedback to adults attending Australian SMART Recovery groups for addictive behaviour(s).

Objective: Details regarding the formative stage of app development is essential, but often neglected. Improved consideration of the end-user is vital for curtailing app attrition and enhancing engagement. This paper provides a pragmatic example of how principles embedded in published frameworks can be operationalised to address these priorities during the design and development of the SMART Track app.

Methods: Three published frameworks for creating digital health technologies (“Person-Based Approach,” “BIT” Model and IDEAS framework) were integrated and applied across two stages of research to inform the development, design and content of SMART Track. These frameworks were chosen to ensure that SMART Track was informed by the needs and preferences of the end-user (“Person-Based”); best practise recommendations for mHealth development (“BIT” Model) and a collaborative, iterative development process between the multi-disciplinary research team, app developers and end-users (IDEAS framework).

Results: Stage one of the research process generated in-depth knowledge to inform app development, including a comprehensive set of aims (clinical, research/organisation, and usage); clear articulation of the target behaviour (self-monitoring of recovery related behaviours and experiences); relevant theory (self-determination and social control); appropriate behavioural strategies (e.g., behaviour change taxonomy and process motivators) and key factors that may influence engagement (e.g., transparency, relevance and trust). These findings were synthesised into guiding principles that were applied during stage two in an iterative approach to app design, content and development.

Conclusions: This paper contributes new knowledge on important person-centred and theoretical considerations that underpin a novel ROM and feedback app for people with addictive behaviour(s). Although person-centred design and best-practise recommendations were employed, further research is needed to determine whether this leads to improved usage outcomes.

Clinical Trial Registration: Pilot Trial: http://anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377336.

Introduction

Routine Outcome Monitoring (ROM), or the systematic, repeated assessment of client progress throughout treatment (1), is an integral component of evidence-based healthcare for a range of chronic conditions (2), including addictive behaviours (3, 4). Traditionally, ROM has been performed using clinician-rated instruments (1). Over recent years, the importance of capturing the client perspective has been acknowledged (5). ROM data is important for treatment planning and quality assurance (6–9) and allows organisations to understand, evaluate and improve service delivery (9, 10). From a research perspective, ROM data provides insight into which clients receive the most benefit from services and how variations in care may impact engagement and outcomes (9).

Meta-analytic and/or systematic reviews largely report ROM has positive clinical benefits (1, 11, 12). Feedback from ROM assessments (rather than the assessment process in and of itself) may be central to these benefits (13), although further evidence is needed (14). Positive outcomes are more likely when feedback is immediate, personalised, delivered in an engaging and collaborative way and reflects change over time (13, 15). The vast majority of existing ROM approaches provide clinician feedback only (16, 17). This is unfortunate as direct client feedback may enhance the positive impact of ROM on treatment outcomes (1). Improving client involvement in the feedback process is an important clinical and research priority (18).

Various approaches to ROM have been implemented across a range of mental health (19) and addiction (20, 21) treatment services, settings and organisations. Mutual support groups are a notable exception. “Mutual support” refers to the reciprocal provision of social, emotional and informational support by group members undergoing recovery from addiction (22). Although accumulating evidence points to the importance and benefit of participating in mutual support (23–25) a major limitation in developing a strong evidence base has been the lack of systematic outcome data evaluating routine service provision. Although 12-step models are traditionally the most well-known and accessed model of mutual support (26), other approaches (e.g., SMART Recovery) are gaining momentum. SMART was originally developed as an alternative to 12-step approaches, with the major distinction being that it employs principles and strategies from cognitive behavioural therapy (CBT) and motivational interviewing [MI (27)]. Furthermore, unlike other clinically endorsed (3, 6) models of mutual support for addictive behaviours (e.g., 12-step approaches), SMART Recovery groups utilise a trained facilitator to oversee the conduct of the group. This provides a unique opportunity for encouraging ROM and feedback as a standard component of the groups.

Integration of ROM and feedback into clinical services is not without challenges (28, 29). Factors that can undermine engagement with ROM and feedback include the “time burden” associated with completing, scoring, interpreting or discussing outcome assessments (28, 30). Scepticism regarding the perceived relevance of the outcome(s) assessed and/or feedback generated has also been noted (31, 32). Within addiction services, client engagement, retention and follow-up are well-documented challenges (33), therefore the introduction of ROM instruments must be brief and the turn-around of feedback rapid (13). Few of the validated tools developed to assess treatment outcome within drug and alcohol settings [e.g., Addiction Severity Index (34)], have been investigated within the context of ROM and feedback (13). The utility of these instruments for continuous monitoring and feedback is therefore unknown. Furthermore, the principles of person centred care and recovery oriented service provision (35) would suggest that ROM should be holistic and multi-dimensional (36). Given the importance of brevity, this further complicates the process of instrument selection.

The idea of using technology to track progress within healthcare settings is not new but current approaches are limited (37), and further research (38) and innovation (39, 40) is needed. Unlike other Health Information Technology approaches (e.g., web-based platforms), mobile health applications (mHealth apps) have the potential to offer a quick, easy, interactive and engaging platform for tracking and accessing information about health and health-related behaviours (41). As smartphone ownership and usage is commonplace (42) smartphone apps have the added benefit of engaging individuals in real-time, in everyday situations and offering moment-to-moment, tailored content and support as needed (43). Immediacy of feedback and personalisation have been identified as factors that may increase responsiveness and ongoing motivation to utilise ROM systems (44). Accordingly, the functional capabilities of a smartphone app makes it the ideal platform to administer ROM and feedback, not only for putting the client at the centre of the ROM and feedback process, but as an engaging and streamlined mechanism for providing timely and personalised feedback across time.

Recent systematic reviews of digital recovery support services (45), digital measurement feedback systems (37, 38) and addiction-related mHealth apps [e.g., (46–49)] point to the promise of utilising technology for the purposes of ROM and feedback. However, a key limitation is the ever-increasing gap between availability of mHealth apps and scientific validation (47, 50–52). It is not surprising then that an international workshop of experts in digital health identified improved evaluation of digital health technologies as a key priority for future development efforts (53). Central to this is an improved focus on theory-informed content and design (53).

Improved attention to user engagement is also warranted, as mHealth apps are characterised by high levels of attrition. According to recent figures, 66% of app users drop out after the first use and 80% drop out within the first month (54). Inadequate consideration of the needs and preferences of the end user throughout the development process has been implicated (53, 55, 56). Fortunately, there are now several frameworks (55, 57, 58) available to ensure that app development is grounded in a sound understanding of the end user; incorporates theoretically-informed and evidence based content; and employs best practise methods for app development.

Objectives and Importance

Details regarding the formative stage of mHealth development are essential (59, 60), but often neglected in published reports (53, 61). Improved consideration of the end user throughout development is vital for curtailing attrition and enhancing engagement (55, 56). Ensuring that appropriate theory is used to guide development is recommended for both mHealth apps (51, 53) and ROM and feedback systems (18, 37). This paper describes how published guidelines were used in the current study to ensure that these priorities were addressed throughout the development of a ROM and feedback mHealth app (“SMART Track”) for people attending Australian SMART Recovery groups (online and/or face to face). The preliminary evaluation of this app using a pilot study with nested qualitative evaluation and economic analysis is reported separately (62, 63).

Materials and Methods

This paper addresses development items from the CONSORT E-Health (60) and mERA (59) checklists. This study has been approved by the University of Wollongong and Illawarra Shoalhaven Local Health District (ISLHD) Health and Medical Human Research Ethics Committee (2018/099; HREC/18/WGONG/34).

Summary of Frameworks Utilised

The development of the SMART Track app was informed by three published frameworks that offer guidance on the creation of digital health technologies. Although each model can be used in isolation, we chose to combine these approaches to ensure that app development was informed by a more comprehensive set of guidelines that included foci related to the end-user [i.e., “person”; “Person-Based” (55)], best practise recommendations for mHealth development [“BIT” Model (57)] and a collaborative, iterative development process between the research team, app developers and participants [IDEAS framework (58)].

Firstly, from a person based perspective, app design, development, and content was informed by an in-depth understanding of SMART Recovery participants and the associated context/environment in which SMART Track will be used (55). Central to this approach was the conduct of preparatory qualitative work and the development of “guiding principles” which ensured that key decisions (e.g., including or rejecting app features and/or content) remained faithful to the understanding of the person, their context and the design objectives (55). Secondly, the BIT-Model represents a mechanism for improving the success with which researchers and developers are able to translate the clinical aim(s) of an intervention into an “implementable” technological solution (57). It was therefore utilised in the current study to ensure that central theoretical and technological considerations were adequately addressed. Finally, the IDEAS framework is designed to enhance the quality, relevance and likely efficacy of digital health interventions (58). It arose from a multi-disciplinary collaboration between researchers, designers and engineers and capitalises upon the strengths from each discipline by integrating behavioural theory, human-centred design and rigorous evaluation (58). Guidance pertaining to the formative stages of app development (“Integrate” and “DEsign”) were employed. It is beyond the scope of the current paper to discuss these frameworks in detail. Rather, we focus on how they were used to inform the design, content and development of SMART Track.

Integrating the Person-Centred, BIT-Model and IDEAS Framework

Our approach to integrating the three frameworks in the current study is described detail in Figure 1, and summarised below.

Figure 1.

Integration of Person-Based, BIT Model, and Ideas Framework.

Stage One: In-depth Understanding

The initial stage of development focused on generating an in-depth understanding of people (i.e., SMART Recovery participants and SMART Recovery group facilitators), theory and behavioural strategies to guide app development. Stage one of development was informed by the guidance offered by the “Why” and “How (Conceptual)” stages of the “BIT Model” (64) and the “Integrate” phase of the IDEAS framework (58). These insights were synthesised into a set of guiding principles [person-based (55)] to inform app development, content, and design.

Procedure

The initial stage of development was an iterative process informed by: (a) qualitative feedback from SMART Recovery participants and SMART Recovery group facilitators; (b) clinical and research expertise from the expert advisory and steering committees; (c) a multi-disciplinary workshop facilitated by the customer experience agency contracted for app development and design (GHO, Sydney); and (d) key findings from the literature.

Qualitative Insights

Qualitative insights were derived via a series of 1:1 semi-structured telephone interviews with a sample of SMART Recovery participants (N = 20) and a semi-structured telephone focus group conducted with a sample of SMART Recovery group facilitators (N = 8). To allow rapid turn-around of feedback, at the end of each interview the qualitative researcher generated a brief summary of interview content (focusing on recommendations for app content, and barriers and facilitators to app use). A detailed account of the qualitative evaluation has been reported separately (65).

Clinical and Research Expertise

The expert advisory committee (n = 12) comprised researchers, clinicians, staff affiliated with SMART Recovery and/or Universities across the states of New South Wales, Queensland, and Victoria in Australia. The expert advisory committee initially met via tele/video conference fortnightly, and then monthly as the study progressed. The steering committee (n = 8) comprised members of the research team (chief investigator, principal investigator at SMART Recovery and trial-coordinator) and representatives from key health, non-government and Aboriginal Community Controlled organisations. Steering committee meetings were scheduled at least twice per year via tele/videoconference. Members of the expert advisory and steering committees also provided additional feedback via email and/or phone between meetings as needed.

Multi-Disciplinary Workshop

The multi-disciplinary workshop was facilitated by customer experience agency GHO at their Sydney office and attended by members of the development team (Executive Strategy Director, Senior UX/UI Designer, Front End Developer, Executive Creative Director, Project Manager) and expert advisory committee (researchers, clinicians and/or SMART Recovery group facilitators). Group discussion explored the objectives of the project (for the ROM and feedback tool, research team, funding body and SMART Recovery); proposed intervention features; factors that could influence engagement; and examples of how technology can be used to increase appeal/engagement with self-monitoring.

Key Findings From the Literature

The trial-coordinator in collaboration with members of the expert advisory committee regularly reviewed the published literature throughout the project to keep abreast of key findings pertaining to ROM and feedback (development, evaluation, engagement, tools, strategies, and/or content). Literature searches were pragmatic and ongoing as a systematic review of the literature was beyond the resources of the current project.

Stage Two: App Development

Stage two focused on applying the learnings from stage one (described below under results) to the design, content and development of the SMART Track app.

Procedure

Development [DEsign phase (58)] proceeded according to an iterative process informed by the outcomes from stage one; the digital user experience (UX) and creative expertise of GHO; and feedback from the research team, SMART Recovery participants and SMART Recovery group facilitators. Accordingly, the features (“what”), timing (“when”), delivery, design, and content (“how”: technical) of SMART Track evolved over time in-line with feedback received.

Multi-Disciplinary Workshop

A second multi-disciplinary workshop was held to refine the proposed content and features of the ROM and feedback (based on the learnings from stage one). This workshop was again facilitated by GHO and attended by members of the development team (Senior UX/UI Designer, Front End Developer, Executive Creative Director, Project Manager) and expert advisory committee (researchers, clinicians, and/or SMART Recovery facilitators).

Prototyping and Feedback

App development was broken into three “sprints,” each focusing on the design and functionality of a discrete section of the app. For each sprint, GHO used the software platforms “Sketch” to create high fidelity designs and “InVision” to present prototypes for feedback. Prototypes were presented to the research team in the first instance (via the video conferencing software “Zoom”) and amended following collaborative input from GHO and the research team.

Amended prototypes were then reviewed during nine individual usability testing sessions (three sessions per “sprint”). Participants comprised a convenience sample of SMART Recovery participants (n = 5), SMART Recovery group facilitators (n = 3), and the research team (n = 1). Sessions were facilitated by the Senior UX/UI Designer at GHO in collaboration with the GHO Project Manager and/or Research trial-coordinator. Within each session the participant was asked to use the prototype (via Smart phone or Web), and notes were taken about how the participant interacted with the prototype (e.g., facial expressions/body language conveying confusion and/or excitement) and any comments made. Participants were given time for spontaneous use and were also prompted with various scenarios (e.g., imagine that you wanted to add a personal motivation for change—where would you go?). The prototype was then amended in line with the verbal and non-verbal feedback collected and re-presented to the research team for final discussion, feedback, amendments and sign-off.

Information Synthesis

Following the methods outlined above, stage one findings were derived from the qualitative interviews with SMART Recovery participants and the focus group with SMART Recovery group facilitators; the clinical, research and development expertise of the research team and our collaborators; and a thorough understanding of relevant literature. Information was synthesised using the following methods. Content summaries generated by the qualitative researcher were reviewed by the trial-coordinator. Information pertaining to usability and acceptability was extracted. This information was summarised alongside knowledge derived from the literature and used to facilitate team discussion pertaining to app content and features. This in turn was used to generate content for each of the key considerations detailed within the Person-Centred, BIT-Model and IDEAS Framework, which was further reviewed and discussed until consensus was reached. Stage two findings were derived from applying stage one findings to the design, development and content of the SMART Track app.

Results

Stage One: In-depth Understanding

The “in-depth understanding” used to inform the development of SMART Track is described in detail in Figure 2 and summarised below according to the relevant components of each framework. Stage one culminated with the development of a set of “guiding principles” (“person based” (55) See Figure 2].

Figure 2.

Stage one findings used to inform app development, as a function of theoretical model.

BIT Model: “Why”

From a funding perspective, the project remit was to develop a ROM and feedback (i.e., self-monitoring) app. From a clinical perspective, the aim was to develop a tool that participants would experience as useful in supporting their recovery (clinical aim). If it was experienced as useful, it seemed likely that participants would be more willing to use the tool beyond the initial download (usage aims), which was important as regular participant engagement with ROM and feedback represents an important opportunity for improving understanding of participant outcomes within SMART Recovery (research/organisation aim).

IDEAS Framework: “Empathise”

Key factors with the capacity to influence participant engagement (thereby perceived utility) of the app are outlined in Figure 2 (see also Supplementary Table 1). Participant feedback and research findings highlighted that trust in the app is of paramount importance. From this it was concluded that the rationale for data collection (and how data will be used, stored, and protected) must be made clear and the relevance to the individual highlighted. The development work also highlighted the importance of ensuring that the app is experienced as unobtrusive and that it both complements and reinforces what people are already doing to support their recovery (e.g., group attendance, working towards meaningful goals, self-management skills). Furthermore, it must be “easy” (to download, set up, navigate and use), user-friendly (e.g., non-stigmatising language, language that is simple, concrete, confident and helpful) and encouraging (e.g., “celebrate successes”). As SMART Recovery group facilitators are central to the conduct of groups, it became apparent that participant engagement may be compromised if the app is perceived by facilitators as unhelpful or disruptive. To support the longevity of the app, team discussions and expert input highlighted the importance of developing the app with future platforms in mind.

IDEAS Framework: “Specify”

The behaviour of interest targeted by the ROM and feedback app is regular self-assessment of progress. To enhance the perceived relevance and utility of the app to SMART Recovery participants our preparatory work demonstrated that the target content for self-assessment must be experienced by participants to be meaningful. Accordingly a range of behaviours and experiences were included to reflect the diversity of SMART Recovery participants. ROM questions were drawn from validated self-report instruments [e.g., Substance Use Recovery Evaluator (66); K-10 (67, 68); Screener for Substance and Behavioural Addictions (69)]; and as needed, wording was modified in line with participant feedback. Further details are available in the published protocol (63).

IDEAS Framework: “Ground”

To optimise the likelihood that participants would be willing to engage in regular self-assessment, this project drew from the program principles of SMART Recovery Australia (SRAU) (70) and insights from Self Determination Theory (71) and Social Control Theory (72). Specifically, we sought to ensure that the app complemented what participants were already doing to support their recovery. Accordingly, guided by the program principles of SRAU, SMART Track was developed so that it could be used alongside SMART Recovery group participation as a way of enhancing competence and confidence (“self-management”); facilitating meaningful discussion with peers (“mutual aid”) and supporting individual recovery goals (“choice”). Consistent with these principles, Self-Determination Theory highlights the importance of utilising competence, positive emotional experience and autonomy to motivate desired behaviour(s) (71). Finally, drawing from Social Control Theory, as connexion with others motivates “desirable” behaviour [e.g., via modelling and accountability (72)] we also sought to include features that would support connexion with and learning from others (e.g., lived experience storeys and prompts for participants to discuss their feedback at their SMART Recovery group).

BIT Model: “How (Conceptual)”

The preparatory work also identified “self-monitoring” and “feedback” (64) as key strategies for facilitating participant engagement in self-assessment. Additional strategies drawn from the Behaviour Change Taxonomy [e.g., action planning, prompts/cues and positive reinforcement (73)] and process motivators [e.g., strategies to facilitate choice, competence and personalization (58)] were also identified as potential ways of enhancing engagement with the process of self-monitoring.

Stage Two: App Development

BIT Model: “What”

The features included in SMART Track [“What”: BIT Elements (57)] are presented in Figure 3 according to the underlying behavioural strategies and process motivators [“How”: Conceptual (57)]. The relationship between these features and learnings from stage one are summarised below.

Figure 3.

Relationship between app features and associated behavioural strategies.

Features to Support Individual Recovery Journey

To enhance compatibility between SMART Track and what participants already do to support their recovery, several SMART Recovery tools are featured. Firstly, a “7-day plan” allows participants to specify details of an action plan and record their progress towards personally meaningful goals. Secondly, an interactive “urge log” allows participants to rate the strength of an urge as it is experienced, receive coping strategies and motivational messages to assist with urge management and to record associated triggers, outcome(s) and any (un)helpful management strategies. Additional SMART Recovery programme content and self-management tools are also available through a “resources” section. Participant expertise in self-management is acknowledged by including “hints, tips and motivations” derived directly from qualitative interviews with SMART Recovery participants and “motivational stories” of self-management and/or recovery.

Features to Respect Participant Autonomy

Respect for participant autonomy is made via several features that promote choice and control. For example, to allow people to monitor personally meaningful outcomes the app includes both multidimensional outcome assessment, and the option of tracking progress towards personally meaningful goals. There is also some flexibility in the timing and frequency of push notifications. Furthermore, the wording of push notifications (e.g., “log now,” “log later”) is such that ultimately it is up to the individual whether or when they choose to engage in the various self-monitoring tasks.

Features to Facilitate Positive Emotional Experience and Relatedness

When participants achieve personally meaningful goals, animations (celebratory confetti) are used to celebrate success. Progress on the various outcome assessments is highlighted and reinforced (graphs across time) and the wording of written feedback is such that all efforts are encouraged. Although a social networking feature was beyond the scope of the current app (due to funding and other pragmatic constraints), participants are encouraged to consider discussing their feedback with trusted others (e.g., SMART Recovery group). They can also use the app to log “helpful contacts” and are prompted to consider accessing support when progress is challenging and/or urges are experienced.

BIT Model: “When”

To balance client autonomy with routine collection of outcome data, the app includes a mix of user-defined workflows and time-based rules (64). For example, participants are prompted to link the completion of the ROM items and their 7-day plan to attendance at a SMART Recovery group; tick off their completed tasks as they are achieved and log their urges following each self-reported experience (i.e., user-defined). ROM items not completed at the end of each 1-week cycle (proposed ROM frequency) will count as “missing data” and the next week of progress monitoring will commence (i.e., time-based rules).

“How”: Technical Characteristics

Consistent with the guiding principles that were developed, and informed by the creative and technical expertise of GHO, decisions regarding medium, complexity, aesthetics and personalisation were made to optimise the likelihood that the app would be experienced as simple, engaging, unobtrusive and confidential.

Medium

Text was the primary medium employed to communicate information within the app. Given the diverse experience of SMART Recovery participants, several strategies were employed to maximise accessibility. Firstly, where possible, written information was supplemented by visual prompts (e.g., when orienting people to the app) or content (e.g., when providing feedback). Efforts were made to convey content using simple, brief sentences and a conversational tone. Where detailed information was required (e.g., privacy policy), key elements were summarised in the app, with more detailed content accessible via a link.

Complexity

To provide context and assist with completion, ROM items are prefaced by a brief description of item content and the rationale behind collecting this information. To simplify use, ROM items are completed using radio buttons, multi-select buttons, drop down menus and sliding scales. Free text response options are used sparingly, and restricted to capturing idiosyncratic outcomes (e.g., 7-day plan) and reflections (e.g., used to capture personal reflections on feedback received).

Aesthetics

The colour scheme and style of SMART Track was guided by the need to ensure that it complemented SMART Recovery branding, whilst not being instantly recognisable as to respect an individual's privacy. Otherwise, aesthetics were selected by GHO with a view to optimise ease of use (e.g., layout, interaction cues, text style, and size) and participant engagement (e.g., colour scheme, graphics).

Personalisation

To optimise ease of use, personal relevance and participant engagement a range of features were included to support personalisation. Participant responses to the ROM items directly inform tailored visual (graphs over time) and written feedback. Participants can elect to add one or more personal motivations for change (visual, video and/or written). This information can be amended as many times as needed and is displayed prominently within the app. A “contacts” section allows participants to enter the details of key support people. The “urge log” allows participants to enter the helpful strategies and/or motivations they would like to see when they next experience an urge.

SMART Track: An Overview of the Resultant Mhealth ROM and Feedback App

In summary, SMART Track consists of core ROM and feedback functionality and several additional features to enhance engagement (Resources; Customisable support(s) and personal motivation(s); Interactive urge log; and Pop up motivations and self-management strategies; Figure 4). Further details are available in the published protocol (63), and briefly summarised below. SMART Track provides multi-dimensional ROM assessment and feedback (ROM items are detailed in Supplementary Table 2 as a function of target domain and assessment frequency). Feedback consists of tailored visual and written feedback across eight domains (7-day plan, behaviour of concern, effect of substance use, self-care, relationships, outlook on life, resources and mental health; Figure 5). The “Resources” screen delivers ten pieces of content. This is distributed across seven self-management resources (including SMART Recovery resources) and three motivational storeys [extracted with permission from the “Lives of Substance” website (74)]. Participants have the option of tailoring app content by uploading key contact number(s)/support services and/or personal motivation(s) for change (photo, audio, video, and/or written) into the “Me” section of the app. In addition to tracking the number, frequency and strength of urges, when the participant experiences an urge, the interactive urge log prompts participants to manage their urges, log triggers, and reflect on how to maintain/improve effective urge self-management. Participants also receive a “pop” up message when they open the app for the first time each day. SMART Track is freely available for Android (75) and Apple (75) devices.

Figure 4.

(A) SMART Track Overview and (B) Participant “Walk Through”.

Figure 5.

(A) Overall progress screen and (B) Progress screen for the individual domain of “self-care”.

Discussion

This paper illustrates how three published frameworks were integrated and applied to the design and development (55, 57, 58) of an app designed to routinely monitor outcomes and provide feedback to participants attending SMART Recovery groups. It adds to the growing body of technology based solutions for ROM and feedback (37, 38, 45). By detailing the formative stage of SMART Track, this paper provides a pragmatic example of how principles embedded in these models can be operationalised to address key priorities in mHealth app development. This paper adds to the growing body of literature that has harnessed technology to overcome traditional barriers to ROM [e.g., scoring and providing tailored feedback (76)].

Strengths

Our user-centred and theory informed approach to the design and development of SMART Track addresses key limitations (51, 53) within the mHealth literature. Evidence from mHealth apps designed to support self-management of other chronic conditions suggest that future research efforts should focus on “a simple and user-friendly-designed mHealth system, data confidentiality, lay language use for structured and automated feedback or advice, positive motivation and improving engagement” (77). When developing tools to assess patient outcomes, early, meaningful and ongoing consultation with key stakeholders is central to maximising acceptability and person engagement (78). End-users want to be actively involved and consulted throughout the development process (79). These principles were at the forefront of the current body of work.

Challenges and Opportunities

The use of cloud functions for collecting and storing data means that SMART Track needs a reliable internet connexion to function. This is a common limitation within the digital health technology literature (37). Reliance on internet connectivity not only impacts the user experience, but may also inadvertently disadvantage certain user-groups (45). Improving offline functionality represents an important challenge for future development and evaluation efforts. Although considerable efforts were made to support a personalised experience (e.g., tailored written and visual feedback derived from participant responses; user led goal setting; choice over when/whether to receive goal-setting notifications; option to add personal motivations to support urge management) funding and time constraints meant that we were unable to action all participant priorities. For example, common requests included the option of being able to share feedback with trusted others (e.g., via shared log-in or direct export options); the option of participating in a discussion board feature (e.g., to share resources and support) and the capacity to complete self-management activities and worksheets within the app. Future research would benefit from continued efforts to improve the user experience and identify the demographic, clinical and contextual factors that may influence engagement with and utility of SMART Track. For example, by using routinely captured data analytics, and/or geospatial data SMART Track could be used to identify participant characteristics, contextual factors and/or app features associated with engagement, attrition and clinical outcomes (39, 40). Such information could have potential clinical benefits, for example providing early warning signs to participants at risk of poor outcomes, thereby allowing them to seek additional support and potentially circumvent a relapse [e.g., see (80)]. Finally, although our integration of three published frameworks ensured that the development and evaluation of SMART Track was informed by the needs and preferences of the end-user (55); best practise recommendations for mHealth development (57) and a collaborative, iterative development process (58), the process was complicated by overlap between these models. The development of new, comprehensive guidelines applicable to the development and evaluation of mHealth apps offer an important solution moving forward [e.g., The Accelerated Creation-to-Sustainment model (81)].

Conclusions

Although the principles derived from models used in the development of SMART Track facilitated the process of app design and development, future research is needed to determine whether this leads to improved outcomes such as uptake, engagement and sustained use. One of the most critical indicators of this app's success will be the frequency that users complete the routine outcome assessments embedded in the app. Review of the extent that various features of the app are utilised will provide insights into which elements of the various models appear to facilitate user engagement. A pilot study has been conducted (62, 63) and provides preliminary evidence for the feasibility, acceptability and utility of SMART Track for ROM and feedback in SMART Recovery groups.

Data Availability Statement

The datasets presented in this article are not readily available because the authors don't have permission to share the data. Requests to access the datasets should be directed to alisonbe@uow.edu.au.

Ethics Statement

The studies involving human participants were reviewed and approved by the University of Wollongong and Illawarra Shoalhaven Local Health District (ISLHD) Health and Medical Human Research Ethics Committee (2018/099; HREC/18/WGONG/34). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AKB: conceptualisation, methodology, formal analysis, investigation, data curation, writing—original draft, writing—review and editing, project administration, and funding acquisition. PK: conceptualisation, methodology, resources, writing—review and editing, supervision, and funding acquisition. FD, ALB, LH, and AS: conceptualisation, methodology, writing—review and editing, supervision, and funding acquisition. VM: conceptualisation, methodology, writing—review and editing, and supervision. JN and JK: conceptualisation, methodology, writing—review and editing, and funding acquisition. RG: methodology, formal analysis, investigation, and writing—review and editing. AA and RM: conceptualisation, methodology, resources, writing—review and editing, and funding acquisition. RC and MM: methodology, software, resources, data curation, and writing—review and editing. Authorship follows ICMJE recommendations (82). All author offered critical revisions to the manuscript for important intellectual content, have approved the final version of this manuscript, agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved, and made substantial contributions to study conception, design, methods, and/or the content and features of the ROM and feedback system under investigation.

Conflict of Interest

RM is Executive Director of SMART Recovery Australia. AA is employed by SMART Recovery as the National Program Manager and Trainer. PK, FD, ALB, AS, LH, VM, AKB, JK, and AA volunteer as members of the SMART Recovery Australia Research Advisory Committee. RC is the Exec. Creative Director and MM is the BBA Technical Lead at GHO, the company contracted to design and develop the app. The potential and/or perceived conflict of interest is negligible. The role of study investigators on the research advisory committee and/or as an employee of SMART Recovery is freely available on the SMART Recovery Australia website (and study participants can be directed to this information as required). The role of GHO is the design and development of SMART Track is clearly detailed in app store description(s). Furthermore, the team responsible for informing study design and overseeing the conduct of the study and data analysis also consists of researchers, clinicians, and statisticians independent from SMART Recovery. An independent qualitative researcher was recruited to collect and analyse the qualitative data. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.677637/full#supplementary-material