Skip to main content
PMC home page
JMIR Research Protocols logoLink to JMIR Research Protocols
. 2018 Nov 12;7(11):e10121. doi: 10.2196/10121

Issues in Child and Adolescent Inpatient Assessment and Evaluation After Discharge: Protocol for App Development and a Randomized Controlled Trial

Kristian Hansson 1,✉, Björn Axel Johansson 1,2, Claes Andersson 3, Maria Rastam 2, Sophia Eberhard 2
Editor: Gunther Eysenbach
Reviewed by: Marianne Bonnert, Armen Arevian, Noor Beckwith
PMCID: PMC6256105  PMID: 30425029

Abstract

Background

New methods are needed for collecting data of in- and outpatients and for improving outpatient compliance after discharge. Mobile technologies, such as smartphone apps, have shown promising results, (eg, helping unwell people by offering support and resources). Screening for the condition, including comorbidities, is a vital part of psychiatric care. Comorbid conditions, especially in emergency evaluation, are often missed, leading to inaccurate diagnosis and treatment. One way of improving diagnostic accuracy is to use a structured diagnostic process. Digitalized screening and follow-up have the advantage of making administration and scoring easier and less time consuming, thereby increasing response rate. To address these problems, we decided to create a smartphone app called The Blue App. The Blue App was developed through 6 steps, described in the manuscript.

Objective

The aim of this paper is to describe (1) the development of The Blue App and (2) 2 planned research studies to evaluate the app.

Methods

Two studies will be performed. Study 1 has a descriptive design, mapping comorbidities before and after the introduction of The Blue App. Study 2 has a randomized controlled design, measuring compliance with outpatient treatments as well as depressive symptoms, rated as changes in Montgomery-Åsberg Depression Scale scores during a 1-year follow-up.

Results

We have described app development. Data collection for Study 1 started in autumn 2017. Study 2 will start in autumn 2018. We expect to have enrolled the 150 patients in Study 2 by December 2019. Final results will be published in a scientific journal.

Conclusions

A technically advanced and easy-to-use Web-based mobile phone app corresponding to the unit’s needs was developed, and 2 studies are planned to evaluate its usefulness.

International Registered Report Identifier (IRRID)

RR1-10.2196/10121

Keywords: app, data collection, mobile phone, research protocol, screening

Introduction

Emergency units with 24/7 evaluation and admission facilities are important services for stabilizing adolescents with mental health crisis [1,2]. On admission to a unit, patients often need extensive mental health resources [3]. Demand for health production is high in emergency units when inpatient stays are short [4].

The largest intervention effect is reached during the first week of admission [5]. Short inpatient stays for acute stabilization are a challenge when it comes to structured diagnostic screening. Despite the availability of validated diagnostic screening instruments, pen-and-paper versions lead to poor response rates and consume resources, making their use less attractive [6]. The current toolbox is outdated for use in our population of adolescents, and new methods are needed for inpatient data collection and for improving outpatient compliance. Mobile technologies, such as smartphone apps, have shown promising results in terms of data collection and as an intervention for behavioral change, but little empirical evidence has been found to date [7,8].

Literature in the field of child and adolescent psychiatric inpatient emergency care is scarce, and studies are limited to characteristics of admitted patients [3]. Therefore, little is known about the course of illness after discharge. In Sweden, there has been no comprehensive evaluation and follow-up of children and adolescents admitted and discharged from psychiatric emergency units.

Diagnosing the condition and screening for comorbidity using validated instruments is a vital part of psychiatric care. Moreover, a structured screening procedure is also perceived as positive and valid by patients and their parents [9].

In psychiatric care, diagnostic interrater reliability among clinicians is weak [10]. Comorbid conditions are commonly missed during psychiatric evaluations, leading to inaccurate diagnosis and treatment. One way of increasing diagnostic accuracy is to use a structured diagnostic process, involving diagnostic screening instruments [11].

Digitalized screening and follow-up make administration and scoring easier and less time consuming for patients, parents, and staff. This helps increase the frequency of completed systematic psychiatric screenings. It also has the advantage of availability, as questionnaires can be sent to and answered on patients’ smartphones, increasing response rates [6]. This structured data collection would enable studies of child and adolescent psychiatric emergency inpatients in further research.

Apps have proved effective in helping unwell people, by offering support, resources, and information [7]. In recent years, several new smartphone-based solutions have been developed for mental health in general and specifically for child and adolescent psychiatry [12-16]. However, apps developed by health care professionals for adolescents must be innovative, useful, and fun to compete with apps not encouraging healthy behaviors [7].

To address these problems, we decided to create a smartphone app: The Blue App (referring to blue as the color of hope, as well as expressing moodiness). The Blue App is a Web-based tool made to resemble and function as a mobile app.

The aim of this paper is to describe the development of The Blue App and to present 2 research protocols aimed at evaluating the app.

Methods

Development of The Blue App

The process of developing The Blue App, from concept to working product, was divided into 6 steps.

Step 1. Identifying the Need for Quick and Easy Information Gathering

Since 2010, members of our group have introduced and developed interactive voice response (IVR) into Swedish child and adolescent psychiatry. This is a technology in which a server is programmed to use scripts in interaction with the user on their cell phone [9,14]. Johansson et al [17] have shown that emergency patients had their own cell phones and that the response rate among participating patients (N=60) was promising as each individual responded on average 91% (365/402) of their calls. An answering frequency of 100% was shown by 71% (30/42).

Our work with IVR inspired us to further develop technology to meet the unit’s needs regarding improved methods for the diagnostic and follow-up processes. In 2013, we published our first IVR paper [17]. Simultaneously, we were discussing the shortcomings at the unit regarding the identification of comorbidities. We were also discussing the development of a more sophisticated follow-up tool that would provide feedback using a visual presentation of changes in symptoms over time. At the same time, smartphones were becoming more common among adolescents. This enabled us to integrate our previous work with our efforts to widen the use of screening questionnaires in a better package than the older cell phones could offer.

To address the issues related to short inpatient stays and need for improved diagnostic screening, we decided to create a smartphone app based on our previous work.

Step 2. Identifying Desired Information and Choosing Adequate Screening Questionnaires

In addition to clinical evaluations, we wanted to screen for and describe the psychiatric conditions that were most frequently presented at emergency units. The most common reasons for admission to the unit were suicidal ideation, severely depressed mood, and acute crises. Our experience corresponds well with literature on child and adolescent psychiatry with regard to frequent comorbid states [2]. We also wanted to map changes in symptom severity before and after admission.

The validated screening questionnaires were chosen in collaboration with national key opinion leaders and with regard to frequent comorbid states in child and adolescent psychiatry [18-20].

The following 10 questionnaires were chosen:

  1. Montgomery-Åsberg Depression Scale (MADRS-S): a 9-item diagnostic self-rating questionnaire measuring depressive symptoms during the previous 3 days [21]

  2. Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID): a semi structured diagnostic interview concerning the core psychiatric disorders among children and adolescents [22]

  3. Alcohol Use Disorders Identification Test-Consumption: a 3-item screening questionnaire regarding alcohol use [23]

  4. Drug Use Disorders Identification Test: an 11-item screening questionnaire regarding drug use [24]

  5. Hopkins Symptom Checklist-10: a symptom inventory that measures signs of anxiety and depression [25]

  6. Sheehan Disability Scale: a scale assessing functional impairment in domains of work or school, social, and family life [26]

  7. Autism Spectrum Screening Questionnaire: a 27-item checklist that assesses symptoms characteristics of high-functioning autism spectrum disorder [27]

  8. Swanson Nolan and Pelham Questionnaire: a 30-item questionnaire assessing attention-deficit/hyperactivity disorder symptoms and behavioral problems [28]

  9. Deliberate Self-Harm Inventory-9: a scale showing the frequency of different ways of self-harming [29]

  10. Treatment Satisfaction Scale 2: a 6-item measure of treatment effectiveness [30] (included to evaluate the satisfaction of treatment but not a part of the research protocols)

Step 3. Examining the Feasibility in an Emergency Psychiatric Unit: Pilot Study

To evaluate the feasibility of the screening procedure, a pilot study was performed with adolescents admitted to the psychiatric emergency unit. The first aim was to assess the feasibility of administration of 10 questionnaires in a small consecutive population. The second aim was to learn more about the clinical relevance of the systematic screening, that is, would the results indicate more comorbidities than found in the medical records? Patients admitted to the unit between February 11, 2014, and March 14, 2014, were invited to participate and complete the 10 questionnaires regarding psychiatric morbidity and comorbidity to be included in The Blue App. All screening questionnaires involved using pen and paper.

We included 16 patients (12 girls and 4 boys; age: mean 15.6 (SD 1.31) years). Of these, 12 completed all the pen-and-paper screening questionnaires, whereas 4 patients could not complete their screening procedures due to early discharge. Of 10, 9 diagnoses at discharge were confirmed using MINI-KID. MINI-KID also identified, on average, 5 diagnostic areas of potential psychiatric interest, including the hazardous use of alcohol or substance abuse, none of which had been clearly addressed during the inpatient stay or mentioned in the psychiatric records.

Step 4. Getting Organizational Acceptance: Feasibility Study

The next step was to secure a way forward for The Blue App. As our aim was to use the tool in daily clinical practice, we decided to apply for funding from the funding body for all public psychiatric health care in Scania, South Sweden. All potential new procedures in Swedish public health care require a thorough assessment procedure to clarify the cost-benefit ratio, safety, and potential risks (feasibility study). The aim of this feasibility study was to clarify the requirements of The Blue App and to elucidate the cost-benefit and safety aspects so as to generate a basis for decision making for the funding body. Another part of the feasibility study was to collect knowledge about alternative solutions allowing the same functionality by investigating if comparable products were available on the market.

The feasibility study included about 15 meetings with the IT and finance departments. Clinicians’ needs were highlighted, which resulted in a thorough description of the technical requirements for 90 user cases, including mock-ups, flowcharts, and wireframes. The study concluded that no existing solution was available on the market. The final document was presented to the funding board with the working title “The Blue App.” The board approved funding in May 2015.

Step 5. App Construction

After approval, the next step was to call for public procurement using the feasibility study documents. In total, 3 tenders were considered, and representatives from each company were interviewed. The clinicians working on The Blue App were involved in decision making. Work with the company chosen, Stretch Öresund, proceeded with workshop sessions, creating a shared language and translating clinicians’ needs to the mindset of app architects. The work continued with meetings being held every 2nd week for a 6-month period, during which mutual feedback on the progress of The Blue App was given. The company’s CEO met with the unit staff to demonstrate the app and collect feedback to increase the chance of successful implementation.

The work resulted in an app corresponding with the unit’s needs. The structure of the development process, with frequent and regular meetings, was necessary to avoid misunderstandings. The development process allowed the app architects to gain a profound understanding of psychiatric care and to adapt solutions to the unit’s requirements.

Step 6. Workshop With Adolescents: What Do the Patients Think?

During the development of The Blue App, feedback was obtained through a workshop with adolescent end users. The main aim was to assess the user-friendliness of the app. We arranged the workshop in spring 2016, during which 10 adolescents, 5 boys and 5 girls aged 14-17 years, participated. Of these, 2 adolescents were undergoing psychiatric outpatient treatment. The participating adolescents were introduced to the app and then navigated through The Blue App follow-up user interface. The participants found the app easy to navigate and suggested some minor improvements, such as the integration of graphic design.

Planned Research Studies to Evaluate The Blue App: Research Studies 1 and 2

The objective was to design several studies to evaluate The Blue App. In the first study (Study 1), we aim to assess whether additional comorbid conditions could be identified using The Blue App. In the second study (Study 2), we want to evaluate whether the use of The Blue App can improve outpatient compliance and treatment outcome through feedback of psychiatric symptoms.

Design

Study 1

In Study 1, we want to map the prevalence of psychiatric comorbidity found in the medical records before and after the introduction of The Blue App. The hypothesis is that more diagnoses will be found in the records after the introduction of The Blue App.

Study 2

At discharge, eligible patients will be offered participation in a 1-year follow-up study using a randomized controlled design (Figure 1). After receiving informed consent, patients will be randomized into 3 groups and asked to answer questions regarding depressive symptoms (MADRS-S) at different time intervals over a period of 1 year after discharge. The time intervals were chosen to minimize dropout while being long enough to detect effects in the patient. We undertook a power calculation, as we found no comparable studies. Based on estimated scores on the depression rating scale (MADRS-S) at discharge, we expect scores at discharge to be around 30, with SD 7.1.

Figure 1.

Figure 1

Open in a new tab

Study design with 3 randomization groups for the follow-up study at the Child and Adolescent Psychiatric Emergency unit, Malmö, Sweden. ICD-10: International Classification of Diseases, 10th revision; MADRS-S: Montgomery-Åsberg Depression Scale.

The null hypothesis for Study 2 is that no significant difference is found in the randomized controlled trial; that is, the feedback given through The Blue App would not improve the outcome or compliance. In the alternative hypothesis, we expect all patients to reduce their MARS-S scores during the study period and expect a difference of about 4 points on MADRS-S between intervention and control groups. With a power of .8, we will therefore need 3 groups of 50 patients each in order to detect differences at a 5% level.

Setting and Participants: Studies 1 and 2

Scania in southern Sweden has a population of over 1 million, of which 280,000 are children and adolescents. The university hospital in Malmö, the major city in the area, provides the only psychiatric emergency unit for children and adolescents, with 11 hospital beds. The hospital admits 350 unique patients annually together with a parent for acute stabilization. The most common reasons for admission are suicidal ideations, severely depressed moods, and acute crises. A majority of the patients are aged 13-17 years, with an even gender distribution.

A treatment plan is drawn up by the unit physician together with patients and parents. Nearly 15% of the patients are treated according to the Swedish Compulsory Mental Care Act. The unit team consists of physicians, psychologists, social workers, nurses, and treatment staff, often in collaboration with open-care units, schools, and social authorities. Before discharge, a rescue plan is drawn up together with patients and parents in the event of a recurrent emergency situation. Nearly 95% of the patients are discharged within a week of admission.

Procedure

Study 1

All eligible patients will be offered participation in Study 1 and will be invited to complete the questionnaire (exclusion criteria: language other than Swedish, admission for <24 hours, severely ill, age <12 years old). We expect about 190 of the 350 patients admitted per year to be eligible for inclusion. The feasibility study indicated little attrition, but we anticipate that about 30% might not want to participate, leaving us with about 130 per year for possible inclusion.

Patients and parents will be asked to complete questionnaires on psychiatric symptoms, treatment satisfaction, and quality of life. Included patients will be registered on The Blue App, allowing them to log in via their smartphone using a personal identity number at the log-in page (Figure 2).

Figure 2.

Figure 2

Open in a new tab

Screenshot of The Blue App login screen. Source: Region Skåne.

The patients will then receive a short message service text message with a one-time password, enabling them to enter The Blue App and answer the questionnaires. It is also possible to generate one-time passwords and complete the questionnaires via The Blue App staff administration view. Parents can use their child’s log-in when answering the parental questionnaires.

The screening process at the unit will be divided into 3 stages (Figure 3). The first stage is defined as the first 24 hours at the unit. The second stage, during treatment, is defined as the time between intake and discharge. The discharge stage is defined as the last 24 hours at the unit. Patients and their parents will be given access to the questionnaires at each stage, which they will complete with the help of staff. At the unit, the questionnaires will be administered mainly on tablets, but can also be filled in via the patients’ own devices.

Figure 3.

Figure 3

Open in a new tab

Questionnaires used at the different stages during the inpatient stay at the Child and Adolescent Psychiatric Emergency Unit, Malmö, Sweden. ASSQ: Autism Spectrum Screening Questionnaire; AUDIT-C: Alcohol Use Disorders Identification Test-Consumption; DSHI-9: Deliberate Self-Harm Inventory-9; DUDIT: Drug Use Disorders Identification Test; HSCL-10: Hopkins Symptom Checklist-10; MADRS-S: Montgomery-Åsberg Depression Scale; MINI-KID: Mini International Neuropsychiatric Interview for Children and Adolescents; SDS: Sheehan Disability Scale; SNAP-IV: Swanson Nolan and Pelham Questionnaire; TSS-2: Treatment Satisfaction Scale 2.

Study 2

At discharge, participation in study 2 will be offered to all patients who complete the questionnaires for Study 1 (Figure 3).

All patients in the study will receive treatment as usual at their outpatient units. Patients will use the same log-in procedure as in Study 1. With app-assisted technology, a link to the MADRS-S questionnaire (Figure 4) will be sent to all participants at predefined intervals depending on randomization.

Figure 4.

Figure 4

Open in a new tab

Screenshot of the MADRS-S (Montgomery-Åsberg Depression Scale) questionnaire item 7. Source: Region Skåne.

After discharge, researchers and staff will not be able to trace the group to which a patient is assigned. Patients readmitted to the ward during follow-up will remain in the study.

In group 1, feedback on depressive symptoms will be presented in a graph, together with a brief recommendation [31] on each follow-up occasion (Figure 5). A text below the graph will describe how patients can contact the emergency unit if they experience very severe symptoms or danger, with a button to make a direct call to the 24/7 emergency facilities.

Figure 5.

Figure 5

Open in a new tab

Screenshot of the MADRS-S (Montgomery-Åsberg Depression Scale) feedback screen for group 1. Source: Region Skåne.

Dropout analysis will examine for any group differences in terms of gender, age, and main International Classification of Diseases, 10th revision diagnoses. Information regarding reasons for missed appointments and dropouts will be taken from the medical records. Outcome measures are changes in MADRS-S scores and treatment compliance, as measured by attended outpatient appointments during the study period.

Ethical Approval

No compensation will be offered. Subjects in the study will be protected by the informed consent process. The Regional Ethical Board in Lund has approved the study (2013-06-19; Nr 423/2013).

Statistical Analysis

The two studies will be analyzed separately, and baseline variables will be analyzed with descriptive statistics. Pearson’s chi-square tests will be used to determine differences between the groups in terms of gender distribution and proportion of participants scoring above cut-off in the questionnaires used.

In Study 1, we will use descriptive statistics (mean, SD, median, interquartile range, continued variables, frequency and counts, categorical variations, and exact CIs for proportions) to describe the cohort and analyze differences in the number of psychiatric diagnoses before and after the introduction (t test) of The Blue App [32].

In Study 2, we will analyze differences between the 3 groups in terms of changes in MADRS-S scores and treatment compliance. Outcome measures (MADRS-S scores and dropout from outpatient care) will be analyzed according to intention to treat. Linear mixed model analyses will be used to identify changes over time in depressive symptoms and to examine compliance with outpatient treatments. In these analyses, group, time, and the intercept between the groups and time will be used as fixed effects and subjects as random intercepts.

Results

The app development is described. Data collection for Study 1 started in autumn 2017. Study 2 will be initiated in autumn 2018. We expect to have enrolled 150 patients for Study 2 by December 2019. Funding for both Study 1 and Study 2 have have been granted by Region Skåne, the regional council for health care in south of Sweden. Final results will be published in a scientific journal.

Discussion

Principal Findings

This study is a process description of a child and adolescent emergency inpatient unit, where an app was developed with the aim of identifying comorbidity and improving outpatient compliance and treatment outcomes.

Strengths

The Blue App is a dynamic tool capable of adjustment to new demands, with variable settings, and to other units with potential to improve the transition between inpatient and outpatient care. The Blue App will help us collect data to make it easier to evaluate the work at the inpatient unit and will lay a foundation for future research in the field.

One important factor behind the development process was the support given to the project group from all levels in the organization, from stakeholders to staff [33]. Other crucial factors were related to the project group’s acceptance in the organization and to the committed team members whose perseverance has extended over several years. The organization benefits from the shared visions and team learning [33]. Finally, we want to highlight the importance of being offered enough time to work on this project.

Limitations

The Blue App has been developed in a Swedish context, which limits its generalizability. Questionnaires are in Swedish, thereby excluding non-Swedish speaking patients. In the next version of The Blue App, the questionnaires will be available in the 4 most common languages at the unit (English, Dhari, Arabic, and French).

The design chosen for Study 2, in which selection is based on participation in Study 1, might affect inclusion and, therefore, could be regarded as a weakness. However, participation in Study 1 gives a thorough assessment of comorbidities at baseline, which is the strength of the design.

Conclusions

A technically advanced and easy-to-use Web-based mobile phone app corresponding to the unit’s needs was developed, and 2 studies to evaluate its usefulness are planned.

Acknowledgments

The authors want to acknowledge Peter Höglund for valuable statistical input, and Jakob Täljemark for language improvements. We thank Stiftelsen Lindhaga and Region Skåne, the regional council for health care in south of Sweden, for grants.

Abbreviations

IVR

interactive voice response

MADRS-S

Montgomery-Åsberg Depression Scale

MINI-KID

Mini International Neuropsychiatric Interview for Children and Adolescents

Footnotes

Authors' Contributions: SE, KH, and BAJ initiated the study. CA and MR made substantial contributions to the concept and design. KH, SE, and BAJ were involved in drafting the paper, and MR and CA made significant contributions when reviewing the manuscript. All 5 authors have approved the final version.

Conflicts of Interest: None declared.

References

  • 1.Stewart C, Spicer M, Babl FE. Caring for adolescents with mental health problems: challenges in the emergency department. J Paediatr Child Health. 2006 Nov;42(11):726–30. doi: 10.1111/j.1440-1754.2006.00959.x. [DOI] [PubMed] [Google Scholar]
  • 2.Hanssen-Bauer K, Heyerdahl S, Hatling T, Jensen G, Olstad PM, Stangeland T, Tinderholt T. Admissions to acute adolescent psychiatric units: a prospective study of clinical severity and outcome. Int J Ment Health Syst. 2011 Jan 06;5(1):1. doi: 10.1186/1752-4458-5-1. https://ijmhs.biomedcentral.com/articles/10.1186/1752-4458-5-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Bettmann JE, Jasperson RA. Adolescents in Residential and Inpatient Treatment: A Review of the Outcome Literature. Child Youth Care Forum. 2009 Apr 28;38(4):161–183. doi: 10.1007/s10566-009-9073-y. [DOI] [Google Scholar]
  • 4.Case BG, Olfson M, Marcus SC, Siegel C. Trends in the inpatient mental health treatment of children and adolescents in US community hospitals between 1990 and 2000. Arch Gen Psychiatry. 2007 Jan;64(1):89–96. doi: 10.1001/archpsyc.64.1.89. [DOI] [PubMed] [Google Scholar]
  • 5.Green J. Provision of Intensive Treatment: In-patient Units, Day Units and Intensive Outreach. In: Rutter M, editor. Rutter's Child and Adolescent Psychiatry. Oxford, UK: Blackwell Publishing Ltd; 2008. pp. 1126–1142. [Google Scholar]
  • 6.Andersson C, Danielsson S, Silfverberg-Dymling G, Löndahl G, Johansson BA. Evaluation of Interactive Voice Response (IVR) and postal survey in follow-up of children and adolescents discharged from psychiatric outpatient treatment: a randomized controlled trial. Springerplus. 2014;3:77. doi: 10.1186/2193-1801-3-77. http://europepmc.org/abstract/MED/24567883. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Weaver ER, Horyniak DR, Jenkinson R, Dietze P, Lim MS. "Let's get Wasted!" and Other Apps: Characteristics, Acceptability, and Use of Alcohol-Related Smartphone Applications. JMIR Mhealth Uhealth. 2013 Jun 25;1(1):e9. doi: 10.2196/mhealth.2709. http://mhealth.jmir.org/2013/1/e9/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Cohn AM, Hunter-Reel D, Hagman BT, Mitchell J. Promoting behavior change from alcohol use through mobile technology: the future of ecological momentary assessment. Alcohol Clin Exp Res. 2011 Dec;35(12):2209–15. doi: 10.1111/j.1530-0277.2011.01571.x. http://europepmc.org/abstract/MED/21689119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Thienemann M. Introducing a structured interview into a clinical setting. J Am Acad Child Adolesc Psychiatry. 2004 Aug;43(8):1057–60. doi: 10.1097/01.chi.0000129221.12305.13. [DOI] [PubMed] [Google Scholar]
  • 10.Miller PR. Inpatient diagnostic assessments: 2. Interrater reliability and outcomes of structured vs. unstructured interviews. Psychiatry Research. 2001 Dec;105(3):265–271. doi: 10.1016/S0165-1781(01)00318-3. [DOI] [PubMed] [Google Scholar]
  • 11.Rettew DC, Lynch AD, Achenbach TM, Dumenci L, Ivanova MY. Meta-analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews. Int J Methods Psychiatr Res. 2009 Sep;18(3):169–84. doi: 10.1002/mpr.289. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Gregory JM, Sukhera J, Taylor-Gates M. Integrating Smartphone Technology at the Time of Discharge from a Child and Adolescent Inpatient Psychiatry Unit. J Can Acad Child Adolesc Psychiatry. 2017;26(1):45–50. http://europepmc.org/abstract/MED/28331503. [PMC free article] [PubMed] [Google Scholar]
  • 13.Dahne Jennifer, Lejuez Carl W. Smartphone and Mobile Application Utilization Prior to and Following Treatment Among Individuals Enrolled in Residential Substance Use Treatment. J Subst Abuse Treat. 2015 Nov;58:95–9. doi: 10.1016/j.jsat.2015.06.017. http://europepmc.org/abstract/MED/26231698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Liss DT, Serrano E, Wakeman J, Nowicki C, Buchanan DR, Cesan A, Brown T. "The Doctor Needs to Know": Acceptability of Smartphone Location Tracking for Care Coordination. JMIR Mhealth Uhealth. 2018 May 04;6(5):e112. doi: 10.2196/mhealth.9726. http://mhealth.jmir.org/2018/5/e112/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Torous J, Chan SR, Yee-Marie TS, Behrens J, Mathew I, Conrad EJ, Hinton L, Yellowlees P, Keshavan M. Patient Smartphone Ownership and Interest in Mobile Apps to Monitor Symptoms of Mental Health Conditions: A Survey in Four Geographically Distinct Psychiatric Clinics. JMIR Ment Health. 2014 Dec;1(1):e5. doi: 10.2196/mental.4004. http://mental.jmir.org/2014/1/e5/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Benarous X, Edel Y, Consoli A, Brunelle J, Etter J, Cohen D, Khazaal Y. Ecological Momentary Assessment and Smartphone Application Intervention in Adolescents with Substance Use and Comorbid Severe Psychiatric Disorders: Study Protocol. Front Psychiatry. 2016 Sep;7:157. doi: 10.3389/fpsyt.2016.00157. doi: 10.3389/fpsyt.2016.00157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Johansson BA, Remvall S, Malgerud R, Lindgren A, Andersson C. Interactive voice response - an automated follow-up technique for adolescents discharged from acute psychiatric inpatient care: a randomised controlled trial. Springerplus. 2013 Dec;2(1):146. doi: 10.1186/2193-1801-2-146. http://europepmc.org/abstract/MED/23641319. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Thapar A, Collishaw S, Pine DS, Thapar AK. Depression in adolescence. The Lancet. 2012 Mar;379(9820):1056–1067. doi: 10.1016/S0140-6736(11)60871-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Marangoni C, De Chiara CL, Faedda GL. Bipolar disorder and ADHD: comorbidity and diagnostic distinctions. Curr Psychiatry Rep. 2015 Aug;17(8):604. doi: 10.1007/s11920-015-0604-y. [DOI] [PubMed] [Google Scholar]
  • 20.Mason MJ, Aplasca A, Morales-Theodore R, Zaharakis N, Linker J. Psychiatric Comorbidity and Complications. Child Adolesc Psychiatr Clin N Am. 2016 Jul;25(3):521–32. doi: 10.1016/j.chc.2016.02.007. [DOI] [PubMed] [Google Scholar]
  • 21.Svanborg P, Åsberg M. A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr Scand. 1994 Jan;89(1):21–28. doi: 10.1111/j.1600-0447.1994.tb01480.x. [DOI] [PubMed] [Google Scholar]
  • 22.Sheehan DV, Sheehan KH, Shytle RD, Janavs J, Bannon Y, Rogers JE, Milo KM, Stock SL, Wilkinson B. Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID) J Clin Psychiatry. 2010 Mar;71(3):313–26. doi: 10.4088/JCP.09m05305whi. [DOI] [PubMed] [Google Scholar]
  • 23.Bush K. The AUDIT Alcohol Consumption Questions (AUDIT-C): An Effective Brief Screening Test for Problem Drinking. Arch Intern Med. 1998 Sep 14;158(16):1789. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
  • 24.Berman AH, Bergman H, Palmstierna T, Schlyter F. Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample. Eur Addict Res. 2005 Dec;11(1):22–31. doi: 10.1159/000081413. [DOI] [PubMed] [Google Scholar]
  • 25.Strand BH, Dalgard OS, Tambs K, Rognerud M. Measuring the mental health status of the Norwegian population: a comparison of the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36) Nord J Psychiatry. 2003 Jul;57(2):113–8. doi: 10.1080/08039480310000932. [DOI] [PubMed] [Google Scholar]
  • 26.Sheehan DV. The Anxiety Disease. New York: Charles Scribner and Sons; 1983. p. 151. [Google Scholar]
  • 27.Ehlers S, Gillberg C, Wing L. A screening questionnaire for Asperger syndrome and other high-functioning autism spectrum disorders in school age children. Journal of Autism and Developmental Disorders. 1999;29(2):129–141. doi: 10.1023/A:1023040610384. [DOI] [PubMed] [Google Scholar]
  • 28.Bussing R, Fernandez M, Harwood M, Wei H, Garvan CW, Eyberg SM, Swanson JM. Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms: psychometric properties and normative ratings from a school district sample. Assessment. 2008 Sep;15(3):317–28. doi: 10.1177/1073191107313888. http://europepmc.org/abstract/MED/18310593. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Lundh L, Karim J, Quilisch E. Deliberate self-harm in 15-year-old adolescents: a pilot study with a modified version of the Deliberate Self-Harm Inventory. Scand J Psychol. 2007 Feb;48(1):33–41. doi: 10.1111/j.1467-9450.2007.00567.x. [DOI] [PubMed] [Google Scholar]
  • 30.Clinton D, Björck C, Sohlberg S, Norring C. Patient satisfaction with treatment in eating disorders: cause for complacency or concern? Eur Eat Disorders Rev. 2004 Jun 24;12(4):240–246. doi: 10.1002/erv.582. [DOI] [Google Scholar]
  • 31.Sarris J, O'Neil A, Coulson CE, Schweitzer I, Berk M. Lifestyle medicine for depression. BMC Psychiatry. 2014 Apr 10;14:107. doi: 10.1186/1471-244X-14-107. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-14-107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Altman D.G. Practical Statistics for Medical Research. Boca Raton: CRC Press; 1990. [Google Scholar]
  • 33.Drucker PF. Practice of Management. Oxford: Butterworth-Heinemann (Elsevier); 2007. [Google Scholar]

Articles from JMIR Research Protocols are provided here courtesy of JMIR Publications Inc.

RESOURCES