Abstract
Background
The rapid extent of digital innovation for the collection of data has transformed the way in which health professionals collect, share, and analyze health information for better clinical decision-making and health care. In the last decade, there has been an increased interest in telemedicine by mental health agencies; the gap between the need for care and both diagnosis and treatment is wide, and digital technology could play an important role in filling this gap. However, there are limited data on the effectiveness of the clinical process and cost-effectiveness of most telemedicine applications.
Objective
This study examined the implementation of the first Italian online, web-based, comprehensive screening tool and described the screening and diagnostic process through the interactive web platform in a child psychopathology clinic. This is a feasibility study that aims to present the design and implementation of the best practices to improve patient experiences and clinical outcomes. Moreover, the paper evaluates the platform with qualitative and quantitative measures.
Methods
We planned, designed, and implemented a web-based system to collect, store, and manage clinical data. The platform was developed by a multidisciplinary team composed of researchers, clinicians, and informatics professionals through different steps. First, we defined the clinical information to be collected. A number of measures were chosen, tapping several clinical risk areas such as neurodevelopmental disorders and emotional and behavioral problems. The web application architecture and process were then designed. The three phases of process design are described in detail: design of the input interface, processing design, and design of the output interface. Finally, the system has been implemented and evaluated. Based on indicators recommended by the National Quality Forum and the Italian National Guidelines, we evaluated the quality of the system and used quantitative measures that were replicable and comparable over time.
Results
We present the implemented architecture and features of Medea Information and Clinical Assessment On-Line (MedicalBIT), and we provide performance measures for the data collected between October 2018 and June 2021. The measured concepts pertain to four domains: access to care, financial impact/cost, experience, and effectiveness.
Conclusions
In this study, we present the successful implementation of an innovative digital tool. The findings of this study show that the implemented web-based platform appears to be an efficient, cost-effective, and feasible way to improve digital care in the field of child psychiatry.
Keywords: digital health, big data, developmental psychopathology, neurodevelopmental disorders, digital data, digital innovation, mental health, screening tool, children, psychopathology, web platform, digital intervention, clinical outcome
Introduction
Digital data collection is an emerging trend in various fields, including the medical and psychological ones. Digital innovation is changing the way health information is collected, shared, and analyzed for better clinical decision-making and health care. Digital innovation is also rapidly expanding in the medical field, significantly improving the quality of health care, reducing health care costs, and enhancing research processes [1]. The rapid evolution of technology has recently promoted web platforms for big data collection in the health care field, and related publications are exponentially increasing [2].
“Big data” refers to a voluminous collection of information taking place quickly without affecting quality, and there are several well-known definitions describing this [3-5]. Pastorino and colleagues [2] stress the importance of defining data as both smart and big because big data presents a substantial potential when it is meaningful. This implies using data for improving health conditions by searching for increasingly clearer and more accurate links between causes, diseases, therapies, and outcomes. In the Study on Big Data in Public Health, Telemedicine, and Healthcare [6], the European Commission identified four macroareas in health care for big data use: (1) early signs for detection, diagnosis, and intervention; (2) identification of risk factors for diseases to improve prevention; (3) enhancing pharmacovigilance and patient safety by communication of real-time information; and (4) improvement in outcome prediction.
All this is possible since a large amount of data is an invaluable resource for epidemiological studies, analyses of general population needs, treatment evaluation, and experimental designs on the target population. If research develops in this direction, it will enable precision medicine that will contribute to optimization of resources: the right care for the right patient at the right time. Therefore, smart use of big data can provide a possible answer to the need for care: socioeconomic and clinical sustainability. In line with this, health technology assessments aim to inform on safe, effective, patient-centered policy-making as well as determine the greatest value for it.
Against this background, telehealth [7] should be seen not only as a complement or alternative to traditional clinical practice but also as an ideal channel to collect and analyze big data. Besides rapid technological progress, COVID-19 spurred an exponential growth in telehealth, and there will be no return to the prepandemic situation according to the American Telemedicine Association [8]. There are several fields of telehealth, just like there are medical specialties. For example, investments in information technology are promoting the development of telepsychiatry. According to Allen [9] and Torous and colleagues [10] one of the most important contributions of artificial intelligence for psychiatry concerns apps, as shown by their innovative project Learn, Assess, Manage, and Prevent (MindLAMP). MindLAMP is a digital platform used both for clinic and research purposes: it receives patient input and aggregates data, it guides reflection, and it helps orient toward treatments. Through the app, patients can input data in an active way (survey, symptom registration, etc) or in a passive way (information collected in the background even if the user is not using the app), and they can receive some advice and mindfulness resources [11]. It is evident that, in the last decade, there has been an increased interest in telemedicine by mental health agencies; the gap between the need for care and possible answers both for diagnosis and treatment is wide, and digital technology could play a fundamental role [12]. In the child and adolescence mental health services, the parents and caregivers of young patients must be involved in the clinical diagnostic process, both as powerful sources of information to be used by clinicians and to obtain a clear understanding about their child’s difficulties. This engagement should be conducted in a comfortable environment for the participants.
Telemedicine seems to be efficient both for the assessment and treatment. For example, among services for the diagnosis of autism spectrum disorder (ASD), the use of telemedicine has shown some promising results in terms of observation [13-15]. As for remote treatment, several recent studies on telemedicine and developmental disorders such as autistic spectrum disorder, specific learning disorder, specific language impairment, dyspraxia, and acquired brain injuries show encouraging results [16-20].
A review of the literature by the National Quality Forum—a no-profit organization for health care development—found a positive effect of telehealth on the quality of processes, outcomes, and costs. It also identified the medical areas where telehealth spread more extensively, with a correspondingly greater increase in scientific publications, namely, dermatology, mental health, rehabilitation, medical management, and chronic diseases. However, despite this topic being current and debated, telemedicine is relatively recent, and there is a lack of studies assessing its quality [21].
In our study, we examined the implementation of the first Italian online, web-based, comprehensive screening tool: Medea Information and Clinical Assessment On-Line (MedicalBIT) [22]. We carried out a feasibility study aimed to present the design and the implementation of the best practices to improve users’ experiences and discuss the evaluation of the platform with qualitative and quantitative measures. Additionally, we intend to describe a web-based screening and diagnostic process in a child psychopathology clinic.
Methods
Overview
This section describes the key elements of the platform implementation.
We planned, designed, and implemented a web-based system to collect, store, and manage clinical data through different steps. The whole eHealth system is composed of users (patients and clinicians), a service provider (the Association La Nostra Famiglia-IRCCS Eugenio Medea, a no-profit organization providing care and rehabilitation to children with disabilities), and a service developer (SE-GE Consulting Company).
The main purpose of the MedicalBIT platform is to support diagnostic flow by systematic data collection.
Patients can answer questionnaires comfortably from home through a user-friendly and easily accessible interface, saving time traveling to and staying in a clinic. Clinicians have access to a real-time description of individual patients’ symptoms and a graphical output of possible alerts through clinically based algorithms. Clinicians then assign patients to different diagnostic paths for principal neurodevelopmental disorders (Attention-Deficit/Hyperactivity Disorder, Autism Spectrum Disorder, Specific Learning Disorders, Language Disorders) and behavioral and emotional disorders based on this output. Taking into account the international gold standard, a clinical assessment is performed for each path. The final diagnostic output may differ from the initial path assignment based on the assessment results.
Finally, the data collected can be easily exported in Excel (Microsoft Corporation) format for research analyses: researchers can use collected data for further analysis to develop predictive models.
The platform was developed by a multidisciplinary team composed of researchers, clinicians, and informatics professionals. Figure 1 provides an overview of the workflow, and Figure 2 displays some picture frames of the web platform home page. A description of each step is reported below:
Figure 1.
The diagnostic flow on Medea Information and Clinical Assessment On-Line (MedicalBIT).
Figure 2.
The main menu and some pictures of the Medea Information and Clinical Assessment On-Line (MedicalBIT) home page.
Defining clinical information to be collected
Web application architecture and process design
System implementation and evaluation
Defining Clinical Information to Be Collected
A number of measures were chosen by tapping several clinical risk areas, such as neurodevelopmental disorders and emotional and behavioral problems. Measures were selected for their feasibility in routine clinical practice (ie, brevity, free availability, validation in children and young people, and translation) and psychometric performance (ie, validity, reliability, and sensitivity to change).
The following questionnaires were selected:
Risk factors: A questionnaire to explore biological and environmental risk factors such as family composition; presence/absence of psychiatric diseases in parents or close relatives; prenatal, perinatal, and postnatal factors; and developmental milestones.
ASD: The Modified Checklist for Autism in Toddlers [23] and the Autism Spectrum Quotient: Children’s Version [24] were used for the detection of ASD. The first is one of the most widely used ASD toddler screening instruments; it is easily accessible and low-cost. The second is a brief, parent-reported, 50-item questionnaire to quantify autistic traits in children aged 4-11 years.
Emotional and behavioral problems: The Strengths and Difficulties Questionnaire [25] is a brief instrument widely used to assess main areas of developmental psychopathology and personal strengths. It consists of 25 items and is available in three forms depending on responders: parents, teachers, and adolescents (self-report).
Other neurodevelopmental disorders: Ad hoc screening tools for specific language impairment and specific learning disorder were implemented by our institute’s research group working on specific learning, language, and communication disorders.
Overall, these tools cover ages ranging from 18 months to 15 years.
After this initial screening step, other standardized assessment tools can be administered too, such as the Child Behavior Checklist (CBCL) [26,27] and the Development and Well-Being Assessment (DAWBA) [28]. Through an interoperability system, links and credentials can be sent to caregivers to complete questionnaires (CBCL thorough ASEBA-Web) and the interview (DAWBA thorough Dawba.net), and results can then be imported by uploading appropriately encrypted files.
Web Application Architecture and Process Design
Figure 3 shows a high-level diagram of the platform development process. The three phases of process design are described in detail below:
Figure 3.
The developed web app architecture design.
Design of the input interface
Processing design
Design of the output interface
Design of the Input Interface
An easy-to-use interface was developed to support the interaction between front-end users and the web-based application; the web interface is simple to learn and easy to use, and it displays information in a consistent and progressive manner and maximizes functionalities. To be displayed properly on most mobile devices, the platform implements a responsive web design.
The system also relies on the active engagement of caregivers who send their data to the informatics system using a PC and mobile device. Patients scheduled for an appointment at the Child Psychopathology Unit (IRCCS Medea) get an email with a token for the first web app. They are then guided through a series of steps: completion of a brief form with primary clinical information, verifying of this information, and filling out of the registration form. They then receive a set of questionnaires according to their age and main characteristics, along with the link to the MedicalBIT assessment for completion.
Users are not asked to complete all forms in one session, they can resume from where they last left. After completing all the selected questionnaires, patients are scheduled for a doctor’s visit.
To support the user’s interaction with the platform, a user-friendly back-end interface was implemented. Through a control panel, staff can handle main functions such as administration rights, patients’ lists, and support for data entry.
Processing Design: Technical Implementation of Data Storage and Automated Scoring Algorithm
At the end of each survey, the system immediately processes the caregiver’s answers using artificial intelligence algorithms. Starting from scores provided for every possible answer and relying on appropriate threshold values, alerts, and key performance indicators are generated, which are useful to health professionals not only to monitor single patients but also for statistics and research purposes.
The whole process is General Data Protection Regulation compliant. The web portal is protected with a digital certificate issued by a certification authority, implements HTTPS (the secure browsing protocol for the World Wide Web), and is hosted on a virtual server in the GARR cloud environment. GARR is the Italian ultrabroadband network for education and research.
The infrastructure provides high reliability, service continuity, and data protection. Backup takes place daily, and we rely on disaster recovery as a service in an alternative environment to ensure availability in case of downtime. We used open source solutions for easier integration with company applications and to avoid vendor lock-in. Our platform is compliant with Italian guidelines [29]: “To ensure an effective support to all company processes, it is necessary to guarantee data univocity and integrity, real-time updating, historicization and audit trail, ergonomics, standardization, integration, stability, availability, security, privacy, innovation, evolvability.”
All data are unique; nothing is duplicated. Information can be accessed only by authorized users and is protected from unauthorized changes; furthermore, confidential data is safeguarded.
There are two different access levels:
-
Front-end user
Caregivers filling out the forms can only view and download the information concerning their child. They have no access to scores generated by the automatic scoring algorithms. Multiple patients can be associated to each caregiver (eg, in the case of siblings).
-
Back-end users
Health care professionals viewing the answers and any alerts enter the diagnoses at the end of the diagnostic process.
Back-office users with full operation rights
The system administrator has access to additional features such as dashboard and data export (see next paragraph).
The integrity of data and documents is guaranteed by the use of appropriate logs of activity and changes. These logs are also useful from a legal point of view. All information relating to a particular patient can also be deleted by users with administrator rights. Documents saved on the server are protected by encryption; to comply with the privacy legislation, the information stored in the database is pseudo-anonymized.
Design of the Output Interface
A real-time graphing, panel, and table interface is available only to back-end users and contains all processed data in real time. This data is also displayed in interactive dashboards available at different access levels. Two different types of reporting are created to display the collected data (Figure 4):
Figure 4.
Different output levels.
Clinical output: an individual-level output, reporting patient outcome scores
Dashboard: group-level data, reporting summarized data in a dashboard
All data stored in the database system can be exported in Excel format for further processing.
System Implementation and Evaluation
This digital system has been on since October 2018 and has been used by 1301 patients (front-end users) and back-end users, including 17 clinicians, 2 back-office operators, and administration operators.
Back-end users are employed by the provider service: back-office operators are office secretaries and administration operators are psychologists and researchers who designed the service in collaboration with the data scientist and consulting company SEGE srl.
Based on indicators recommended by the National Quality Forum [21] and the Italian National Guidelines [29], we evaluated the quality of the system and used quantitative measures that were replicable and comparable over time. As shown in Textbox 1, the selected measures pertain to four areas: access to care, financial impact/cost, experience, and effectiveness.
Telehealth measurement framework domains and subdomains.
Access to care
Access for parent/caregiver
Access to information
Effectiveness
System effectiveness
Operational effectiveness
Technical effectiveness
Costs
The financial impact to family/caregiver
The financial impact to care team
Experience
Patient, family, or caregiver experience
Results
According to the Italian National Guidelines [29] and the National Quality Forum [21], we provide performance measures available for data collected between October 2018 and June 2021 (Table 1 and Figure 5). This process is a work in progress, and in the future, we aim to capture more indicators about different domains and subdomains.
Table 1.
Platform performance measures.
| Domains and subdomains | Qualitative measures | Quantitative measures | |||
|
Access to care
| |||||
|
|
|
|
|
||
| Effectiveness | |||||
|
|
|
|
|
||
|
|
|
|
|
||
|
|
|
|
|
||
| Costs | |||||
|
|
|
|
|
||
| Experience | |||||
|
|
|
|
|
||
aN/A: not applicable.
Figure 5.
(a) Number of accesses of new patients for each semester in the last 3 years: multiple accesses by 1 patient are counted once. (b) Average and SD per month of accesses by new patients in the last 3 years. (c) Dynamic dimension value = (August 2020 to July 2021) / (August 2019 to July 2020) = 560 / 410 = 1.36. This enabled us to compare differences in accesses in the last 2 years. (d) The graph shows the trend in Medea Information and Clinical Assessment On-Line (MedicalBIT) during the different phases of the pandemic. (e) Data show that the average time between the completion of questionnaires and receiving a diagnosis has clearly decreased over the 3-year period: 2019 (mean 150, SD 179.6), 2020 (mean 82, SD 77.3), and 2021 (mean 65, SD 34.9).
Discussion
Principal Results
In this study, we present the design and implementation of the MedicalBIT web platform, which allows patients to complete questionnaires comfortably from home through a user-friendly and easily accessible interface.
This work is based on the European Commission’s recommendations in the Study on Big Data in Public Health, Telemedicine, and Healthcare [6] that identified four major areas in health care where big data may be used: (1) early signs for detection, diagnosis, and intervention; (2) identification of risk factors for disease to improve prevention care; (3) enhancing pharmacovigilance and patient safety by communication of real-time information; and (4) improvement in outcomes prediction.
The MedicalBIT platform, as described in this paper, allows for the timely collection of clinical data to support the diagnostic process and subsequent steps (1 and 3), and it enables the development of predictive models to improve preventive care and outcome prediction (2 and 4). A core component of the proposed model is caregivers’ perspective of their child’s mental health, laying the foundation for evidence-based practice and person-centered care. The MedicalBIT web platform was introduced into the clinical workflow, making patients’ and caregivers’ care experiences more comfortable. As mentioned, health information technology can save time and reduce costs, thereby improving the health care experience for both patients and clinicians. For this purpose, we also collected customer satisfaction feedback.
Furthermore, performance measures enable us to describe the platform with an objective measurement that allows for comparisons both within the same platform over time and between different web platforms used for similar purposes. According to reports so far, available quantitative data shows general upward trends for platform use from 2018 to 2021 despite the apparent drop during the first months of the COVID-19 pandemic, which required a complete reorganization of care provision.
This trend is in line with research confirming the value of telepsychiatry [30,31] not only during the pandemic period but also before and, most importantly, after the emergency. As described in the Introduction, use of telemedicine in the psychiatry and developmental psychiatry fields can offer several benefits such as improved efficiency and effectiveness of mental health services: it can reach more people with fewer resources [32].
We hope that these changes will lead to filling the care gap in the field of child psychiatry, which has been appropriately defined as “one of the most difficult and crucial challenges of the next decade” [12].
Our project fits into this still uncertain and experimental scenario, with its pros and cons. Telemedicine in the child psychiatry clinical routine can help both patients and health care providers save time and provide them easy access to the care system. Furthermore, because of its fast and advantageous features, it facilitates the hospital and clinic workflow, as processing and automatic output of clinical data enables the collection of large amounts of data for research purposes. Nevertheless, there are some limitations to consider such as the impact of physical distance on human relationships and ethical and coroner issues [29].
Further studies are clearly necessary to establish evidence-based telepsychiatry-specific standards of care, following guidelines proposed by the American Telemedicine Association [33] and the American Academy of Child and Adolescent Psychiatry [34].
Limitations and Further Development
Based on the current limitations, we are working on a customer satisfaction questionnaire for both families and clinicians to collect feedback to make users feel more engaged in this process and help us as back-office operators to improve services. Moreover, we intend to improve our platform performance measurement tools to obtain more quantitative indicators about the four domains presented in Table 1.
According to Waller and Stotler [35], one of the next steps should be the implementation of a measurement and methodology evaluation of the impact of telehealth on clinical outcomes (in our case, on the diagnostic outcome as an indicator of clinical effectiveness). Our intent is to also implement a platform-based service that could follow and provide assistance to patients after diagnosis (information, psychoeducational materials, rehabilitation tools).
Conclusions
This study describes the successful implementation of an innovative digital tool. According to our results, the web-based platform appears to be a feasible, efficient, and cost-effective method for enhancing digital care in the field of child psychiatry. It also contributes to the collection of big and smart data, in line with European guidelines. This work shows how large data sets may enhance the accuracy and timing of diagnosis, and contribute to more effective interventions based on predictive models.
Acknowledgments
The authors wish to thank clinicians and back-office staff who took part in the web platform co-design and all clinicians using Medea Information and Clinical Assessment On-Line (MedicalBIT). This work has been possible due to the patients’ families, clinicians, researchers, and back-office staff who trust this innovative system.
Abbreviations
- ASD
autism spectrum disorder
- CBCL
Child Behavior Checklist
- DAWBA
Development and Well-Being Assessment
- MedicalBIT
Medea Information and Clinical Assessment On-Line
- MindLAMP
Learn, Assess, Manage, and Prevent
Footnotes
Authors' Contributions: PC, SBC, and MM conceptualized the study. SBC gathered the resources and curated the data. SBC, SP, and PC prepared the original draft. PC, SBC, SP, and SC reviewed and edited the manuscript. MM supervised the study.
Conflicts of Interest: None declared.
References
- 1.Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. World Health Organization. 2016. [2022-06-27]. https://apps.who.int/iris/bitstream/handle/10665/252183/?sequence=1 .
- 2.Pastorino R, De Vito C, Migliara G, Glocker K, Binenbaum I, Ricciardi W, Boccia S. Benefits and challenges of Big Data in healthcare: an overview of the European initiatives. Eur J Public Health. 2019 Oct 01;29(Supplement_3):23–27. doi: 10.1093/eurpub/ckz168. http://europepmc.org/abstract/MED/31738444 .5628051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Manyika J, Chui M, Brown B, Bughin J, Dobbs R, Roxburgh C, Hung Byers A. Big Data: the Next Frontier for Innovation, Competition, and Productivity. Washington, DC: McKinsey Global Institute; 2011. [Google Scholar]
- 4.Beyer M, Laney D. The importance of 'Big Data': a definition. Gartner. 2012. [2022-06-27]. https://www.gartner.com/en/documents/2057415#:~:text=%22Big%20data%22%20warrants%20innovative%20processing,to%20business%20goals%20and%20objectives .
- 5.McAfee A, Brynjolfsson E. Big data: the management revolution. Harv Bus Rev. 2012 Oct;90(10):60–6, 68, 128. [PubMed] [Google Scholar]
- 6.Study on Big Data in Public Health, Telemedine and Healthcare: final report. European Commission. 2016. [2021-09-01]. https://ec.europa.eu/health/sites/health/files/ehealth/docs/bigdata_report_en.pdf%0A%0A .
- 7.Telehealth: defining 21st century care. American Telemedicine Association. 2020. [2022-06-27]. https://marketing.americantelemed .org/hubfs/Files/Resources/ATA_Telehealth_Taxonomy_9-11-20.pdf .
- 8.The adoption of telehealth. American Telemedicine Association. 2021. [2022-06-27]. https://www.americantelemed.org/wp-content/uploads/2021/05/Adoption-of-Telehealth.pdf .
- 9.Allen S. Artificial intelligence and the future of psychiatry. IEEE Pulse. 2020;11(3):2–6. doi: 10.1109/MPULS.2020.2993657. [DOI] [PubMed] [Google Scholar]
- 10.Torous J, Jän Myrick K, Rauseo-Ricupero N, Firth J. Digital mental health and COVID-19: using technology today to accelerate the curve on access and quality tomorrow. JMIR Ment Health. 2020 Mar 26;7(3):e18848. doi: 10.2196/18848. https://mental.jmir.org/2020/3/e18848/ v7i3e18848 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Torous J, Wisniewski H, Bird B, Carpenter E, David G, Elejalde E, Fulford D, Guimond S, Hays R, Henson P, Hoffman L, Lim C, Menon M, Noel V, Pearson J, Peterson R, Susheela A, Troy H, Vaidyam A, Weizenbaum E, Naslund JA, Keshavan M. Creating a digital health smartphone app and digital phenotyping platform for mental health and diverse healthcare needs: an interdisciplinary and collaborative approach. J Technol Behav Sci. 2019 Apr 27;4(2):73–85. doi: 10.1007/s41347-019-00095-w. [DOI] [Google Scholar]
- 12.Fatori D, Polanczyk GV. The role of digital technology in bridging the child mental health care gap. Eur Child Adolesc Psychiatry. 2019 Apr;28(4):425–426. doi: 10.1007/s00787-019-01316-6.10.1007/s00787-019-01316-6 [DOI] [PubMed] [Google Scholar]
- 13.Corona L, Hine J, Nicholson A, Stone C, Swanson A, Wade J, Wagner L, Weitlauf A, Warren Z. What is the TELE-ASD-PEDS? Vanderbilt Kennedy Center. 2020. [2022-03-25]. https://vkc.vumc.org/vkc/triad/tele-asd-peds .
- 14.Corona L, Wagner L, Wade J, Weitlauf A, Hine J, Nicholson A, Stone C, Vehorn A, Warren Z. Toward novel tools for autism identification: fusing computational and clinical expertise. J Autism Dev Disord. 2021 Nov;51(11):4003–4012. doi: 10.1007/s10803-020-04857-x. http://europepmc.org/abstract/MED/33417138 .10.1007/s10803-020-04857-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wagner L, Weitlauf AS, Hine J, Corona LL, Berman AF, Nicholson A, Allen W, Black M, Warren Z. Transitioning to telemedicine during COVID-19: impact on perceptions and use of telemedicine procedures for the diagnosis of autism in toddlers. J Autism Dev Disord. 2022 May;52(5):2247–2257. doi: 10.1007/s10803-021-05112-7. http://europepmc.org/abstract/MED/34085153 .10.1007/s10803-021-05112-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Sutherland R, Trembath D, Roberts J. Telehealth and autism: a systematic search and review of the literature. Int J Speech Lang Pathol. 2018 Jun;20(3):324–336. doi: 10.1080/17549507.2018.1465123. [DOI] [PubMed] [Google Scholar]
- 17.Hodge MA, Sutherland R, Jeng K, Bale G, Batta P, Cambridge A, Detheridge J, Drevensek S, Edwards L, Everett M, Ganesalingam C, Geier P, Kass C, Mathieson S, McCabe M, Micallef K, Molomby K, Pfeiffer S, Pope S, Tait F, Williamsz M, Young-Dwarte L, Silove N. Literacy assessment via telepractice is comparable to face-to-face assessment in children with reading difficulties living in rural Australia. Telemed J E Health. 2019 Apr;25(4):279–287. doi: 10.1089/tmj.2018.0049. [DOI] [PubMed] [Google Scholar]
- 18.Wales D, Skinner L, Hayman M. The efficacy of telehealth-delivered speech and language intervention for primary school-age children: a systematic review. Int J Telerehabil. 2017;9(1):55–70. doi: 10.5195/ijt.2017.6219. http://europepmc.org/abstract/MED/28814995 .ijt-09-55 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Miyahara M, Butson R, Cutfield R, Clarkson JE. A pilot study of family-focused tele-intervention for children with developmental coordination disorder: development and lessons learned. Telemed J E Health. 2009 Sep;15(7):707–12. doi: 10.1089/tmj.2009.0022. [DOI] [PubMed] [Google Scholar]
- 20.Corti C, Oldrati V, Oprandi M, Ferrari E, Poggi G, Borgatti R, Urgesi C, Bardoni A. Remote technology-based training programs for children with acquired brain injury: a systematic review and a meta-analytic exploration. Behav Neurol. 2019;2019:1346987. doi: 10.1155/2019/1346987. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Creating a framework to support measure development for telehealth. National Quality Forum. 2017. [2021-09-01]. https://www.qualityforum.org/Publications/2017/08/Creating_a_Framework_to_Support_Measure_Development_for_Telehealth.aspx .
- 22.MedicalBIT. [2022-06-27]. https://www.medicalbit.com/it/home/
- 23.Robins DL, Casagrande K, Barton M, Chen CA, Dumont-Mathieu T, Fein D. Validation of the modified checklist for Autism in toddlers, revised with follow-up (M-CHAT-R/F) Pediatrics. 2014 Jan;133(1):37–45. doi: 10.1542/peds.2013-1813. http://europepmc.org/abstract/MED/24366990 .peds.2013-1813 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Auyeung B, Baron-Cohen S, Wheelwright S, Allison C. The Autism Spectrum Quotient: Children's Version (AQ-Child) J Autism Dev Disord. 2008 Aug;38(7):1230–40. doi: 10.1007/s10803-007-0504-z. [DOI] [PubMed] [Google Scholar]
- 25.Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry. 1997 Jul;38(5):581–6. doi: 10.1111/j.1469-7610.1997.tb01545.x. [DOI] [PubMed] [Google Scholar]
- 26.Achenbach T. Child Behavior Checklist (CBCL), Achenbach System of Empirically Based Assessment. Burlington, VT: ASEBA; 2001. [Google Scholar]
- 27.Achenbach T, Rescorla L. Manual for the ASEBA School-Age Forms and Profiles. Burlington, VT: University of Vermont; 2001. [Google Scholar]
- 28.Goodman R, Ford T, Richards H, Gatward R, Meltzer H. The Development and Well-Being Assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry. 2000 Jul;41(5):645–55. [PubMed] [Google Scholar]
- 29.Linee di indirizzo nazionali sulla telemedicina. Ministero della Salute. 2014. [2022-06-27]. https://www.salute.gov.it/portale/ehealth/dettaglioContenutiEHealth.jsp?lingua=italiano&id=5525&area=eHealth&menu=telemedicina .
- 30.Chang BP, Kessler RC, Pincus HA, Nock MK. Digital approaches for mental health in the age of covid-19. BMJ. 2020 Jun 29;369:m2541. doi: 10.1136/bmj.m2541. [DOI] [PubMed] [Google Scholar]
- 31.Garofalo M, Vaithilingam S, Ferrando S. Telemedicine for psychiatry and mental health. In: Latifi R, Doarn CR, Merrell RC, editors. Telemedicine, Telehealth and Telepresence: Principles, Strategies, Applications, and New Directions. Cham: Springer; 2021. pp. 365–378. [Google Scholar]
- 32.Moock J. Support from the internet for individuals with mental disorders: advantages and disadvantages of e-mental health service delivery. Front Public Health. 2014;2:65. doi: 10.3389/fpubh.2014.00065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Yellowlees P, Shore J, Roberts L, American Telemedicine Association Practice guidelines for videoconferencing-based telemental health - October 2009. Telemed J E Health. 2010 Dec;16(10):1074–89. doi: 10.1089/tmj.2010.0148. [DOI] [PubMed] [Google Scholar]
- 34.American Academy of Child and Adolescent Psychiatry (AACAP) Committee on Telepsychiatry and AACAP Committee on Quality Issues Clinical update: telepsychiatry with children and adolescents. J Am Acad Child Adolesc Psychiatry. 2017 Oct;56(10):875–893. doi: 10.1016/j.jaac.2017.07.008.S0890-8567(17)30333-7 [DOI] [PubMed] [Google Scholar]
- 35.Waller M, Stotler C. Telemedicine: a primer. Curr Allergy Asthma Rep. 2018 Aug 25;18(10):54. doi: 10.1007/s11882-018-0808-4.10.1007/s11882-018-0808-4 [DOI] [PubMed] [Google Scholar]