Abstract
Digital therapeutics is an evidence-based intervention using high-quality software, with the sole purpose of treatment. As many healthcare systems are encountering high demands of quality outcomes, the need for digital therapeutics is gradually increasing in the clinical field. We conducted review of the implications of digital therapeutics in the treatment of neurological deficits for stroke patients. The implications of digital therapeutics have been discussed in four domains: cognition, speech and aphasia, motor, and vision. It was evident that different forms of digital therapeutics such as online platforms, virtual reality trainings, and iPad applications have been investigated in many trials to test its feasibility in clinical use. Although digital therapeutics may deliver high-quality solutions to healthcare services, the medicalization of digital therapeutics is accompanied with many limitations. Clinically validated digital therapeutics should be developed to prove its efficacy in stroke rehabilitation.
Keywords: Digital therapeutics, Stroke, Cognition, Aphasia, Hemianopia, Hemiplegia
Introduction
The use of mobile digital technology in clinical practice is expanding continuously. By 2020, it is estimated that approximately 80% of the world adult population will own smartphones, mirroring the exceptional growth in technology access [1]. This progression is affecting many aspects of our lives including communication, finance, education, and healthcare. Through the development of technology, healthcare professionals and patients are becoming capable of monitoring and managing both acute and chronic conditions.
Digital therapeutics is a newly emerging concept of therapeutic approach in the healthcare system. It is a subdivision of digital health, which defines a healthcare system driven using any form of digital technology to enhance the efficacy of healthcare delivery, such as robotics and artificial intelligence, genomics, mobile phone applications, and telemedicine. These technologies are applied in the clinical field to aid in diagnosis, clinical decision making, treatment, and care delivery [2]. Meanwhile, digital therapeutics is an evidence-based intervention using high-quality software and focuses on treatment [3,4]. The components of digital therapeutics include smartphones, personal digital assistants, virtual reality (VR), and tablet computers that converge with software algorithms. The use of computing power and connectivity capabilities of smartphones such as specific mobile applications, personalized patient education, Internet-based cognitive-behavioral therapy, and virtual health coach is an important feature of digital therapeutics [5,6].
Stroke is the leading cause of chronic disability and mortality in many countries [7]. Brain injuries from stroke result in decreased neuroplasticity of the brain, which is the process of reorganization within the brain to recover itself from functional impairments. This process is time dependent and associated with progressive and continuous learning and practice. Rehabilitation therapy comprising stepwise training focuses on providing stimulation for neurons in the brain such that the brain remains active and forms neuroplasticity. It aims to restore the function of patients by promoting neuroplasticity to form new connections within the brain [8]. Digital therapeutics that perform similarly and focus on progressive repetitive practice to stimulate cortical reorganization are being developed. Various technologies used in digital therapeutics can deliver repetitive, goal-oriented tasks with variability while providing performance feedback. Task difficulty can be adjusted and scaled to a patient’s capability, and remote social interaction is facilitated within the system [9].
Although evidence suggests that post-stroke rehabilitation lessens disabilities [10], the number of patients receiving rehabilitation therapy is less than one-third, and the frequency of therapy varies by patient [11]. Many healthcare systems struggle to deliver high-quality outcomes owing to the greater incidence of chronic diseases resulting from an increasing ageing population [12]. Access to rehabilitation has become limited and dependent on patients’ socioeconomic status [11]. Disengagement, inactivity, boredom, and social isolation are typical in stroke rehabilitation, thus adversely affecting functional outcomes [13]. Digital therapeutics can provide a high-quality cost-benefit solution to these challenges in stroke rehabilitation, thus providing a rapid and accurate management program. The use of iPads, gaming technology, and VR enables productive training with enhanced sensory stimulation, thereby optimizing the restoration of neuroplasticity and provides ease of use and navigation for patients [13]. Reduced level of boredom and feelings of isolation have been reported by rehabilitation therapists, and satisfaction rates have increased in patients after the introduction of iPad-based training [14]. Technology is allowing patients to become more active in managing their own health, thus increasing patient motivation and engagement; therefore, healthcare will transition out of a clinic and into a home.
Previous studies demonstrated that the use of digital therapeutics is gradually expanding in the clinical field. Digital therapeutics is currently widely used as a treatment for psychiatric conditions such as depression and anxiety disorders. Following these studies, many trials have been conducted to investigate the feasibility of digital therapeutics in neurological disorders such as epilepsy, multiple sclerosis, Alzheimer’s disease, and stroke. These conditions represent challenging and costly problems, where many treatment options have limited efficacy and require time-intensive interactions with therapists. The aim of this study is to provide a review regarding the implication of digital therapeutics in the treatment of neurologic deficits following stroke in four domains—cognition, speech and aphasia, motor, and vision.
Interventions for cognitive function
Current cognitive rehabilitation involves conventional exercises using pencil and paper assisted by therapists and computerassisted training; they broadly target attention, working memory, and executive functions such as problem-solving and planning [15]. Restitution rehabilitation restores cognitive functions through repeated practice and specifically targets impaired cognitive functions. Compensation rehabilitation reduces the effects of cognitive impairment on functional abilities, and training is focused on achieving activities of daily living. Patients are trained to minimize demands on impaired functions through increasing awareness and adapting to functional loss [16]. Memory rehabilitation especially focuses on compensation training; it teaches patients to use internal aids such as mnemonics and mental imagery and external aids such as writing diaries and lists to help patients remember important information [17].
Cognitive rehabilitation is an area where online platform therapeutics has been broadly developed, and many studies have demonstrated the benefit of computerized therapeutics as a rehabilitation program. Evidence suggests that the best method to enhance neuroplasticity and to rehabilitate the brain is by playing video games [8]. In a study, the brain magnetic resonance image of patients who played video games showed an increase in grey matter in areas of the brain responsible for executive and memory functions [18]. Several video game-based rehabilitation tools are being developed, including online platforms, iPad applications, and VR trainings. A summary of the reviewed studies is shown in Table 1.
Table 1.
Study | Study design | Sample size (n) | Intervention | Study description | Stroke stage | Frequency of intervention | Follow-up | Outcome measures | Results |
---|---|---|---|---|---|---|---|---|---|
Akerlund et al. (2013) [23] | RCT | IG: 20 | Online platform Cogmed QM | IG: standard rehabilitation with Cogmed training | Sub-acute | 30–45 min for 5 day/wk | 5 wk intervention | WAIS-III | Both IG and CG improved after their training in working memory, BNIS and digit span; however, a greater improvement in was seen in IG compared to CG. |
CG: 18 | Digit span | ||||||||
CG: standard cognitive rehabilitation | 6, 18 wk post-intervention | Spatial span | |||||||
Working Memory sub-scale | |||||||||
BNIS | |||||||||
DEX | |||||||||
Westerberg et al. (2007) [20] | Randomised pilot study | IG: 9 | Online platform Cogmed QM | IG: Cogmed | Chronic | 40 min, 5 day/wk | 5 wk intervention | CFQ | A significant improvement seen in working memory and attention, and decrease in cognitive symptoms |
CG: 9 | CG: no training | Digit span | |||||||
Span board | |||||||||
PASAT | |||||||||
Raven’s progressive matrices | |||||||||
Claeson-Dahl test | |||||||||
Richter et al. (2015) [24] | RCT | IG: 18 | Online platform WOME | IG: WOME training | Chronic | 1 hr/session, 9 sessions | 4 mo post-intervention | CFQ | Significant improvement in working memory in IG |
CG: 18 | CG: standard rehabilitation | Digit span | |||||||
Lundqvist et al. (2010) [21] | A controlled experimental study with a cross-over design | Group 1: 10 | Online platform Cogmed QM | Both groups received training program but at different times | Chronic | 45–60 min, 5 day/wk | 5 wk intervention | PASAT | A significant improvement in the working memory tasks after training and at follow-up |
Group 2: 11 | 4, 20 wk post-intervention | CWIT | |||||||
WAIS-III | |||||||||
Listening Span Task | |||||||||
The Picture Span | |||||||||
Johansson et al. (2012) [22] | Prospective cohort study | 18 | Online platform Cogmed QM | Cognitive assessment at baseline, after training, and at follow-up was compared | Chronic | 30–45 min, 3 day/wk | 8 wk intervention | CFQ | A significant improvement in working memory tasks post-training, effect was maintained at follow-up |
6 mo post-intervention | COPM | ||||||||
Des Roches et al. (2015) [25] | Clinical controlled study | IG: 42 | iPad based program | Both groups received 1 hr clinic session with a clinician | Chronic | 1 hr, once/wk | 10 wk intervention | WAB-R-CQ | Both groups showed an improvement over time, IG showed a greater positive change in accuracy and latency on the tasks than CG. |
CG: 9 | Constant Therapy | IG also received Constant Therapy at home | CLQT | ||||||
PAPT | |||||||||
De Luca et al. (2018) [27] | RCT | IG: 6 | VR training with BTs-Nirvana using I-SIP | IG: VR training with BTs-Nirvana | Chronic | 45 min, 3 day/wk | 8 wk intervention | MoCA | Immediately after treatment, IG presented a greater improvement in TCR, MoCA, attention assessment, the verbal memory, and the visuo-spatial abilities compared to CG. Improvement persisted at follow-up only in IG. |
CG: 6 | FIM | ||||||||
CG: standard cognitive treatment presented by using a paper-and-pencil modality | 1 mo post-intervention | FAB | |||||||
AM | |||||||||
TCT MI |
RCT, randomized controlled trial; IG, intervention group; CG, control group; WAIS-III, Wechsler Adult Intelligence Scale-Third Edition; BNIS, Barrow Neurological Institute Screen for Higher Cerebral Functions; DEX,dysexecutive index; CFQ, Cognitive Failures Questionnaire; PASAT, Paced Auditory Serial Attention Test; WOME, Working Memory; CWIT, Color Word Interference Test; COPM, Canadian Occupational Performance Measure; WAB-R-CQ, Revised Western Aphasia Battery CQ; CLQT, Cognitive Linguistic Quick Test; PAPT, Pyramids and Palm Trees; VR, virtual reality; I-SIP, interactive-semi-immersive program; MoCA, Montreal Cognitive Assessment; FIM, Functional Independence Measure; FAB, Frontal Assessment Battery; AM, attentive matrices; TCT, Trunk Control Test; MI, Motricity Index scale.
Online platform interventions
Cogmed Working Memory Training (Pearson Education Inc., Stockholm, Sweden) is an online platform involving a digital rehabilitation program implemented to improve working memory functions in cognitively impaired patients. The program consists of visuo-spatial and verbal working memory tasks, where patients are required to remember the position of a stimulus and reproduce the stimulus in the same order and in the reverse order, thus challenging the patient’s working memory capacity [19]. A trial using Cogmed training at home in stroke patients showed improvement in working memory and attention in patients who received training. Furthermore, these patients reported fewer cognitive problems after training [20]. Many studies have reported similar results that demonstrated an improvement in working memory and executive function in patients using Cogmed [21-23].
The Working Memory (WOME) software (RehaCom, Hasomed GmbH, Berlin, Germany) is a computer software that provides therapy-assisted training to enhance working memory. The memory training in this software include card game tasks where patients are asked to remember a short series of cards and reproduce the card in the reverse order, and symbol matching tasks and exercises using letters and words. In a study, individualized memory training using the WOME software and standard memory training in the clinical setting was compared. The results of the study showed significant improvement in working memory in patients who received the training, thus demonstrating the effectiveness of the WOME software as a cognitive rehabilitation software [24]. Other online platforms included Lumosity (Lumos Lab, San Francisco, CA, USA), CogniPlus (https://www.schuhfried.com), CogniFit Software (https://www.cognifit.com), and INSIGHT (Posit Science Corporation, San Francisco, CA, USA).
iPad-based interventions
An iPad-based application, Constant Therapy (The Learning Corp., Newton, MA, USA), was developed as a rehabilitation program for improving both language and cognitive skills in stroke patients with aphasia. Tasks were classified into either language or cognitive tasks: language tasks included naming, reading, writing, and sentence planning, whereas cognitive tasks included visuo-spatial processing, memory, attention, problem solving, and executive function. The cognitive tasks included symbol matching, calculation, functional reading, and memory tasks using pictures and words. In a study, patients using Constant Therapy demonstrated greater positive changes especially in the memory task compared with the patients who did not, demonstrating the feasibility of using tablet-based software to deliver tailored language and cognitive therapy [25]. A further study was performed using Constant Therapy to directly compare software therapy with clinic therapy. The results of the study showed similar outcomes for patients using software therapy and clinic therapy. However, patients using software therapy could master the task faster compared with patients who received clinic therapy, thus demonstrating the potential of the software rehabilitation program for stroke patients [26].
VR-based interventions
The use of VR for cognitive rehabilitation is necessitated, as controlled trials and available training using VR systems are lacking. VR training with BTs-Nirvana using the interactivesemi-immersive program (I-SIP) created a series of interactive exercises targeting attention, memory, spatial cognition, and executive function. VR-programmed movements that correlate to virtual touch aim to move or manipulate specific objects in different directions to form a dynamic interaction in a virtual environment. A study compared VR-based cognitive training and conventional cognitive rehabilitation in stroke patients. The results showed greater improvement in attention and memory assessments in patients who received VR training. This study demonstrated that VR training can be considered as a useful complementary treatment to improve visuo-spatial deficits and motor functions as well as cognitive functions [27].
Interventions for speech and aphasia
Two types of aphasia are common in stroke patients: fluent aphasia where the patient’s ability to understand the meaning of spoken words is significantly impaired and nonfluent aphasia where speech output becomes severely reduced and limited in speaking and writing. Semantic training is provided to patients with fluent aphasia, which is a meaning-based approach to therapy; it focuses on training based on synonyms and antonyms, completion of sentences, and word description tasks. Phonological training is provided to patients with nonfluent aphasia, which focuses on auditory or articulatory word forms, such as emphasizing word sounds and speech productions [28]. Typically, both semantic and phonological trainings are used in combination for aphasia rehabilitation, targeting different language domains such as naming, comprehension, sentence processing and production, and writing skills [29].
Evidence suggests that a high intensity rehabilitation program is associated with better outcomes [30]. Recent studies have also shown that intensive repetitive treatment may result in better outcomes; however, sustained training is required at a lower intensity to maintain the improvements from the therapy [31]. The current traditional rehabilitation for aphasia requires high levels of therapist involvement, and patients often receive limited support to maintain their progress. The use of computer technology has since been investigated increasingly in aphasia rehabilitation. The use of digital therapeutics in either home or clinical settings can facilitate high-intensity programs and support the discharge of patients while maintaining the improvements from rehabilitation [29].
iPad-based training has been developed the most for speech and language rehabilitation; additionally, the use of iPads and computers has been shown effective owing to their applicability for repetitive task training using images [13]. The independent use of a technology-based rehabilitation program using a computer, iPad tablet, VR, and iPod has been evaluated to investigate the efficacy of therapy in patients with aphasia. A summary of the reviewed studies is shown in Table 2.
Table 2.
Study | Study design | Sample | Intervention | Study description | Stroke stage | Frequency of intervention | Follow-up | Outcome measures | Results |
---|---|---|---|---|---|---|---|---|---|
Palmer et al. (2012) [33] | RCT | IG: 15 | Computer-based program StepByStep | IG: StepbyStep training with usual care—reading, and writing activities with therapists | Chronic | 20 min, 3 day/wk | 5 mo intervention | OANB | IG showed more gains in naming ability after treatment than CG, with the difference of 19.8%. |
CG: 13 | CG usual care only | 3 mo post-intervention | |||||||
Doesborgh et al. (2010) [35] | RCT | IG: 8 | Computer-based program Multicue | IG: Multicue | Chronic | 30–45 min, 2/3 day/wk | 2 mo intervention | BNT | Only IG improved in the BNT. Mean improvement did not differ between IG and CG. |
CG: 10 | CG: no treatment | ANELT-A | |||||||
Fink et al. (2002) [36] | Clinical controlled study | Group 1: 3 | Computer-based program Moss Talk Words | Group 1: full clinician guidance | Chronic | 12 sessions for 30–45 min each | 4 wk intervention | PNT | Both groups showed gains on trained words in PNT. |
Group 2: 3 | Group 2: partial independence | PRT | |||||||
PORT | |||||||||
Ramsberger et al. (2007) [37] | Clinical controlled study | Intensity | Computer-based program Moss Talk Words | Low intensity: 2/wk | Unspecified | Unspecified | 4 wk intervention | Performance on Moss Talk Words | Patients showed gains in naming, regardless of intensity. |
Low: 2 | High intensity: 5/wk | ||||||||
High: 2 | |||||||||
Des Roches et al. (2015) [25] | Clinical controlled study | IG: 42 | iPad-based program Constant Therapy | Both groups received 1 hr clinic session with a clinician. | Chronic | 1 hr, 1 day/wk | 10 wk intervention | WAB-R-CQ | Almost all patients showed gains on treatment tasks, IG showed more gains on standardized measures than CG. |
CG: 9 | IG also received Constant Therapy at home. | CLQT | |||||||
PAPT | |||||||||
Steele et al. (2014) [38] | Pre-poststudy | 9 | iPad-based program Lingraphica TalkPath | Participants received individual and group speech-language teletherapy services, and also used on-line language exercises to practice from home between therapy sessions. | Chronic | 21 hr over 12 wk | 12 wk intervention | WAB | Participants showed gains in CETI and NOMS on most items, RIC-CCRSA also showed gains in one item and in the overall score. |
CETI | |||||||||
NOMS | |||||||||
RIC-CCRSA | |||||||||
Kurland et al. (2014) [40] | Pre-poststudy | 5 | iPad-based program iBooks | Intensive 2-wk aphasia treatment program prior to beginning the individualised home practice programs | Chronic | 2 hr/wk | 6 mo intervention | BDAE | All patients maintained previous improvements and showed a further improvement in new trained words. BDAE and BNT scores were equal to or better than baseline. |
BNT | |||||||||
Thompson et al. (2010) [41] | Clinical controlled study | IG: 6 | VR training Sentactics | IG: Sentactics | Chronic | 1 hr, 4 day/wk | 8 wk intervention | NAVS | IG showed more improvements in NAVS than CG, no difference to the clinician delivered therapy. |
CG: 6 | CG: no treatment | ||||||||
Cherney et al. (2010) [43] | RCT | IG: 11 | Computer-based program ORLA | IG: computer ORLA | Chronic | 1 hr, 2–3 day/wk | 8 wk intervention | WAB-AQ | Groups had equal gains in both WAB-R and WAB-AQ and showed no difference between the two groups. |
CG: 14 | CG: clinician delivered therapy | WAB-R | |||||||
Cherney et al. (2012) [44] | RCT | IG: 19 | VR training Webbased ORLA | IG: Web-based ORLA | Chronic | 9 hr/wk | 6 wk intervention | WAB-AQ | An improvement in language performance measured by WAB-R-AQ and WAB-R writing in IG; however, insignificant changes in WAB-R-AQ and WAB-R reading. Improvements were maintained in follow-up. |
CG: 13 | CG: placebo-computer treatment | 6 wk post-intervention | WAB-R | ||||||
Fridriksson et al. (2012) [46] | Pre-posttest | 13 | iPod-based program | Aphasia assessments evaluated before and after the training. | Chronic | 30 min/wk | 6 wk intervention | WAB-R | Patients were able to produce more than twice as many words during the speech entrainment–audio visual compared with the speech entrainment–audio only therapy. |
BNT |
RCT, randomized controlled trial; IG, intervention group; CG, control group; OANB, Object and Action Naming Battery; BNT, Boston Naming test; ANELT-A, Amsterdam Nijmegen Everyday Language Test scale A; PNT, Philadelphia Naming Test; PRT, Philadelphia Repetition Test; PORT, Philadelphia Oral Reading Test; WAB-R-CQ, Revised Western Aphasia Battery CQ; CLQT, Cognitive Linguistic Quick Test; PAPT, Pyramids and Palm Trees; CETI, Communicative Effectiveness Index; NOMS, National Outcome Measurement System; RIC-CCRSA, Rehabilitation Institute of Chicago–Communication Confidence Rating Scale for Aphasia; BDAE, Boston Diagnostic Aphasia Examination; VR, virtual reality; NAVS, North-western Assessment of Verbs and Sentences; ORLA, Oral Reading for Language in Aphasia; WAB-AQ, Western Aphasia Battery Aphasia Quotient.
Computer-based interventions: semantic training and phonological training
The StepByStep (Steps Consulting Ltd., South Gloucestershire, UK) computer program includes over 10,000 language exercises ranging from listening to target words; it produces words with visual, semantic, and phonemic cues for writing words or producing sentences [32]. A trial was conducted to investigate the efficacy of self-managed computer treatments for patients with chronic post-stroke aphasia. It was demonstrated that patients who received StepByStep training demonstrated greater improvement in naming ability compared with patients who received the standard speech and language therapy [33]. Multicue is another computer-based word finding therapy for patients with post-stroke aphasia. The program comprises a series of pictures that are randomly presented to patients to target naming and verbal communication skills. Self-administered cues are presented to patients to stimulate independence by encouraging them to discover the most helpful cues [34]. A study that investigated Multicue as a rehabilitation program demonstrated significant improvement in naming abilities measured through the Boston Naming Test in patients who received the training; however, no significant improvement was shown in verbal communication skills. The study suggested that in chronic aphasia patients, Multicue may exhibit a positive effect on word finding in picture naming but not on verbal communication [35]. MossTalk Words (Moss Rehabilitation Research Institute, Elkins Park, PA, USA) offers training that comprises picture-naming tests with multiple auditory and visual cues to facilitate the training. Two studies have been performed to test the effectiveness of the MossTalk Words program [36,37]. These studies demonstrated that independent computerized therapies can be as effective as clinician-guided therapies [36]. Furthermore, improvement in naming ability has been shown after training, regardless of the training intensity [37].
iPad-based interventions: semantic training and phonological training
Additionally, iPad-based aphasia rehabilitation therapy has been explored, including Constant Therapy. Constant Therapy includes 37 evidence-based cognitive and language tasks of different difficulty levels. In a study, patients who received the Constant Therapy training demonstrated greater improvements in accuracy and latency [25]. Lingraphica TalkPath Therapy (Lingraphica, Princeton, NJ, USA) is another iPad-based program for stroke patients; it includes exercises for rebuilding speech and language skills. The exercise targets listening, speaking, reading, and writing abilities. In a study, patients who used TalkPath Therapy demonstrated significant improvements in spontaneous speech, auditory verbal comprehension, repetition, and naming tasks [38]. Language TherAppy (https://tactustherapy.com) is a self-delivered iPad-based speech therapy. In a clinical trial, patients who used Language TherAppy as an aphasia rehabilitation tool demonstrated significant improvements in reading, writing, naming, and comprehension tasks [39]. Another tablet-based technology was used to maintain and augment a 2-week intensive clinical speech and language therapy for patients with post-stroke communication deficits. Interactive iPad Books, which includes self-managed tasks, was created using the iBooks Author software; it targets object and verb naming that is to be practiced in a home setting. The tasks included confrontation, naming, repeating words out loud after a video model, and matching pictures with words presented on an iPad. A speech and language therapist was also in contact with the participants weekly via the GoToMeeting application. All the participants maintained the improvements from the intensive clinical therapy and could learn new words after training in a home setting using the iBooks [40]. Other iPad-based aphasia rehabilitation applications include iAphasia and SentenceShaper (https://sentenceshaper.com).
VR-based interventions: phonological training
Sentactics (Sentactics Corporation, Concord, CA, USA) aims to deliver computer-automated treatment for underlying forms, which is a linguistic treatment for improving nonfluent aphasia. Interactive software enables the delivery of speech therapy for improving disorganized sentence production and comprehension through a virtual clinician. Patients are trained by repeating after the virtual clinician, reading sentences and describing pictures presented on the screen aloud. Thompson et al. [41] conducted a study to test the efficacy of Sentactics as an aphasia rehabilitation tool. Computer-delivered Sentactics was compared with a clinician-delivered therapy. The results showed that patients who received Sentactics training demonstrated a significant improvement in comprehension and speech production. The outcomes of Sentactics and the clinician-guided therapy were similar, suggesting that technologydelivered therapy is a viable option for aphasia rehabilitation programs [41]. Another VR aphasia therapy is Web Oral Reading for Language in Aphasia (ORLA, Rehabilitation Institute of Chicago, Chicago, IL, USA). ORLA is a therapy program where patients repeatedly read aloud sentences, first in unison with a clinician and then independently. The program was developed to improve a patient’s reading comprehension skill by providing practice in phonological and semantic reading routes [42]. A study comparing computer-delivered-ORLA and clinician-delivered ORLA indicated no significant difference between the two therapies. A small difference between the two groups was deemed to be caused by the difference in opportunity for participants to observe oral-motor movements of the therapist. To overcome this difference, a newer version of Web-ORLA with a virtual therapist whose lips movements are visible with speech has been developed [43]. The virtual ORLA incorporated a threedimensional (3D) animated clinician with a digitalized female voice. A trial was conducted regarding the efficacy of Web-ORLA. Patients demonstrated a statistically significant improvement in language performance such as auditory comprehension, oral expression, and written expression after training. Analysis of individual subject data indicated that almost 80% of the participants receiving Web-ORLA exhibited a clinically significant change in at least one language modality. Recently, more versions of virtual therapists with visible speech has been developed to be compatible with most personal computers, web browsers, and smartphones, thus increasing compatibility on a range of devices [44,45].
iPod-based interventions: phonological training
Earlier studies of speech and language therapy reported that delivering an audio-visual speech stimulation on an iPod screen significantly increased the fluency of verbal output of patients with nonfluent aphasia. The study used an iPod application, “visually assisted speech technology,” where the mouth of the speaker was shown on the iPod screen; additionally, using the audio-visual feedback, the patient attempted to mimic the speaker. The results of the study showed that the use of audiovisual feedback resulted in a greater improvement in speech production compared with auditory-only speech stimuli [46].
Interventions for motor function
Motor deficits are a main symptom in stroke patients. Rehabilitation focuses on muscle strengthening, walking and balance training, arm movement training, and fine movement training [47]. The primary focus of the currently used motor rehabilitation is the relearning of specific motor skills and daily task training. Especially in the rehabilitation of stance and gait, a task-specific repetitive approach is increasingly being used as well as intensive preparatory training in the sitting and standing positions. The number of steps practiced per session is shown to be important during rehabilitation trainings and gaittraining. Software and machineries are recommended, as the physical work required to practice a large number of steps is difficult to achieve for unaided physiotherapists. The use of gait-training software enables the early verticalization of patients and is more effective when compared with physiotherapist trainings alone in terms of improving the motor function and increasing patient adherence to and enjoyment of exercise, although it does not replace the physiotherapist [48]. The use of video games and the third person view of patients themselves in VR trainings render the technology highly suitable for lower and upper limb motor trainings. However, these trainings do not prevent patients from using compensatory body mechanisms nor do they provide fine motor trainings such as finger or wrist movements, which are areas for development.
Similar principles are applied in the rehabilitation of the upper limb motor function. Physiotherapists train a patient’s arm and hand function with repetitive practice, focusing on strength, coordination, and speed of movements [48]. Robots, software, and VR trainings have been proven effective as rehabilitation therapy. Evidence shows that the neuroplasticity following stroke is related to intensity, frequency, and duration of therapy [49]. Therefore, tablet-based and VR-based trainings could provide an individualized and repetitive practice environment. A summary of the reviewed studies is shown in Table 3.
Table 3.
Study | Study design | Sample size (n) | Intervention | Study description | Stage of stroke | Frequency of intervention | Follow-up | Outcome measures | Results |
---|---|---|---|---|---|---|---|---|---|
Subramanian et al. (2013) [55] | RCT | IG: 16 | VR training CAREN | IG: VR training | Chronic | 45 min, 3 day/wk | 4 wk intervention | RPSS | Both groups showed a significant improvement post-intervention, a greater improvement was seen in shoulder adduction and flexion in IG compared to CG. |
CG: 16 | CG: conventional training—targeting reaching and pointing movements similar to the VR training | WMFT | |||||||
MAL-AS | |||||||||
McEwen et al. (2014) [54] | RCT | IG: 30 | VR training IREX | IG: standard rehab+VR training (in standing) | Chronic | 30 min, 10–12 sessions/day, | 3 wk intervention | TUG | IG showed a greater effect on TUG and TMWT. |
CG: 29 | CG: standard rehab+VR training without the challenge (in sitting) | 1-mo post-intervention | TMWT | More individuals in IG than in CG showed a decreased impairment in the lower extremity, measured by CMSA. | |||||
CMSA | |||||||||
Broeren et al. (2004) [56] | Single case study | 1 | VR training with haptic device | Changes in the pre-posttraining assessments and follow-up assessments were evaluated | Unspecified | 90 min/session, 12 sessions in 4 wk | 4 wk intervention | Purdue Pegboard Test | Improvements were seen in fine manual dexterity, grip force, and motor control of the affected upper extremity. |
20 wk post-intervention | Dynamometer handgrip strength | ||||||||
Cannell et al. (2018) [52] | RCT | IG: 35 | VR training JRS | IG: JRS WAVE | Subacute | 1 hr/wk | 8 wk intervention | MMAS, FRT, ST, 10 MWT, TUG | Both groups improved on standing balance after training. No difference was seen between the two groups in primary or secondary outcomes. |
CG: 38 | WAVE | CG: usual care—individualised and group physical exercise program | Sitting balance test | ||||||
Ikbali Afsar et al. (2018) [57] | RCT | IG: 19 | VR training with | IG: conventional physical rehab+VR | Unspecified | 30 min/session, 5 times/wk | 4 wk intervention | BBT | The functional measures showed a significant improvement post-treatment in both groups. IG showed a significantly higher result in the Brunnstrom stage and in BBT compared to CG. |
CG: 16 | Microsoft Xbox | CG: conventional rehab only | Brunnstorm stage | ||||||
360 Kinect system | FMA | ||||||||
Emmerson et al. (2017) [58] | RCT | IG: 30 | iPad-based home exercise program | IG: iPad program | Chronic | Unspecified, completion of program in 4 wk | 4 wk intervention | WMFT | There was no difference between IG and CG. |
CG: 32 | CG: standard program | ||||||||
Carabeo et al. (2014) [59] | Cohort study | - | Android-based game app FINDEX | 3 | Chronic | 30 min/session, total 9 sessions | 6 wk intervention | Performance on FINDEX | Improvement in patients’ dexterity |
Schneider et al. (2007) [63] | Clinical controlled study | IG: 20 | MST | IG: conventional therapy+MST | Chronic | 15 sessions, 30 min/session | 3 wk intervention | ARAT | Patients showed an improvement in speed and precision of movements. IG showed an improvement in motor control. |
CG: 20 | CG: conventional therapy only. | BBT | |||||||
9HPT |
RCT, randomized controlled trial; IG, intervention group; CG, control group; VR, virtual reality; CAREN, computer-assisted rehabilitation environment; RPSS, Reaching Performance Scale for Stroke; WMFT, Wolf Motor Function Test; MAL-AS, Motor Activity Log Amount Scale; IREX, Interactive Rehabilitation Exercise software; TUG, Timed Up and Go test; TMWT, Two-Minute Walk Test; CMSA, Chedoke-McMaster Stroke Assessment; JRS, Jintronix Rehabilitation System; MMAS, Modified Motor Assessment Scale; FRT, Functional Reach test; ST, step test; 10 MWT, 10-m walk test; BBT, Box & Block Test; FMA, Fugl-Meyer assessment; MST, musicsupported therapy; ARAT, Action Research Arm Test; 9HPT, 9 Hole Pegboard Test.
VR-based interventions
Lower limb motor function: walking and balance
Jintronix Rehabilitation System (JRS WAVE; http://www.jintronix.com/) is a Microsoft-Kinect-based VR software program. JRS WAVE provides one-on-one therapy sessions using the Microsoft Kinect infrared camera sensor device, which recognizes and tracks the movements of patients in real time. Interactivemotion-capture-based rehabilitation technology targets sitting and standing balance, coordination and gait rehabilitation, as well as upper limb motor, by using various therapeutic games in both home and outpatient settings. By increasing patient motivation and adherence using interactive video games, JRS WAVE aims to increase the amount of exercise completed by patients, as well as enable exercise both during and outside of clinical therapy sessions. The interactive gesture-controlled motion-capture games included sitting and standing tasks that move the center of mass toward the limits of stability, and sitting and standing lower limb activities that improve stepping and gait. Furthermore, the system measures changes in speed and motion range to provide feedback regarding therapy progress [50]. A trial evaluated the feasibility of JRS WAVE for use in stroke patients by comparing VR training under therapist supervision with standard physiotherapy, in a hospital setting. Both types of training showed an improvement in sitting and standing balance, walking and mobility tasks, and upper-limb tasks, with no significant difference [51,52].
Another VR-based rehabilitation includes the Interactive Rehabilitation Exercise software (IREX, GestureTek technologies, Toronto, ON, Canada). Through VR exercise regimes such as balancing, rotation, leaning, crouching, stair stepping, and exertion, IREX is designed to improve trunk and limb control, balance control, as well as fine and gross motor movements in stroke patients. The physical tasks include snowboarding, swimming, and goalkeeping, where the motion range is measured by a virtual goniometer [53]. A study was conducted using IREX to investigate the feasibility of motor rehabilitation training in hospital setting. The study demonstrated that patients who received the training showed a reduction in impairments of the lower extremity compared with patients who did not receive the training; this establishes the efficacy of video game training in a VR environment [54].
Upper limb motor function: gross and fine motor skills
A trial explored the advantages of using a virtual environment, i.e., a computer-assisted rehabilitation environment (CAREN) for post-stroke arm motor rehabilitation compared with using the traditional physical exercise training over 4 weeks. The training simulated a reach-to-grasp task in a supermarket shopping scene, incorporating visual illusions to create 3D visual cues. Patients received training in the sitting position to encourage the full range of movement at the shoulder and elbow. Electromagnetic sensors were used to track arm and trunk positions while patients pointed at the target on a supermarket shelf such as ice cream, water, and vegetables, as fast and as accurately as possible following the instructions. Both groups showed increased upper limb use and activity level, as well as performance improvement in the reach-to-grasp task; however, only the VR training group demonstrated improvement in shoulder adduction, flexion, and elbow extension. The study suggested that the better outcomes in VR training group may be attributed to the more efficient use of feedback provided by the VR system [55].
A single case study investigated the efficacy of VR training using a haptic device and a 3D video game for motor rehabilitation in post-stroke patients with the left hemi-paretic arm. Patients played a 3D ball-and-bricks game similar to a pool ball game displayed on a screen, in which the ball was controlled with the left arm using the haptic device. Participants who received the training showed improvement in fine manual dexterity, grip force, and motor control of the affected upper extremity [56]. Another study investigated the use of VR training using the Microsoft Xbox 360 Kinect video game system in sub-acute stroke patients. The training program focused on the active movements of the upper extremity, where patients performed bilateral shoulder abduction and adduction, elbow flexion, and extension in video games. The study reported that patients who received both VR training and conventional physiotherapy showed significant improvement in Fugl-Meyer scores compared to patients who only received conventional therapy; thus, providing evidence that Kinect-based game training in conjunction with conventional therapy may have supplemental benefits for stroke patients [57].
iPad-based interventions
Upper limb motor function: gross motor skills
Home exercise rehabilitation using videos and automated reminders on an iPad was compared with a paper-based conventional home exercise program in stroke patients with upper limb deficits. The iPad contains home exercise programs such as stretching exercises to improve the motion range, as well as strengthening and coordination exercises of the upper limb. Shoulder flexion, extension, abduction, and wrist extension exercises for stretching and training against gravity are part of the strengthening exercise. Between patients who used an iPad program and those who used the conventional program, individual home-based training in these patients showed no difference in the improvement of the upper limb function. The trial demonstrated that the use of a home exercise program in both the standard and electronic methods is a viable choice of treatment for upper limb motor rehabilitation [58].
Upper limb motor function: fine motor skills
An Android-based tablet game, FINDEX, has been investigated as a tool for motor rehabilitation in stroke patients with impaired fine motor skills [59]. FINDEX assesses and monitors a patient’s fine finger dexterity such as finger control, and coordination and range of motions. Patients train their finger control using dragging tasks, where they are asked to place pizza toppings in the corresponding spaces; meanwhile, finger coordination is trained using a tapping task, where patients are asked to tap piano keys displayed on the screen to play the correct chords to accompany the music. To improve the motion range of fingers, a stretch task trains patients to move around a finger as far as possible while maintaining one finger at one location on the screen. Three patients were trained in an in-patient setting using the application, and the result of the study showed improvement in dexterity in all patients. The findings from the study suggested that mobile games such as FINDEX may be used as an effective therapeutic tool for fine motor rehabilitation in a clinical setting. Other iPadbased therapeutic tools developed to improve fine motor skills include Fruit Ninja [60] and Dexteria [61].
Music-supported therapy
Music therapies have been used in stroke rehabilitation to stimulate motor functions, ranging from the use of rhythmic auditory stimulation to music creation and performance on an instrument. Music physiologically motivates and entrains movement, where the rhythm in music acts as a stimulus for motor function. The repetitive and predictable structures of music are suitable cues for learning in motor rehabilitation [62]. Music-supported therapy (MST) presents a series of exercises using electronic musical instruments such as a piano and a drum to train fine and gross motor skills by inducing an integration of the auditory-sensorimotor pathway. Patients started the exercises in a clinical setting by playing single tones, and the number of tones increased in the subsequent levels until the patients could play a tune comprising five to eight notes in various sequences with the paretic hand. The patients showed a significant improvement after the treatment in terms of speed, precision, and smoothness of movements [63]. In a study, MST was combined with neuroimaging techniques to understand the neuroplastic changes observed after the rehabilitation program using functional magnetic resonance imaging (fMRI). The study demonstrated that the music listening task in patients showed a bilateral activation of the auditory-motor circuit, highlighting that following MST, connectivity in the auditory-motor circuit increased [64]. Another technology with a musical framework was introduced as physical rehabilitation in stroke patients. A digital musical instrument involving drumming to the beat using digital drum pads synced to an iPad application was used to assess the improvement of the upper limb movements in a clinical setting. Patients would select their favorite music and correspondingly tap the image of drum pads on the iPad screen to match the beat. The patients involved in this study showed a significant increase in functional measures after the intervention period [65].
Interventions for vision
Rehabilitation for visuo-spatial neglect is necessitated for digital therapeutics. Although visual impairments following stroke have profound effects on patients, a standard systematic approach for comprehensive visual rehabilitation does not exist in contrast to cognitive, speech, and motor rehabilitation [66]. Visual rehabilitation following stroke focuses on the visual field defect, strabismus and ocular motility, central vision impairment, visuo-spatial neglect, and visual perceptual deficits [67]. The developed digital therapeutics are focused on visual field defects. iPad-based or VR-based interventions for visual impairments are not widely available, thus prompting the further development of digital therapeutics for post-stroke visual rehabilitation.
The approaches to traditional rehabilitation for visual field defects include substitution therapy using visual aids such as prisms, compensation therapy using intact residual abilities, and restitution therapy, which is focused on regenerating the plasticity of neural tissues by continuously presenting stimuli in the blind field [68,69]. Recently, visual perceptual learning (VPL) has been introduced for vision restitution therapy. Repetitive visual training stimulates damaged visual cortices, thus resulting in a long-term enhancement of perceptual tasks [70]. The use of computer-based digital therapeutics in compensation and restitution therapy including VPL has been investigated. A summary of the reviewed studies is shown in Table 4.
Table 4.
Study | Study design | Sample size (n) | Intervention | Study description | Stage of stroke | Frequency of intervention | Follow-up | Outcome measures | Results |
---|---|---|---|---|---|---|---|---|---|
Sahraie et al. (2016) [72] | Clinical controlled study | IG: 16 | NEC Computer-based compensatory therapy | IG: NEC training | Chronic | 45 min/level, total of 12 levels | 2 wk intervention | TÜP | An improvement of around 31% in scanning speed in the cancellation task and 21% in the visual search task. |
CG: 16 | CG: no training | 11 wk post-intervention | Pen and paper version of the cancellation tasks and visual search tasks | ||||||
Aimola et al. (2014) [73] | RCT | IG:28 | Computer-based compensatory therapy | IG: reading and writing training | Chronic | 1 hr/day | 5 wk intervention | OculusTwinfield 2 perimeter | IG demonstrated a significant improvement in exploration and reading compared to CG. |
CG:24 | CG: control training | Visual search tasks | |||||||
TEA,SART1VFQ-251VIQ | |||||||||
Sato et al. (2014) [74] | Pre-poststudy | 9 | VISIOcoach computer-based compensatory therapy | The visual performance measures compared in pre-training and post-training | Unspecified | Unspecified | 8 wk intervention | HVF30-2, ETDRSVA, Peiii-Robson Contrast Sensitivity chart, MP-1 microperimetry, video-recording of eye movements | Performance in reading tasks did not differ significantly after training. However, head and eye movements improved in all patients. |
Kasten et al. (1995) [76] | Clinical controlled study | IG: 11 | Computer-based VRT | IG: computer-based VRT | Unspecified | 1 hr/day | 80-300 hr intervention | HRP | IG demonstrated a significant improvement in the detection of small light stimuli and an increased ability to discriminate colors in the blind field, whereas CG showed a decrease in the visual field. |
CG:3 | CG: little or no therapy | TAP | |||||||
Kasten et al. (1998) [77] | RCT | IG: 19 | Computer-based VRT | IG: computerized VRT | Chronic | 1 hr/day except Sundays | 6 mo intervention | HRP | VRT led to a significant improvement (29.4%) in the ability to detect visual stimuli in IG. 4.9°一5.8° visual-field expansion was observed. CG did not show comparable improvements. |
CG: 19 | CG: placebo-no train-ing | TAP | |||||||
Reinhard et al. (2005) [79] | Clinical controlled study | 17 | Computer-based VRT | The visual performance measures compared in pre-training and post-training | Chronic | 1 hr/day, 6 day/wk | 6 mo intervention | SLO | None of the patients showed a significant change in the visual field defect after training. |
HRP | |||||||||
TAP | |||||||||
Schreiber et al. (2006) [80] | Clinical controlled study | 16 | Computer-based VRT | The visual performance measures compared in pre and post-training | Chronic | 1 hr/day, 6 day/wk | 6 mo intervention | SAP | VRT has little effect in rehabilitation of HVFD. |
Huxlin et al. (2018) [70] | Clinical controlled trial | IG 17 | VPL | IG:VPL | Chronic | 300 trials/day, 5 day/wk | 3-14 mo intervention | HVF 24-2 and 10-2 | IG recovered 108 degrees [2] of vision on average, while CG improved over an area of 16 degrees [2]. |
CG5 | CG: no training | Discrimination task performance | |||||||
Das et al. (2014) [81] | Clinical controlled trial | Group 1:3 | VPL | Group 1: static orientation training | Chronic | 300 trials for 5 day/wk | - | HVF | Moving stimuli and double training may be superior training tools for inducing visual recovery in CB. |
Graup 2:6 | Group 2: motion and static orientation discrimination training | ||||||||
Huxlin et al. (2009) [82] | Clinical controlled study | 7 | VPL | The visual performance measures were compared in pre-training and post-training | Chronic | Unspecified | 9-18 mo intervention | HVF 24-2 and 10-2 | Perceptual relearning of complex visual motion processing is possible with damaged primary visual cortex when VPL is presented in the blind field. |
Discrimination task performance |
IG, intervention group; CG, control group; NEC, NeuroEyeCoach; TÜP, Tübingen perimeter; RCT, randomized controlled trial; TEA, Test of Everyday Attention; SART, Sustained Attention to Response task; VFQ-25, Visual Functioning Questionnaire-25; VIQ, Visual Impairments Questionnaire; HVF, Humphrey Visual field; ETDRS VA, Early Treatment Diabetic Retinopathy Study Visual Acuity; MP-1, MicroPerimeter 1; VRT, visual restoration therapy; HRP, High-resolution perimetry; TAP, Tübinger automatic perimeter; SLO, scanning laser ophthalmoscope; SAP, static automated perimetry; HVFD, homonymous visual field defect; VPL, visual perceptual learning.
Computer-based visual compensation therapy
Compensation therapy is a search-and-scan training program that allows patients to compensate for impairments by training the speed and accuracy of eye movements [67]. NeuroEyeCoach (NEC) is a compensation therapy developed by NovaVision (NovaVision Inc., Boca Raton, FL, USA) [71]. It is a software program that uses saccadic training such as finding objects to improve visual search efficiency and navigation skills. Patients are asked to search the computer screen for words, numbers, or pictures and decide if a particular item is present. The level of difficulty is altered by changing the target positions and by a more complex search with multiple distractors. In a study, participants showed significant improvements in computerized tasks such as cancellation tasks and simple visual search tasks after receiving individual NEC training in clinical settings. The results of this study demonstrated that NEC training can be used as an effective compensatory therapy in patients with visual field defects [72]. Another pilot study showed significantly improved performances in exploration and reading tasks in patients with visual field defects following individualized home-based compensatory training using a computer software [73]. The training comprised the visual exploration task, where patients had to search for a defined target on the screen among distractors, and the reading task, which required searching for a nonword target among a number of different words. VISIOcoach (www.visiocoach.de) suggested another compensation therapy for patients with hemianopia or quadrantanopia. It is a software rehabilitation composed of saccade training to improve natural search and spatial orientation. A study exploring the effect and feasibility of VISIOcoach showed that the saccade amplitude and eye movement in a blind hemifield improved in patients after a home-based self-training, although no significant difference was shown in reading tests after the training [74]. The study concluded that VISIOcoach rehabilitation training is a feasible option and can be used as an effective home-based saccadic training tool. It was also suggested that compensatory therapy may induce neuroplasticity in stroke patients, following an fMRI study that demonstrated the significant activation of the oculomotor cortical field after compensatory training [75].
Computer-based visual restoration therapy
In the 1990s, Kasten and Sabel [76] published data regarding the efficacy of a computer-based visual restoration therapy (VRT). The system presents repetitive light stimuli in the border zone between the blind field and the intact visual field. Computerized VRT aims to strengthen the information processing of residual neurons by activating cells repeatedly. Kasten and Sabel [76] demonstrated the use of computer-based VRT in patients with homonymous visual field defects (HVFDs), where significant improvements in light stimuli detection and color discrimination in the blind field were shown. Another study by Kasten et al. [77] investigated the feasibility of computer-based training in patients with optic nerve or post-chiasmatic brain injury. The computer screen flashed light to a patient’s blind spot border zone to stimulate and expand the visual field. The study reported up to a 5° enlargement of the visual field in 95% of the participants. Furthermore, patients reported a general sense of improved vision such as an improvements in detecting and localizing stimuli, and reading [77]. However, the results of these studies were later criticized for the fact that the patients may have developed compensatory eye movements instead of vision restoration, as the VRT program in the study failed to allow a firm central fixation of eye movements [78]. A following study repeated the trial using a scanning laser ophthalmoscope to ensure exact eye fixation, and the results showed no significant improvement in the visual field of patients after a computerized VRT [79]. Another study using computer-based VRT also demonstrated a minute effect on patients with HVFD [80].
Huxlin add Cavanaugh [70] demonstrated that VPL training improves performance in visual field tests in chronic stroke patients. VPL training involves a direction discrimination task and orientation discrimination task, where participants are asked to discriminate the left or right direction of random dot stimuli or the vertical-horizontal orientation discrimination of static Gabors presented on a computer screen. Prior to and following the training, the Humphrey visual fields in patients were evaluated. An online eye tracking software was used to ensure a fixed eye position on the stimuli. The results showed a significantly greater improvement in the Humphrey perimetry in trained patients compared with untrained patients [70]. Using the intensive repetitive VPL training, it was demonstrated that stroke patients with damaged primary visual cortices can perceptually relearn complex visual motion processing. The study demonstrated that a thorough localized training with a direction integration task can improve the blind field vision of patients with cortical blindness (CB) [81,82].
A VPL training program developed by Nunaps Inc. (https://nunaps.com) also demonstrated the efficacy of software-based VPL training in chronic CB patients following stroke. In an unpublished exploratory study, VPL occurrence resulted in a significant improvement in visual field defects, manifesting an increase in the resting state functional connectivity between the ipsilateral cuneus and the contralateral hippocampus. A following unpublished small randomized trial showed a significantly improved outcome in those who acquired performance improvements of VPL after training. Subsequently, a pivotal trial will be conducted to determine the efficacy of digital therapeutics, “Nunap-Vision” to treat visual field defects after stroke.
Limitations and future perspectives
Although digital therapeutics may influence the current healthcare system, the medicalization of digital therapeutics is accompanied with many limitations. Currently, high-quality evidence that supports the clinical benefits of digital therapeutics is lacking [83]. Although the number of digital therapeuticsrelated publications is increasing gradually, the published evidence in support of its clinical use is limited. Several recent meta-analyses and systemic reviews conclude that high-quality evidence is lacking for the use of digital therapeutics to manage healthcare delivery [84,85]. Most clinical trials with digital therapeutics for neurological disorders and stroke are still at the proof-of-concept stage. The reliability of these studies is limited owing to inadequate sample sizes and short follow-up periods. If these studies are repeated in the future with a larger sample size in a randomized manner, the currently significant results may become nonsignificant. Other studies have used a different evaluation tool for the outcome measures, which resulted in variable degrees of outcomes. Most importantly, the majority of these trials did not investigate the underlying neural mechanism of the positive effect of digital therapeutics. Future proof-of-concept trials using digital therapeutics should incorporate neuroimaging or electrophysiological tools to search for biomarkers for mediating treatment effects.
A benefit of digital therapeutics is the ease of access to healthcare data. However, this may create privacy and security concerns. The medical ethics and legal acts governing the privacy and security of these data do not extend outside the healthcare setting, rendering it possible for technology developers to share or sell patients’ data to third parties [83]. For digital therapeutics to be utilized to its full potential, the patients and the healthcare system must be able to trust the digital technology with the data that they manifest. However, privacy and security risk is another challenge that must be overcome to improve healthcare delivery.
Currently, the regulatory framework of digital therapeutics is highly ambiguous. Some have suggested that digital technology should not necessitate the standards of evidence-based medicine as new drugs should. The process of evidence-based medicine is slow and struggles to keep pace with the dynamic nature of digital technology, which may stifle the development and delivery of the technology. Others responded that digital therapeutics should not be exempted from the standards required of other treatments. An objective evidence-based evaluation of digital therapeutics is required to set the standards for developing digital therapeutics in the future [86].
Conclusions
Current studies on digital therapeutics in stroke rehabilitation were reviewed herein. Previous studies and reviews have successfully demonstrated the benefits of using different digital technologies as rehabilitation tools and the feasibility of digital therapeutics as the rehabilitation training of neurological deficits in stroke patients. As the concept of digital therapeutics is relatively new, the development of digital therapeutics is still at its infancy. Digital therapeutics has the potential to develop both quantitatively and qualitatively, as it as a promising therapy for the rehabilitation for stroke patients. “Clinically validated” digital therapeutics should be developed to prove their efficacy in the prevention and treatment of various neurological disorders.
Acknowledgments
This research was supported by a grant of the Korea Health Technology R & D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health and Welfare, Republic of Korea (grant number: HI18C2383).
Footnotes
Disclosure
The authors have no financial conflicts of interest.
References
- 1.Topol E. Digital medicine: empowering both patients and clinicians. Lancet. 2016;388:740–741. doi: 10.1016/S0140-6736(16)31355-1. [DOI] [PubMed] [Google Scholar]
- 2.Fairburn CG, Patel V. The impact of digital technology on psychological treatments and their dissemination. Behav Res Ther. 2017;88:19–25. doi: 10.1016/j.brat.2016.08.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kaufman N, Ferrin C, Sugrue D. Using digital health technology to prevent and treat diabetes. Diabetes Technol Ther. 2019;21(S1):S79–S94. doi: 10.1089/dia.2019.2506. [DOI] [PubMed] [Google Scholar]
- 4.Cho CH, Lee HJ. Could digital therapeutics be a game changer in psychiatry? Psychiatry Investig. 2019;16:97–98. doi: 10.30773/pi.2019.01.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Panayi ND, Mars MM, Burd R. The promise of digital (mobile) health in cancer prevention and treatment. Future Oncol. 2013;9:613–617. doi: 10.2217/fon.13.42. [DOI] [PubMed] [Google Scholar]
- 6.Heeren A. Commentary: the impact of digital technology on psychological treatments and their dissemination. Front Psychol. 2018;9:1571. doi: 10.3389/fpsyg.2018.01571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Johnston SC, Mendis S, Mathers CD. Global variation in stroke burden and mortality: estimates from monitoring, surveillance, and modelling. Lancet Neurol. 2009;8:345–354. doi: 10.1016/S1474-4422(09)70023-7. [DOI] [PubMed] [Google Scholar]
- 8.Stocco L, Valentin S. Can digital games be a way of improving the neuroplasticity in stroke damage? Can the adult brain grow new cells or rewire itself in response to a new experience? Open J Med Psychol. 2017;6:153–165. [Google Scholar]
- 9.Fu MJ, Knutson JS, Chae J. Stroke rehabilitation using virtual environments. Phys Med Rehabil Clin N Am. 2015;26:747–757. doi: 10.1016/j.pmr.2015.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cramer SC. Treatments to promote neural repair after stroke. J Stroke. 2018;20:57–70. doi: 10.5853/jos.2017.02796. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dobkin BH, Dorsch A. New evidence for therapies in stroke rehabilitation. Curr Atheroscler Rep. 2013;15:331. doi: 10.1007/s11883-013-0331-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Kvedar JC, Fogel AL, Elenko E, Zohar D. Digital medicine’s march on chronic disease. Nat Biotechnol. 2016;34:239–246. doi: 10.1038/nbt.3495. [DOI] [PubMed] [Google Scholar]
- 13.Ameer K, Ali K. iPad use in stroke neuro-rehabilitation. Geriatrics (Basel) 2017;2 doi: 10.3390/geriatrics2010002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Suchak P, Ameer K, Gaylard J, Newman J, Ali K. Improving patients experience in a neuro-rehabilitation unit using tablet computers. Eur J Pers Cent Healthc. 2016;4:289–295. [Google Scholar]
- 15.Loetscher T, Lincoln NB. Cognitive rehabilitation for attention deficits following stroke. Cochrane Database Syst Rev. 2013;5:CD002842. doi: 10.1002/14651858.CD002842.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Chung C, Pollock A, Campbell T, Durward B, Hagen S. Cognitive rehabilitation for executive dysfunction in adults with stroke or other adult nonprogressive acquired brain damage. Stroke. 2013;44:e77–e78. doi: 10.1161/STROKEAHA.113.002335. [DOI] [PubMed] [Google Scholar]
- 17.das Nair R, Cogger H, Worthington E, Lincoln NB. Cognitive rehabilitation for memory deficits after stroke. Cochrane Database Syst Rev. 2016;9:CD002293. doi: 10.1002/14651858.CD002293.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kühn S, Gleich T, Lorenz RC, Lindenberger U, Gallinat J. Playing Super Mario induces structural brain plasticity: gray matter changes resulting from training with a commercial video game. Mol Psychiatry. 2014;19:265–271. doi: 10.1038/mp.2013.120. [DOI] [PubMed] [Google Scholar]
- 19.Cogmed Working Memory Training Pearson Education. https://www.cogmed.com. 2016. Accessed September 23, 2019.
- 20.Westerberg H, Jacobaeus H, Hirvikoski T, Clevberger P, Ostensson ML, Bartfai A, et al. Computerized working memory training after stroke: a pilot study. Brain Inj. 2007;21:21–29. doi: 10.1080/02699050601148726. [DOI] [PubMed] [Google Scholar]
- 21.Lundqvist A, Grundström K, Samuelsson K, Rönnberg J. Computerized training of working memory in a group of patients suffering from acquired brain injury. Brain Inj. 2010;24:1173–1183. doi: 10.3109/02699052.2010.498007. [DOI] [PubMed] [Google Scholar]
- 22.Johansson B, Tornmalm M. Working memory training for patients with acquired brain injury: effects in daily life. Scand J Occup Ther. 2012;19:176–183. doi: 10.3109/11038128.2011.603352. [DOI] [PubMed] [Google Scholar]
- 23.Akerlund E, Esbjörnsson E, Sunnerhagen KS, Björkdahl A. Can computerized working memory training improve impaired working memory, cognition and psychological health? Brain Inj. 2013;27:1649–1657. doi: 10.3109/02699052.2013.830195. [DOI] [PubMed] [Google Scholar]
- 24.Richter KM, Mödden C, Eling P, Hildebrandt H. Working memory training and semantic structuring improves remembering future events, not past events. Neurorehabil Neural Repair. 2015;29:33–40. doi: 10.1177/1545968314527352. [DOI] [PubMed] [Google Scholar]
- 25.Des Roches CA, Balachandran I, Ascenso EM, Tripodis Y, Kiran S. Effectiveness of an impairment-based individualized rehabilitation program using an iPad-based software platform. Front Hum Neurosci. 2015;8:1015. doi: 10.3389/fnhum.2014.01015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Godlove J, Anantha V, Advani M, Des Roches C, Kiran S. Comparison of therapy practice at home and in the clinic: a retrospective analysis of the constant therapy platform data set. Front Neurol. 2019;10:140. doi: 10.3389/fneur.2019.00140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.De Luca R, Russo M, Naro A, Tomasello P, Leonardi S, Santamaria F, et al. Effects of virtual reality-based training with BTs-Nirvana on functional recovery in stroke patients: preliminary considerations. Int J Neurosci. 2018;128:791–796. doi: 10.1080/00207454.2017.1403915. [DOI] [PubMed] [Google Scholar]
- 28.Galletta EE, Barrett AM. Impairment and functional interventions for aphasia: having it all. Curr Phys Med Rehabil Rep. 2014;2:114–120. doi: 10.1007/s40141-014-0050-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Des Roches CA, Kiran S. Technology-based rehabilitation to improve communication after acquired brain injury. Front Neurosci. 2017;11:382. doi: 10.3389/fnins.2017.00382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Doogan C, Dignam J, Copland D, Leff A. Aphasia recovery: when, how and who to treat? Curr Neurol Neurosci Rep. 2018;18:90. doi: 10.1007/s11910-018-0891-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kiran S, Thompson CK. Neuroplasticity of language networks in aphasia: advances, updates, and future challenges. Front Neurol. 2019;10:295. doi: 10.3389/fneur.2019.00295. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Aphasia Software Steps Consulting Ltd] https://aphasiasoftware.com. 2019. Accessed September 23, 2019.
- 33.Palmer R, Enderby P, Cooper C, Latimer N, Julious S, Paterson G, et al. Computer therapy compared with usual care for people with long-standing aphasia poststroke: a pilot randomized controlled trial. Stroke. 2012;43:1904–1911. doi: 10.1161/STROKEAHA.112.650671. [DOI] [PubMed] [Google Scholar]
- 34.Van Mourik M, Van De Sandt-Koenderman WME. Multicue. Aphasiology. 1992;6:179–183. [Google Scholar]
- 35.Doesborgh S, Van De Sandt-Koenderman M, Dippel D, Van Harskamp F, Koudstaal P, Visch-Brink E. Cues on request: the efficacy of multicue, a computer program for wordfinding therapy. Aphasiology. 2004;18:231–222. [Google Scholar]
- 36.Fink RB, Brecher A, Schwartz MF, Robey RR. A computer-implemented protocol for treatment of naming disorders: evaluation of clinician-guided and partially self-guided instruction. Aphasiology. 2002;16:1061–1086. [Google Scholar]
- 37.Ramsberger G, Marie B. Self-administered cued naming therapy: a single-participant investigation of a computerbased therapy program replicated in four cases. Am J Speech Lang Pathol. 2007;16:343–358. doi: 10.1044/1058-0360(2007/038). [DOI] [PubMed] [Google Scholar]
- 38.Steele RD, Baird A, McCall D, Haynes L. Combining teletherapy and on-line language exercises in the treatment of chronic aphasia: an outcome study. Int J Telerehabil. 2015;6:3–20. doi: 10.5195/ijt.2014.6157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Stark BC, Warburton EA. Improved language in chronic aphasia after self-delivered iPad speech therapy. Neuropsychol Rehabil. 2018;28:818–831. doi: 10.1080/09602011.2016.1146150. [DOI] [PubMed] [Google Scholar]
- 40.Kurland J, Wilkins AR, Stokes P. iPractice: piloting the effectiveness of a tablet-based home practice program in aphasia treatment. Semin Speech Lang. 2014;35:51–63. doi: 10.1055/s-0033-1362991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Thompson CK, Choy JJ, Holland A, Cole R. Sentactics®: computer-automated treatment of underlying forms. Aphasiology. 2010;24:1242–1266. doi: 10.1080/02687030903474255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Software Products for People with Aphasia Shirley Ryan Ability Lab. https://www.sralab.org/oral-reading-languageaphasia-orla. 2019. Accessed September 23, 2019.
- 43.Cherney LR. Oral reading for language in aphasia (ORLA): evaluating the efficacy of computer-delivered therapy in chronic nonfluent aphasia. Top Stroke Rehabil. 2010;17:423–431. doi: 10.1310/tsr1706-423. [DOI] [PubMed] [Google Scholar]
- 44.Cherney LR, van Vuuren S. Telerehabilitation, virtual therapists, and acquired neurologic speech and language disorders. Semin Speech Lang. 2012;33:243–257. doi: 10.1055/s-0032-1320044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Cherney LR. Summary: aphasia treatment over the internet: a randomized placebo-controlled clinical trial. http://aphasiology.pitt.edu/2326/1/151-249-1-RV-Cherney.pdf. 2011. Accessed September 23, 2019.
- 46.Fridriksson J, Hubbard HI, Hudspeth SG, Holland AL, Bonilha L, Fromm D, et al. Speech entrainment enables patients with Broca’s aphasia to produce fluent speech. Brain. 2012;135(Pt 12):3815–3829. doi: 10.1093/brain/aws301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Brazzelli M, Saunders DH, Greig CA, Mead GE. Physical fitness training for stroke patients. Cochrane Database Syst Rev. 2011;11:CD003316. doi: 10.1002/14651858.CD003316.pub4. [DOI] [PubMed] [Google Scholar]
- 48.Knecht S, Hesse S, Oster P. Rehabilitation after stroke. Dtsch Arztebl Int. 2011;108:600–606. doi: 10.3238/arztebl.2011.0600. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Adamovich SV, Fluet GG, Tunik E, Merians AS. Sensorimotor training in virtual reality: a review. NeuroRehabilitation. 2009;25:29–44. doi: 10.3233/NRE-2009-0497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Sense Your Progress Jintronix. http://www.jintronix.com. 2019. Accessed September 23, 2019.
- 51.Bird ML, Cannell J, Callisaya ML, Moles E, Rathjen A, Lane K, et al. “FIND technology”: investigating the feasibility, efficacy and safety of controller-free interactive digital rehabilitation technology in an inpatient stroke population: study protocol for a randomized controlled trial. Trials. 2016;17:203. doi: 10.1186/s13063-016-1318-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Cannell J, Jovic E, Rathjen A, Lane K, Tyson AM, Callisaya ML, et al. The efficacy of interactive, motion capture-based rehabilitation on functional outcomes in an inpatient stroke population: a randomized controlled trial. Clin Rehabil. 2018;32:191–200. doi: 10.1177/0269215517720790. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.IREX is Gesturetek Health’s flagship product for the healthcare arena, with over 750 units installed in rehabilitation facilities worldwide. Gesturetek Health. http://www.gesturetekhealth.com/node/27. 2016. Accessed September 23, 2019.
- 54.McEwen D, Taillon-Hobson A, Bilodeau M, Sveistrup H, Finestone H. Virtual reality exercise improves mobility after stroke: an inpatient randomized controlled trial. Stroke. 2014;45:1853–1855. doi: 10.1161/STROKEAHA.114.005362. [DOI] [PubMed] [Google Scholar]
- 55.Subramanian SK, Lourenço CB, Chilingaryan G, Sveistrup H, Levin MF. Arm motor recovery using a virtual reality intervention in chronic stroke: randomized control trial. Neurorehabil Neural Repair. 2013;27:13–23. doi: 10.1177/1545968312449695. [DOI] [PubMed] [Google Scholar]
- 56.Broeren J, Rydmark M, Sunnerhagen KS. Virtual reality and haptics as a training device for movement rehabilitation after stroke: a single-case study. Arch Phys Med Rehabil. 2004;85:1247–1250. doi: 10.1016/j.apmr.2003.09.020. [DOI] [PubMed] [Google Scholar]
- 57.Ikbali Afsar S, Mirzayev I, Umit Yemisci O, Cosar Saracgil SN. Virtual reality in upper extremity rehabilitation of stroke patients: a randomized controlled trial. J Stroke Cerebrovasc Dis. 2018;27:3473–3478. doi: 10.1016/j.jstrokecerebrovasdis.2018.08.007. [DOI] [PubMed] [Google Scholar]
- 58.Emmerson KB, Harding KE, Taylor NF. Home exercise programmes supported by video and automated reminders compared with standard paper-based home exercise programmes in patients with stroke: a randomized controlled trial. Clin Rehabil. 2017;31:1068–1077. doi: 10.1177/0269215516680856. [DOI] [PubMed] [Google Scholar]
- 59.Carabeo CGG, Dalida CMM, Padilla EMZ, Rodrigo MMT. Stroke patient rehabilitation: a pilot study of an androidbased game. Simul Gaming. 2014;45:151–166. [Google Scholar]
- 60.Halfbrick Studios https://halfbrick.com. 2019. Accessed September 23, 2019.
- 61.BinaryLabs, Inc; The dexteria family of apps for occupational therapy and education: iPad apps for fine motor skills, autism, and dyslexia. http://www.dexteria.net. 2019. Accessed September 23, 2019. [Google Scholar]
- 62.Magee WL, Clark I, Tamplin J, Bradt J. Music interventions for acquired brain injury. Cochrane Database Syst Rev. 2017;1:CD006787. doi: 10.1002/14651858.CD006787.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Schneider S, Schönle PW, Altenmüller E, Münte TF. Using musical instruments to improve motor skill recovery following a stroke. J Neurol. 2007;254:1339–1346. doi: 10.1007/s00415-006-0523-2. [DOI] [PubMed] [Google Scholar]
- 64.Rodriguez-Fornells A, Rojo N, Amengual JL, Ripollés P, Altenmüller E, Münte TF. The involvement of audio-motor coupling in the music-supported therapy applied to stroke patients. Ann N Y Acad Sci. 2012;1252:282–293. doi: 10.1111/j.1749-6632.2011.06425.x. [DOI] [PubMed] [Google Scholar]
- 65.Kirk P, Grierson M, Bodak R, Ward N, Brander F, Kelly K, et al., editors. Motivating stroke rehabilitation through music. CHI ‘16 Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems; 2016 May 7-12; San Jose, CA. New York. 2016. pp. 1781–1785. [Google Scholar]
- 66.Roberts PS, Rizzo JR, Hreha K, Wertheimer J, Kaldenberg J, Hironaka D, et al. A conceptual model for vision rehabilitation. J Rehabil Res Dev. 2016;53:693–704. doi: 10.1682/JRRD.2015.06.0113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Hanna KL, Hepworth LR, Rowe FJ. The treatment methods for post-stroke visual impairment: a systematic review. Brain Behav. 2017;7:e00682. doi: 10.1002/brb3.682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Grunda T, Marsalek P, Sykorova P. Homonymous hemianopia and related visual defects: restoration of vision after a stroke. Acta Neurobiol Exp (Wars) 2013;73:237–249. doi: 10.55782/ane-2013-1933. [DOI] [PubMed] [Google Scholar]
- 69.Howard C, Rowe FJ. Adaptation to poststroke visual field loss: a systematic review. Brain Behav. 2018;8:e01041. doi: 10.1002/brb3.1041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Huxlin KR, Cavanaugh MR. Author response: visual discrimination training improves Humphrey perimetry in chronic cortically induced blindness. Neurology. 2018;90:437. doi: 10.1212/WNL.0000000000005024. [DOI] [PubMed] [Google Scholar]
- 71.NeuroEyeCoach-Vision compensation therapy: home-based systematic eye movement training program for stroke and brain injury patients. NovaVision https://novavision.com/neuroeyecoach. 2019. Accessed September 23, 2019.
- 72.Sahraie A, Smania N, Zihl J. Use of NeuroEyeCoach™ to improve eye movement efficacy in patients with homonymous visual field loss. Biomed Res Int. 2016;2016:5186461. doi: 10.1155/2016/5186461. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Aimola L, Lane AR, Smith DT, Kerkhoff G, Ford GA, Schenk T. Efficacy and feasibility of home-based training for individuals with homonymous visual field defects. Neurorehabil Neural Repair. 2014;28:207–218. doi: 10.1177/1545968313503219. [DOI] [PubMed] [Google Scholar]
- 74.Sato G, Villani G, Piccolo E, Tiso F. Modified visiocoach training in hemianopia. Invest Ophthalmol Vis Sci. 2014;55:4132. [Google Scholar]
- 75.Nelles G, Pscherer A, de Greiff A, Forsting M, Gerhard H, Esser J, et al. Eye-movement training-induced plasticity in patients with post-stroke hemianopia. J Neurol. 2009;256:726–733. doi: 10.1007/s00415-009-5005-x. [DOI] [PubMed] [Google Scholar]
- 76.Kasten E, Sabel BA. Visual field enlargement after computer training in brain-damaged patients with homonymous deficits: an open pilot trial. Restor Neurol Neurosci. 1995;8:113–127. doi: 10.3233/RNN-1995-8302. [DOI] [PubMed] [Google Scholar]
- 77.Kasten E, Wüst S, Behrens-Baumann W, Sabel BA. Computer-based training for the treatment of partial blindness. Nat Med. 1998;4:1083–1087. doi: 10.1038/2079. [DOI] [PubMed] [Google Scholar]
- 78.Horton JC. Disappointing results from Nova Vision’s visual restoration therapy. Br J Ophthalmol. 2005;89:1–2. doi: 10.1136/bjo.2004.058214. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Reinhard J, Schreiber A, Schiefer U, Kasten E, Sabel BA, Kenkel S, et al. Does visual restitution training change absolute homonymous visual field defects? A fundus controlled study. Br J Ophthalmol. 2005;89:30–35. doi: 10.1136/bjo.2003.040543. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Schreiber A, Vonthein R, Reinhard J, Trauzettel-Klosinski S, Connert C, Schiefer U. Effect of visual restitution training on absolute homonymous scotomas. Neurology. 2006;67:143–145. doi: 10.1212/01.wnl.0000223338.26040.fb. [DOI] [PubMed] [Google Scholar]
- 81.Das A, Tadin D, Huxlin KR. Beyond blindsight: properties of visual relearning in cortically blind fields. J Neurosci. 2014;34:11652–11664. doi: 10.1523/JNEUROSCI.1076-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Huxlin KR, Martin T, Kelly K, Riley M, Friedman DI, Burgin WS, et al. Perceptual relearning of complex visual motion after V1 damage in humans. J Neurosci. 2009;29:3981–3991. doi: 10.1523/JNEUROSCI.4882-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Steinhubl SR, Muse ED, Topol EJ. The emerging field of mobile health. Sci Transl Med. 2015;7:283rv3. doi: 10.1126/scitranslmed.aaa3487. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med. 2013;10:e1001362. doi: 10.1371/journal.pmed.1001362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Prgomet M, Georgiou A, Westbrook JI. The impact of mobile handheld technology on hospital physicians’ work practices and patient care: a systematic review. J Am Med Inform Assoc. 2009;16:792–801. doi: 10.1197/jamia.M3215. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 86.Greaves F, Joshi I, Campbell M, Roberts S, Patel N, Powell J. What is an appropriate level of evidence for a digital health intervention? Lancet. 2019;392:2665–2667. doi: 10.1016/S0140-6736(18)33129-5. [DOI] [PubMed] [Google Scholar]