Abstract
Purpose
Smartphone applications (SPApps) have become a key tool for the self-management of low back pain (LBP). However, the scientific evidence behind the outcome measures used in SPApps for LBP is never investigated before. Therefore, this systematic review firstly assess the quality of the free SPApps for LBP, secondly examines the outcome measures used and thirdly evaluates the outcome measures against the International Classification of Functioning, Disability and Health (ICF) core set classifications for LBP.
Methods
A systematic scoping review was conducted in the iTunes and Google Play™ on-line stores for LBP SPApps which are free to download. These searches were conducted using keywords suggested by the Cochrane Back and Neck Group. SPApps were screened and downloaded to assess the quality using the Mobile App Rating Scale (MARS). SPApps using outcome measures were reviewed separately to evaluate whether their outcome measures represented any of the ICF components for LBP.
Results
The overall quality of the apps has a mean MARS score of 2.5/5. Out of 74 apps reviewed, only four apps had outcome measures that could be linked to ICF components for LBP. Two of the four categories comprising the LBP core set were well represented.
Conclusion
The overall quality of the SPApps for LBP is low. Only very few SPApps offer outcome measures to monitor their effectiveness in the management of LBP. There is very limited evidence to show that the outcome measures used in the apps represents all the four core sets of LBP criteria set by ICF.
Keywords: Low back pain, Mobile health, Outcome measure, Care quality, Patient
Introduction
According to the reports from Global Burden of Disease 2010, low back pain (LBP) is the greatest cause of disability and the greatest contributor to overall disease burden [1]. The Health and Safety Executive (2017) estimated that 3.2 million working days were lost due to back disorders in the UK between 2016 and 2017 with the yearly total costs attributable to LBP have been estimated at £12.3 billion [2, 3]. The National Institute for Health and Care Excellence (NICE) clinical guidelines recommend that, for a long-term solution, patients should be provided with advice and information to help them self-manage their LBP [4]. More recently, smartphone application (SPApp) is recommended as one of the management strategies to effectively engage the patients in the self-management of LBP [5, 6].
Mobile health (mHealth) is the term used for the practice of medicine and public health supported by mobile devices through SPApps [7]. mHealth has been suggested to be an efficient way of delivering healthcare services through health care apps for a variety of patient populations, particularly those on a low income or in less accessible locations [8, 9]. While the patients and healthcare professionals are increasingly using health-related applications, concerns about ethical issues, bias, conflicts of interest and privacy are emerging [8]. The NICE, (2016) low back pain guidelines advocate directing LBP care based on a person’s multidimensional risk profile and considering psychosocial factors from the start of an intervention [10]. However, there is no reported scientific evidence on how many of the SPApps designed to provide LBP care have considered the patients multidimensional risk profile in the design and development of mHealth applications.
The International Classification of Functioning, Disability and Health (ICF) framework (World Health Organization, 2001) suggests that capturing patients multidimensional disease profile by measuring how people with a health condition such as LBP function in their daily life is more important than focusing on their diagnosis or the presence or absence of disease [11]. The ICF combines domains of function, disability and health into a core set of a useful and valid framework to help structure rehabilitation plans and patient goal setting in adults with musculoskeletal conditions such as LBP [12]. Based on the typical spectrum of problems in functioning among people with LBP, the ICF framework presents a comprehensive ICF core set for LBP which includes 79 ICF domains and classifies it under four categories viz., body structures, body functions, activity limitations and participation restrictions, and environmental factors [13, 14]. Therefore, it would be appropriate to consider these domains as outcome measures for LBP management in the SPApps related with LBP management. Currently, it is not known whether the outcome measures in the LBP SPApps consider the ICF domains in smart phone based management of LBP.
The mHealth applications are used by 76% of the smartphone owner populations [15] and approximately an average of 7481 new health and fitness apps are released every day by the iTunes and Google Play™ [16]. Almost 90% of the SPApps are free to download and use, and it has been reported that free apps have a much higher download rate than paid applications [16]. There is very little regulatory control over the content and the quality of these SPApps have not been evaluated [17]. Therefore, the current review intends to focus on the free SPApps for LBP management. For people who suffer from LBP, a SPApp may help patients to manage the low back pain in several ways according to the different operating features available in the SPApp. According to the design and the features, a SPApp may provide patient education, create health awareness, evaluate body function, monitor the progress of the disease and advocate exercises to the patients to manage the LBP. Figure 1 illustrates the different underlying features and explains the rationale behind how SPApp works in LBP management. For example, a patient with LBP can learn different types of pain-relieving exercises from a SPApp named MSK Help and the patient can perform the exercises to help themselves to manage the LBP. Another example is where a SPApp named Lumbago evaluates the body function among LBP patients through measuring different functional activities such as lifting, carrying, walking, etc. Thus, one can understand how LBP affects an individual’s day to day functions. Therefore, the clinical guidelines recommend SPApps as an efficient approach to deliver health care services for patients with LBP [4, 10]. A recent systematic review on LBP SPApps confirmed lack of research into the effectiveness of the interventions and the outcome measures that these apps offer [6], and another systematic review reports a large gap between the scientific and commercial faces of the SPApps used in pain management [8]. No systematic review was found that had appraised the outcome measures used in SPApps for management of LBP. Therefore, the main aims of this systematic review are (i) to assess the quality of the free SPApps for LBP in terms of the design and function, (ii) to examine the outcome measures used by smartphone apps in the management of LBP and (iii) to determine whether the outcome measures characterise ICF core sets. The outcome of this review may help inform and guide health professionals and people with LBP in making informed choices when selecting SPApps for the management of LBP.
Fig. 1.

Features and rationale on how SPApps works in management of LBP
Methods
Study design
This systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [18] with a three steps review design: Step one; the SPApps meeting the prior broad inclusion criteria were assessed for the quality of their content. Step two; outcome measures were extracted from the relevant SPApps and described in detail. Step three; linking the concepts contained within the outcome measures of the SPApps to the corresponding categories of the ICF comprehensive core set for LBP. As none of the free SPApps for LBP have been published in the scientific databases, a systematic review of the app platforms instead of the scientific databases was adopted as a pragmatic approach to evaluate the outcome measures in the SPApps. The PRISMA guidelines was a recommended methodology to systematically search, retrieve and analyse the scientific evidence and hence, the protocol from the PRISMA was used in this review [18].
Search strategy
A comprehensive search strategy was developed and tested in a pilot search, for identifying SPApps relevant to this review. In October to November 2018 a systematic search of the Apple iTunes and Google Play™ stores was conducted following the PRISMA guidelines for systematic literature reviews [19]. The search terms used were selected from key words recommended by the Cochrane Back and Neck Group [20]. The following search terms were employed; “Back pain” OR “Low back pain” OR “Lumbago” OR “Spinal pain” and a comprehensive list of LBP related mobile apps was created. The Android apps were identified by a search of the Google Play™ store via the internet web page and the Apple iTunes apps were identified via the iPhone operating system (iOS®) App Store® search app on the iPhone. To ensure saturation, apps listed from a previous systematic review was identified and screened manually [17]. The review authors only had access to United Kingdom app stores and so the searches were limited to the UK. The initial screening of the SPApps was conducted based on the name and description of the apps against the inclusion criteria to identify potentially relevant apps. Apps that met this inclusion criteria were recorded and then downloaded on to their respective devices (iPhone 5S iOS® 11.1.2 and Nexus 5X Android 8.0) for a full evaluation of their content.
Inclusion criteria
The SPApps designed to be used in the management of LBP only were included for the review. Apps were also to provide a self-management strategy, which is a key recommendation in the current clinical guidelines provided by NICE [4]. The type of self-management intervention offered or the expected frequency of intervention was not restricted. Interventions that offer educational information or advice were also included as this is considered a self-management strategy for LBP. Interventions could also be of a combined nature, such as education and exercise, with no limit to the number of combinations and strategies advised. Apps must be offered in the English language and available in the UK Apple iTunes and Google Play™ stores, be free of charge and targeted for any age groups. All apps must be published or updated from 2016 or later to ensure they were compatible with the latest updates on Android; Android 8.0 Oreo™, and iPhone; iOS® 11.0, to guarantee full functionality. Apps must also be a self-contained product i.e. apps must not require another app to access their full content.
Exclusion criteria
Apps were excluded if they have in app charges that restricts the full content of the app. Apps that were unrelated to LBP management i.e. apps for booking appointments were excluded. If only general information about LBP, such as anatomy and common risk factors were provided by the app, they were excluded as they did not provide a management plan to be followed. Apps that offered treatment for ankylosing spondylitis or other medical health conditions, or apps for general health and wellbeing were also excluded. Where the same app was available on both platforms (iOS® and Android), only one version of the app was kept for analysis on the iOS® platform. This decision was made as the iOS® platform requires a more stringent set of guidelines to be followed for publishing (Apple, 2018) [21]. Table 1 shows a summary of the inclusion and exclusion criteria for the selection of the LBP SPApps.
Table 1.
Inclusion and exclusion criteria for the selection of the SPApps for the review
| Inclusion criteria | Exclusion criteria |
|---|---|
|
• Provided a self-management strategy for low back pain • Designed for patient use • Targeted at patients of any aged groups • Written in English • Free of charge • Self-contained product • Available on the UK Apple iTunes and Google Play stores • Compatible with the latest software updates |
• Apps that are unrelated to LBP management • Appointment booking apps • Apps designed for healthcare practitioners • General health and wellbeing apps • Apps with in-app purchases that restrict the full content of the app |
Data extraction and analysis
The lead reviewer performed the data extraction independently, and any concerns regarding app eligibility were discussed and reflected upon with the other reviewers. The information extracted from the apps included; the name of the app, version, developer, update date and platform availability. Where available, the number of consumer reviews and the average rating, given as a 5-star rating system, was also extracted. This data was recorded on an electronic spread sheet (Microsoft Excel, 2016) and duplicated apps, available on both platforms, were identified and removed. Whenever an app offered an outcome measure, a description and type of outcome measure was also extracted.
Quality assessment of apps
The quality of the SPApps were assessed using the Mobile Application Rating Scale (MARS) which has five broad categories that include engagement (fun, interesting, customisable, interactive and well targeted to the chosen audience), functionality (functioning, easy to use, navigation, logical flow, and gestural design of the app), aesthetics, (graphic design, overall visual appeal, colour scheme, and stylistic consistency), information quality (quality and quantity of information, credibility of developer) and an overall quality scale [22]. The apps were assessed independently by one trained reviewer using the MARS to classify and assess the quality of the mHealth apps. Apps were scored in subcategories for the quality of their engagement, functionality, aesthetics and quality of information and a mean score was recorded as the rating for that category. The overall total for the app was then calculated by finding a mean of the mean scores. Each MARS item used a 5-point scale (1-Inadequate, 2-Poor, 3-Acceptable, 4-Good, 5-Excellent) [22], therefore, the higher the score, out of five, the higher the quality of the app. Data was presented as a mean score for each category and the total score of the MARS for each app.
Prior to the quality rating of the SPApps, ten randomly selected apps were assessed at two separate time points to determine the intra-rater reliability of the reviewer who reviewed the app. Intraclass correlation coefficient (ICC2,1) estimates with a 95% confidence interval were calculated using the Statistical Package for the Social Science (SPSS) for Windows version 24.0 (SPSS Inc., Chicago, IL, USA). For intra-rater reliability analyses, ICC(2,1) values were interpreted as 0.90–1.00 = very high reliability, 0.70–0.89 = high reliability, 0.50–0.69 = moderate reliability, 0.26–0.49 = low reliability and 0.00–0.25 = little, if any reliability [23].
Procedure of assessing the outcome measures in SPApps with ICF core sets for LBP
The ICF comprehensive core set for LBP includes the categories of body functions (physiological and psychological functions of the body systems), body structures (anatomical parts of the body), activities (execution of a task or an action) and participation (involvement in a life situation), and environmental factors (the social, physical and attitudinal environment that people live in). The linking process was performed following a previously established protocol [13] and the outcome measures in the SPApps were assessed using the linking concepts of ICF LBP core sets. The outcome measures used by the apps were extracted and defined and this descriptive data was recorded on an electronic spread sheet (Microsoft Excel, 2016). Then, the outcome measure questions were identified and linked to the most appropriate ICF domain within the LBP comprehensive core set. The outcome measures from the SPApps were coded and recorded in a table according to the categories in the comprehensive core set for LBP from the ICF framework [24, 25]. The overall percentages for compliance of the SPApps was calculated to demonstrate the degree to which the app has covered each domain and therefore to what degree they reflect the ICF model of health outcomes.
Results
Search results
The search yielded a total of 897 SPApps out of which a total of 120 apps were identified to be eligible for further evaluation after exclusion of 777 applications. The remaining 120 apps were downloaded on to their respective devices for full evaluation with a further 46 apps being excluded at this point. After further evaluation, a total of 74 apps were included in this review. The flow chart for the selection of LBP SPApps was shown in the Fig. 2. Of the 74 apps included in this review, 9 were available exclusively on iTunes and 56 were available only on Google Play™ with 9 apps available on both stores When each of these 74 applications were individually evaluated, the final results revealed that only 4/74 (5.4%) included outcome measures as part of the applications.
Fig. 2.
Flowchart of the selection of smartphone apps for LBP
Quality assessment of the LBP SPApps
The intra-rater reliability analysis showed a very high reliability for the quality assessment of the LBP SPApps by the reviewer with the ICC(2,1)—0.97. The mean MARS total score, on a 0–5 scale, for the 74 apps was 2.54 (SD, 0.43) with a range of 1.63–3.57. The result of the quality assessment of the SPApps using the MARS quality rating scale and any available consumer ratings are presented in Table 2. The apps were rated on their engagement, functionality, aesthetics and information, with the lowest mean score for the category of information. The quality of information scored a mean of 1.72 (SD, 0.58) with a range of 0.43–3.71. None of the apps had been trialled or tested in published scientific literature. The second lowest score was for engagement with a mean of 1.77 (SD, 0.57) and a range of 1.0–3.4. The MARS subscale category of aesthetics had a mean score of 2.59 (SD, 0.80) with the largest range of 1.0–4.33. This was influenced by the apps having either high or low graphic resolution and good or no visual appeal in terms of their layout and colouring. The subscale of functionality had the highest mean score of 4.06 (SD 0.43) and the smallest range of 3.0–5.0. While the lowest scoring LBP apps were ‘Yoga exercises (juliusapps)—V.1.0’ and ‘Slipped Disc Exercise—V.1.1’ with the mean MARS scores of 1, the highest total MARS scoring app for LBP was: “NHS 24 MSK help” V. 2.0.4, with a mean score 3.57 and was developed by the National Health Services, United Kingdom. This app scored highly in the categories of Functionality and Information, in which it posted the highest score out of the 74 apps.
Table 2.
The mean scores and consumer rating of the LBP SPApps evaluated by the Mobile Application Rating Scale (MARS)
| App name-version (developer) | Engagement | Functionality | Aesthetics | Information | Total mean score | Consumer rating |
|---|---|---|---|---|---|---|
| IREHAB Back Pain—V.1.4.4a | 2.6 | 5 | 4.33 | 1.71 | 3.41 | 4.3 |
| 6 Minute Back Pain Relief—V.1.02b | 2.6 | 4 | 3 | 2 | 2.90 | – |
| NHS 24 MSK help—V.2.0.4b | 2.6 | 4.5 | 3.33 | 3.71 | 3.57 | – |
| 10 Min Lower Back Therapy Workout—V.1.5a | 2 | 4 | 2.66 | 1.14 | 2.45 | – |
| Manage Your Health—V.1.08b | 3 | 3.75 | 3 | 3 | 3.19 | – |
| Free Pain Relief Guidea | 1.8 | 3.75 | 2.33 | 1.57 | 2.36 | – |
| Vivira: pain-free movement—V.1.9.4b | 3.4 | 5 | 4.33 | 1.43 | 3.54 | – |
| Straighten Up Canada—V.1.6b | 2.6 | 3.75 | 3 | 1.86 | 2.80 | – |
| Lower Back Challenge Workout—V.1.4a | 2.2 | 3.75 | 2.66 | 1.43 | 2.51 | – |
| Yoga_For_Back_Pain_Reliefa | 1.2 | 4.5 | 1.33 | 1.71 | 2.19 | – |
| Back Doctor Pain Relief—V.1.03.24b | 2.6 | 4.5 | 3 | 2.57 | 3.17 | – |
| Yoga for pain reliefa | 1.2 | 4.5 | 1.33 | 1.71 | 2.19 | – |
| Dr. T—V.1.3a | 1.8 | 4 | 3 | 1.14 | 2.49 | – |
| Exercise Programs—V.1.2.3a | 1.8 | 4 | 3.66 | 1.57 | 2.76 | – |
| Physera—V.3.5b | 3.4 | 3.75 | 4 | 1.43 | 3.15 | – |
| My Back Injury—V.1.3a | 2 | 3.75 | 3.33 | 2.29 | 2.84 | – |
| Lumbago (Ad Supported)b | 2.8 | 4 | 3.33 | 1.71 | 2.96 | – |
| WellMama Lite Post Pregnancyb | 2 | 4.25 | 4 | 1.57 | 2.96 | – |
| Back pain relieving exercises—V.1.0.0 | 1.6 | 5 | 2.33 | 2.29 | 2.80 | 4.5 |
| Lower Back Pain Relief—V.1.0 | 1.6 | 4.5 | 2.33 | 1.14 | 2.39 | 4.4 |
| Healthy Spine and Straight Posture—V.2.3 | 3.2 | 3.5 | 4 | 2.43 | 3.28 | 4.7 |
| Back Pain Relief Yoga Poses (Proven Digital Web Solutions)—V.17.0 | 1.4 | 5 | 3 | 1.86 | 2.81 | 3.8 |
| Upper and Lower Back Pain Relief—V.1.2 | 1.4 | 4.25 | 1.33 | 1.57 | 2.14 | 3.5 |
| Back Pain: Cause and Treatment—V.1.1.0 | 1.4 | 4.25 | 2 | 1.57 | 2.31 | 4.4 |
| Back Pain Protocols—V.0.4 | 1.4 | 4.25 | 2 | 2 | 2.41 | 5 |
| Yoga Poses and Asanas for Relief of Back Pain—V.2.0.2 | 2.36 | 4 | 4 | 1.29 | 2.97 | 4.2 |
| Yoga Poses for Lower Back Pain Relief—V.1.7 | 3.2 | 3.75 | 3.33 | 2.43 | 3.18 | 4.6 |
| Yoga Back Pain Relief 2017—V.2.0 | 2.4 | 4 | 2 | 2.29 | 2.67 | 4.6 |
| Low Backpain Protocols—V.0.3 | 1.6 | 4 | 2 | 2 | 2.40 | 4.5 |
| Back Pain Relief Exercises Videos—V.1.0 | 1.4 | 3.75 | 2 | 1.14 | 2.07 | 3.2 |
| Back Pain Exercises 1—V.2.0 | 1.6 | 4 | 2.67 | 2.71 | 2.75 | 5 |
| Yoga for Back Pain Relief (vishapp)—V.1.0 | 1.4 | 4 | 3 | 2.14 | 2.64 | 5 |
| 30 Day Back Workout Challenge—V.1.0.0G | 1.8 | 4 | 3.33 | 2.14 | 2.82 | 4.3 |
| Back Pain Tips—V.1.0 | 1.6 | 5 | 2.33 | 1.57 | 2.63 | 5 |
| Yoga Poses for Lower Back Pain—V.4.0 | 1.4 | 4 | 2 | 2 | 2.35 | – |
| Back Pain Relief Tips—V.1.0 | 1.4 | 4.75 | 4 | 2.29 | 3.11 | – |
| Slipped Disc Exercise—V.1.1 | 1 | 3.75 | 1.33 | 0.43 | 1.63 | 4.2 |
| Yoga Asanas for Spine and Back—V.1.2 | 2.8 | 4 | 3 | 2 | 2.95 | 4.5 |
| Yoga For Back Pain (ACG apps)—V.1.1 | 1.4 | 4.25 | 3 | 2 | 2.66 | 3.8 |
| Back Pain (Galaxy Studio Digital)—V.1.0 | 1.4 | 4 | 2.66 | 1.71 | 2.45 | – |
| Back Pain (Expert Health Studio)—V.1.0 | 1.4 | 4 | 2.66 | 2 | 2.52 | 2.9 |
| Back Pain (Free Mobile Shop Apps)—V.1.0 | 1.6 | 4 | 3.33 | 1.43 | 2.59 | – |
| 10 Minute Lower Back Workout—V.1.0.0 | 1.6 | 3.25 | 2.66 | 1.45 | 2.24 | 3.7 |
| Back Pain Blueprint—V.1.0 | 1.6 | 3 | 2 | 1.43 | 2.01 | – |
| Back Pain and How To Prevent It—V.40.0 | 1.4 | 4.5 | 2 | 1.43 | 2.33 | 3.4 |
| Back Pain Exercise Guide (Snippy Apps)—V.1.0 | 1.6 | 5 | 3.33 | 2.14 | 3.02 | – |
| Back Pain (NABIOM SOFT)—V.1.0 | 1.6 | 5 | 3.33 | 2.14 | 3.02 | 4 |
| Back Pain (Moe Game)—V.1.0 | 1.4 | 4 | 2 | 1.43 | 2.21 | 2.2 |
| Back Pain (Free Entertainment Apps)—V.1.0 | 1.4 | 4 | 2 | 1.43 | 2.21 | 4 |
| Yoga Stretches for Back Pain—V.1.0 | 1.4 | 3.75 | 1 | 1.29 | 1.86 | 3.7 |
| Yoga for Back Pain Relief (Joey Morque)—V.1.0 | 1.4 | 4 | 1.66 | 1 | 2.02 | – |
| Back Pain: Everything You Need to Know—V.1.21 | 1.2 | 3.75 | 2 | 1.86 | 2.20 | – |
| Treat Back Pain—V.1.0 | 1.6 | 4.25 | 2.33 | 2.14 | 2.58 | – |
| Yoga Poses for Back Pain—V.1.0 | 1.6 | 4.25 | 2.33 | 2.14 | 2.58 | 4.1 |
| Yoga Tips For Back Pain (SnippyApps)—V.2.0 | 1.6 | 4.5 | 3.33 | 2 | 2.86 | – |
| Backpain (Moonlight inc)—V1.1.0 | 1.4 | 4 | 3 | 1.14 | 2.39 | 5 |
| Back Pain Exercise Guide (Noel Barton)—V.2.0 | 1.6 | 3.5 | 1.33 | 0.57 | 1.75 | – |
| Backpain Remedy—V.1.4 | 1.8 | 4 | 2.33 | 1.57 | 2.43 | – |
| Back Pain—causes, symptoms, treatments—V.1.0.0 | 1.6 | 4 | 2 | 2 | 2.40 | – |
| Yoga For Back Pain (Frozenwaves apps)—V.1.4 | 1.4 | 3.25 | 2.66 | 1.71 | 2.26 | – |
| Back Pain Relief Yoga Poses (Henasy)—V.1.0 | 1.6 | 4 | 3.33 | 1.71 | 2.66 | – |
| Lower Back Pain (Sun Media Soft)—V.1.1.0 | 1.6 | 3.75 | 3 | 1.14 | 2.37 | – |
| Back Pain Exercise Guide (Harwell publishing)—V.2.0 | 1.4 | 4 | 2.33 | 0.71 | 2.11 | – |
| Get rid of Back Pain—V.1.0 | 1.6 | 4 | 2.66 | 1.71 | 2.50 | – |
| Back Pain Exercises 2—V.2.0 | 1.6 | 4 | 2.67 | 2.71 | 2.75 | 5 |
| Yoga Tips For Back Pain (Apps Viva)—V.2.0 | 1.4 | 4 | 2 | 0.71 | 2.03 | – |
| Back Pain Exercise Guide (Apps Viva)—V.2.0 | 1.4 | 4 | 2 | 0.71 | 2.03 | 3 |
| Yoga Stretches For Back Pain—V.2.0 | 1.4 | 4 | 2 | 0.71 | 2.03 | – |
| Lower Back Muscle Pain—V.1.0 | 1.6 | 3.75 | 2.66 | 2.14 | 2.54 | – |
| Back Stretches for Pain Relief—V.1.0 | 1.4 | 3.75 | 1 | 1.29 | 1.86 | 3.7 |
| Yoga Lower Back Excercises—V.2.0 | 1.6 | 3.5 | 2 | 2.14 | 2.31 | – |
| Physiotherapy Exercises by Dr. Huma Ibrar Abbasi—V.1.0 | 1.4 | 3.5 | 2 | 2 | 2.23 | 4.1 |
| Yoga exercises (juliusapps)—V.1.0 | 1 | 4 | 1.33 | 0.86 | 1.79 | 4.2 |
| Back pain during pregnancy | 1.4 | 3.5 | 2 | 1.57 | 2.12 | 4.2 |
Apps in bold type were the three highest scoring apps in this review
aApps available exclusively on iTunes
bApps available on iTunes and Google Play™
Characteristics of outcome measures
Out of the 74 apps included in this review only four apps provided the user with outcome measures. The outcome measures used by all four apps are person reported questions and questionnaires, focusing on ratings of pain, activities of daily living (ADLs), and psychological factors. A full description and list of the outcome measures used by the four apps is available in Table 3. Only one app, out of the 74, used an outcome measure that has been published in the scientific literature, the Back Pain Functional Scale. Three of the apps posed questions regarding the patients’ pain. One app posed the question of pain relief, the second app posed the question of the rating of pain, using a sliding scale. The third app covered the location, severity, description, frequency, irritability and relieving factors of the patients’ pain. In terms of the persons’ wellbeing, two apps recorded how the user is feeling. One app required the user to answer by choosing one of the following answers: Good, Tired, Sore, Sick. The other app asked specifically about emotional and physical wellbeing, supplying the options of answering on a scale of one to five with the ratings starting at bad going up to very good. Two apps questioned the impact to the user’s daily work life. One app questioned the user’s views on back pain and work and the other app asked how their work is affected by their back pain. The quality rating of the four apps that provided outcome measures scored a mean total score of 3.31/5 for the MARS. This value was higher than the overall mean value for the 74 apps at 2.54/5.
Table 3.
Outcome measures questions and questionnaires posed by the apps
| App | Outcome measure |
|---|---|
| 1. NHS 24 MSK Help |
Self-help Questionnaire: helping my problem. Answers yes/no 1. Duration; I am off work or struggling to stay at work because of this problem 2. Pain relief; I have no or poor pain relief for this problem 3. Exercise; I would like some advice on what exercises can help with this problem 4. Resting; I would like to know if I should be resting or moving 5. Work; I want to know a bit more about this problem Back Beliefs Calculator: Answers numerical 1–5 scale (completely disagree–completely agree) 1. There is no real treatment for back trouble 2. Back trouble will eventually stop you from working 3. Back trouble means periods of pain for the rest of one’s life 4. Back trouble makes everything in life worse 5. Back trouble may mean you end up in a wheelchair 6. Back trouble means long periods off work 7. Once you have had back trouble there is always a weakness 8. Back trouble must be rested 9. Later in life back trouble get progressively worse |
| 2. Lumbago |
Daily form: Questionnaire 1. Does your pain radiate? Select affected areas 2. How intense is your pain? VAS 1–10 3. What best describes your pain? Select description 4. How often do you experience pain? Constant–Rarely 5. What affects your pain? Select ADLs 6. Do you have trouble with balance? Yes/No 7. What helps alleviate your pain? Select option 8. What percentage of improvement have you experienced? Percentage. Weekly form: The Back Pain Functional Scale: a self-report measure that evaluates functional ability in people with BP. Answers: Graded rating (no difficulty–unable to perform) 1. Usual work, housework, or school activities: How difficult is it to perform the following? 2. Usual hobbies, recreational or sporting activities? 3. Performance of heavy activities around your home? 4. Bending or stooping 5. Putting on shoes or socks 6. Lifting a box of groceries from the floor 7. Sleeping 8. Standing for 1 h 9. Walking for 1 mile 10. Going up or down 2 flights of stairs (about 20 steps) 11. Sitting for 1 h 12. Driving for 1 h |
| 3. Physera |
Questions after workout: 1. How are you feeling? Good/Tired/Sore/Sick 2. Pain? Sliding scale: no pain–worst pain imaginable |
| 4. Vivira: pain-free movement |
Wellbeing questions: 1–5 (bad–very good) 1. How are you emotionally? 2. How are you physically? |
Evaluation of outcome measures in SPApps against the ICF categories
Table 4 shows the results on how each outcome measure was linked to the most appropriate ICF domain from the comprehensive core set for LBP. The LBP core set consists of four categories titled; body structures, body functions, activities and participation, and environmental factors. The aim was to see how well each of these four categories was represented by the outcome measures chosen by each app.
Table 4.
Evaluation findings of the outcome measures used in LBP SPApps against the ICF LBP core set domains
| App | Outcome measure | Question number. ICF domain (coding) | Additional information |
|---|---|---|---|
| MSK Help NHS | Self-help: Questionnaire: Helping my problem |
1. Remunerative employment ((G) d850) Work and employment (d859) |
Duration off work |
| 2. Sensation of pain ((G) b280) | Pain relief | ||
| 3. Recreation and leisure (d920) | Exercise advice | ||
| 4. No coding | |||
| 5. Work and employment (d859) | |||
| Back Beliefs Calculator |
1. Remunerative employment ((G) d850) 2. Work and employment (d859) 3. Sensation of pain ((G) b280) 4.–9. No coding. |
||
| Lumbago | Daily form: Questionnaire | 1.–5. and 7. Sensation of pain ((G) b280) | Radiate, intensity, description |
| 6. Proprioceptive function (b260) | Balance | ||
| 8. No coding | |||
| Weekly Form: The back pain functional Scale |
1. Housework (d640) Work and employment (d859) Remunerative employment ((G) d850) 2. Recreation and Leisure (d920) |
||
| 3. and 6. Lifting and carrying objects (d430) | Heavy activities, lifting groceries | ||
| 5. Dressing (d540) | Sock and shoes | ||
| 7. Sleep functions (b134) | |||
| 8., 9. and 11. Maintaining a body position (d415) | Standing, sitting | ||
| 9. and 10. Gait pattern functions (d770) | |||
| Walking ((G) d450) | Walking 1 mile | ||
| Moving around ((G) d455) | |||
| Exercise tolerance functions (d455) | |||
| Moving around in different locations (d460) | |||
|
12. Hand and arm use (d445) Driving (d475) |
Driving for 1 h | ||
| Physera | Pain: rate from no pain–worst pain imaginable | 2. Sensation of pain ((G) b280) | Rating |
| Vivira |
Wellbeing Log How are you emotionally? How are you physically? |
1. Emotional functions ((G) b152) | Emotionally |
| 2. Experience of self and time functions (b180) | Physically |
(G)—Items from the Generic set
Body functions
All four apps addressed the category of body function for LBP. Three of the apps presented questions that link to the concept of ‘Sensation of pain’. Among the outcome measures, approximately 38% (11/29) of questions were referred to the ‘Sensation of pain when compared to overall questions in the apps. One app, Vivira, provided an outcome measure that did not address the person’s pain but rather asked questions relating to the user’s emotional functions. The Vivira app attempted to establish a link between the questions between the ‘emotional functions’ and the LBP core set specific concept of ‘Experience of self and time functions’.
Body structures
None of the apps addressed the Body structure category from the comprehensive LBP core set. This category covers domains relating to how much impairment the person has in their organs, limbs and their components.
Activity limitations and participation restrictions
Only two of the four apps had questions relating to activity limitations and participation restrictions. However, overall the largest percentage (55% 16/29) of questions were directed at addressing this category. One app that addressed the largest variety of the domains from this category was Lumbago, which assessed three different measures by asking ‘remunerative employment’, ‘moving around’ and ‘walking’. Also, this app assessed many of the specific domains in the core set for LBP, namely; ‘housework’, ‘work and employment’, ‘exercise tolerance’, ‘moving around in different locations’, ‘maintaining a body position’, ‘hand and arm use’, ‘driving’, ‘gait pattern functions’, ‘lifting and carrying objects’, ‘dressing’, ‘changing basic body position’ and ‘recreation and leisure’.
Environmental factors
None of the four apps addressed any of the environmental factors domains in their outcome measures. Overall, only four apps of the 74 included in this review provided outcome measures. All four of these apps had outcome measures that addressed items from the ICF comprehensive core set for LBP. Among these four apps, only one app, Lumbago, addressed specific LBP core set items covering two of the four main categories. Only the Lumbago app scored the most number of measures as indicated by the ICF core set for LBP.
Discussion
Given the rapid construction and accessibility of smartphone apps, it is increasingly difficult for users, health care professionals, and researchers to easily identify and assess the quality of apps. It is even harder for the patients to identify the most appropriate apps to manage their LBP. While there are some information available to understand the effects of intervention from SPApps to manage LBP [6], there is no information available on the quality and types of outcome measures used in the free SPApps for management of LBP. Only few of the evaluated apps included outcome but it was not clear how valid these outcome measures are. In order for health professionals to confidently recommend SPApps for their clients, there was a need to evaluate these measures against the established ICF comprehensive core sets for LBP [24, 25]. Therefore, the current review (i) investigated the quality of the free apps used in self-management of LBP, (ii) examined the outcome measures used by these SPApps and (iii) evaluated these outcome measures against ICF core sets for LBP. The overall findings of this review suggested that the health care professionals and consumers should be aware of the general low quality of free apps available for management of LBP.
The review identified some new key aspects of information in the LBP SPApps that had never been studied before. The outcome of the review contributes valuable information to inform agencies such as US Food and Drug Administration (FDA), the European Medical Device Directive (MDD), Medicines and Healthcare Products Regulatory Agency (MHRA), United Kingdom and other international agencies that regulate SPApps and mobile health devices. It is doubtful that the LBP SPApps that are free to download meet relevant regulatory control procedures and health certification measures. The quality regulations management system and codes of practice to update information provided by the SPApps is not explicitly mentioned in any of the apps description. While some of the free SPApps for LBP may undermine the quality of health care and safety, it is unknown who is responsible for any health liabilities and it is not addressed in any of the SPApps description. Therefore, based on the current review, health care apps regulatory procedures and certification measures for the LBP SPApps is recommended and these regulatory measures need to be written clearly in the description of the SPApps to assist the end-users to make informed choises. Further discussion is focused on highlighting the challenges faced by patients in using SPApps for management of LBP and a rationalize need for thorough scientific research and evidence based approach to enhance the quality of SPApps for management of LBP in clinical practice so that the consumers can make better choices and use better products.
The review findings highlighted some significant challenges present in the free LBP SPApps and the consequent difficulties which the patient might encounter using these apps. Firstly, the sheer volume of 74 SPApps available for LBP management demonstrated the practical difficulties faced by the users or the patients to identify and select an appropriate app to manage their LBP. Secondly, only 3 out of 74 apps scored an acceptable quality score as indicated by MARS score. As the quality of the apps are not usually disclosed by the app developers, it makes it even more challenging for the patients to identify a good quality app. In this context, the findings of the review might help Health care and exercise professionals to consider the quality of the apps and their suitability in terms of established clinical measures when they advise and recommend their clients to use those apps. In other words, the SPApps developers should make the description, contents and quality of the apps clearer to the users and the patients to help them identify and use the effective app. Thirdly, only 4 out of 74 SPApps had included outcome measures that might be meaningful for the patients, but none of these could be linked to all the four categories that form the ICF comprehensive core sets for LBP. This might suggests a limited knowledge by the app developer in this area and it warrants a more comprehensive screening of outcome measures and education for app developers in order for them to fully understand LBP and its impacts on patients’ day to day life events. Perhaps, an evidence based scientific approach with deeper insights on the ICF core sets of LBP categories might lead to the inclusion of more appropriate outcome measures and makes the LBP SPApps more relevant and meaningful to patients. Also, collaboration between health care and exercise professionals with app developers may enhance the quality of these apps.
The scientific evidence to support the development of SPApps for management of LBP is currently limited. There are numerous published studies regarding smartphone apps covering many different health conditions from asthma to cancer [26, 27]. However, there are very little research evidence available on SPApps for LBP. Therefore, there are many great opportunities for the health care clinicians, researchers and app developers to engage through multi-disciplinary collaborative projects to design and develop SPApps for management of LBP. With collaboration, evidence-based practice can be implemented guided by the best available scientific evidence to ensure patients receive the best quality of care. In the future, health care professionals could assist SpApps developers to find standardized instruments specifically designed to measure and monitor LBP. One recent systematic review from Australia examined the content of LBP apps in reference to the interventions recommend by the current NICE guidelines [6]. The findings of the review indicate that there is currently a gap in the literature focusing on outcome measures used by smartphones for LBP and in most areas of healthcare. This review therefore adds to the current literature as no previous studies were found that compare LBP smartphone apps outcome measures to the ICF core set for LBP.
One of the implications of this review for patients looking to use SPApps to self-manage their LBP is that the patient needs to be aware and confident that the quality of the apps is of high standard and it cannot be gauged from the customer review scores only. This implies that the reviews posted by consumers may not be a reliable indicator of the quality of the app. In one hand, the ratings and reviews available for an app is one measure that consumers often use to distinguish ‘good’ apps from ‘bad’ ones [28]. However, apps consumer ratings can be very subjective and provide little scientific value to health literature and may be misleading to patients [29]). Currently the content and quality of mobile health apps is poorly regulated [30] and it is a concern that apps can be released offering rehabilitation advice and protocols for health conditions without evidence of their effectiveness [8, 17].and therefore this review can aid LBP patients to make a more informed decision when selecting an app of quality. Alongside standardized app quality ratings, trustworthiness and whether the app has any scientific underpinning are essential factors that should be published alongside the app description, and therefore tighter control and governance should be implemented to meet our health care standards [31]. If patients are looking to use an app they should be encouraged to discuss these with their health professionals and seek those apps that have a good quality rating with outcome measures for improved long-term solutions and to monitor any changes to their symptoms [32]. App users should be made aware of these factors and health care providers should evaluate them carefully when recommending apps to their patients.
With respect to the limitations of the study, the current review focused only on the free SPApps for LBP due to the financial constraints. It is therefore recommended that all apps whether free of occurring a fee could be evaluated together, and small funding to be allocated for this by the health care policy makers. This would give a more comprehensive account of the quality of apps available to public in the self-management of LBP. It would also help to establish whether the paid apps are of a higher quality, offer outcome measures and have professional healthcare providers involved. A strength of this study was the use of a valid established procedure to evaluate the outcome measures through ICF comprehensive core sets for LBP [33, 34]. mHealth technology is a promising and accessible platform for patients to improve overall health and to manage the chronic problems effectively [35]. Therefore, this was the first study evaluating and reporting the outcome measures on SPApps for management of LBP and it could contribute to the knowledge base and encourage further research in the ever expanding mHealth in LBP conditions, which may help policy makers, health care professionals, app developers and patients with low back pain.
Conclusion
The review has shown that the overall quality of the free SPApps for LBP are rather low and only 5% of the apps employ valid outcome measures. The review also highlighted that the few outcome measures used in the apps do not represents all the four core sets of LBP criteria set by ICF. Future research and development of mHealth apps for LBP may consider a scientific evidence base and engage in a multidisciplinary approach between app developers, health care professionals and patients.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Hoy D, March L, Brooks P, Blyth F, Woolf A, Bain C, Williams G, Smith E, Vos T, Barendregt J, Murray C. The global burden of low back pain: estimates from the Global Burden of Disease 2010 study. Ann Rheum Dis. 2014;73(6):968–974. doi: 10.1136/annrheumdis-2013-204428. [DOI] [PubMed] [Google Scholar]
- 2.The Health and Safety Executive, 2017. Work-related musculoskeletal disorders (WRMSDs) statistics in Great Britain 2017. United Kingdom: The Health and Safety Executive (HSE). 2017. http://www.hse.gov.uk/Statistics/causdis/musculoskeletal/msd.pdf. Accessed 15 Dec 2017.
- 3.Maniadakis N, Gray A. The economic burden of back pain in the UK. Pain. 2000;84(1):95–103. doi: 10.1016/S0304-3959(99)00187-6. [DOI] [PubMed] [Google Scholar]
- 4.National Institute for Health and Clinical Excellence (NICE), 2016. Low back pain and sciatica in over 16s: assessment and management. London: National Institute for Health and Care Excellence (NICE): clinical guidelines. 2016. https://www.nice.org.uk/guidance/ng59/resources/low-back-pain-and-sciatica-in-over-16s-assessment-and-management-pdf-1837521693637. Accessed 10 Jan 2019.
- 5.Oliveira VC, Ferreira PH, Maher CG, Pinto RZ, Refshauge KM, Ferreira ML. Effectiveness of self-management of low back pain: systematic review with meta-analysis. Arthritis Care Res. 2012;64(11):1739–1748. doi: 10.1002/acr.21737. [DOI] [PubMed] [Google Scholar]
- 6.Machado GC, Pinheiro MB, Lee H, Ahmed OH, Hendrick P, Williams C, Kamper SJ. Smartphone apps for the self-management of low back pain: a systematic review. Best Pract Res Clin Rheumatol. 2016;30(6):1098–1109. doi: 10.1016/j.berh.2017.04.002. [DOI] [PubMed] [Google Scholar]
- 7.Selter A, Tsangouri C, Ali SB, Freed D, Vatchinsky A, Kizer J, Sahuguet A, Vojta D, Vad V, Pollak J, Estrin D. An mHealth App for self-management of chronic lower back pain (Limbr): pilot study. JMIR Mhealth Uhealth. 2018;6(9):e179. doi: 10.2196/mhealth.8256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.de la Vega R, Miró J. mHealth: a strategic field without a solid scientific soul. A systematic review of pain-related apps. PLoS ONE. 2014;9(7):e101312. doi: 10.1371/journal.pone.0101312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.West D. How mobile devices are transforming healthcare. Issues Technol Innov. 2012;18(1):1–11. [Google Scholar]
- 10.O’Sullivan K, O’Keeffe M, O’Sullivan P. NICE low back pain guidelines: opportunities and obstacles to change practice. Br J Sports Med. 2017;51:1632–1633. doi: 10.1136/bjsports-2017-097810. [DOI] [PubMed] [Google Scholar]
- 11.World Health Organization. International Classification of Functioning, Disability and Health: ICF. Geneva: World Health Organization (WHO). 2001. https://apps.who.int/iris/bitstream/handle/10665/42407/9241545429.pdf;jsessionid=1E3720813105A127AE35B58435C81602?sequence=1. Accessed 29 Mar 2019.
- 12.Lohmann S, Decker J, Mueller M, Strobl R, Grill E. The ICF forms a useful framework for classifying individual patient goals in post-acute rehabilitation. J Rehabil Med. 2011;43(2):151–155. doi: 10.2340/16501977-0657. [DOI] [PubMed] [Google Scholar]
- 13.Cieza A, Stucki G, Weigl M, Disler P, Jackel W, van der Linden S, Kostanjsek N, de Bie R. ICF Core Sets for low back pain. J Rehabil Med. 2004;36:69–74. doi: 10.1080/16501960410016037. [DOI] [PubMed] [Google Scholar]
- 14.Ewert T, Fuessl M, Cieza A, Andersen C, Chatterji S, Kostanjsek N, Stucki G. Identification of the most common patient problems in patients with chronic conditions using the ICF checklist. J Rehabil Med. 2004;36(1):22–29. doi: 10.1080/16501960410015362. [DOI] [PubMed] [Google Scholar]
- 15.Deloitte Touche Tohmatsu Limited. mHealth in an mWorld How mobile technology is transforming health care. Washington DC, USA: Deloitte Center for Health Solutions. 2012. https://www2.deloitte.com/content/dam/Deloitte/us/Documents/life-sciences-health-care/us-lhsc-mhealth-in-an-mworld-103014.pdf. Accessed 20 Jan 2019.
- 16.Matters. App Market Stats. 2018. https://42matters.com/stats. Accessed 21 Jan 2019.
- 17.Haute Autorité de Santé. Good practice guidelines on health apps and smart devices. France: Haute Autorité de Santé (HAS). 2016. https://www.has-sante.fr/portail/upload/docs/application/pdf/2017-03/dir1/good_practice_guidelines_on_health_apps_and_smart_devices_mobile_health_or_mhealth.pdf. Accessed 2 Nov 2018.
- 18.Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, the PRISMA-P Group Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647. doi: 10.1136/bmj.g7647. [DOI] [PubMed] [Google Scholar]
- 19.Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. doi: 10.1186/2046-4053-4-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Furlan AD, Malmivaara A, Chou R, Maher CG, Deyo RA, Schoene M, Bronfort G, Van Tulder MW. 2015 updated method guideline for systematic reviews in the Cochrane Back and Neck Group. Spine. 2015;40(21):1660–1673. doi: 10.1097/BRS.0000000000001061. [DOI] [PubMed] [Google Scholar]
- 21.Apple, 2018. Guidelines. 2018. https://developer.apple.com/app-store/guidelines/. Accessed 18 Jan 2019.
- 22.Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth uHealth. 2015;3(1):e27. doi: 10.2196/mhealth.3422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Stanton R, Wintour SA, Kean CO. Validity and intra-rater reliability of MyJump app on iPhone 6s in jump performance. J Sci Med Sport. 2017;20(5):518–523. doi: 10.1016/j.jsams.2016.09.016. [DOI] [PubMed] [Google Scholar]
- 24.Glocker C, Kirchberger I, Gläßel A, Fincziczki A, Stucki G, Cieza A. Content validity of the comprehensive international classification of functioning, disability and health (ICF) core set for low back pain from the perspective of physicians: a Delphi survey. Chronic Illn. 2013;9(1):57–72. doi: 10.1177/1742395312451280. [DOI] [PubMed] [Google Scholar]
- 25.ICF Research Branch. ICF generic and rehabilitation sets. 2017. https://www.icf-research-branch.org/icf-core-sets-projects2/musculoskeletal-conditions/icf-core-set-for-low-back-pain. Accessed 28 Nov 2018.
- 26.Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 2012;10(1):144. doi: 10.1186/1741-7015-10-144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Pandey A, Hasan S, Dubey D, Sarangi S. Smartphone apps as a source of cancer information: changing trends in health information-seeking behavior. J Cancer Educ. 2013;28(1):138–142. doi: 10.1007/s13187-012-0446-9. [DOI] [PubMed] [Google Scholar]
- 28.Kuehnhausen M, Frost VS. Trusting smartphone apps? To install or not to install, that is the question. In: 2013 IEEE international multi-disciplinary conference on cognitive methods in situation awareness and decision support (CogSIMA). IEEE. App Review Service™. 2013. pp. 30–37. http://appiness.io/. Accessed 12 Dec 2018.
- 29.BinDhim NF, Hawkey A, Trevena L. A systematic review of quality assessment methods for smartphone health apps. Telemed e-Health. 2015;21(2):97–104. doi: 10.1089/tmj.2014.0088. [DOI] [PubMed] [Google Scholar]
- 30.McCartney M. How do we know whether medical apps work? BMJ. 2013;346:f1811. doi: 10.1136/bmj.f1811. [DOI] [PubMed] [Google Scholar]
- 31.Rodríguez MS, Vázquez SC, Casas PM, de la Cuerda RC. Neurorehabilitation and apps: a systematic review of mobile applications (English Edition) Neurología. 2016;33(5):313–326. doi: 10.1016/j.nrl.2015.10.005. [DOI] [PubMed] [Google Scholar]
- 32.Santana MJ, Haverman L, Absolom K, Takeuchi E, Feeny D, Grootenhuis M, Velikova G. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res. 2015;24(7):1707–1718. doi: 10.1007/s11136-014-0903-5. [DOI] [PubMed] [Google Scholar]
- 33.Valdes K, Naughton N, Algar L. Linking ICF components to outcome measures for orthotic intervention for CMC OA: a systematic review. J Hand Ther. 2016;29(4):396–404. doi: 10.1016/j.jht.2016.06.001. [DOI] [PubMed] [Google Scholar]
- 34.Fayed N, Cieza A, Edmond Bickenbach J. Linking health and health-related information to the ICF: a systematic review of the literature from 2001 to 2008. Disabil Rehabil. 2011;33(21–22):1941–1951. doi: 10.3109/09638288.2011.553704. [DOI] [PubMed] [Google Scholar]
- 35.Kaplan AL, Cohen ER, Zimlichman E. Improving patient engagement in self-measured blood pressure monitoring using a mobile health technology. Health Inform Sci Syst. 2017;5(1):4. doi: 10.1007/s13755-017-0026-9. [DOI] [PMC free article] [PubMed] [Google Scholar]

