Accuracy of a Smartphone Software Application Compared With a Handheld Goniometer for Measuring Shoulder Range of Motion in Asymptomatic Adults

Abstract

Background:

Typically, shoulder range of motion (ROM) measurements are performed in a clinic setting, where physicians, physical therapists (PTs), and occupational therapists make use of a goniometer.

Purpose:

To determine the accuracy of a smartphone-based ROM software application (app) in assessing active shoulder ROM and compare the measurements with traditional goniometry as measured by a PT.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Methods:

This prospective, nonblinded study was conducted at a single institution with adult asymptomatic participants with full active ROM of both shoulders. Participants were enrolled between June 1 and 15, 2021. Each participant self-assessed their active shoulder ROM using the PeerWell smartphone app. A single PT concurrently measured each participant’s active shoulder ROM using a handheld universal goniometer. Bilateral shoulder ROM (forward flexion, abduction [AB], external rotation, internal rotation [IR], and extension) was measured 2 times for each participant. Interrater reliability between the smartphone app and PT measurements as well as intrarater reliability for each method of measurement were assessed using the intraclass correlation coefficient (ICC), and limits of agreement were analyzed for the difference in measurement methods.

Results:

Data were analyzed for 60 shoulders (30 right, 30 left) from 30 participants (mean age, 31.4 ± 11.7 years; 73% female). The interrater reliability between the 2 methods was excellent for all movements (ICCs, 0.90-0.96). For all movements except shoulder AB, the mean difference in the measurements between the 2 methods was <1.3°; the mean difference in AB ROM was 2.08°. For all movements except IR, both PTs and the app showed excellent intrarater reliability (ICCs >0.90); for IR, good intrarater reliability (ICC ≥0.75) was observed.

Conclusion:

The PeerWell smartphone app provided measurements comparable with manual measurements taken by a PT using a goniometer. These data provide evidence that the smartphone app is a reliable and valid tool for measuring shoulder ROM and show promise for measuring and monitoring patient ROM remotely.

Health care providers often use goniometry to obtain objective measurements of joint range of motion (ROM) to determine baseline functional status and to measure the effectiveness of treatment interventions. The original gold standard for measuring joint ROM was radiography; however, due to practicality, safety, and accessibility, the universal handheld goniometer has been the tool used most commonly by providers in the clinic to measure joint ROM. ⁴

Inclinometers have been investigated as an alternative method for measuring joint ROM. Green et al ⁶ evaluated the interrater and intrarater reliability of measuring several different shoulder movements using an inclinometer and found that the reliability varied according to joint movement but still demonstrated an acceptable level of reliability to be used as an outcome measure for shoulder pathologies. Considering the potential of inclinometers as an acceptable alternative for the universal goniometer and the growing number of mobile goniometric applications (apps) available on the market, it is important to investigate the reliability, validity, and usability of these goniometric mobile apps for use in the clinic and for remote patient monitoring.

Mobile apps for goniometry use different methods to measure joint ROM. These include image-based measurements of joint motion (measurements taken from photos or videos) or using the sensors that exist in most mobile phones, such as the accelerometer, which measures the displacement of an object; the gyroscope, which measures spin movement; and the magnetometer, which provides a reference point for the displacement or orientation by measuring the strength or direction of magnetic fields. ¹³

Previous research exploring the reliability and validity of goniometric measurements via mobile apps has been promising. Mitchell et al ¹⁴ compared active shoulder external rotation (ER) measurements using an inclinometer-based app, a photo-based app and the universal goniometer with both novice and expert users. The authors found both apps to be reliable and comparable with the universal goniometer. Shin et al ¹⁷ compared measurements of shoulder forward flexion (FF), abduction (AB), ER, and internal rotation (IR) using both a universal goniometer and a smartphone inclinometer app and showed satisfactory interrater reliability (intraclass correlation coefficients [ICCs], 0.76-0.87) except for IR at 90° of AB (ICCs, 0.66-0.67). Intrarater reliability was reported as excellent (ICCs, 0.96-0.99) except for IR at 90° of AB (ICCs, 0.79-0.99). ¹⁷ Similarly, Tremblay et al ¹⁸ studied the concurrent criterion validity of the gyroscopic functions of a mobile device as compared with inclinometer in healthy subjects and reported excellent criterion validity for shoulder FF, AB, extension (EX), and ER (ICCs, 0.907-0.996), and mean difference for all movements was between -1.9° and 2.0°, except for EX, where the mean difference was between 3.8° and 4.7°.

The aforementioned mobile apps had to be utilized by trained providers, as they either require manual placement of the device in alignment with bony landmarks or moving segments according to traditional goniometric measurement procedures, or the start and stop points of motion are not automatically recorded. Patients in a remote setting would need the assistance of another person to perform a ROM measurement, which may not always be feasible. With the rapid expansion of telehealth utilization and remote monitoring of patients, there is a need for the development and testing of mobile goniometric apps that can be used independently by untrained patients in the comfort of their own homes.

The purpose of this study was to determine the accuracy of a patient-centric smartphone-based app in assessing active shoulder ROM compared with traditional goniometry measured by a physical therapist (PT) in a clinic setting. Our hypothesis was that this remote smartphone app would effectively measure active shoulder ROM in all planes with good intrarater reliability.

Methods

Institutional review board approval was obtained for the study protocol, and all participants provided informed consent. Included in this prospective, nonblinded study conducted at a single institution between June 1 and June 30, 2021 were 30 asymptomatic adult participants who were screened to ensure they did not have a shoulder injury that would limit ROM. In addition to ROM, patient age, sex, height, weight, and body mass index (BMI) were recorded.

ROM Measurements

Measurements were made using a smartphone app designed for remote ROM monitoring (PeerWell). Before each test, the subject viewed a short video demonstrating hand placement, body positioning, and motion. The subject then practiced the movement to replicate it. All subjects were asymptomatic and had full range of shoulder active ROM at baseline. Subjects were instructed to move the arm in each direction as tolerated. Bilateral shoulder active ROM was measured for each participant. Two PTs (R.S. and N.B.) collected all goniometer measurements. ¹⁶ Both PTs were faculty at our institution and licensed in the state of Texas during the period of the study. For each participant, the goniometer measurements of active shoulder ROM were taken by a single PT.

The protocol for the app (Figure 1) involved holding the smartphone in one hand and, when instructed by the app, performing the movement. During use, the app guides the user by vocalizing instructions on the proper technique for maximal accuracy. When the movement is completed, the app calculates and states the ROM measurement. For maximum efficiency and to minimize bias, a single PT took concurrent measurements using the goniometer as the participant was using the smartphone app (ie, measurements from both methods were taken on the same movement). The subject performed each movement 2 times, the second immediately after the first, maintaining the original position. To ensure consistency of the motion, the movement was restricted to a certain angle each time by using a yoga block under the wrist, so that the distance between the hand and the plinth was maintained at the end ROM, across all subjects. The ROM movements measured were FF, AB, ER, IR, EX. All measurements were taken in the supine position except for EX, which was taken in the prone position (Figure 2).

Figure 1.

Workflow for the PeerWell smartphone app.

Figure 2.

Measuring shoulder active ROM movements using the PeerWell smartphone app: (A) abduction, (B) extension, (C) forward flexion, (D) external rotation, and (E) internal rotation. Direction of movement is shown by the blue dotted lines and arrowheads. app, application; ROM, range of motion.

Sample Size and Statistical Analysis

At an alpha level of 0.05, we calculated that a sample size of approximately 30 patients (with 2 measurements for each patient) would provide the study with greater than 80% power to detect an ICC of 0.90 compared with the null hypothesis of 0.60. Power was calculated using the NCSS PASS 2019, Version 19.0.2 (NCSS, LLC) with module “tests for intraclass correlation.”

All statistical analyses were performed using R software Version 3.4.4 (R Foundation for Statistical Computing). Statistical significance was set at P < 0.05. ICCs were calculated to determine (1) the intrarater reliability for the PT measurements and the smartphone app measurements by comparing the 2 repeated measures (1 immediately after the other) as performed by the same rater within each method and (2) the agreement between the 2 methods (interrater reliability) by comparing the average of the 2 repeated measures from each method. ICC values were interpreted as poor (<0.50), moderate (0.50-0.75), good (0.75-0.90), or excellent (>0.90). ¹¹ The standard error of measurement (SEM) was calculated as described in Eliasziw et al. ²

Limits of agreement (LoA) were calculated as the mean difference ±1.3 standard deviation of difference. The interpretation of LoA was that the 80% LoA will contain the difference between the 2 methods being compared, or 80% of future measurement pairs.

Results

Subject Demographics

A total of 30 participants (22 female, 8 male) were included in this study. Data were analyzed for 60 shoulders (30 right, 30 left). The mean age (±SD) was 31.4 ± 11.7 years and the mean BMI was 24.2 ± 6.7 kg/m². Additional details of height, weight, sex, and BMI are provided in Table 1.

Table 1.

Characteristics of the Study Participants (N = 30) ^a

Characteristic	Value
Age, years
Mean ± SD	31.4 ± 11.7
Median (range)	27 (23-66)
Sex, n (%)
Male	8 (26.7)
Female	22 (73.3)
Height, cm
Mean ± SD	185.2 ± 65.3
Median (range)	168.1 (157.4-438.7)
Weight, kg
Mean ± SD	73.1 ± 14.5
Median (range)	70.3 (49.9-124.7)
BMI, kg/m2
Mean ± SD	24.2 ± 6.7
Median (range)	24.2 (4.1-36.3)
BMI category (kg/m2), n (%)
≤24.9	17 (56.7)
25-29.9	8 (26.7)
30-34.9	4 (13.3)
≥35	1 (3.3)

^a BMI, body mass index.

Intrarater and Interrater Reliability

For all motions except IR, both PT and app showed excellent intrarater reliability (ICCs >0.90), which ranged from 0.93 to 0.97 for PT and 0.90 to 0.95 for the app (Table 2). For IR, both methods showed good intrarater reliability (ICCs ≥0.75), with an ICC of 0.84 for PT and 0.75 for the app. The interrater ICC indicated excellent agreement between the PT and app measurements (ICCs ≥0.90). The intrarater SEM for the app as well as PT ranged from 4° to 7°, and the interrater SEM between PT and app ranged from 2° to 7°.

Table 2.

Intrarater and Interrater Reliability of the Smartphone App and Physical Therapist Measurements for Shoulder ROM ^a

	Intrarater ICC (95% CI)
ROM Measurement	Physical Therapist	Smartphone App	Interrater ICC (95% CI)
Forward flexion	0.95 (0.92-0.97)	0.94 (0.90-0.96)	0.96 (0.94-0.98)
Abduction	0.97 (0.95-0.98)	0.95 (0.93-0.97)	0.90 (0.85-0.94)
Internal rotation	0.84 (0.76-0.90)	0.75 (0.62-0.84)	0.93 (0.89-0.95)
External rotation	0.96 (0.93-0.97)	0.93 (0.88-0.95)	0.95 (0.92-0.97)
Extension	0.93 (0.90-0.96)	0.90 (0.84-0.93)	0.96 (0.93-0.97)

^a app, application; ICC, intraclass correlation coefficient; ROM, range of motion.

Limits of Agreement

The difference in measurements between PT and app and the LoA are shown in Table 3. For all motions except AB, the mean difference in the measurements between PT and app was less than 1.3°, and the difference was within 6° of the mean differences for 80% of the participants. For AB, the mean difference was 2.08, and the difference was within 15° of the mean difference for 80% of the participants.

Table 3.

Difference and LoA in the Measurements Between PT and Smartphone App ^a

ROM Measurement	Difference, Mean ± SD b	80% LoA
Forward flexion	0.73 ± 3.84	-4.2 to 5.65
Abduction	-2.08 ± 9.54	-14.31 to 10.15
Internal rotation	0.59 ± 3.1	-3.39 to 4.57
External rotation	1.21 ± 3.18	-2.87 to 5.29
Extension	0.43 ± 3.72	-4.34 to 5.19

^a The unit of measure is degrees. app, smartphone app; LoA, limits of agreement; PT, physical therapist; ROM, range of motion.

^b Measurement by the PT minus the measurement by the app.

Discussion

In a direct comparison between a smartphone app and a goniometer, our study demonstrated that a smartphone-based digital ROM tool was comparable with traditional goniometry in accuracy and reproducibility. The main findings of our study are as follows: (1) for all planes of motion, the ICC demonstrated good-to-excellent interrater reliability for the app when compared with the gold standard of goniometric measurements. For all motions except IR, both PT and app showed excellent intrarater reliability (ICCs ≥0.90). For IR, both methods showed good intrarater reliability (ICCs ≥0.75). (2) For all planes of motion except AB (mean difference of -2.08°), the mean difference in measurements between the goniometer and the app was less than 1.3° with an LoA within 6° of the mean differences. For AB, there was more variance between subjects holding the smartphone, which could explain the slightly higher mean difference.

Previous studies with similar numbers of subjects have demonstrated the accuracy of goniometric measurements for various joints in the body. ^1,12,7,15 Cleffken et al ¹ tested the reproducibility of elbow goniometric measurements with a sample size of 42 patients; Lenssen et al ¹² reported on the reproducibility of goniometric measurements of the knee in 30 patients; Holm et al ⁷ reported the reliability of these measurements on hip ROM in 25 patients; the sample size in these similar studies was sufficient to establish good agreement between the methods being compared. In a study by Muir et al ¹⁵ on shoulder movements, intrarater and interrater reliability and measurement error in glenohumeral ROM measurements were evaluated in 17 subjects using a standard goniometer, and the SEM and minimum clinically important difference (MCID) of goniometry were calculated. The SEM reported by Muir et al ¹⁵ ranged from 4° to 7° for intrarater and from 6° to 9° for interrater agreement. The MCID ranged from 11° to 16° for a single evaluator and from 14° to 24° for 2 evaluators. In our study, the intrarater SEM for the app ranged from 4° to 7°, and the interrater SEM between the PT and app ranged from 2° to 7°, which is well within the SEM ranges reported in Muir et al. ¹⁵ This demonstrates that measuring accurate shoulder ROM using the app can be useful in a remote setting where accurate measurement can be challenging.

While goniometry has remained the measurement standard in previous studies and in clinical practice, for clinicians and health care professionals, there is a need for alternative and easier-to-use tools to measure ROM that can be reproduced by patients to track their recovery as health care moves toward a virtual environment. With the COVID-19 pandemic forcing social distancing, travel restrictions, and difficulty seeing patients in a clinical setting, telemedicine has become a tool used more commonly to evaluate patients. Although orthopaedic surgeons often prefer an in-person clinical evaluation, remote methods such as telehealth and smartphone apps are becoming an acceptable way to evaluate patients during initial and follow-up encounters, provided patient function can be assessed objectively. ^5,8

Werner et al ²⁰ first tested the smartphone “clinometer”, which is a readily available, inexpensive app, for shoulder ROM and compared this with the gold standard of goniometric measurement and clinician visual estimation. With 24 healthy subjects, and 15 symptomatic shoulders, they compared the accuracy of measurements between various levels of training and found that the smartphone clinometer had excellent agreement with the goniometric gold standard to measure shoulder ROM in both healthy and symptomatic subjects. There was also a good correlation among different skill levels of providers, indicating its reproducibility for use amongst all clinical team members. In addition, there were no differences in reliability for measurements in pathologic shoulders, demonstrating clinical utility. The study supported the premise for use of smartphone apps in shoulder ROM measurements.

While the use of a smartphone clinometer was reproducible and accurate, an important advantage of the smartphone app used in this study is the ability for the patient to measure their own ROM without a trained professional. This data can then be taken to any number of providers, including PTs, occupational therapists, massage therapists, primary care physicians, or surgeons. Furthermore, this app has the potential for integration of data collection into larger-scale health care systems.

A more recent study assessed the effectiveness of a remote on-screen app-based method of shoulder ROM measurement through telehealth. ¹⁹ Their cohort consisted of a similar distribution to that of Werner et al, ²⁰ with 24 healthy volunteers and 16 symptomatic patients. Shoulder ROM was first examined using a goniometer in the clinic and then over Zoom remote conferencing using a digital protractor EX app. The ICC between both measurement modalities and the intraobserver reliability between examiners of different training levels was greater than 0.9 (ie, excellent), indicating potential for reproducibility of this technique in a remote setting. ¹⁹

Knapp et al ⁹ performed a study with total hip arthroplasty and total knee arthroplasty patients to assess their engagement using the same smartphone app that was used in this study. The content available to the patients in the app included physical therapy videos/exercises and other educational materials. They found almost 50% patient engagement among enrollees, concluding that this app could be a reliable form of patient engagement in the postoperative period. In a separate study, the authors compared the app with traditional goniometric measurements by a physician and a PT for knee ROM. ¹⁰ They found a reliable correlation between clinicians with goniometers and the app in assessing knee ROM, validating its clinical application.

While patient evaluations are performed routinely at outpatient clinics and hospitals, many patients face challenges, including (1) the ability to afford transportation to follow-up visits and/or (2) moving to another community. In a 2010 study, direct transportation made up approximately 3% of surgical costs postoperatively within 1 year after surgical treatment of proximal humerus fractures. ³ The ability to objectively record ROM in a remote setting can make a huge difference for patients who cannot find transportation due to distance, health status, or financial difficulty. As for patients lost to follow-up due to a move, telehealth remote monitoring with ROM measurements can give a surgeon the ability to assess postoperative progress more readily from a distance and maintain the continuity of the care needed for successful clinical outcomes. Our findings confirm our hypothesis that a smartphone app has the potential to be a reliable tool for shoulder ROM measurement in a remote setting and is comparable with the existing gold standard.

Limitations

Our study is not without limitations. Many therapists and surgeons would argue there is no substitute for in person-evaluation and visual clinical examination of shoulder pathology. For this reason, patients with structural or evolving pathology causing compensatory scapular and trunk motion may not be accurately assessed using a digital app that they self-administer. In addition, the smartphone app does require patient training to accurately use the ROM tool to avoid potential inaccuracies in measurements. This limitation is overcome with a clear instructional video. Another limitation is that all the patients were asymptomatic volunteers without any pathology. Assessing patients with shoulder pathology would be beneficial to ensure accurate measurements of all shoulder states. Further, for each participant, a single PT measured the shoulder ROM, thus interrater reliability between the PTs could not be calculated. Finally, all these patients were evaluated in a controlled environment (ie, an in-office setting). Evaluation in the patients’ homes may introduce additional challenges such as difficulties of holding the phone or computer at the proper angle for the patient or caregiver.

Conclusion

The PeerWell smartphone app used in this study provided measurements that were comparable with manual measurements made by a PT using a goniometer for shoulder ROM in asymptomatic subjects. Measurements were comparable for FF, AB ER, IR, and EX for shoulder ROM. These data provide evidence that the smartphone app is a reliable and valid tool for measuring shoulder ROM and shows promise for measuring and monitoring patients’ ROM remotely.