RELIABILITY AND ACCURACY OF A GONIOMETER MOBILE DEVICE APPLICATION FOR VIDEO MEASUREMENT OF THE FUNCTIONAL MOVEMENT SCREEN DEEP SQUAT TEST

Abstract

Background/Purpose

The squat is a fundamental movement of many athletic and daily activities. Methods to clinically assess the squat maneuver range from simple observation to the use of sophisticated equipment. The purpose of this study was to examine the reliability of Coach's Eye (TechSmith Corp), a 2‐dimensional (2D) motion analysis mobile device application (app), for assessing maximal sagittal plane hip, knee, and ankle motion during a functional movement screen deep squat, and to compare range of motion values generated by it to those from a Vicon (Vicon Motion Systems Ltd) 3‐dimensional (3D) motion analysis system.

Methods

Twenty‐six healthy subjects performed three functional movement screen deep squats recorded simultaneously by both the app (on an iPad [Apple Inc]) and the 3D motion analysis system. Joint angle data were calculated with Vicon Nexus software (Vicon Motion Systems Ltd). The app video was analyzed frame by frame to determine, and freeze on the screen, the deepest position of the squat. With a capacitive stylus reference lines were then drawn on the iPad screen to determine joint angles. Procedures were repeated with approximately 48 hours between sessions.

Results

Test‐retest intrarater reliability (ICC_3,1) for the app at the hip, knee, and ankle was 0.98, 0.98, and 0.79, respectively. Minimum detectable change was hip 6°, knee 6°, and ankle 7°. Hip joint angles measured with the 2D app exceeded measurements obtained with the 3D motion analysis system by approximately 40°. Differences at the knee and ankle were of lower magnitude, with mean differences of 5° and 3°, respectively. Bland‐Altman analysis demonstrated a systematic bias in the hip range‐of‐motion measurement. No such bias was demonstrated at the knee or ankle.

Conclusions

The 2D app demonstrated excellent reliability and appeared to be a responsive means to assess for clinical change, with minimum detectable change values ranging from 6° to 7°. These results also suggest that the 2D app may be used as an alternative to a sophisticated 3D motion analysis system for assessing sagittal plane knee and ankle motion; however, it does not appear to be a comparable alternative for assessing hip motion.

Level of Evidence

3

INTRODUCTION

The squat is a fundamental movement at the basis of many athletic and physical activities. Performing an efficient squat requires mobility and stability of the ankle, knee, hip, and spine.¹ Abnormal lower extremity kinematics while performing a deep squat may be due to restricted mobility.^2,3 Abnormal lower extremity movement patterns during a squat may also be associated with an increased risk for injury. Hence, assessing squat mechanics may provide insight on functional capabilities as well as risk for injury.

The Functional Movement Screen (FMS™) is a battery of seven fundamental movement tests, including the deep squat, that is used to evaluate and categorize functional movement patterns.⁴ Cook et al⁵ developed a 4‐point scoring system to classify movement patterns observed during performance of FMS tests. A score of (3) is given if the movement is performed correctly, a (2) if compensation occurs, a (1) if the subject is unable to perform the movement, and a (0) if pain occurs with testing. These scores are used to guide interventions and may predict risk for injury.^4,6 The scoring system has been shown to be reliable and can be used by examiners with varying levels of experience. Minick et al⁴ has reported that examiner experience does not affect the reliability of scoring the FMS™ deep squat test. Although reliable, the scoring system has a degree of subjectivity, with broad movement pattern classifications not reflecting specific joint mobility. Motion analysis allows greater objectivity in the assessment of joint mobility observed during the squat maneuver. Using a Vicon motion analysis system (Vicon Motion Systems Ltd, Oxford, UK) to record specific joint motion in comparison to FMS™ scoring categories, Butler et al⁷ reported those who scored higher on the FMS™ squat had greater peak range of motion (ROM) values at the hip and knee and greater ankle dorsiflexion excursion than those who scored low. While the FMS™ scoring assesses the quality of motion, precise measurement of specific joint performance can provide additional information on which to base interventions.

Three‐dimensional (3D) motion analysis has been cited as the gold standard for kinematic joint analysis,⁸ however, these systems are expensive, require time‐consuming procedures, and are not readily available to all clinics. Conversely, 2‐dimensional (2D) motion analysis is now widely available, ranging from camera systems to mobile device applications (apps). Researchers have previously compared 2D to 3D motion analysis systems, and report varying levels of agreement between the two types of systems.^9-13 To the authors’ knowledge, the analysis of the FMS™ deep squat maneuver with the simultaneous use of 2D and 3D motion analysis has not been reported.

The primary objective of this study was to examine the reliability of the Coach's Eye (TechSmith Corporation, Okemos, MI), a 2D motion analysis tablet computer app, for goniometric assessment of maximal sagittal plane hip, knee, and ankle motion during the FMS™ deep squat. A secondary objective was to compare ROM values obtained with the app to ROM values obtained with the 3D Vicon motion analysis system.

METHODS

Subjects

Healthy subjects were recruited through flyers and word‐of‐mouth publicity to participate in the study. Inclusion criteria included being able to perform a pain free squat as described per the FMS™ procedure. Exclusion criteria, which were assessed via questioning by an examiner, included any injury, surgery, or self‐reported musculoskeletal abnormality of a lower or upper extremity that would prevent the subject from performing a deep squat. The methods and procedures used in this study were approved by the Mayo Clinic institutional review board. Each subject was informed of procedures and risks and signed an informed consent form prior to participation.

Instrumentation

2D Analysis

The 2D analysis was performed using a second‐generation tablet computer (iPad by Apple Inc., Cupertino, CA) running the iOS 7 operating system with the Coach's Eye app version 4.0. The app utilized video recording of the subject's performance, which was captured and analyzed at 30 frames per second.

3D Analysis

The 3D analysis of the lower extremity was recorded by a Vicon motion analysis system (Vicon Motion Systems Ltd, Oxford, UK) consisting of five Vicon MX20+ infrared motion capture cameras running at 100 Hz, a Vicon MX control unit, and a Vicon MX Ultranet unit. The five wall mounted cameras were at a height of 300 cm and spaced around the periphery of the room, thereby providing the necessary coverage to ensure that each reflective marker placed on the subject was viewable from at least two cameras at all times. Data were captured and analyzed on a desktop computer using Vicon Nexus software version 1.8.5.

PROCEDURES

Subjects were briefed on the procedures prior to participating. To allow adequate skin exposure for placement of the reflective markers, the male subjects wore compression shorts and the female subjects wore compression shorts and a support top. Using a measurement caliper, each subject's inter‐anterior superior iliac spine (ASIS) distance, knee width at the medial and lateral femoral condyles, and ankle width at the medial and lateral malleoli were measured. Leg length was measured from the ASIS to the medial malleoli using a standard retractable measuring tape. These measurements were taken to facilitate kinematic calculations, as described below. Height and weight were also recorded. All anthropometric measurements were taken by the same tester for each trial.

The Vicon motion analysis system was configured and calibrated by the same tester before each data collection, according to the manufacturer's instructions (Vicon Plug‐in‐Gait Product Guide for use with Plug‐in‐Gait Version 1.9 in Vicon Nexus). Sixteen 10‐mm reflective markers were placed on the subject's lower extremities using the Vicon Plug‐In Gait template. A single examiner, trained in procedures by an individual with over 20 years of experience with motion analysis systems, placed markers on the subjects. Markers were located at the head of the second metatarsal, the posterior calcaneus, the lateral malleolus, the lateral fibula, one cm proximal to the knee joint axis laterally, and the lateral femur. Four markers were also placed on the pelvis at each anterior and posterior superior iliac spine bony landmark. The ASIS markers were adjusted 2.5 cm laterally to the ASIS proper to prevent loss of view of the markers between the torso and the thigh when the subject performed the deep squat. The inter‐ASIS measurement recorded for the Plug‐in marker set reflected this adjustment. Markers were placed by the same tester for each trial.

The tablet was placed at a standard height (floor to camera) of 43 cm and distance (camera to center of the subject's stance) of 280 cm in the sagittal plane to the right of the subject. For consistent subject positioning, the subject was positioned on a nonreflective dark surface marked with a nonreflective line. The subject stood with arms crossed over the chest and a static image was recorded using the 3D motion analysis system software. These data were reconstructed, each reflective marker was labeled, and the Vicon Static Plug‐In Gait process was performed to create a Vicon skeleton of the subject. As the static trial was being calibrated and saved, standardized instructions describing how to perform the deep squat maneuver of the FMS™ were given at each trial by the same tester, who read a prepared script. Subjects were instructed to start with their feet shoulder‐width apart, with feet pointed forward (in line with the sagittal plane). The subject was given a dowel to hold in both hands. The dowel was raised such that the subject's elbows were extended and the shoulders maximally flexed. The subject was instructed to keep both feet flat on the floor and keep the chest forward while slowly squatting as far as possible. While the authors used standardized FMS™ instructions, the squat was not modified by placing a board under the subject's heels based on performance as would be done with a formal FMS™ screen as the primary objective was to compare the 2D and 3D measurement systems. Once the subject received instructions and verbally reported an understanding of them, the subject performed three practice deep squats. After the three practice squats, the subject was allowed a rest period of one minute. The tester then instructed the subject to perform three squats, of which only the third was subsequently analyzed. Specifically, the tester said “ready,” thus prompting the 2D app and the 3D motion analysis system to begin recording, and then said “go,” thus prompting the subject to perform three squats to the best of his or her ability. When the subject returned to an erect stance after completing the third squat, both recordings were stopped, checked for any recording errors, and saved. The procedures were repeated with approximately 48 hours between sessions for each subject.

A single examiner performed the 2D analysis after data collection using the app. The app allowed the video to be collected and analyzed at a frame rate of 30 frames per second. The built‐in angle tool was used to determine joint angle values for the subject as viewed from the subject's right side. Measurements were determined by moving each subject's video frame by frame to the lowest portion of the third squat. On the selected image, the examiner used a capacitive stylus to draw lines to establish specific joint angles. For the hip, a line was used to bisect the trunk and a line was used to bisect the thigh. For the knee angle, a line was drawn to bisect the thigh and a line was drawn to bisect the leg. For the ankle, a line was drawn to bisect the leg, and a line was drawn to bisect the lateral plantar border of the foot (Figure 1).

Figure 1.

Screen Shot of the Coach's Eye (TechSmith Corporation, Okemos, MI) Tablet Computer Goniometer Application.

Motion analysis system data were reconstructed using the Vicon Nexus software, and the Vicon Dynamic Plug‐in Gait process was run on the recorded data to output the recorded and calculated joint angles. Marker trajectories were filtered with a Woltring quintic spline filter, at a mean square error of 20 mm. Right lower‐extremity joint angle data from the deepest portion of the third squat were recorded.

Data Analysis

Test‐retest reliability was assessed using model 3 form 1 intraclass correlation coefficients (ICC_{3, 1}) as described by Shrout and Fleiss.¹⁴ Minimum detectable change (MDC) was calculated with the following equation from Schmitt and Di Fabio¹⁵:

$$MDC=z\times SD\times \sqrt{2\left(1-ICC\right)}$$

where z is the z‐score associated with a 95% level of confidence (z=1.96) and the SD is from a representative sample of data. Bland‐Altman analysis 95% limits of agreement were used to assess agreement between the results from the motion analysis system and the results from the app. Data were analyzed with IBM SPSS 21.0 statistical software (IBM Corp).

RESULTS

Twenty‐six healthy subjects (age range, 21‐26 years) were recruited to participate in the study. There were 11 men and 15 women, with a mean (SD) weight of 72.15 (12.64) kg and a mean height of 175.8 (10.61) cm.

The mean joint angles recorded at the termination of the third squat with the 3D motion analysis system and the 2D app are summarized in Table 1. Angles obtained via the Vicon motion analysis system were 87.3˚ for hip flexion, 109.9˚ for knee flexion and 24.1˚ for ankle dorsiflexion. Measurements derived with the Coaches Eye app were 127.2˚, 114.9˚and 27.2˚ for the hip, knee and ankle respectively. The reliability of measurement using the motion analysis system to evaluate the deep squat is summarized in Table 2, and the reliability of measurement using the app is summarized in Table 3. Reliability of measurement ranged from 0.62 to 0.88 for the motion analysis system, and from 0.79 to 0.98 for the app. The MDC using the app was 6° at the hip, 6° at the knee, and 7° at the ankle.

Table 1.

Mean Joint Angles Derived From the Vicon Motion Analysis System and the Coach's Eye Tablet Computer App^a

Joint	3D Vicon Motion Analysis System	2D Coach’s Eye Tablet Computer App
Hip Flexion	87.3˚ (10.5˚)	127.2˚ (15.5˚)
Knee Flexion	109.9˚ (17.6˚)	114.9˚ (15.9˚)
AnkleDorsiflexion	24.1˚ (6.6˚)	27.2˚ (5.3˚)

Abbreviations: 2D= 2‐dimensional; 3D= 3‐dimensional; App= application.

^aValues are reported as mean (SD), in degrees.

Table 2.

Test‐Retest Reliability of the 3‐Dimensional Vicon Motion Analysis System

Joint	Mean (SD) (Degrees)	ICC (3, 1)a	95% CI
Hip 1	87.33 (10.52)	.615	.306‐.807
Hip 2	87.83 (8.33)
Knee 1	109.87 (17.58)	.879	.748‐.944
Knee 2	108.17 (17.65)
Ankle 1	24.10 (6.60)	.628	.324‐.814
Ankle 2	21.77 (6.55)

Abbreviation: ICC= intraclass correlation coefficient.

^ap<.05.

Table 3.

Test‐Retest Reliability of the Coach’s Eye 2‐Dimensional Smart Tablet Goniometer Application

Joint	Mean (SD) (Degrees)	ICC (3, 1)a	95% CI
Hip 1	127.15 (15.54)	.978	.953‐.990
Hip 2	127.96 (14.53)
Knee 1	114.88 (15.91)	.984	.964‐.993
Knee 2	115.69 (15.50)
Ankle 1	27.23 (5.29)	.792	.589‐.901
Ankle 2	27.10 (5.57)

Abbreviation: ICC= intraclass correlation coefficient.

^ap<.05.

Joint angles measured with the app consistently overestimated joint angles measured with the motion analysis system. Most prominently, hip angle measurements obtained with the app (mean, 127°±16°) exceeded measurements obtained with the motion analysis system (mean, 87°±11°) by approximately 40°. Differences at the knee and ankle were of lower magnitude, with mean differences of 5° and 3°, respectively. A systematic bias was found with the Bland‐Altman analysis at the hip, with the 95% coefficient of agreement −10.3° to −69.3° (Figure 2).

Figure 2.

Bland‐Altman 95% Limits of Agreement Between the Coach's Eye (TechSmith Corp) Tablet Computer Goniometer Application and the Vicon (Vicon Motion Systems Ltd) 3‐Dimensional Motion Analysis System for the Hip, Knee, and Ankle.

DISCUSSION

The primary objective of this study was to examine the reliability of the 2D Coach's Eye goniometer app during the performance of the deep squat. According to the values previously described by Shrout and Fleiss¹⁴ (ICC >0.75, excellent reliability; ICC 0.40‐0.75, fair to good reliability; and ICC <0.40, poor reliability), the current data suggest that sagittal plane hip, knee, and ankle angle measurements with the app demonstrate excellent test‐retest reliability. Clinically, the ability to reproduce measurements accurately is valuable in assessing outcomes of joint mobility interventions.¹⁶ Test‐retest reliability at the hip, knee, and ankle for this app was 0.978, 0.984, and 0.792, respectively, suggesting that it is a reliable tool for assessment of sagittal lower extremity joint ROM. To the authors’ knowledge, the reliability of measurements obtained with this app has not been described previously. ICC values of comparable goniometer apps have been reported. Werner et al¹⁷ reported an interrater reliability of 0.65 using five raters with varying degrees of experience to compare the use of a standard goniometer to the Clinometer (Plaincode Software Solutions, Germany), a smartphone app, at the shoulder. Likewise, Ferriero et al^11,12 reported test‐retest reliability of 0.958‐0.996 and of 0.998 for the knee and elbow, respectively, when results obtained with a standard goniometer were compared to those obtained with the DrGoniometer (Dr.Goniometer, Milano, Italy smartphone app). In a validation of a photography‐based goniometry method to measure elbow ROM, Blonna et al¹⁸ reported test/retest reliability ranging from 0.930 to 0.990. The current results support the utilization of the Coach's Eye app as a simple and reliable means to measure sagittal plane hip, knee, and ankle motion.

When compared to the 3D motion analysis system, the 2D app had greater reliability, which may be due to several factors. While the procedures outlining the placement of the reflective markers for the motion analysis system were well defined and a single examiner was used to place the markers, there was most likely some variation in the exact positioning of the markers from subject to subject or from session to session.¹⁹ In addition, it is possible that there was some movement of the markers during the squat because of skin movement.²⁰ In contrast, with the app, there may be less comparative variation in the placement of the reference lines.

MDC represents the smallest change in measurement over time that reflects a true threshold change rather than simple measurement error.¹⁵ MDC is used to indicate the level of responsiveness of a measurement. It is a function of both the SD of a set of measurements and the reliability coefficient, or ICC, for a given measurement.¹⁵ The lower the MDC, the more responsive an instrument is to smaller levels of change in measurement over time, which allows the clinician to be confident that noted changes in measurements represent true change. More specifically, if a subsequent measure is greater than the MDC, then the change is likely not due to measurement error.^21,22 The current study found MDC values for joint ROM measurements using the app of 6°, 6°, and 7° for the hip, knee, and ankle, respectively. In terms of practical use, repeated measurements at the ankle when using the app during a deep squat would have to be greater than 7° to indicate true change. Similarly, when measuring ankle dorsiflexion during a weight bearing lunge using a standard goniometer, Konor et al²³ reported MDC values of 5° on the left and 7.7° on the right.

The secondary objective of this study was to compare the kinematic data obtained via the 3D motion analysis system with that obtained with the 2D app. The mean average differences between these two systems were small at the knee and ankle joints; however, they were considerable at the hip joint. Using the 95% limit of agreement, calculated as the mean difference between devices ±1.96×SD of the mean difference, one can predict the expected range of measurement error between the 2D and 3D goniometric measurements. On the basis of the Bland‐Altman statistic for determining the 95% limit of agreement, the knee had a mean average difference of 5°, with a 95% limit of agreement ranging from −17.6° to 7.6°. The ankle had a mean average difference of 3.1°, with a 95% limit of agreement ranging from −14.6° to 8.3°. These results at the knee and ankle parallel differences found in previous app‐based goniometer studies.^12,24,25

A systematic bias was noted between the 2D and 3D systems for hip flexion angles, with the app overestimating the motion analysis system measurements by an average of 39.8°. This discrepancy may be due to a difference in the reference points used to calculate or measure the hip joint angle. Nussbaumer et al²⁴ suggested that current goniometer measurements of hip flexion measure a combination of “true” hip flexion and pelvic tilt, resulting in a thigh flexion angle on the trunk in contrast to a true Coxafemoralangle. Consequently, when hip ROM is acquired without consideration of normal pelvis motion, as was the case during the 2D measurement, recorded hip flexion values are likely greater than the true coxafemoral angle. According to Norkin and White,²⁶ the hip flexion angle (thigh to trunk measurement) specified by the American Academy of Orthopaedic Surgeons is 120°. This is similar to the measured hip flexion angle of 127˚ in this study using the 2D app. Conversely, Elson and Aspinall²⁷ reported a mean hip flexion angle of 85° using a standard goniometer. Included in their methods was monitoring of pelvic motion via palpation. Thus, their values may be a better reflection of true coxafemoral motion. Likewise, Hemmerich et al²⁸ used a six‐degree‐of‐freedom electromagnetic tracking system to measure hip, knee, and ankle ROM during activities of daily living, including squatting, and found a mean coxafemoral angle of 95.4°. The values of these two studies are similar to our measurement of 87.3° found with the 3D motion analysis system in the current study. The results for the app appear to systematically overestimate the hip flexion angle from measurement of a trunk‐thigh angle, whereas the motion analysis system more accurately measures true coxafemoral flexion. Thus, although both the 2D and 3D capture systems are reliable, clinicians should be aware that they likely represent two distinctly different measurements that are not interchangeable.

This study has limitations. The frame rate of the mobile app was 30 frames per second, which may have limited the ability of the researchers to capture the exact low point of a subject's squat. Additionally, while the two systems were used simultaneously to record performance, they were not time sequenced with specific time markers, which could compromise the ability to analyze the exact same squat position between the two systems. Given the speed of the squat performance, the authors do not believe that the frame‐rate limitation or the lack of sequencing had a meaningful impact on the current findings. Lastly, analysis of human movement using 3D or 2D technology is subject to instrument errors, errors in identifying anatomical landmarks, and soft tissue movement artifact. Although this study analyzed only sagittal‐plane lower‐extremity motion, future research should investigate other planes of lower‐extremity motion, as well as other joints and other movement patterns.

CONCLUSION

This study examined the reliability of a 2D goniometric tablet computer app to assess sagittal plane kinematic data at the hip, knee, and ankle joints during a deep squat and compared the results with those from a 3D motion analysis system. The current findings suggest that this app provides a reliable means of assessing sagittal plane kinematics during a deep squat maneuver. Measurements obtained at the knee and ankle are comparable to those obtained with the more sophisticated 3D motion analysis system. However, hip flexion measurements do not appear to be interchangeable between the two systems. Although each system is reliable, for clinical use, if an individual is assessed over time, consistent use of a single system is recommended for measuring sagittal plane hip motion. If it is important for the clinician to understand the motion occurring between the pelvis and the femur, the Vicon measurement is preferred, whereas if a thigh‐to‐trunk angle meets the needs of the clinician, the 2D analysis is a reliable tool to acquire this information.