Development and evaluation of a vision pose-tracking based Beighton score tool for generalized joint hypermobility in individuals with suspected Ehlers-Danlos syndromes

Abstract

Background

Generalized joint hypermobility (GJH) is often challenging to assess, but its presence could suggest a syndromic diagnosis of Ehlers-Danlos Syndromes (EDS).

Objective

An automated and objective method for estimating joint hypermobility with Beighton score using short video clips is proposed.

Method

A total of 225 adults (91.8% female, median age 32.0, range 18–64) referred to a specialized EDS clinic were recruited for this study. A video-based method relying on pose-estimation libraries was developed to predict per-joint hypermobility of both elbows, knees, fifth fingers, thumbs, and spine; as well as the overall Beighton score. The system was developed on the first 100 individuals (training set), and validated on the remaining 125 individuals (test set).

Results

The system screened out 31.9% of the training set and 32.0% of the test set as not having GJH, while recalling 89.1% and 91.9% of the true positives on the train and test set, respectively. The consistency of the system between the training and test sets suggests that it generalizes well to unseen individuals. The system was tuned to be with a focus on sensitivity to avoid screening out individuals with GJH. As such, the specificity of the system is 52.1% on the training set and 42.4% on the test set.

Conclusion

The proposed system can objectively screen individuals for possible GJH and also screen out those without GJH during the referral process, reducing the burden on specialized EDS clinics while providing early diagnostic triage. Future research will focus on deploying the tool as a mobile application.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12938-026-01527-4.

Introduction

Generalized joint hypermobility (GJH) describes the wide-spread presence of hypermobility in the body, usually affecting upper and lower extremities and torso [1]. The presence of GJH is typically assessed using the Beighton score, whereby a clinician performs a physical examination of joint hypermobility and assigns a score out of 9 tested joints, with one point for each of the joints assessed that meet the clinical criteria for hypermobility [2]. The movements assessed as part of the Beighton exam are the active hyperextension of the knees and elbows past 10º, the passive hyperextension of the fifth fingers over 90º, the ability of the thumb to be touched to the forearm, and the ability to touch the floor with the palms while the knees remain extended [2]. The movements are evaluated independently for each side of the body, and the overall Beighton score is incremented by one point for each movement where the clinical threshold for hypermobility is met.

While GJH may represent a normal genetic or physiologic trait with minimal impact on an individual’s quality of life, it can also be linked to inherited connective tissue disorders (CTD) such as Marfan Syndrome, osteogenesis imperfecta, and Ehlers Danlos Syndromes (EDS) [1, 3–5]. Individuals with GJH may develop chronic pain and the ones who do not meet the strict criteria required for a narrower diagnosis of a specific inherited CTD are said to have generalized hypermobility spectrum disorders (G-HSD) [1]. The objective identification of GJH is thus important to provide support and appropriate medical care to individuals on the disease spectrum.

A Beighton score of 4 or greater out of 9 is generally considered suspicious of GJH in adults and warrants further clinical evaluation [5–7]. It is important to note that age-specific thresholds are recommended in clinical practice for formal diagnosis of GJH (6 or greater for pre-pubertal children, 5 or greater from puberty to age 50, and 4 or greater for individuals over 50 years of age) [3].

Despite the existence of a clearly established standard procedure for evaluating the Beighton score [8, 9], practitioners often lack knowledge of the exact steps required or don’t have access or time to use a goniometer for conducting this assessment. This flawed evaluation has the potential to cause unwarranted concern in susceptible patients, prompting them to pursue unneeded investigations and seek a diagnosis for a genetic disorder. While some studies have found that the intra- and inter-rater reliability of the Beighton exam is high, these assessments were conducted using a small number of clinicians who were allowed time to discuss and calibrate their approaches prior to beginning the experiment [10]. Outside of a research setting, such calibration is not possible between clinicians practising at different locations, leading to discrepancies in the Beighton score assessed by different clinicians.

One way to improve the objectivity of the Beighton exam is to use a deterministic algorithm for screening for joint hypermobility from short video clips. Smartphones are widely available today and can thus be used to record short videos of individuals performing the movements of the Beighton exam in home or primary care settings. Human pose-estimation libraries [11–13], which use machine learning models to estimate the location of body joints in an image or video, can be used to determine how each joint of interest moves during assessments and predict hypermobility. Individuals who exhibit low hypermobility in videos can be screened out as being unlikely to have GJH.

Conversely, individuals whose algorithmically calculated Beighton score is above the screening threshold of 4/9 can be evaluated by a clinician and referred to a specialized clinic for formal diagnosis, if appropriate.

Automated landmark extraction utilizing the OpenPose and MediaPipe human pose-estimation libraries has been used to test vision-based goniometry for measuring upper body joint and hand ROM in healthy individuals [14, 15]. Fan et al. showed a strong connection between their automated vision-based tool and clinician goniometer results, with a mean difference of less than 3º for most maneuvers [15]. Our lab has investigated multiple human pose-estimation libraries for estimating upper and lower body joint maximum ROM from video in suspected EDS/G-HSD patients [16]. In this cohort, video angles and ground-truth goniometer measurements correlated moderately to strongly (Spearman's rho = 0.608–0.786 on joints examined as part of the Beighton exam) [16]. Video-based goniometry can automatically measure ROM, but it has not been validated for clinical joint hypermobility screening.

Therefore, in this study we developed and validated an automated vision-based system for assessing joint hypermobility using the Beighton score. The primary goal of the system was to facilitate the screening-out of individuals without GJH, while retaining those with potential or likely GJH. The performance of the system was compared to ground truth determinations of joint hypermobility as assessed by clinicians through a physical examination.

Results

A detailed description of the methodology and approach for the development of the vision-based screening system is presented in the Methods section at the end of the manuscript. A total of the 225 individuals participated in this study. The first 100 participants formed the training set, and the last 125 formed the test set. Four individuals withdrew from the study after data collection, four individuals were missing all clinical data, and one individual was missing both clinical and video data. The full cohort was 91.8% female (train set: 91.8%; test set: 91.9%). The median age of the entire cohort was 32.0 (range: 18–64; IQR: 16.0). The median age of the train set was 34.0 (range: 18–64; IQR: 14.8) while the median age of the test set was 31.0 (range: 18–56; IQR: 16.5).

Table 1 presents the number of recorded videos and clinical assessments of joint hypermobility for each joint, as well as the prevalence of clinically hypermobile joints in the training and test sets.

Table 1.

Number of recorded videos, clinical data records, and the prevalence of positive hypermobility findings by joint

	Elbow				Knee				Fifth finger				Thumb				Spine		Overall
	Left		Right		Left		Right		Left		Right		Left		Right
	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
Number of participants with recorded video of joint assessment	95	120	96	120	95	123	96	121	97	123	96	122	97	118	92	120	96	120	97	123
Number of participants with recorded clinical data (angle measurement/ Beighton score)	94	122	94	122	94	122	94	122	94	122	94	122	94	122	93	122	94	122	94	122
Percent of participants with positive hypermobility finding (as determined by clinicians)	12.8	9.0	14.9	8.2	34.0	21.3	35.1	20.5	36.2	17.2	22.3	11.5	72.3	71.3	64.9	64.8	63.8	59.0	48.9	30.3

As seen in Table 1, the training set (the first 100 participants) included more hypermobile joints for all joints except thumbs. The training set included 48.9% of people with a clinician-assessed Beighton score of 4 or more out of 9, while 30.3% of participants in the test set met this threshold.

Selection of hyperparameters for spine and thumb joints

For the spine and thumb joints, the parameters for identifying hypermobile examples were tuned experimentally. The best-performing algorithm for identifying hypermobility in the spine achieved a recall of 85.0% and a precision of 83.6% on the training set, with a class-weighted F1-score of 0.843. The best configuration for the thumb movement attained a recall of 97.0%, and precision of 94.0% on the training set, with a class-weighted F1-score of 0.960.

Selection of thresholds for elbow, knee, and fifth finger joints

After the parameters for the algorithms for identifying hypermobility in the thumb and spine movements were selected (as described in Sect. "Selection of hyperparameters for spine and thumb joints"), a grid search was conducted to select the thresholds for the elbows, knees, and fifth fingers that yielded the best overall system performance. Based on these experiments, an angle of hyperextension greater than 3˚ (as calculated from the videos) was used to identify hypermobility of the knee, an angle of hyperextension greater than 15˚ was used to identify hypermobility of the elbow, and an angle of hyperextension greater than 77.5˚ was used to identify hypermobility in the fifth fingers.

Per-joint hypermobility assessment on test set

After selecting the thresholds and hyperparameters for distinguishing between positive and negative hypermobility findings for each joint on the training set, the performance of the data analysis workflow was assessed on the test set. Table 2 presents the precision, recall, and class-weighted F1 score for each joint on the test set after interpolating any missing values.

Table 2.

Precision, Recall, and F1 Score by Joint on Test Set

	Test set missing % (requiring interpolation	Prevalence of positive class on test set	Model performance on test set
			Precision	Recall	F1 score
Spine	0.8%	59.0%	0.704	0.792	0.745
Knee—Left	0.8%	21.3%	0.280	0.885	0.426
Knee—Right	0.8%	20.5%	0.324	0.880	0.473
Elbow—Left	0.8%	9.0%	0.350	0.636	0.452
Elbow—Right	0.8%	8.2%	0.350	0.700	0.467
Fifth Finger—Left	5.7%	17.2%	0.364	0.762	0.492
Fifth Finger—Right	6.6%	11.5%	0.375	0.643	0.474
Thumb—Left	5.1%	71.3%	0.809	0.874	0.840
Thumb—Right	9.2%	64.8%	0.703	0.810	0.753

Vision-based Beighton score calculation and generalized joint hypermobility screening

The overall Beighton score was calculated by summing the hypermobility scores for each of the nine individual joints assessed. Figure 1 presents the confusion matrix for the distribution of the total Beighton score calculated by the vision-based system as compared to the clinicians’ assessments on the test set.

Fig. 1.

Confusion matrix comparing the Beighton scores assessed from video to those determined by a clinician through a physical examination on the test set

As is done with the clinician-assessed Beighton exam, a score of 4 or greater generally warrants further investigation, and was thus chosen as the threshold to be a considered a borderline positive exam. Individuals scoring 4 or more on the vision-based assessment were “screened-in” as having potential GJH by the system, while those with a video-assessed Beighton score of 3 or less were “screened-out”. Table 3 presents the number of participants screened out by the system for the selected top-performing configuration. The screen-out rate, sensitivity, specificity, negative predictive value (NPV), and precision are presented separately for the train and test sets.

Table 3.

Number of exams before and after screening using automated vision-based Beighton assessment tool

	Starting count	Starting positive count	Positive rate before screening	Count of negatives screened out	Count of positives screened out	Screen-out rate	Recall/sensitivity	Specificity	Negative predictive value	Precision/positive rate after screening
Train	94	46	48.9%	25	5	31.9%	89.1%	52.1%	83.3%	64.1%
Test	122	37	30.3%	36	3	32.0%	91.9%	42.4%	92.3%	41.0%

As seen in Table 3, the starting rate of positive Beighton exams (score of 4 or more as determined by the clinicians) was significantly higher for the training set (48.9%) than the test set (30.3%). However, the screen-out rate and sensitivity were similar across both data splits, suggesting the vision-based screening method generalizes well to unseen data. The precision is lower on the test set, meaning a higher proportion of individuals with a clinician-assessed Beighton score < 4 are screened-in by the vision-based system. This is due to the lower starting rate of positive clinician-assessed GJH findings in the test set. For the same reason, the NPV (the percentage of true negatives in the screened out individuals) is lower on the train set than on the test set.

As a comparison to the proposed vision-based system, an experiment in which participants were randomly screened-out was conducted. The screen-out rate used for this simulation was 32.0%, which corresponded to the screen-out rate of the video-based method on the test set (Table 4). Table 4, below, presents the average recall, precision, and specificity over 100,000 simulated trials of the random removal experiment. The percentile of the performance of the proposed vision-based system on the test set when compared to the simulated random removal trials is also presented for each statistic.

Table 4.

Simulated random removal recall, precision, and specificity; and comparison to vision-based system

	Recall		Precision		Specificity
Percent of participants randomly screened-out	Random removal mean ± standard deviation	Vision-based system percentile	Random removal mean ± standard deviation	Vision-based system percentile	Random removal mean ± standard deviation	Vision-based system percentile
32.0%	68.0 ± 6.4%	99.9%	30.3 ± 2.9%	100%	24.6 ± 1.6%	99.9%

As presented in Table 4, the mean recall, precision, and specificity achieved through the random removal process was significantly lower than when the vision-based system was used. The vision-based system ranked above the 99.9th percentile when comparing to the random removal trials across all metrics.

Discussion

Summary of key findings

In this work, we proposed an automated vision-based system for screening for potential joint hypermobility using the Beighton score. This system was developed with the aim of screening individuals out who do not meet the criteria for potential GJH (Beighton score < 4) to facilitate triaging of those scoring 4 or greater for further evaluation and potential diagnosis by clinicians.

In our previously published protocol manuscript, we established a performance target of a recall of 70% while screening out half of the individuals, assuming that the rate of positive Beighton exams in the cohort was 15% [17]. To account for the higher prevalence of positive Beighton exams in this study (48.9% and 30.9% on the train and test sets, respectively) when compared to previously anticipated positive rate of 15%, a lower proportion of individuals were screened out (32.0% of the test set—Table 3). At this screen-out rate, our vision-based system achieved a recall of 89.1% on the train set and 91.9% on the test set (Table 3).

Through our experiments, we found that vision-based angle cut-offs of 3° at the knees, 15° at the elbows, and 77.5° at the fifth fingers were effective for identifying hypermobility at these joints. It is important to note that these angle measurements obtained from video analysis are not directly comparable to those measured using a physical goniometer. As demonstrated in our prior work [16], video-based joint angle measurements correlate moderately to strongly with goniometric measurements (Spearman’s ρ = 0.608–0.786), but they are influenced by sources of error including camera positioning, 2D projection of 3D movement, and pose-estimation model performance. Because video-based pose estimation is fundamentally limited compared to a physical examination and is subject to perspective effects and less precise keypoint definitions, what registers as 10° of hyperextension during clinical goniometry may translate to a different numerical value when computed from 2D skeletal keypoints extracted from video. To account for these differences, we conducted a systematic grid search optimization to empirically derive video-based thresholds that maximized agreement with clinician-determined Beighton assessments, which served as the gold standard throughout this study. The resulting thresholds should therefore not be interpreted as direct deviations from the standard Beighton criteria (10° for knees and elbows, 90° for fifth fingers), but rather as calibrated values in the video measurement space that best distinguished hypermobile from non-hypermobile findings at each joint in our training cohort. The system’s final output remains a standard Beighton score interpreted according to established clinical guidelines, where a score of four or greater indicates potential generalized joint hypermobility (GJH), warranting further evaluation. This calibration enables the automated system to produce screening results consistent with existing clinical practice.

In some instances it may be preferable to screen-out a smaller or larger proportion of the starting cohort. In this work, a cut-off threshold on 4 was used, but it is possible to select a different threshold on the Beighton score calculated by the vision-based system used to denote the boundary between positive and negative hypermobility exams. Table B of the Supplement presents the discard rate, recall, specificity, and precision of the system when the cut-off score for differentiating between a positive and negative Beighton exam was varied from 0 to 9. As seen in Table B, when the threshold is set to 3, a recall of 97.3% was achieved by the system, at the trade-off of a much lower screen-out rate of 13.1%. Conversely, if the threshold is raised to 5, the recall is lowered to 75.7%, but 54.9% of the participants are screened out. The same trend holds for the other Beighton score thresholds where higher cut-off thresholds result in more individuals being screened-out at the expense of lower recall values.

Comparison with related work

Recent studies have demonstrated similar feasibility of vision-based joint assessment systems. For example, Cholewa et al. [18] achieved concordance correlations of 0.89–0.98 for camera-based knee ROM assessment in healthy volunteers, while Dy et al. [19] reported excellent inter-rater reliability (0.80–0.99) for video-based shoulder and elbow assessment in brachial plexus injury patients. Critically, systematic differences between video-based and goniometer measurements have been documented by other groups. Nedović et al. [20] found that AI-powered pose estimation overestimated ROM by 8–10 degrees compared to digital goniometry despite excellent inter-repetition reliability (ICC 0.91–0.96), with only moderate inter-method agreement (ICC 0.66–0.71) in knee flexion and hip extension. Similarly, in our previous work [16], we found that angles of hyperextension calculated from video were correlated, but not calibrated, with goniometric measurements taken by expert clinicians. For this reason, the video-based thresholds (3° for knees, 15° for elbows, 77.5° for fifth fingers) identified in this work differ numerically from goniometer criteria, but successfully identify the same individuals as hypermobile as clinicians.

To the best of our knowledge, the only other published work assessing Beighton maneuvers from video is by Kudithi et al. [21], who developed a custom HybridPoseNet model for directly assessing joint hyperextension angles. Their proposed model (combining MobileNet-V2 with HyperLSTM) was used to estimate joint angles for thumb, elbow, knee, and fifth finger from video in 50 samples with high correlation to goniometric measurements (Spearman’s rho 0.822–0.896). The authors divided their dataset into train (40 samples) and test (10 samples) sets, but it is unclear how many total videos were used to train the models or their content of the videos (joint hypermobility movements vs. skin extensibility). Furthermore, the focus of Kudithi et al.’s [21] work was on comparing video-derived angles of hyperextension to manual goniometry measurements rather than evaluating the system's ability to digitize the Beighton assessment or replicate clinical screening decisions. Conversely, our work presents the first comprehensive validation of an automated Beighton scoring system evaluated against clinician assessments rather than goniometry.

Clinical applicability and implementation of proposed system

The primary clinical utility of our vision-based system lies in its ability to screen out individuals who are unlikely to have generalized joint hypermobility (GJH), thereby reducing unwarranted referrals to specialized EDS clinics that often face long wait times and limited capacity. The tertiary centre where this study was conducted receives an average of 60–80 referrals per day for suspected EDS. As there are only a few centres providing specialized EDS/CTD care, most comparable centres experience similarly high referral volumes, leading to resource constraints and delays in assessing patients in a timely manner. These large numbers of referrals also prolong wait times, creating uncertainty and frustration for individuals who ultimately do not have EDS/CTD and may instead have an alternative medical condition requiring appropriate evaluation and management. Our vision-based system was developed to help address these challenges.

In our test set, the system successfully identified 32.0% of referred individuals as unlikely to have GJH while maintaining high sensitivity (91.9%) and negative predictive value (92.3%) for those with clinician-suspected hypermobility. This screening capability is particularly valuable in the context of our study population, which consisted entirely of individuals already referred to a specialist clinic by other healthcare providers. The ability to objectively identify approximately one-third of these pre-selected referrals as unlikely to benefit from specialist evaluation represents a meaningful reduction in clinic burden and could allow specialists to focus their expertise on individuals more likely to require their care.

It is critical to emphasize that our system is designed as a screening tool, not a diagnostic instrument. Individuals who screen positive (with a vision-based Beighton score of 4 or greater) require comprehensive clinical evaluation by an experienced clinician. The Beighton score represents only one component of the diagnostic criteria for conditions such as EDS, which also require assessment of symptoms, family history, skin characteristics, and other clinical features which are not captured by our video analysis system. Furthermore, the current specificity of our system (42.4% on the test set) means that among screen-positive individuals, approximately 58% will not meet clinical criteria for GJH upon formal assessment. While this false-positive rate may seem high, it reflects our deliberate design choice to prioritize sensitivity over specificity, which is a critical construct of medical screening tools, ensuring that very few individuals with true GJH are incorrectly screened out. For a triage tool used in already-referred populations, this trade-off is clinically appropriate.

The implementation of our proposed system would fit within existing referral pathways, potentially serving as an intermediate step between primary care referral and specialist appointment. Primary care physicians could use the system to support their referral decisions, or specialist clinics could request video submissions from referred patients prior to scheduling in-person appointments. The proposed system should be viewed as a decision-support tool rather than a replacement for clinical expertise, with the understanding that its value lies in confidently identifying individuals who do not require specialist evaluation rather than definitively diagnosing those who do.

Limitations and future work

There are several limitations of this work that will be addressed in future stages of the project. As previously discussed, the proposed system frequently failed to assess hypermobility of the thumb joint from video (Table 5). This was often due to occlusion (Fig. 2a) or poor positioning of the thumb (Fig. 2b). In this study, data was collected when participants visited the clinic, but analysis of the collected data was performed at a later date. Therefore, it was not possible to identify and re-record the videos which could not be analyzed by the system. Future work will focus on deploying the video-analysis system in a smartphone application which will analyze the data immediately after recording and allow the user to re-record any videos which are not captured correctly.

Table 5.

Percent of videos where Beighton scores could not be calculated from video

Joint	Percentage of videos where Beighton score could not be calculated
	Train set	Test set
Spine	1.1%	0.8%
Knee—Left	2.1%	0.8%
Knee—Right	1.1%	0.8%
Elbow—Left	2.1%	0.8%
Elbow—Right	1.1%	0.8%
Fifth Finger—Left	17.0%	5.7%
Fifth Finger—Right	19.1%	6.6%
Thumb—Left	33.0%	5.1%
Thumb—Right	36.2%	9.2%

Fig. 2.

a Unobstructed (left) and obstructed (right) frames of the thumb flexion movement. Individuals were verbally instructed to try to perform the movement while not obstructing their thumb, however some individuals were unable to achieve the same level of hyperflexion without a firm grasp of the thumb; b Examples of correct (left) and incorrect (right) framing of the thumb flexion movement. Participants were instructed to perform the thumb flexion movement with their elbow extended and starting with their palm facing the floor. If the hand was rotated such that the fingers were not pointed directly at the floor, the index finger appears shorter from the perspective of the camera. This apparent shortening of the index finger can be detected, and the vision-based system will not use the frames where the hand is rotated to make a determination of thumb hypermobility

An additional limitation is the lack of formal standardization of force applied during passive movements (thumb and fifth finger assessments) between the clinical and video-recorded Beighton assessments. While participants were given standardized verbal instructions for both assessments, the actual force applied may have varied between repetitions, potentially introducing measurement variability. This reflects a general challenge in Beighton score assessment, as force standardization is difficult to achieve even in traditional clinical settings. Furthermore, the repetition of the assessment that was video recorded was always performed approximately 30 min after the repetition that was assessed by the clinician. It is possible that the participants’ joints were temporarily loosened due to the clinical assessment and were thus more mobile in the recorded repetition.

Another limitation is that all videos in this study were recorded in a controlled clinic environment by trained research personnel using standardized equipment and protocols. This controlled setup ensured consistent video quality, camera positioning, and lighting conditions, which may not reflect the variability encountered when patients record videos independently at home. Patients attempting to self-record may face challenges such as inconsistent camera angles, inadequate lighting, or difficulty positioning themselves appropriately in frame. We are currently conducting a study in which we deploy our proposed system in a smartphone application and evaluate its performance when individuals use it independently in a home setting. This ongoing work will assess the system's performance when data is collected by many different individuals, using different mobile devices in a variety of physical environments, and will compare the reliability of clinic-recorded versus patient-recorded assessments.

A further limitation of the current study is that we did not collect detailed anthropometric or demographic data such as height, weight, body mass index (BMI), or ethnicity in this study. While our use of geometric angles calculated from joint positions (rather than absolute distances) should theoretically make measurements largely invariant to body size, the potential influence of these factors on pose estimation accuracy remains unexplored in this population. Individuals with higher BMI may present challenges for accurate keypoint detection, particularly when soft tissue obscures anatomical landmarks. Similarly, pose estimation algorithms trained predominantly on certain demographic groups may perform differently across diverse populations. Future work should explicitly investigate whether body size, BMI, and ethnicity affect system performance and should include validation in more diverse cohorts to ensure the system generalizes appropriately across different patient populations.

A final limitation is that our study population consisted entirely of individuals referred to an EDS specialist clinic with suspected hypermobility conditions. While this population represents the intended clinical use case for our screening system, the absence of healthy controls limits our ability to evaluate system performance across the full spectrum of joint hypermobility. Including a general population cohort without suspected hypermobility would provide important context for interpreting our specificity results and would help establish whether the system's performance differs between highly selected clinical populations and unselected community samples. Such validation in healthy controls would also enable assessment of the system's positive predictive value in lower-prevalence populations, which is particularly relevant if the system were to be deployed for broader population screening rather than triage of already-referred individuals.

Beyond addressing these limitations, future work will focus on technical improvements to improve system performance. We plan to leverage the heterogeneous data from our ongoing home recording study to train more robust models across varied recording conditions. Additionally, we will investigate 3D pose estimation libraries, which could provide more accurate joint angle calculations that are invariant to camera positioning and account for depth information. Finally, we will further fine-tune our hand pose estimation models by expanding our training dataset with additional manually annotated landmarks, particularly for challenging cases such as hand occlusion and varied body positions. We believe these improvements to keypoint localization will improve overall screening performance.

Conclusion

In this work, we presented the development and validation of a vision-based system for assessing GJH using the Beighton scale. The proposed system is explainable, providing a prediction of whether each of the nine joint movements assessed as part of the Beighton exam meet the cut off threshold for joint hypermobility. The system was tuned to screen-out individuals who likely do not have GJH, while retaining those who are borderline or have GJH for formal examination and diagnosis by a clinician. This system delivered a screen-out rate of 32.0% while screening-in 91.9% of the positive cases (as determined by the clinician) on a dataset of unseen participants who were previously referred to the clinic with a Beighton score of 4 or more.

Methods

Data collection

A total of 225 individuals undergoing an initial evaluation of joint hypermobility at the GoodHope EDS clinic at Toronto General Hospital were recruited to participate in this study [22]. All patients seen at the clinic had a suspected or previously confirmed diagnosis of EDS/G-HSD by another health care provider (Beighton score > = 4, as assessed by the referring clinician). Note that less than half of the recruited cohort was found to have a Beighton score > = 4 when assessed by the clinicians at the EDS clinic (Table 1).

During a single clinical visit, participants were asked to perform the Beighton maneuvers twice: the first in which the clinician assessed their joint hypermobility, using a goniometer (if appropriate), and the second in which the participant was verbally instructed to perform the Beighton assessment while a tripod-mounted smartphone camera was used to record their movements. The second (video-recorded) repetition was performed at the end of each participant’s clinical appointment, approximately 30 min after the first repetition. For both assessments, participants performed the Beighton maneuvers without direct assistance, using active movement for the knees, elbows, and forward flexion, and self-applying force using their other hand for the passive extension of the thumbs and fifth fingers. A Motorola Moto G Pure or Motorola E5 Play (1080 × 1920 pixels, 30 Hz) smartphone was used to record short videos of each joint movement, with the orientation and distance from the camera controlled between participants. The recorded videos were downsampled spatially to 540 × 960 pixels before processing to speed-up the analysis time.

This study was approved by the Institutional Research Ethics Board on 26 April 2022 (Study ID—22–5073.2) and conducted in accordance with the Declaration of Helsinki. The participants were introduced to the study in-person and were given as much time as they needed to ask questions and review the consent form. If they were interested in participating in this research study, the participants provided written consent.

The participants were reminded of their right to decline to undergo assessment of any joint for the purposes of this study if they were experiencing any pain or discomfort, leading to differences in the number of recorded videos by joint. A detailed description of the data collection protocol is available in our protocol manuscript [17].

To facilitate development and evaluation of an automated, vision-based Beighton assessment tool, the collected data was divided into two non-overlapping sets. The first 100 participants were delegated to the training set, while the subsequent 125 participants were recruited as part of the test set. Clinician-assessed Beighton scores of 4 or greater were classified as positive exams requiring further clinical assessment, consistent with standard clinical practice in this adult population.

Prediction of Beighton score from video

As a first step towards assessing joint hypermobility from video, the locations of important body joints and landmarks were extracted using appropriate pose-estimation machine learning models. The input to these models is the image or video frame for which a prediction is needed, and the output is the (x, y) positions of each keypoint in the image, as well as a confidence score describing how certain the model is of its predicted location for each joint.

After identifying the key points in each video, the predictions for each frame were combined to create a continuous trajectory of the movement of each joint over time. Joint positions at timesteps where the model was unsure about its estimate were interpolated using the keypoint locations at datapoints that were close in time. The smoothed trajectories were then used to determine if the joint exhibited hypermobility. A more detailed description of this process is presented in Appendix A of the Supplement. The entire data processing workflow (including human pose-estimation and angle calculation, described below) is deterministic, ensuring that the same input video will consistently yield the same hypermobility assessment.

Elbow, knee, fifth finger

The in-clinic assessment of joint hypermobility at the elbows, knees, and fifth fingers requires the measurement of the maximum angle of extension, and then comparing this value to a predetermined threshold. Analogously, the angle of extension at the elbows, knees, and fifth fingers was calculated geometrically using the smoothed trajectories at each joint of interest. A detailed description of how the joint angles were calculated is presented in our prior work [16].

Spine

The clinical assessment of hypermobility at the spine requires observation of whether an individual is able to place their palms flat on the floor in front of their feet while their knees remain straight. This general approach was replicated when evaluating the hypermobility of the spine joint from video. The vertical position of the wrist keypoints was compared to the vertical position of the ankle keypoints, while simultaneously evaluating whether the knee closest to the camera was straight. To reduce the number of false positives due to jitter in the pose-estimation output, a minimum confidence per joint, as well as a minimum number of timesteps for which the criteria were met were also imposed.

The algorithm was tuned to recall a minimum of 80% of the positive cases in the training set, with the configuration yielding the highest F1-score selected.

Thumb

The clinical determination of hypermobility of the thumb involves assessing whether it can be pushed back to touch the forearm while the elbow is straight and the arm is extended in front of the body. The thumb is more challenging to assess from video as the movement requires the participant to use their other hand to manipulate the thumb which can obscure the camera’s view of the thumb being assessed. While participants were given explicit instructions to avoid obstructing the camera’s view, many individuals were not able to demonstrate their maximum level of hypermobility if their thumb was not obstructed by their other hand. Figure 2, below, provides examples of unobstructed and obstructed views of this movement.

Thumb hypermobility was assessed from video by training a specialized pose-estimation model to track keypoints in the hand and distal forearm and determining whether the distance between the thumb and forearm keypoints were sufficiently close together.

The configuration of parameters which yielded the highest F1-score while recalling at least 80% of the positive cases on the train set was selected.

Prediction of hypermobility in joints where heuristic algorithms were unsuccessful

There were instances where patients experienced pain and could not perform specific Beighton maneuvers or where the vision-based system could not determine if a joint satisfied hypermobility criteria. These cases require imputing missing numbers to calculate a Beighton score of 9. Table 5 shows how many videos per joint needed missing value interpolation.

We interpolated missing values using Random Forest (RF) models. Nine RF models were developed, each using the determinations of hypermobility of the other eight joints of the Beighton exam (as calculated from the video) as input and returned a binary decision on the remaining joint's hypermobility. Any missing values of the eight joints used to make the prediction were interpolated using the mean prevalence of the positive class at that joint in the training set. Each model was fit on the entire training set and used to interpolate missing values on the test set. More detail about the hyperparameter tuning of the RF models is presented in Appendix C of the Supplement.

Calculation of total Beighton score

The total video-estimated Beighton score was calculated by summing all of the estimated per-joint scores of joint hypermobility.

Ablation experiments for selection of best system performance

Ablation experiments were conducted whereby the angle thresholds for the knee, elbow, and fifth finger joints were varied and the performance of the overall system in screening for GJH was assessed. As demonstrated in our previous work [15], while angles of hyperextension calculated from video correlate with goniometric measurements, they are not calibrated to the same scale, necessitating empirical determination of appropriate video-based thresholds. Knee angles of hyperextension (as calculated from video) were varied from 1° to 6°, elbow angles of hyperextension were varied from 11° to 15°, and fifth finger angles of 75°–80° were assessed. A grid search approach was used to evaluate all combinations of parameters.

The combination of screen-out rate, recall/sensitivity, and specificity was optimized on the training data set through consultation with a clinical expert who provided guidance with respect to the appropriate minimum levels of each for the system to have clinical usefulness.

Statistical analysis and software information

Performance of the vision-based system was evaluated using standard classification metrics: sensitivity (recall), specificity, precision, and the F1 score. The F1 score is the harmonic mean of precision and recall, providing a balanced measure of system performance that accounts for both false positives and false negatives and is calculated as: F1 score = 2 × (precision × recall)/(precision + recall). We report the class-weighted F1 score, which weights the positive and negative classes equally, and accounts for class imbalance in the dataset.

All data processing and statistical analyses were performed in Python 3.8.12. The Shapiro–Wilk test function in the Scipy package (v1.8.0) was used to evaluate whether the participant ages were normally distributed. The calculation of system performance (sensitivity, specificity, preparation of confusion matrices) was done using the scikit-learn package (v.1.0.2). Scikit-learn was also leveraged to interpolate missing values using its RF model and grid search capabilities.

Author contributions

NM, AD, and BT conceived and organized the study. All authors contributed to the design of the study and preparation of study procedures. AS and NM collected the data. AS processed the data, performed the statistical analysis, and drafted the manuscript. NM, AD, and BT interpreted the results and critically revised the manuscript. All authors approved the final version.

Funding

This research is funded by a philanthropic donation from Canerector Foundation. The funding body above did not have any role in the design of the study or collection, analysis, and interpretation of data.

Data availability

The data collected as part of this study are not publicly available due to the identifying and personal nature of the videos.

Declarations

Ethics approval and consent to participate

This study was approved by the University Health Network Research Ethics Board Toronto on April 26, 2022 (protocol 22–5073) and was conducted in accordance with the Declaration of Helsinki. Written consent was obtained from all participants prior to data collection.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.