A machine learning model for frailty based on wearable device measurements

Abstract

Background

Frailty is an important factor in human aging associated with a broad range of adverse outcomes. Frailty metrics are time intensive to collect making them difficult for larger scale application.

Methods

We apply machine learning to predict these frailty metrics, associated risk factors, and adverse outcomes from activity data. We use activity data collected using Actigraphy wearable accelerometer sensors, which are devices that measure acceleration along three axes of movement. Models were evaluated using Area Under the receiver operator Curve (AUC), Area Under Precision Recall Curve (AUPRC), Spearman rank test, Mann-Whitney U test, or Kruskal-Wallis test on repeated subsampling of train and test sets. All statistical tests are reported using -log₁₀(P-value).

Results

Machine learning models show strong predictive performance even with small amounts of accelerometry data available. They are also able to better determine adverse outcomes such as hospitalization and mortality than frailty metrics themselves in our geriatric population.

Conclusions

This approach of wearable activity data-based prediction of frailty offers a surrogate (proxy or estimate) for determining frailty metrics in a scalable manner. It can also be used to determine adverse outcomes such as hospitalizations and mortality, allowing frailty to be used as a metric in other studies or medical practices.

Plain language summary

Frailty occurs during human aging and is associated with a broad range of unfavourable outcomes. Frailty is measured using various scores but these often rely on subjective information, are labor intensive to measure, and are not assessed over time. This work presents objective measures indicative of frailty, based on wearable sensors that measure movement. This was tested in a group of people with an average age of 75. Application of a computational model using this data enabled long-term outcomes, including hospitalization and death, to be more accurately predicted than using existing frailty measures. This work demonstrates that 48 hours of data collection per patient is sufficient. This type of system could be used on a larger number of people, enabling those at risk of unfavourable outcomes to be targeted with medical interventions or support.

Culos, Manas et al. construct machine learning surrogate frailty metrics from activity data collected in a nominally intrusive manner. They show that a limited amount of activity data is necessary to model frailty metrics allowing for an increased proliferation of their application along with a robust source of data for other age-related outcomes.

Introduction

According to World Population Prospects 2019¹, there are 703 million people aged 65 and over in the world, while the number of older people is projected to double to 1.5 billion in 2050. The most problematic manifestation of aging societies is the clinical condition of frailty^2,3. Campbell and Buchner defined the term of frailty in 1997⁴ as ‘a condition or syndrome which results from a multi-system reduction in reserve capacity to the extent that a number of physiological systems are close to, or past, the threshold of symptomatic clinical failure. As a consequence, the frail person is at increased risk of disability and death from minor external stresses’. Frailty has since arisen as a priority in daily clinical practice and in the care of older adults⁵, thus numerous frailty measures or scales have been proposed. Although there is no accepted gold standard frailty metric, there are two widespread approaches, the frailty phenotype⁶ model and the frailty index of cumulative deficits⁷. The former is used in daily clinical practice and is based on a physical function model, while the latter extensively evaluates possible health deficits to quantify the cumulative effect⁷. Despite widespread use, these two scales have been criticized for their definition, methodology, risk classification, and ability to predict adverse events^8,9. Overcoming some of the limitations of these frailty measures a new operative scale named Frailty Trait Scale (FTS), based on characteristics of the biological trait of frailty syndrome has been developed¹⁰. The FTS incorporates new and relevant frailty components according to recent findings on frailty pathophysiology and has been suggested as a more sensitive instrument for detecting changes in the individual’s biological status than the previously validated scales¹⁰. Authors also designed a short form derived from the FTS, that can be used in clinical settings in daily practice and detect changes in frailty status after an intervention (FTS₅)¹¹. Despite the development and use of all these scales, the implementation of frailty assessment in clinical settings remains a challenge^5,12.

Regular physical activity has been shown to protect against the condition of frailty and reflect an individual’s frailty^13,14. Wearable device derived measures^15,16 of basal activity levels can then inform us about physical characteristics predictive of frailty or other outcomes of interest. Previous studies have associated activity with individual health characteristics¹⁷, classes of frailty^18–20, or biological age^21,22. However, models directly estimating multiple frailty metrics, to our knowledge, have not yet been developed.

This study presents Extreme Gradient Boosted (XGBoost) Machine Learning (ML) models that leverage wearable accelerography technologies to predict frailty metrics, risk factors, and long term outcomes in a geriatric population (Supplemental Fig. 1) from the Toledo Study for Healthy Aging (TSHA). We show models accurately reproduce multiple frailty metrics using a limited window of data collection. Furthermore, activity based models for identifying adverse outcomes, like hospitalization and mortality, often outperform traditional frailty metrics. These findings show that activity data can serve as a robust, objective, and scalable surrogate for assessing frailty in clinical and research settings while also empowering other clinically relevant predictive models.

Methods

Measurement of frailty

Participants were classified using 4 different frailty scales: (1) Fried Frailty Phenotype (FP) scale; (2) Rockwood Frailty Index (FI), (3) Frailty Trait Scale (FTS), and (4) the Frailty Trait Scale-short form (FTS5). In the FP scale, weakness and slowness were measured and low energy, low physical activity, and loss of weight were reported. Each item was assigned a point⁶. According to this, participants were classified as robust (0 points), prefrail (1‒2 points), and frail (>3 points). The FI is a count of 40 clinical deficits, including the presence and severity of current disease, ability in activities of daily living, and physical and neurologic signs of clinical examinations²³. The presence of each item is scored as a 1. The FTS includes 7 dimensions (balance of energy nutrition, physical activity, nervous system, vascular system, strength, endurance, and walking speed). Each item is scored from 0 (best) to 4 (worst), except for the “chair test,” which scores from 0 to 5 points¹⁰. The FTS5 is a short scale that is easy to administer and has a similar performance to the FTS, and it can be used in clinical settings for frailty diagnosis and evolution¹¹. The 5-item (FTS5) (range 0-50) provides an opportunity for tracking trajectories inside each category (frail or non-frail) and between categories, supporting the fact that frailty is not a “categorical” issue but a continuous, discrete one.

Study design and participants

Data were drawn from the Toledo Study for Healthy Aging. The full methodology of the TSHA has been previously described²⁴. Briefly, the TSHA is a population prospective cohort study originally conceived to examine the determinants and consequences of aging and frailty in individuals older than 65 years from Toledo, Spain (Table 1). In the first stage a team of trained psychologists conducted computer assisted face to face interviews with subjects. In the second stage, a physical examination followed by clinical and performance tests at the subject’s home was performed by experienced nurses. In the third stage, participants went to their health center to provide a blood sample in a fasted state. In the fourth stage, anthropometry data and body composition were obtained with dual energy X-ray absorptiometry (DXA). At this later stage, participants were asked to wear an accelerometer for a week (Supplemental Fig. 2). The current study used data from participants with valid accelerometer records from the second (2012 to 2014) and third (2015 to 2017) TSHA waves²⁵. Signed informed consent was obtained from all participants. The study and subsequent analysis of collected data was approved by the Clinical Research Ethics Committee of the Toledo Hospital (approval code: 2010/93) and was conducted in accordance with the Declaration of Helsinki for human studies. As this approval includes the analysis of the deidentified TSHA data no additional IRB approval was necessary.

Table 1.

Summary Statistics of TSHA Cohort: mean and standard deviation of TSHA cohort (n = 437) separated by gender and admitted wave. Only wave two and three included actigraphy measurements

	Cohort Wave	n	Age	Weight (Kg)	Height (cm)	BMI	Waist Circumference (cm)	Hip Circumference (cm)
Male	2	85	77.48 (5.06)	77.24 (10.91)	163.19 (7.55)	28.98 (3.36)	98.93 (9.62)	103.39 (7.77)
	3	121	74.50 (4.90)	77.50 (12.42)	163.86 (6.26)	28.86 (4.40)	103.09 (12.08)	103.62 (8.57)
Female	2	104	76.38 (4.10)	69.79 (11.78)	150.11 (6.53)	30.97 (4.88)	91.09 (11.87)	107.84 (10.44)
	3	127	75.35 (5.41)	69.80 (12.08)	151.02 (5.09)	30.59 (5.00)	95.49 (11.21)	109.57 (10.06)

Accelerometry Data Processing

Physical activity measured with ActiGraph smartwatch (ActiGraph, LLC, FL, USA) was processed to extract features representing various aspects of daily physical activity (e.g., step count, vigorous bouts of activity, and sedentary bouts.)²⁶. ActiGraph’s actigraphy platform ActiLife v6.13.3 was used to extract features which were later used to build predictive models of frailty. These devices and data types have proven to be efficacious when compared against laboratory tests of physical activity but have shown a tendency to underestimate total wake time and sleep onset but is generally considered a good choice without any clear superior option^27–30.

Extraction of sleep and activity features is detailed in the associated ActiLife v6.13.3 documentation. Exact features including names and values can be located in the included github repository (see Code & Data Availability section in supplemental) “data/actigraph_export/*” including extracted data and the variables used in their generation.

Predictive Models

The ML algorithm used for all supervised learning tasks were gradient boosted decision trees via the XGboost package V1.5.2.1³¹ in R version 4.1.2. This tree based method produces multiple classification or regression trees which are fit and then added to an ensemble. Specifically, individual trees are added to an ensemble as weak learners to improve the shortcomings of the previous weak learners, which is based on the gradient boosting framework by^32,33. As the primary objective of this study was to demonstrate the potential of ML predictors for frailty XGboost was selected for its history of good performance, ease of application for subject matter experts, and ability to handle mixed and missing data.

Individual trees are represented as functions, $f_k$ that exist in the space of all possible classification and regression trees $F$. All $k$ trees are then summed to form the final predictions on some observation $x_i$.

$$\widehat{y_i}={\sum }_{k=1}^K{f}_k\left(x_i\right),\mathrm{where}{f}_k\in F$$

This differs from traditional random forest models through the additive training procedure used as opposed to the trees being independently generated. Broken up into distinct steps the process of building an XGboost model can be seen as the following:

$${\widehat{y}}_i^{\left(0\right)}=0$$

$${\widehat{y}}_i^{\left(1\right)}=f_1\left(x_i\right)={\widehat{y}}_i^{\left(0\right)}+f_1\left(x_i\right)$$

$${\widehat{y}}_i^{\left(2\right)}=f_1\left(x_i\right)+f_2\left(x_i\right)={\widehat{y}}_i^{\left(1\right)}+f_2\left(x_i\right)$$

$${\widehat{y}}_i^{\left(t\right)}={\sum }_{k=1}^t{f}_t\left(x_i\right)={\widehat{y}}_i^{\left(t-1\right)}+f_t\left(x_i\right)$$

Such models with 50 classification or regression trees were built for all predictions made throughout this work; all other parameters were set to the default of the R package.

Model Robustness and Evaluation

A repeated subsampling holdout method, without replacement and stratification, was used to evaluate the model³⁴. The data was randomly divided into two equal sized subsets for training and testing using a uniform random sampler. Models were trained on one half and tested on the other to ensure that the predictions are made on patients previously blinded to the algorithm. This process was then repeated 50 times to test the robustness of the models. This stringent process of predicting on previously unseen data shows if results are generalizable to future populations through variation in train and test splits. Results generated in this manner are similar to bootstrap estimates (difference being sampling with or without replacement) of model performance. For the exact process of model training, prediction, and evaluation see the script “actigraphy-frailty/scripts/analysis.R” (Code & Data Availability section in supplemental).

These test set predictions were then averaged for final evaluation and presented using a Spearman rank, Mann-Whitney U, or Kruskal-Wallis test: Application of these statistical tests were dependent on the target used for ML modeling being continuous, binary classes, or multiple classes respectively. Supplementary data files 1-8 contain model evaluation of risks, outcomes, frailty metrics, and components of the frailty trait scales as the prediction target; Including evaluation for each of the 50 repeated model predictions (Supplementary data files 1,2,4 and 8) and the average model predictions across repeats (Supplementary data files 3, 5, 6, and 7).

Statistics and reproducibility

Statistical analyses were performed on a total cohort of 437 participants using R version 4.1.2. A repeated subsampling holdout method, without replacement and stratification, was used to evaluate the model³⁴. That is, the dataset was randomly divided into two equal sized subsets for training and testing (50/50 split) to ensure that the predictions are made on patients previously blinded to the algorithm. This procedure was repeated for 50 independent replicates to test the robustness of the models. This stringent process of repeated predicting on previously unseen data shows if results are generalizable to future populations through variation in train and test splits. As models were specified a priori using default parameters no hyperparameter tuning was conducted and no validation set was required. Generalization error was therefore estimated directly using the repeated train/test procedure³⁵. Results generated in this manner are similar to bootstrap estimates (difference being sampling with or without replacement) of model performance. For the exact process of model training, prediction, and evaluation see the script “actigraphy-frailty/scripts/analysis.R” (Code & Data Availability section in supplemental).

Test set predictions were averaged across the 50 replicates for final evaluation. Model evaluation was performed using the Spearman rank, Mann-Whitney U, or Kruskal-Wallis test: Application of these statistical tests were dependent on the target used for ML modeling being continuous, binary classes, or multiple classes respectively. All statistical tests are two-sided and reported as either -log10(P-value) or their associated rho value. For correlation network visualizations, pairwise correlations were calculated, and p-values were adjusted for multiple hypothesis testing using the Bonferroni correction method. Since each predictive model is an independent hypothesis trained and evaluated on only one outcome it does not form a single family of tests, therefore multiple hypothesis correction is unnecessary.

Correlation network visualizations

All datasets were visualized using correlation network structures. Each actigraphy feature is represented by a node and the network layout is determined by the t-SNE algorithm applied to the complete correlation network³⁶. All correlation p-values are adjusted for multiple hypothesis testing using the Bonferroni correction method.

Results

XGBoost models accurately predicted FTS on test data (8.70×10⁻³⁶ Spearman p-value and 0.55 Spearman rho) (Fig. 1); Similar analyses were done on Fried Score, Rockwood Index, FTS₅ and FTS₅ based frailty class (Supplemental Results and Supplemental Fig. 3, 4, 5, 6 and 7). Restricting training data to days of the week showed model performance was related to the day of measurement (Fig. 2a). Averaging data from each hour of the day across days exhibited a similar model behavior (Fig. 2b). Increasing the amount of data aggregated in one hour increments showed models built on 48 hours of data produced similar results to those built on all available data (Fig. 2c). Data aggregated from consecutive days showed a 48 hour window of collection sufficiently effective, with a slight improvement when highly predictive days aggregated together (Fig. 2d). Suggesting that two days of data collected in a targeted fashion are highly effective at determining FTS.

Fig. 1. Actigraphy Data used to predict FTS.

Model features, shown as t-SNE of pairwise correlation, with higher predictive importance visualized by larger nodes with a darker purple colour. Correlations are calculated on the 50 model repeats a. Model predictions of FTS on unseen data compared to actual measurements b. Comparison of actigraphy predicted FTS values vs actual FTS values as predictors of risk factors. Correlations are calculated on the entire cohort n = 437 for all 69 risk factors c.

Fig. 2. Daily and Hourly importance to prediction of FTS.

Model performance when data is collected at specific days for the FTS measure a. Similarly, averaging activity from a specific hour across each day shows how specific hours can determine frailty of individuals b. Through incremental inclusion of actigraphy data by the hour we can see the relationship between additional data and predictive performance c. Given the similarity in performance between the first 48 hours and the total data model aggregation of two consecutive days was performed to investigate performance given a more targeted data sampling strategy d. All statistical tests were two-sided Spearman’s Correlation. All results are reported on the full n = 437 cohort.

Additional ML models of various age related risk factors including the Mean Corpuscular Hemoglobin Concentration (MCHC), Maximum Corpuscular Volume (MVC), and grip strength (Fig. 3a) produced robust predictions. Comparing predictive capabilities of FTS with actigraphy derived ML models showed the ML approach superior for a large number of risk factors (Fig. 3b). Actigraphy predicted FTS and FTS₅ as predictors of risk factors perform similarly (Fig. 1c & Supplemental Fig. 4c). Ultimately, actigraphy-based frailty predictions recapitulate frailty metrics, their associations with risk factors, and generally enhance predictions through direct analysis. Additionally, activity based prediction of long term outcomes, specifically hospitalization³⁷ and mortality, have an observable improvement over FTS and FTS₅ as predictors (Fig. 3c) once again showing the malleability of our analytic approach.

Fig. 3. Actigraphy models reproduce frailty scores and predict mortality and hospital admission.

The correlation network displays actigraphies predictive ability where node size and intensity of colour indicate p-value of model prediction (two-sided Spearman) a. Comparison of the 113 actigraphy risk predictions, and FTS as a predictor of risks via -log10 two-sided Spearman correlation p-values b. Predictive models for hospitalization c and mortality d, compare the predictive capabilities of actigraphy data with XGBoost (visualized curves) to FTS and FTS₅. All AUC and correlation results are reported on the full n = 437 cohort.

Discussion

Frailty is a clinically relevant syndrome in the elderly population associated with increased risk for falls, incident disability, hospitalization, and mortality^6,38–41. Consequently, reducing the burden of frailty is one of the most essential challenges faced by public health authorities¹² as the global population ages. However, current methods developed for assessing frailty have not been widely adopted especially in clinical settings and often do not adequately reflect the biological condition of frailty. Wearable technologies present a unique solution which can assess frailty, risk factors, and long term health outcomes simultaneously.

This work demonstrates the feasibility and potential clinical application of wearable sensors and machine learning models to predict patient frailty, age related risk factors, and long term outcomes. Previous work using data derived from wearable devices to predict frailty have focused on frailty classes (frail vs non-frail)^18–20 which can obfuscate variations of frailty within a given class, whereas this work directly estimates multiple frailty metrics. Additionally, studies using patient activity to determine mortality and hospitalization often focus on specific causes of mortality such as heart failure, or on specific at risk populations^37,42–44, this work makes no distinction based on cause. The use of actigraphy to predict frailty, risk factors, and long term outcomes presented here offers a unified approach for determining characteristics of health and potential adverse events. The broad scope of targets presented show the capabilities and limitations of actigraphy data, particularly in its ability to characterize and assess frailty.

Using observational patient activity data, as opposed to tests of fitness or self reported data, allows for an easy to collect, objective, scalable, and biologically relevant evaluation of frailty metrics as well as an abundant data source for ML models. ML models to predict Fried score, Rockwood index, FTS, and FTS₅ have varied performance and demonstrate how each metric relates to a patient’s physiological capacity for movement (Supplemental Fig. 2). Such models need a small amount of data to sufficiently recreate FTS values allowing for easy inclusion of frailty metrics in resource and time limited settings. Additional analysis of age related risk factors and long-term outcomes as targets show the versatility of the actigraphy measurements in multiple predictive settings (Fig. 3). Like frailty, predictive quality for risk factors and long term outcomes relate to an individual’s activity levels, with risk factors and outcomes such as MVC, MCHC (Fig. 3b), hospitalization, and mortality (Fig. 3c, d) being highly dependent on physiological state.

Some limitations of this work include the TSHA cohort, as ethnicity and socioeconomic status are not included impacting its generalizability. While the XGBoost method proved effective, other methods are worth considering for future work both modern over parameterized deep learning and traditional approaches. Deep neural networks for instance are a powerful ML paradigm with standout performance in numerous problem settings but offer little post-hoc inferential ability and require vast amounts of data. Alternatively, traditional time to event models such as Cox regression offer more interpretable results potentially at the cost of predictive ability. Specific considerations towards this would be that non-linearities need explicit specification in Cox regression, the handling of high dimensional data (important for incorporation of biological modalities), and data availability as other methods require significantly more. Future exploration of other analytical approaches between canonical time-to-event models, other statistical ML models, and deep neural networks would elucidate which approach is most appropriate. Appropriateness being a balance of the model’s ability to be interpreted and performance. Canonical models often allow for direct interpretation, XGboost like other ML models can be interpreted through application of additional methods like Shapley additive explanations, while deep learning approaches often require more experimental methods of interpretation like sparse autoencoders^45,46. Finally, the lack of a consensus frailty metric limits the applicability of our model as some features of frailty are not reflected in activity measures but offer an important perspective; Such as the Romberg Balance Test (Supplemental Fig. 8). Ideally, a comprehensive approach incorporating physiological, biological, and clinical features is essential for developing a robust model of frailty and aging.

While determination of frailty, risk factors, and long term outcomes from actigraphy data has direct clinical applications⁴⁷, further connection to biology is needed to better understand the biological determinants and consequences of aging. The immunological status of patients has been previously associated with actigraphy as a surrogate of recovery and a similar systems biology approach to the study of frailty and its metrics would elucidate the relationship between cellular function and frailty⁴⁸. Additional biological modalities⁴⁹ would allow for the investigation of domain specific frailty biomarkers and potential modifiable targets to relieve the burden of frailty and aging. The Stanford 1000 immunomes project and UCSF 10,000 immunome project are exemplary of such a comprehensive project, with the former being of particular interest due to its focus on aging^50,51. Ultimately the integration of wearable technologies with multiple biological modalities would provide a robust, scalable, and objective perspective on frailty and a broad range of age-related characteristics.

Author contributions

A.C.—Conceptualisation, Methodology, Analysis Design, Analysis, Writing, Visualization, Data Curation. A.M.—Conceptualisation, methodology, Data Acquisition, Writing. K.S., F.J.G.-G., J.L.-R., L.M.A., and L.R.-M.—Writing, Data Interpretation. A.L.C., C.E., D.D.F., T.P., M.B., M.X., N.G.R., R.F., B.G., and M.A.—Review and Editing, Analysis Design. I.A., and N.A.—Conceptualisation, Review and Editing, Supervising, Project Administrating

Peer review

Peer review information

Communications Medicine thanks Björn Friedrich and Emanuele Seminerio for their contribution to the peer review of this work.

Data availability

Requests for original data can be handled through (www.ciberfes.es/) via a review committee. The hospital reviews and determines the purposes for the data requests and what data can be released. Data requests can be sent to: Research and teaching unit, Virgen del Valle Hospital Ctra. Cobisa S/N, 45071 Toledo – Spain, info@estudiotoledo.com. All procedures were approved by the Clinical Research Ethics Committee of the Toledo Hospital and were conducted in accordance with the Declaration of Helsinki for human studies.

Supplemental data can be accessed either as the included.csv files or as.rda files available at the github repository in the results folder https://github.com/tripodlaboratories/actigraphy-frailty/tree/main/results⁵². The included supplementary data files 1, 2, 4 and 8 contain results (rho, p-values, and RMSE) for each individual model repeat of FTS components, risk factors & outcomes, FTS₅ components, and the various frailty metrics respectively. Supplementary Data 3, 5, 6 and 7 contain the average model repeat prediction evaluations for risk factors & outcomes, FTS₅ components, FTS components, and the various frailty metrics respectively. Actigraphy activity data and descriptions of clinical outcomes, risks, and frailty components are provided as.csv’s in the data folder of the associated github https://github.com/tripodlaboratories/actigraphy-frailty/tree/main/data⁵².

Code availability

Code necessary to reproduce results, figures, and models can be found at https://github.com/tripodlaboratories/actigraphy-frailty52. This code was run using R 4.1.2 and xgboost 1.5.2.1 on a macOS 12.3.1 system. There are no restrictions on its access or use.

Competing interests

All authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s43856-026-01419-7.