Toward telemedicine‐compatible physical functioning assessments in kidney transplant candidates

Abstract

Frailty is associated with adverse kidney transplant outcomes and can be assessed by subjective and objective metrics. There is increasing recognition of the value of metrics obtainable remotely. We compared the self‐reported SF‐36 physical functioning subscale score (SF‐36 PF) with in‐person physical performance tests (6‐min walk and sit‐to‐stand) in a prospective cohort of kidney transplant candidates. We assessed each metric's ability to predict time to the composite outcome of waitlist removal or death, censoring at transplant. We built time‐dependent receiver operating characteristic curves and calculated the area under the curve [AUC(t)] at 1 year, using bootstrapping for internal validation. In 199 patients followed for a median of 346 days, 41 reached the composite endpoint. Lower SF‐36 PF scores were associated with higher risk of waitlist removal/death, with every 10‐point decrease corresponding to a 16% increase in risk. All models showed an AUC(t) of 0.83–0.84 that did not contract substantially after internal validation. Among kidney transplant candidates, SF‐36 PF, obtainable remotely, can help to stratify the risk of waitlist removal or death, and may be used as a screening tool for poor physical functioning in ongoing candidate evaluation, particularly where travel, increasing patient volume, or other restrictions challenge in‐person assessment.

1. INTRODUCTION

Frailty is a pathobiological process characterized by loss of physiologic reserve and increased vulnerability to stressors. ¹ Originally described as an adverse consequence of aging, frailty overlaps with but is separate from comorbidity and disability. In clinical practice, frailty is often difficult to define or measure, and there is little consensus regarding specific elements to be included in operational definitions. Thus, we frequently utilize surrogate measures, especially physical functioning defined by the ability to perform specific physical tasks. Due to the high prevalence of comorbidities, including diabetes mellitus and cardiovascular disease, as well as pathophysiologic pathways of uremia including inflammation and protein energy wasting, frailty is prevalent among kidney transplant candidates and associated with unfavorable waitlist and posttransplant outcomes. ² , ³ , ⁴ Despite these well‐documented associations, recent data indicate that 75% of United States kidney transplant programs do not assess frailty as part of routine pretransplant evaluation, ⁵ representing lost opportunities for therapeutic engagement.

Part of the challenge of operationalizing pretransplant frailty assessments pertains to confusion surrounding the myriad surrogate measures ² and how to act on them. Thus, we previously detailed the utility of physical performance measures, namely the 6‐min walk test (6MWT) and the sit‐to‐stand test (STS), in kidney transplant candidates near the top of the waitlist. ⁶ These tests objectively and reproducibly assess patients’ ability to perform standardized physical tasks and are routine tests in physical medicine and rehabilitation. ⁷ , ⁸

As valuable as these in‐person assessments of physical performance may be, there is also a role for metrics that capture elements of frailty without obligating in‐person evaluation. For example, self‐reported physical health and function may be evaluated remotely using well‐validated questionnaires. ⁹ This approach can be particularly advantageous when considering the distances and burdens endured by patients when traveling to transplant centers for evaluation or, as waitlist management becomes increasingly complicated (and prolonged), re‐evaluations.

At our program, for instance, patients transplanted between September 2015 and March 2019 travelled a median distance of 31 miles for their transplant evaluation appointments, with 16% traveling 100 miles or more. The median travel time was 87 min with 36% traveling for 2 or more hours. Our instance is not unique: the distance to transplant programs has been reported to be 23 to 46 miles, with marked variation by region. ¹⁰ , ¹¹ Distance itself may be an obstacle to accessing the waitlist or, thereafter, achieving active status. ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ Validating a physical functioning assessment tool that can be administered remotely could prove valuable in alleviating burdens on patients and caregivers and focusing transplant program resources.

The self‐reported SF‐36 physical functioning subscale score (SF‐36 PF) ⁹ , ¹⁶ is a candidate screening tool for remote assessment by transplant centers. Previous work by Reese et al have shown a robust association between SF‐36 PF results and pretransplant and posttransplant outcomes. ¹⁷ , ¹⁸ Furthermore, since 2008, the Centers for Medicare & Medicaid Services (CMS) have mandated that dialysis units collect SF‐36 PF as part of the 36‐item Kidney Disease Quality of Life (KDQOL‐36) survey, a kidney disease‐specific measure of health‐related quality of life. ¹⁹ , ²⁰ SF‐36 PF is therefore a widely available but under‐appreciated resource for transplant programs. In this study, following on previous work, ⁶ , ¹⁸ we aimed to directly compare SF‐36 PF to objectively obtained physical performance tests in a cohort of kidney transplant candidates approaching the top of the waitlist. Our main interest was the ability of each metric to identify patients at risk for adverse waitlist outcomes. With this knowledge, we propose ways to use these complementary metrics in kidney transplant candidacy evaluation and waitlist management.

2. METHODS

2.1. Cohort assembly

We included all patients evaluated through our program's Transplant Readiness Assessment Clinic (TRAC) from October 2017, when we implemented SF‐36 PF assessment, through December 2018. We included transplant candidates with concurrent 6MWT, STS, and SF‐36 PF results. The operational details of TRAC, our deceased donor transplant candidate waitlist management strategy, have been published previously. ⁶ , ²¹ Briefly, after deceased donor transplant candidates are added to our program's waitlist (~1800 patients), two dedicated transplant nephrologists along with two dedicated transplant nurse coordinators evaluated in‐person those with sufficiently high allocation priority (as determined by kidney allocation scores) to be near the top of the waitlist.

2.2. Physical performance and functioning testing

At each visit, the nurse coordinators performed the 6MWT and STS. The SF‐36 PF questionnaire was obtained over the phone prior to the scheduled visit by a patient care coordinator or obtained during the visit. The 10‐item SF‐36 PF questionnaire is the physical functioning subset of the Medical Outcomes Study Short Form 36‐item questionnaire. ¹⁶ , ²²

It asks survey respondents to answer questions regarding their ability to perform physical tasks, for example, walking multiple blocks, with the responses of “not limited,” “limited a little,” or “limited a lot”. These responses are scored as 10, 5, and 0 respectively. The scores for each question were aggregated to give a total score from 0 to 100: the higher the score, the better the self‐reported physical functioning. An in‐person language interpreter was available to accommodate patients whose preferred language was not English. The research and the clinical transplant teams (including physicians, nurses, dietitians, and social workers) were blinded to the results of the SF‐36 PF throughout the duration of the study.

2.3. Outcomes

From the time of first SF‐36 PF and physical performance assessment, we ascertained waitlist outcomes including transplant (at Stanford or another program), removal from our waitlist, and death. To reduce reporting bias (ie, adverse events are more likely to be reported outside the follow‐up schedule), we censored data at the time of last scheduled follow‐up. The primary study outcome was a composite of death or waitlist removal. We have shown previously ⁶ that 6MWT and STS were strongly associated with this composite outcome and death and provided information beyond commonly collected clinical characteristics. We therefore chose to compare SF‐36 PF scores to 6MWT and STS scores by building a series of nested survival models that included clinical characteristics alone as well as clinical characteristics plus each physical functioning metric (6MWT, STS, and SF‐36 PF). We compared model fit by determining the Akaike information criterion (AIC) for each model, with lower values indicating better model fit.

2.4. Baseline characteristics

We compared baseline characteristics, stratified by SF‐36 PF score < 75 or ≥75. We utilized a threshold score of 75 on the SF‐36 PF, as a score < 75 has previously been demonstrated to be independently associated with higher mortality in incident patients with end‐stage kidney disease (ESKD). ²³ Categorical variables were compared by chi‐square tests and continuous variables by Kolmogorov‐Smirnov, a non‐parametric test. We expressed categorical variables as frequencies (percentages) and continuous variables as medians (25^th–75th percentile range), given skewed distributions.

2.5. Correlation of physical performance test results with self‐reported physical functioning results

We estimated correlation between SF‐36 PF, STS, and 6MWT by linear regression, except in the case of STS where correlation was estimated by the zero inflated Poisson model because of the high occurrence of a zero score for the STS test. ²⁴

2.6. Predictive ability of SF‐36 PF, 6MWT, and STS results

We assessed the ability of each metric to predict our primary outcome by modeling time to waitlist removal/death, censoring at transplant and end of follow‐up. We verified that censoring at transplant and end of follow‐up yielded comparable estimates to the Fine‐Gray model which accounts for competing risk (data not shown). Using the Kaplan‐Meier method, we built a time‐dependent receiver operating characteristic [ROC(t)] curve and calculated the 1‐year area under the curve [AUC(t)] for each metric. The time‐dependent ROC is a statistic that generalizes the traditional ROC, developed for binary outcomes, to time‐to‐event analyses, by allowing the AUC to vary over time [AUC(t)]. ²⁵ , ²⁶ We chose to compute the AUC(t) at 1 year, as SF‐36 PF is repeated in dialysis units annually. Essentially, the AUC(t) at 1 year quantifies the ability of the measure to predict the primary outcome within 1 year. We included multivariable adjustment for demographic factors and comorbidities including dialysis vintage, diabetes status, presence of atherosclerotic disease, and assistive walking device. Twenty‐one percent of patients (n = 42) used an assistive device, which may have differentially affected their abilities to perform daily tasks as well as the 6MWT and STS. We therefore performed a sensitivity analysis excluding these patients.

2.7. Validation

We do not have an external validation cohort. We performed internal validation using bootstrapping to compute a measure of optimism. ²⁷ Briefly, the bootstrap method used sampling with replacement to generate 1000 separate bootstrap datasets with N = 199. The predictive model was applied to each bootstrap dataset to arrive at an AUC(t). Because each bootstrap dataset contains a random sample of patients from the original derivation dataset, the model fit could be less accurate and the AUC(t) will be lower. Thus, we can estimate our degree of over‐fitting (optimism) by subtracting the measure from the AUC(t) computed in the original derivation dataset ²⁸ (See Appendix 1 for details).

We used SAS Enterprise version 7.1 (Cary, NC). The Stanford Institutional Review Board approved this project (protocol #43639) which we conducted in adherence with the Declaration of Helsinki. The clinical and research activities being reported are consistent with the Principles of the Declaration of Istanbul as outlined in the “Declaration of Istanbul on Organ Trafficking and Transplant Tourism.” Because the study used data being collected for routine clinical purposes and was defined as “minimal risk,” the Institutional Review Board waived the need to obtain individual‐level consent.

3. RESULTS

Our cohort consisted of 199 patients, 70 with SF‐36 PF < 75 and 129 with SF‐36 PF ≥ 75 (Table 1). Patients with lower SF‐36 PF scores were older, more likely to be female, had longer dialysis vintage, more likely to have diabetes mellitus, coronary artery disease, and peripheral artery disease, and more likely to use assistive devices for ambulation. The median physical performance testing results (6MWT and STS) were 287 meters (25^th–75th percentile range 152–354) and 11 repetitions (25^th–75th percentile range 3–17) in patients with lower SF‐36 PF scores, compared to 437 meters (25^th–75th percentile range 390–485) and 20 repetitions (25^th–75th percentile range 15–23) in patients with higher SF‐36 PF scores. We observed a direct correlation between SF‐36 PF results and the two physical performance results, with R ² values of .33 and .53 for the STS and the 6MWT, respectively (Figure 1).

Table 1.

Baseline characteristics on date of Transplant Readiness Assessment Clinic (TRAC) evaluation, stratified by short form 36‐question physical functioning subscale score (SF‐36 PF)

Baseline characteristics	SF‐36 PF < 75	SF‐36 PF ≥ 75	P‐value*
	N = 70	N = 129
Demographics and follow‐up
Age (years)	61 (56, 67)	54 (45, 61)	<.0001
Sex (% male)	28 (40%)	76 (59%)	.01
Race/ethnicity (%)
White	8 (11%)	14 (11%)	.6
Black	3 (4%)	10 (8%)
Hispanic, non‐black	38 (54%)	65 (50%)
Asian	15 (21%)	34 (26%)
Other/Mixed	6 (9%)	6 (5%)
Time from listing a (years)	6.5 (4.7, 8.6)	5.9 (3.3, 7.9)	.3
Follow‐up time b (days)	277 (70, 527)	381 (197, 457)	.05
Comorbidities
Dialysis vintage (years)	7.4 (5.8, 8.7)	6.8 (4.3, 8.3)	.4
Diabetes mellitus (%)	48 (69%)	52 (40%)	.0001
Hypertension (%)	66 (94%)	121 (94%)	.9
Atherosclerotic disease c (%)	32 (46%)	32 (25%)	.003
Coronary artery disease d (%)	17 (24%)	20 (16%)	.2
Peripheral artery disease e (%)	16 (23%)	13 (10%)	.02
Lower extremity amputation (%)	4 (6%)	6 (5%)	.7
Measures of physical functioning
Karnofsky score at listing a	80 (70, 90)	90 (70, 90)	.9
STS result (repetitions)	11 (3, 17)	20 (15, 23)	<.0001
6MWT (meters)	286 (152, 354)	437 (390, 485)	<.0001
Assistive walking device (%)	31 (44%)	11 (9%)	<.0001

^aListing refers to initial transplant evaluation.

^bTime from initial TRAC visit to waitlist outcome or censoring.

^cCoronary artery disease/equivalent (ischemic cerebrovascular accident or clinical peripheral artery disease) present at TRAC visit.

^dCoronary intervention/revascularization present at TRAC visit.

^ePeripheral artery intervention/revascularization, amputation, or clinical symptoms present at TRAC visit.

^*Continuous variables are represented as median (25‐75th percentile range). Categorical variables are represented as count (percentage). P‐values refer to non‐parametric tests since the data were not normally distributed. Data are 100% complete unless stated otherwise.

Figure 1.

Correlation between the short form 36‐question physical functioning subscale score (SF‐36 PF) and physical performance test results (sit‐to‐stand test [STS] on the left and 6‐min walk test [6MWT] on the right)

Over a median follow‐up period of 346 days, 31 patients were removed from the waitlist, 10 died on the waitlist, 67 were transplanted, and 91 were alive on the waitlist with survival time censored. Lower SF‐36 PF scores or lower results on the 6MWT and STS tests were associated with a higher risk of waitlist removal or death (Table 2, Figure 2). For every 10‐point decrease in the SF‐36 PF score, there was a 16% increase in the risk of waitlist removal/death. All three models showed a 1‐year AUC(t) of 0.83–0.84 that did not contract substantially after internal validation, indicating good discriminatory ability of all three models in predicting waitlist removal or death. All nested survival models that included physical functioning metrics demonstrated better fit than the model including demographic and clinical characteristics alone. Specifically, the model for SF‐36 PF had an AIC of 262 compared to 274 for the model including only demographic clinical characteristics.

Table 2.

Ability of short form 36‐question physical functioning subscale score (SF‐36 PF) and physical performance test results (sit‐to‐stand test [STS] and 6‐min walk test [6MWT]) to predict time to waitlist removal or death

	Univariate analysis		Multivariate analysis a
	Hazard ratio (95% CI)	AUC(t)	Adjusted hazard ratio (95% CI)	AUC(t)	Bias‐corrected AUC(t)
SF‐36 PF (per 10 points)	0.75 (0.67–0.83)	0.76	0.84 (0.75–0.95)	0.83	0.80
6MWT (per 50 meters)	0.69 (0.62–0.76)	0.79	0.71 (0.60–0.83)	0.84	0.83
STS (per 5 repetitions)	0.61 (0.51–0.73)	0.80	0.74 (0.59–0.92)	0.83	0.81

Abbreviations: AUC(t), Based on time‐dependent ROC at 1 year follow‐up; Bias‐corrected AUC(t), AUC(t) corrected after performing bootstrapping for internal validation.

^aModels adjusted for Age, Sex, Dialysis vintage, Diabetes status, Atherosclerotic disease, and Assistive device use.

Figure 2.

Time‐dependent receiver operating characteristic [ROC(t)] curves measured at 1 year, for short form 36‐question physical functioning subscale score (SF‐36 PF) model on the left, 6‐min walk test (6MWT) model in the middle, and sit‐to‐stand test (STS) model on the right. The outcome of interest was death or removal from the waitlist

After performing a sensitivity analysis excluding patients who use assistive devices, we still found similar ability of the SF‐36 PF model to predict waitlist outcomes with an AUC(t) of 0.83. Figures S1 and S2 demonstrate the proportion of patients in each tertile of 6MWT and STS score, stratified by SF‐36 PF results.

4. DISCUSSION

Previously, we demonstrated that the physical performance metrics of 6MWT and STS were closely associated with waitlist outcomes in kidney transplant candidates at the top of the waitlist and provide information relevant to risk stratification beyond that of commonly collected clinical variables. ⁶ In this study, we demonstrate that SF‐36 PF subscale, a questionnaire that relies on patient self‐report, gave scores that were directly correlated with 6MWT and STS results and exhibited similarly very good discriminatory capacity for distinguishing patients who will die or be removed from the waitlist from those who will not at 1 year. While in‐person evaluation is certainly an invaluable component of the pretransplant assessment, we submit that the SF‐36 PF could be highly informative as an adjunct to physical performance testing and, given its compatibility with remote and telehealth technologies, may be deployed as an initial screening tool for remotely monitoring patients with barriers to in‐person evaluation. A particular feature of our cohort is the high proportion of patients with low self‐reported physical performance testing: 35% versus 24% meeting the definition of only self‐reported frailty in Johansen et al ⁹ and 23% with SF‐36 PF scores in the lowest quartile in Reese et al. ¹⁷ This is a consequence of our cohort‐selection process: our cohort represents a group of patients near the top of the waitlist in an area where the median wait time is 5–10 years; hence, they carry a higher burden of disease and impairment than other cohorts studied to date.

The SF‐36 PF, 6MWT, and STS measure different domains of physical functioning, which in turn is subsumed under and frequently conflated with frailty (Figure 3). We submit that the SF‐36 PF assesses the patient's global physical functioning, or ability to perform physical tasks on a day‐to‐day basis, whereas the STS and 6MWT measure more specific aspects contributing to this global functioning, that is, lower extremity strength, balance, and cardiorespiratory fitness. Our result therefore shows that, unsurprisingly, SF‐36 PF scores and STS/6MWT results are moderately but imperfectly correlated. The SF‐36 PF can therefore be a screen to identify patients at risk. Once identified to be at risk, 6MWT and STS can provide more detailed information on specific limitations (eg, deconditioning, or lower extremity weakness) that might be ameliorated with prehabilitation. The STS is a sub‐component of the SPPB, which measures lower extremity strength and balance. ²⁹ For example, a patient with poor STS results may benefit from physical therapy targeted at balance and lower extremity strength training. The 6MWT incorporates elements of lower extremity strength and gait speed, but also assesses cardiorespiratory fitness ³⁰ , ³¹ in persons with end‐stage kidney disease ³² , ³³ and non‐kidney solid organ transplantation. ¹ Therefore, a patient with poor 6MWT but good STS results may warrant investigations into cardiopulmonary diseases, fluid status optimization, and better anemia management.

Figure 3.

Schematic demonstrating the relation of frailty to metrics in various physical functioning domains

The greatest value of the SF‐36 PF lies in its availability in dialysis‐dependent patients and ease of administration. We therefore envision its use at the referral stage of kidney transplant, prior to the in‐person evaluation. Kidney transplantation is facing a dynamic landscape. A primary goal of the recent Advancing American Kidney Health (AAKH) initiative is to treat 80% of patients newly diagnosed with ESKD with home dialysis or kidney transplant by 2025. The AAKH initiative provides incentives to increase kidney transplant rates, ³⁴ a laudable goal. Incentives for dialysis facilities to refer all patients below the age of 75 for transplant evaluation and the desire for more patients with advanced chronic kidney disease to be treated with preemptive transplantation rather than dialysis will likely cause sizeable increases in referral to transplant programs. ³⁵ , ³⁶

The recent global pandemic has also set forth an expansion of telemedicine services. ³⁷ As patients and providers become increasingly accustomed to telehealth, expanding telehealth to kidney transplant evaluation, especially for those who live far away from transplant programs, is a natural next step. Telehealth modalities may help to diminish the deluge of transplant evaluations, ³⁸ , ³⁹ but should best incorporate remote access physical functioning assessments. For programs like ours that care for a diverse, multi‐lingual population, remote use of the SF‐36 PF would be facilitated by the use of translated forms, including Spanish, Tagalog, Mandarin, and Vietnamese, the languages most commonly encountered in our institution. Employing the SF‐36 PF in a telehealth setting as a screening tool may enhance the effectiveness of the telehealth visits and help transplant programs triage patients for in‐person evaluations, thus mitigating some of the burdens inherent in travel, guiding physical therapy interventions in evaluated patients, and helping transplant programs manage their pretransplant resources.

The main limitation to our study is the lack of an external validation cohort. To address the degree to which our data may be transportable, we performed internal validation with standard bootstrapping techniques, as outlined in our Methods with details in our Appendix 1. ²⁸ Furthermore, our study cohort is likely different from other areas in the country. Patients whose preferred language is not English, for instance, are overrepresented in our cohort (47%) compared to the rest of the country. Many of these patients have poor health literacy. That we are still able to obtain good agreement between SF‐36 PF and physical performance measures bodes well for the applicability of SF‐36 PF elsewhere in the country. Nonetheless, whether our single‐center observations are generalizable to a wider population warrants follow‐up studies and caution in interpretation. The evaluating transplant nephrologists were not blinded to the results of the 6MWT or STS and used the information to provide specific guidance on transplant optimization. However, in terms of delisting versus remaining on the waitlist, the decision is made at the level of the Transplant Evaluation Committee, in accordance with UNOS regulations, and the committee was given a global report of patient physical performance, including assistive device use, rather than details of the 6MWT or STS test. We did not collect other measures of physical functioning or frailty in our patients, including the Fried Frailty Index. Nevertheless, we have outlined the unique advantages of SF‐36 PF, STS, and 6MWT and postulated a framework with which to approach the kidney transplant candidate with these three measures.

In summary, we evaluated the relation between the physical functioning subscale of the SF‐36 with two physical performance metrics, found direct correlations with both, and we established the ability of the SF‐36 PF subscale to predict adverse waitlist outcomes. This brief, validated questionnaire can be used—in‐person or remotely—to screen and evaluate progress of kidney transplant candidates. External validation of our results in other cohorts will help to refine the optimal strategy and timing of its application.

CONFLICT OF INTEREST

The authors of this manuscript have no conflicts of interest to disclose as described by Clinical Transplantation.

AUTHOR CONTRIBUTIONS

XSC and JCT designed the study. XSC and DJW collected data. XSC, J.H, and MRS analyzed the data. XSC, DJW, J.H, MRS, and JCT interpreted the data and made the figures. XSC, DJW, MRS, GMC, and JCT contributed to the intellectual contents of the manuscript. Each author contributed important intellectual content during manuscript drafting or revision and agrees to be personally accountable for the individual's own contributions and to ensure that questions pertaining to the accuracy or integrity of any portion of the work, even one in which the author was not directly involved, are appropriately investigated and resolved, including with documentation in the literature if appropriate.

Supporting information

Fig S1

Fig S2

Appendix 1.

Internal validation was performed to correct the AUC(t) for over‐fitting of the survival models in Table 2.

Let T_i be the time until waitlist removal or death, M_i be a linear combination of the patient (i = 1,…n) characteristics and the performance score, c be a criterion for classifying the linear combination as positive or negative. $\widehat{S}(t)$ is the estimated survival function, ${\widehat{F}}_M(c)$, is the empirical distribution function for the linear combination M_i. From this, sensitivity (Se) and specificity (Sp) can be estimated using the conditional Kaplan‐Meier approach:

$$\widehat{\mathit{Se}}(c,t)=\frac{\left\{1‐\widehat{S}(t|M_i>c)\right\}\left(1‐{\widehat{F}}_M(c)\right)}{1‐\widehat{S}(t)}$$

$$1‐\widehat{\mathit{Sp}}(c,t)=\frac{\left\{\widehat{S}(t|M_i>c)\right\}\left(1‐{\widehat{F}}_M(c)\right)}{S(t)}$$

Area under the curve, AUC(t), can be estimated by trapezoidal rule:

$$\widehat{\mathit{AUC}}(t)=\underset{‐\infty }{\overset{\infty }{\int }}\widehat{\mathit{Se}}(c,t)d[1‐\widehat{\mathit{Sp}}(c,t)]$$

To implement the internal validation, we estimated the $\widehat{\mathit{AUT}}(t)$ on 1000 bootstrapped samples using the following steps:

• 1Using the original dataset, estimate time‐dependent sensitivity $({\widehat{\mathit{Se}}}_0)$ and specificity $({\widehat{\mathit{Sp}}}_0)$ from the Cox Proportional Hazards model.
• 2Select 1000 independent bootstrap samples, by sampling the original data with replacement 1000 times. For each b = 1…1000 bootstrapped sample:

  1. Evaluate the bootstrap sample using the Cox PH model and apply the parameter estimates to the original data to estimate M_bi and c_b.

  2. Calculate the time‐dependent sensitivity $({\widehat{\mathit{Se}}}_{Boo{t}_b})$ and specificity $({\widehat{\mathit{Sp}}}_{Boo{t}_b})$ for each bootstrap sample.

  3. Estimate $\widehat{\mathit{Se}}$ optimism and $\widehat{\mathit{Sp}}$ optimism by the sample average of the 1000 replications

$${\widehat{\mathit{Se}}}_{\mathit{op}}=\sum _{b=1}^{1000}({\widehat{\mathit{Se}}}_{Boo{t}_b}‐{\widehat{\mathit{Se}}}_0)/1000$$

$${\widehat{\mathit{Sp}}}_{\mathit{op}}=\sum _{b=1}^{1000}({\widehat{\mathit{Sp}}}_{Boo{t}_b}‐{\widehat{\mathit{Sp}}}_0)/1000$$

• 3Final biased‐correlated estimates of sensitivity and specificity are used to estimate the bias‐corrected AUC(t).

$$\widehat{\mathit{Se}}={\widehat{\mathit{Se}}}_0‐\left|{\widehat{\mathit{Se}}}_{\mathit{op}}\right|$$

$$\widehat{\mathit{Sp}}={\widehat{\mathit{Sp}}}_0‐\left|{\widehat{\mathit{Sp}}}_{\mathit{op}}\right|$$

REFERENCES

Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005;61(1):92‐105.

Shiang K, The SAS^®–Calculations of Areas Under the Curve (AUC) for Multiple Metabolic Readings, WUSS Conference Proceedings Oct 13‐15 2004.

Kamarudin AN, Cox T, Kolamunnage‐Dona R. Time‐dependent ROC curve analysis in medical research: current methods and applications. Med Res Methodology. 2017;17:53.

Efron B, Tibshirani RJ. An Introduction to the Bootstrap. Chapman & Hall/CRC; 1994.

Watford DJ, Cheng XS, Han J, Stedman MR, Chertow GM, Tan JC. Toward telemedicine‐compatible physical functioning assessments in kidney transplant candidates. Clin Transplant.2021;35:e14173. 10.1111/ctr.14173

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.