Prediction of Prolonged Length of Stay for Stroke Patients on Admission for Inpatient Rehabilitation Based on the International Classification of Functioning, Disability, and Health (ICF) Generic Set: A Study from 50 Centers in China

Abstract

Background

This study aimed to develop a risk prediction model for prolonged length of stay (LOS) in stroke patients in 50 inpatient rehabilitation centers in 20 provinces across mainland China based on the International Classification of Functioning, Disability, and Health (ICF) Generic Set case mix on admission.

Material/Methods

In this cohort study, 383 stroke patients were included from inpatient rehabilitation settings of 50 hospitals across mainland China. Independent predictors of prolonged LOS were identified using multivariate logistic regression analysis. A prediction model was established and then evaluated by receiver operating characteristic (ROC) curve analysis and the Hosmer-Lemeshow test.

Results

Multivariate logistic regression analysis showed that the type of medical insurance and the performance of daily activities (ICF, d230) were associated with prolonged LOS (P<0.05). Age and mobility level measured by the ICF Generic Set demonstrated no significant predictive value. The prediction model showed acceptable discrimination shown by an area under the curve (AUC) of 0.699 (95% CI, 0.646–0.752) and calibration (χ²=11.66; P=0.308).

Conclusions

The risk prediction model for prolonged LOS in stroke patients in 50 rehabilitation centers in China, based on the ICF Generic Set, showed that the scores for the type of medical insurance and the performance of daily activities (ICF, d230) on admission were independent predictors of prolonged LOS. This prediction model may allow stakeholders to estimate the risk of prolonged LOS on admission quantitatively, facilitate the financial planning, treatment regimens during hospitalization, referral after discharge, and reimbursement.

Background

Stroke is associated with a heavy economic and health burden for patients, caregivers, and health service systems, and can be associated with prolonged length of stay (LOS) in hospital and rehabilitation centers as well as long-term disability [1–3]. In the United States, the annual direct and indirect cost of stroke has been estimated at $33 billion US dollars annually [4]. Stroke and prolonged hospital LOS has a major impact on healthcare expenditure in China, with approximately 40 billion RMB in annual direct costs for stroke care [5]. As a primary determinant of the cost of stroke rehabilitation, it is important to identify independent predictors for LOS to improve resource allocation and cost-efficiency [6].

Several prediction models have been proposed to estimate the associations between increased LOS for stroke patients according to baseline evaluations of function. The Functional Independence Measure (FIM™) [7] was identified as a significant predictor for LOS following stroke [6,8–10]. The FIM™ is a functional measurement applied to various conditions and situations during patient rehabilitation, which has been supported by extensive studies to show adequate psychometric properties [11,12]. However, the FIM™ is rarely used in mainland China due to the cost and excessive amount of time spent on its use, given the large number of patients affected by stroke. Also, there have been no large-scale validation studies of the use of FIM™ reported in mainland China, except for two studies that included 68 patients [13], and 29 patients, respectively [14]. Therefore, an alternative tool that is less costly and easier to use for functional evaluation is needed to predict LOS, readmission, and other function-associated events in patients with stroke undergoing rehabilitation in China.

The International Classification of Functioning, Disability, and Health (ICF) (Figure 1) defines functioning as all the body functions and activities, and defines disability as a term for impairment or limitation in activity, restriction in participation in activities, and also includes environmental factors [15]. Positive interactions in the ICF Generic Set framework promote health, while negative interactions lead to disability [16]. This holistic and dynamic concept of the ICF has become popular, and since 2001 the ICF has become a standard assessment method used to describe the health and function of patients [17]. Previously published studies have shown the feasibility of implementing the ICF in clinical and rehabilitation practice [18,19], service provision and payment [20,21], and policy planning [22,23]. Since the ICF is a comprehensive classification of more than 1,450 categories, all of these applications require the development of tools for routine use and to ensure data comparability.

Figure 1.

The International Classification of Functioning, Disability, and Health (ICF) Generic Set theoretical framework.

The ICF Generic Set is a new standard for the assessment of patient functioning that has been used in China and is a simple and practical self-reporting tool that describes the general health of individuals with different health conditions using seven categories [24]. Previous studies showed that the average assessment time of the ICF Generic Set was 6.5 min, with categories sensitive to functioning changes, except for d850 [25]. The interobserver and intra-observer reproducibility were adequate when used in rehabilitation departments [26]. Due to the presence of minimal risk of methodological bias in its use, the ICF Generic Set could serve as a clinically useful and practical predictive assessment method for use in patients with stroke before the start of rehabilitation [24].

In 2018, Liu et al. showed that the total score of the ICF Generic-6 (which excluded item d850) was significantly associated with the probability of prolonged LOS in hospitalized patients across several diagnostic groups [27]. However, this previous study did not distinguish between the impact of the specific categories of the ICF Generic Set on LOS nor did the study report the estimated probability of prolonged LOS for individual patients [27]. Therefore, stakeholders, including caregivers, hospital management staff, and medical insurance companies, did not participate in the shared decision-making, due to a lack of statistical evidence [27]. However, there have been no other reports on the possible use of the ICF Generic Set to improve clinical decision-making and health management, despite several previous studies on the implementation of the ICF [20,28].

Therefore, this study aimed to develop a risk prediction model for prolonged LOS in stroke patients in 50 inpatient rehabilitation centers in 20 provinces across mainland China based on the ICF Generic Set case mix on admission.

Material and Methods

Data sources and study participants

This study was conducted according to the reporting guidelines for the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) [29]. Data were collected from a prospective cohort from 50 hospitals in 20 provinces across mainland China. Figure 2 shows the geographical distribution of the participating hospitals. Data collection was approved by the Chinese Association of Rehabilitation Medicine. This study was exempt from individual hospital ethics approval according to Chinese regulations as it involved routine non-invasive clinical assessment. Informed consent was obtained from all the study participants. Scientific analysis and publication of anonymous data was approved by the Ethics Committee of the Southern Medical University Affiliated Shenzhen Hospital, which was the study center where the data were analyzed (No: NYSZYYEC20170013).

Figure 2.

Geographical distribution of the 50 hospitals included in this study.

Between November 2014 and February 2015, there were 4,510 patients admitted to the study centers, and a total of 383 hospitalized patients with stroke were identified in the rehabilitation departments who were included in the present study. The inclusion criteria included patients who were 18 years or older, with a confirmed diagnosis of stroke by computed tomography (CT) or magnetic resonance imaging (MRI), who had complete clinical data on hospital admission and discharge. The study exclusion criteria were patients less than 18 years of age and patients with recurrent stroke.

Predictors of length of stay (LOS) and outcome

On admission, all patients were evaluated using the seven categories of the International Classification of Functioning, Disability, and Health (ICF) Generic Set (Table 1) [24]. Each ICF Generic Set category had a value of between 0–10, with 0 being no functional difficulty, and 10 being completely dysfunctional. The ICF Generic Set categories were assessed with reference to a simple clinical intuitive description developed in China [30]. For instance, item d230 in the previous ICF system has a long description [31]. However, following the previously published findings of the ease of use of the ICF Generic Set categories in China [26,27], we used the intuitive description for item d230 as actions of planning, managing, and completing activities of daily living.

Table 1.

International Classification of Functioning, Disability and Health (ICF) categories in the Generic Set.

Categories	Functioning description
b130	Energy and drive functions
b152	Emotional functions
b280	Sensation of pain
d230	Performing daily activities
d450	Walking
d455	Moving around
d850	Remunerative employment

Trained nurses undertook the assessment of the ICF Generic Set on the admission of each patient in the study. All nurses received comprehensive training on how to assess functioning using the ICF Generic Set according to a validated manual (Supplementary Material). Assessors rated each category of the ICF Generic Set with the required relevant clinical information, while data collected in review and clinical assessment, such as the Barthel Index of Activities of Daily Living and the visual analog scale (VAS) were included [31]. LOS and baseline data, including gender, age, hemiplegic side, stroke subtype, type of medical insurance, and time after stroke onset, were collected from the hospital information system in each hospital. The ICF Generic Set and other accessible baseline data were candidate predictors. Prolonged LOS was defined by the highest quartile [32–34]. The outcome was coded as a dichotomous variable with 1 being prolonged LOS and 0 being the average LOS.

Statistical analysis

Logistic regression analysis was performed on prolonged LOS for stroke patients using the predictors on admission. Since patients with good performance in walking (item d450) were likely to exhibit no or mild dysfunction on moving (item d455), mobility was introduced as a predictor which was equivalent to the sum of these two categories (items d450+d455) for regression analysis. As the extent of the difficulty in participating in remunerative employment could not be assessed [25], the d850 category was divided, with 0 being no difficulty and 1 being any degree of difficulty. For clinical relevance, age was divided into >65 years and ≤65 years. Univariate analysis was performed to screen for related variables with P<0.05, while age and gender were included in the model. Multivariate analysis was performed to identify independent predictors of prolonged LOS in stroke patients and to establish a prediction model. The optimal model was selected according to the Akaike information criterion (AIC) stepwise variable selection of data to provide the best fit [35]. One variable was removed if the corresponding AIC was reduced [36].

Receiver operating characteristic (ROC) curve analysis [37] was performed, and the area under the curve (AUC) was calculated to evaluate the level of discrimination. The AUC values ranged from 0 to 1, with 0.5 being equivalent to chance prediction and 1 being fully predictive. We also calculated the empirical optimal cut-off point using the Youden index [38] and identified the cut-off point of predicted probability of prolonged LOS and the sensitivity and specificity at the cut-off point. The difference between the predicted probability and actual probability was examined by the Hosmer-Lemeshow test [39] and the calibration plot [40] to assess the accuracy of the model. The Hosmer-Lemeshow chi-squared (χ²) value was calculated by first dividing the data into ten groups (deciles) by the probabilities predicted by the model in ascending order. Then, the average predicted probabilities were compared with the actual event rate estimated for each decile, and the calibration plot was performed accordingly. A value of P<0.05 from the Hosmer-Lemeshow test was considered a significant difference. Statistical analysis was performed using Stata version 15.1 (StataCorp LP, College Station, TX, USA), and a nomogram for visual presentation was mapped using R version 3.5.0 software (www.r-project.org) and the rms package. Complete case analysis was performed, given that the evaluation of the ICF Generic Set at baseline and LOS were not likely biased by missing values.

Results

This study included 383 patients with stroke in inpatient rehabilitation settings, including 52.48% with left hemiplegia and 47.52% with right hemiplegia, 31.33% with hemorrhagic stroke and 68.67% with ischemic stroke. Medical insurance status for urban medical insurance accounted for 55.61% of the study population, including 21.93% in the new cooperative medical scheme (NCMS) [41], 11.23% were self-funded, while 11.23% of the patients were entitled to free medical service or commercial insurance. The mean time after stroke onset at admission was 71.68±49.71 days. The mean patient age was 59.82±14.25 years. There were 251 patients older than 65 years, and 132 patients aged <65 years. The mean length of stay (LOS) for stroke patients in the rehabilitation departments of 50 hospitals in China was 22.73±9.75 days. According to the study protocol, patients who were hospitalized for no less than 28 days were defined as having prolonged LOS. A total of 110 patients were in the prolonged LOS group while the others had normal LOS. The mean, standard deviation, median, and interquartile intervals of the seven categories of the International Classification of Functioning, Disability, and Health (ICF) Generic Set are shown in Table 2.

Table 2.

Baseline characteristics in stroke patients included in the study.

Baseline data	N	Mean±SD	Median	Interquartile interval
Age
≤65 years	132
>65 years	251
Gender
Male	277
Female	106
Hemiplegic side
Left	201
Right	182
Stroke subtype
Hemorrhagic	120
Ischemic	263
Insurance
Urban medical/NCMS/at own expense	340
Free medical service or commercial	43
Time after onset	383	71.68±49.71	72	[32, 110]
b130	383	4.76±3.11	5	[2, 8]
b152	383	4.09±2.93	4	[2, 6]
b280	383	2.39±2.69	2	[0, 4]
d230	383	6.72±2.99	8	[5, 9]
d450	383	7.05±3.55	9	[4, 10]
d455	383	3.18±2.97	10	[8, 10]
d850
0	189
1–10	194
Mobility*		15.23±6.14	18	[11, 20]
Length of stay		22.73±9.75	21	[16, 28]

^*Mobility=items d450 and d455.

NCMS – new cooperative medical scheme.

Table 3 shows the results of univariate analysis. The types of medical insurance, three categories, items b130, b152, and d230 in the ICF Generic Set, and mobility were associated with prolonged hospital stay (P<0.05). Multivariate regression analysis was performed to identify the variables. As shown in Table 4, the type of medical insurance and item d230 (performance of daily activities) were independent predictors of prolonged LOS. An odds ratio (OR) >1 indicated an increased risk of prolonged LOS compared with that for the reference group. Patients with free medical service or commercial medical insurance were more susceptible to prolonged LOS than other patients. Also, the risk of prolonged LOS in stroke patients was associated with the ICF Generic Set index d230 on admission.

Table 3.

Univariate analysis of prolonged length of stay (P-LOS) (n=383).

Baseline data	Normal LOS (n=273)	P-LOS (n=110)	p-value
Age
≤65 years	98	34	0.353
>65 years	175	76
Gender
Male	199	78	0.694
Female	74	32
Hemiplegic side
Left	144	57	0.869
Right	129	53
Stroke subtype
Hemorrhagic	78	42	0.067
Ischemic	195	68
Insurance
Urban medical/NCMS/at own expense	248	92	0.043
Free medical service or commercial	25	18
Time after onset	68.36±50.09	73.59±53.26	0.575
b130	4.45±3.09	5.54±3.02	0.002
b152	3.76±2.93	4.91±2.79	0.001
b280	2.28±2.67	2.66±2.75	0.217
d230	6.15±3.15	8.12±1.95	0.000
Mobility	14.25±6.54	17.68±4.09	0.000
d850
0	142	47	0.100
1–10	131	63
Length of stay (LOS)	17.77±5.43	35.06±6.68	0.000

P-LOS – prolonged length of stay; NCMS – the new cooperative medical scheme.

Table 4.

Multivariate stepwise regression analysis for the prediction of prolonged length of stay (P-LOS).

Model		Model 1	Model 2	Model 3	Model 4
Variables		P-LOS	P-LOS	P-LOS	P-LOS
Age	≤65 years
	>65 years	1.568*	1.572*	1.592*	1.618*
		(0.424)	(0.425)	(0.430)	(0.436)
Gender	Female
	Male	0.841	0.835	0.819	0.809
		(0.230)	(0.228)	(0.223)	(0.220)
Insurance	Urban medical/NCMS/at own expense
	Free medical service or commercial	2.742***	2.750***	2.755***	2.587**
		(1.033)	(1.035)	(1.036)	(0.962)
b130		0.986
		(0.054)
b152		1.061	1.052	1.051
		(0.060)	(0.048)	(0.048)
d230		1.252***	1.247***	1.307***	1.338***
		(0.097)	(0.095)	(0.073)	(0.069)
Mobility		1.032	1.032
		(0.038)	(0.038)
AIC		426.108	424.178	422.951	422.159

Values in the table were odds ratio with standard errors in parenthesis;

^***p<0.01,

^**p<0.05,

^*p<0.1.

NCMS – new cooperative medical scheme; AIC – Akaike information criteria.

Following stepwise regression analysis, relevant variables were selected to develop an optimal model. Model 1 was the initial model with a total of seven variables that included age, gender, type of medical insurance, items b130, b152, d230, and mobility in the ICF Generic Set. Among the seven variables, item b130 gave the highest P-value (0.79). After deleting item b130 from model 1, we constructed model 2. In the same way, model 3 was subsequently examined with the removal of the item of mobility. Finally, model 4 with the four predictors of age, gender, type of medical insurance, and item d230 (performing routine daily activities) provided the best fit. The model was then represented with a nomogram, as shown in Figure 3 [42]. Using this nomogram, the risk of prolonged LOS for an individual patient could be estimated with the available information at admission. For example, a 66-year-old female patient entitled to NCMS with ischemic stroke and a baseline score of 10 in the d230 category corresponded to a total score of 124, which indicated an approximate risk of prolonged LOS of 0.54.

Figure 3.

Nomogram for predicting prolonged length of stay (LOS) in stroke patients using the International Classification of Functioning, Disability, and Health (ICF) Generic Set case mix. The insurance_bi of 0 indicates urban medical insurance, new cooperative medical scheme (NCMS) or no medical insurance (patients paid their own medical costs). A value of 1 indicates free medical service or commercial insurance.

Figure 4 shows that the area under the ROC curve (AUC) was 0.699, and the 95% confidence interval (CI) was 0.646–0.752, indicating that the model could effectively identify 69.9% of patients with prolonged LOS. We identified the cut-off point of the predicted probability of prolonged LOS as 0.20. At this optimal cut-off point, the sensitivity of the ROC classifier was 0.84, and the specificity was 0.49. The Hosmer-Lemeshow test (Figure 5) showed that the predicted probabilities were not significantly different from the actual observations, with a P-value of 0.308 (χ²=11.66), and the calibration plot supported this finding.

Figure 4.

Receiver operating characteristic (ROC) curve to evaluate the discrimination of the prediction model.

Figure 5.

Calibration plot to evaluate the accuracy of the prediction model.

Discussion

In this study, the International Classification of Functioning, Disability, and Health (ICF) Generic Set [24,44] was applied to develop a risk prediction model for a prolonged length of stay (LOS) in stroke patients in 50 inpatient rehabilitation centers in 20 provinces across mainland China. The results indicated that the score for the ICF Generic Set item d230 (performing routine daily activities) at admission was an independent predictor of prolonged LOS. Therefore, the ability to perform routine daily activities or the ability to participate in activities was more likely to affect the LOS in stroke patients when compared with other categories, emphasizing the clinical significance of participation in the ICF framework [43]. Also, the type of medical insurance for stroke patients was identified as an independent predictor of prolonged LOS, which was consistent with the findings from a previously published retrospective study in China [44]. Patients with free medical service or commercial insurance tended to stay hospitalized longer than other patients, although they may not have required more treatments because they paid less for their own medical treatment. Most patients with stroke in Chinese rehabilitation settings prefer not to be discharged because they believe that they would be better cared for in the hospital. As long as cost was not an issue, patients tended to stay in hospital until their recovery was complete.

The findings from the present study showed that there was no significant predictive effect on prolonged LOS from baseline indicators such as age, stroke subtype, time after stroke onset, and mobility as measured by ICF Generic Set on admission. While age has previously been identified as a significant contributor to LOS from retrospective studies in the United States [8], its predictive value on LOS was not detected in the present study, and in previous studies in mainland China and in Taiwan [3,44]. These differing published findings might be due to the differences in the healthcare systems between countries or regions. Also, the nomogram in this study showed that patients with increased age tended to stay hospitalized for longer. Also, it is notable that the ICF Generic Set index of mobility in model 1 and model 2 showed odds ratio (OR) values of >1, indicating that mobility measured by ICF Generic Set contributed to prolonged LOS, but this finding did not reach statistical difference. Stroke severity is recognized to have an impact on LOS [45], but the category of mobility in the ICF Generic Set may not be a significant indicator of stroke severity.

In this study, after screening the variables by multivariate stepwise logistic regression analysis, a predictive model was established, and the nomogram was mapped to quantify the probability of prolonged LOS visually. Compared with previous studies that investigated the predictors of LOS using functioning information [6,8–10,44,46,47], for the first time, the present study reported a feasible prediction model for prolonged LOS in stroke patients. This model may be used and applied by stakeholders in real-world clinical settings using the ICF Generic Set case mix. This study was also the first rehabilitation prediction model to study LOS in stroke patients that was reported in compliance with the reporting guidelines for the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) [29]. In terms of the requirements for the study sample size for this binary outcome study, minimal events per variable (EPV) of 10 was previously supported in several simulation studies for valid predictions [48,49]. The EPV calculated in the present study reached 27.5 (110 P-LOS/4 variables), indicating that the study included a sufficient sample size to make the predictions. Also, receiver operating characteristic (ROC) analysis demonstrated that the present model had acceptable discrimination and high sensitivity for predicting prolonged LOS. The predictive accuracy was also validated with a calibration plot.

Previous studies have identified several baseline factors associated with LOS, such as comorbidity [50,51], cognitive impairment [6], and severity as measured by the National Institute of Health Stroke Scale (NIHSS) [3,45]. Therefore, a possible limitation of this study was the number of factors included in the baseline data of this study. The ICF provides a universal method to describe functioning [52], and the goal of prediction models is that knowledge gaps across different groups might be bridged with simple and accessible tools [53]. Although this model embedded only two independent predictors, it showed acceptable discrimination and accuracy. More importantly, this model may allow healthcare stakeholders, including physicians, patients, and caregivers, to quantitatively estimate the risk of prolonged LOS using available patient information and minimal functioning assessment at the time of admission. Application of this practical model may enable shared clinical and cost decision-making, and decisions on treatment regimen during hospitalization and referral after hospital discharge. For patients with high probabilities of prolonged LOS, hospital management departments may adjust strategies accordingly to promote hospital bed turnover and improve the allocation of health resources. Also, insurance companies may tailor their reimbursement policy for individual patients at an increased risk of prolonged LOS.

This study had several limitations. An optimal predictive model requires two datasets, one for model development and the other for validation, also known as external validation [49]. External validation uses data collected by participants from the same investigator at a different period, or data collected by other researchers in other hospitals or countries, typically with the same predictors and outcome definitions and measurements [54]. The data used in this study were collected from 383 hospitalized stroke patients in 50 hospitals across 20 provinces of mainland China. The samples were representative in a prospective manner. The provision of rehabilitation services across different hospitals did vary to some degree. An approach to compensate for the heterogeneity would be to include a sufficient sample size for model development and to include external validation. However, with the single dataset available, in this study, only internal validation was performed. A split-sample analysis was not applied because splitting one dataset into two parts for model development and validation was considered an inefficient form of internal validation that would have reduced the sample size of the dataset for model development [55]. Therefore, the predictive performance of the model was subject to further validation of external data. Also, the data used in this study were derived from a large multicenter cohort study covering 60 diagnostic groups in various departments such as cardiology, surgery, and rehabilitation. Considering that the general LOS of the rehabilitation department was longer than that of other clinical departments, such as the department of surgery, the subgroup of the rehabilitation medicine department was used to investigate the LOS. The data showed that more than 40% of patients in the rehabilitation departments were admitted due to stroke, and the rest of the patients had up to 15 diagnoses including spinal cord injury, fracture, and traumatic brain injury. In the patients studied, to ensure homogeneity, the data was confined to stroke patients. Such data processing simplified the statistical analysis and also confined the study participants to hospitalized stroke patients in the rehabilitation department, limiting generalization of the prediction model. Also, to validate the performance of the predictive model in other diseases, further studies with a focus on expanding the sample size of specific diseases for external validation is recommended.

Conclusions

The aim of this study was to develop a risk prediction model for prolonged length of stay (LOS) in stroke patients in 50 inpatient rehabilitation centers in 20 provinces across mainland China based on the International Classification of Functioning, Disability, and Health (ICF) Generic Set case mix on admission. The risk prediction model for prolonged LOS in stroke patients showed that the scores for the type of medical insurance and the performance of daily activities (item d230) on admission were independent predictors of prolonged LOS. This prediction model may allow stakeholders to estimate the risk of prolonged LOS on admission quantitatively, facilitate the financial planning, treatment regimens during hospitalization, referral after discharge, and reimbursement.