Algorithm for predicting death among older adults in the home care setting: study protocol for the Risk Evaluation for Support: Predictions for Elder-life in the Community Tool (RESPECT)

Abstract

Introduction

Older adults living in the community often have multiple, chronic conditions and functional impairments. A challenge for healthcare providers working in the community is the lack of a predictive tool that can be applied to the broad spectrum of mortality risks observed and may be used to inform care planning.

Objective

To predict survival time for older adults in the home care setting. The final mortality risk algorithm will be implemented as a web-based calculator that can be used by older adults needing care and by their caregivers.

Design

Open cohort study using the Resident Assessment Instrument for Home Care (RAI-HC) data in Ontario, Canada, from 1 January 2007 to 31 December 2013.

Participants

The derivation cohort will consist of ∼437 000 older adults who had an RAI-HC assessment between 1 January 2007 and 31 December 2012. A split sample validation cohort will include ∼122 000 older adults with an RAI-HC assessment between 1 January and 31 December 2013.

Main outcome measures

Predicted survival from the time of an RAI-HC assessment. All deaths (n≈245 000) will be ascertained through linkage to a population-based registry that is maintained by the Ministry of Health in Ontario.

Statistical analysis

Proportional hazards regression will be estimated after assessment of assumptions. Predictors will include sociodemographic factors, social support, health conditions, functional status, cognition, symptoms of decline and prior healthcare use. Model performance will be evaluated for 6-month and 12-month predicted risks, including measures of calibration (eg, calibration plots) and discrimination (eg, c-statistics). The final algorithm will use combined development and validation data.

Ethics and dissemination

Research ethics approval has been granted by the Sunnybrook Health Sciences Centre Review Board. Findings will be disseminated through presentations at conferences and in peer-reviewed journals.

Trial registration number

NCT02779309, Pre-results.

Strengths and limitations of this study.

• Risk Evaluation for Support: Predictions for Elder-life in the Community Tool (RESPECT) will be developed using a large, routinely collected, population-level home care data set in Ontario, Canada (over 1.3 million home care assessment records between 2007 and 2014).

• RESPECT will improve the identification of people who are and are not near the end of life, by including a wider range of predictors than in prior studies.

• RESPECT will enhance immediate care planning by presenting outcomes as survival time, which is inherently more relatable than a probability of death.

• Developing a single predictive tool that can be applied across low-risk and high-risk home care users is challenging, and our final predictive algorithm may not be well calibrated given the wide-ranging set of relevant risk factors among community-dwelling older adults.

• Our data set contains predictors that may have a time-varying effect on mortality and could violate the proportional hazards assumption. These issues will be extensively assessed and addressed.

Introduction

Although people commonly receive home care prior to death, few receive palliative care and other early interventions that have been shown to improve the quality of dying, such as advance care planning.^1–3 In Ontario, for example, ∼40% of decedents received home care in their last year of life. However, about half of all palliative care was delivered in the last 2 months of life, despite existing recommendations for an earlier introduction of support for those who are frail.^{4 5} Prognostic tools have the potential to help older adults living in the community, their informal caregivers and the healthcare team recognise their need for supportive and palliative care early on. The purpose of this study is to develop and validate a prognostic model of death for people in the home care setting, who may be in their last 1–2 years of life.

Ontario, Canada, is well suited for the development of prognostication tools in the community setting. The public funding and provision of home care services in Ontario means that most individuals with long-term home care needs will receive a structured assessment of their health and functional capacity using the standardised Resident Assessment Instrument for Home Care (RAI-HC). The mandatory administration of the RAI-HC by case managers and nurses involved in care planning has generated a wealth of health information in this province. In addition, the RAI-HC has been linked, at the individual level, to other health administrative databases and vital statistics (ie, deaths) by the Institute for Clinical Evaluative Sciences (ICES). The linked data create opportunities for us to examine individuals' use of health services across multiple sectors over time, and the association between health measures (captured by the RAI-HC) and outcomes. In this study, we used RAI-HC assessments collected from ∼486 000 older adults in Ontario who had at least one RAI-HC assessment over a 7-year period (2007–2014).

Objectives

The primary objective of this study is to develop a mortality risk prediction model to estimate the conditional survival time for community-dwelling older adults who may be nearing the end of life. A secondary objective is to develop a risk communication tool—the Risk Evaluation for Support: Predictions for Elder-life in the Community Tool (RESPECT)—using our final validated algorithm. RESPECT will be implemented as a web tool and accessible to the public from http://www.projectbiglife.ca. It is designed for use by home care clients, their informal caregivers and their healthcare providers. As this algorithm will be developed from a validated instrument that is widely used in the home care setting,^{6 7} it could also readily transform routinely collected data into a risk communication aid for home care providers—especially when deciding whether palliative and end-of-life care should be initiated. The algorithm can also be used for health system planning, by assessing the needs of older adults living in the community at an aggregated level.

Existing algorithms and methodological issues

Previous algorithms for predicting death in people who are frail or near the end of life have generally performed well, with most presenting good discrimination (c-statistics≥0.75).^8–10 However, we have noted several areas for improvement. First, many algorithms present the outcome as a probability of death or a risk score on a mortality-based index.^8–13 Probabilities and index scores are informative for research and to healthcare professional, but they may be difficult for patients and caregivers to comprehend.¹⁴ Presenting a range of measures—including those that are inherently more intuitive, such as survival time—may be helpful for care planning.

Second, many existing algorithms developed for older adults living in the community predict relatively long survival times (eg, up to 10 years).^{8 11 12} Long-term predictions have limited application in immediate care planning, especially for home care recipients who may be in their final months of life. In this study, we propose the estimation of median survival time at 6 and 12 months following a health assessment among older adults living in the community. We will also examine the predicted survival time across risk groups, such as individuals with cancer, dementia or at various levels of functional dependence.

Third, most existing prognostic tools are not adaptive and were not developed as web applications.^{9–12 15} They often ask the same questions of all respondents to facilitate the paper-based application and calculation. Some contain too many items¹⁶ and are not accessible to community-dwelling older adults.¹⁷ Electronic tools can have an adaptive design, with the questions appearing in stages and tailored to prior responses. For example, a person may not need to be asked about challenges with activities of daily living (ADLs), such as eating, if they indicate no limitations in performing instrumental ADLs (IADLs), such as shopping and meal preparation. Additionally, an adaptive implementation can improve the efficiency in administration and the completion rate by reducing response burden on low-risk individuals, while adding discriminatory power to separate medium-risk and high-risk individuals with further queries.¹⁸ Furthermore, we will mostly use risk factors that patients and their family can report, presented in simple terms, to enable self-assessment by people in the community.

Finally, all algorithms have shortcomings concerning their development and validation. A series of recent publications have called for substantial improvements in the design, conduct, analysis and reporting of prognostic studies.^19–22 Several threats to validity are particularly noteworthy for risk algorithms, including inadequate sample sizes, data-driven or arbitrary categorisation of continuous predictors and failure to check model assumptions. Many studies have failed to report or to assess, formally, the underlying assumptions of estimating proportional hazards models.^{23 24} Furthermore, the use of tests of association for selecting predictor variables, data-driven categorisation or specification of functional form and stepwise variable selection procedures can result in models with an overstated predictive ability and increase the risk of type I error.

Contribution of this study

This protocol to develop RESPECT adheres to a recommended checklist of items to include in protocols for prognostic studies.^{25 26} We will capitalise on several opportunities to develop mortality risk algorithms for older people in the community setting. First, we are presenting our study protocol to improve transparency in research, protect against bias and ensure reproducibility.²⁶ We have prespecified our analysis plan and predictive variables before examining their relationship to our outcome of interest. Given the goal of generalising to other population-based settings, this is especially important to avoid overfitting.²⁶ Predictors will be chosen based on their clinical relevance, prior findings of an association with our outcome of interest and the overall number of df. Second, we will use a population-based, routine assessment system to build and evaluate our algorithm. The RAI-HC is a standardised tool that has been used in many other populations and settings,^{6 27} which suggest a high potential for application and validation of our model in other jurisdictions. Third, our large data set (n≈437 000 individuals) will allow for the inclusion of a greater number of predictive variables than in prior studies.²⁸ Fourth, we will test model assumptions with particular attention to the violation of the proportional hazards criteria. Fifth, we seek to develop an algorithm that is discriminating and well calibrated for moderate-risk and high-risk home care clients, who may be in their last months of life. Sixth, we will include metrics of mortality beyond categories of fragility and/or fixed-time mortality probabilities (eg, a 6-month or 12-month probability of death). Our primary outcome will be survival time, because of its intuitively understandable nature and relevance to care planning. However, we will generate probabilities of death and evaluate RESPECT against other predictive models (eg, the Changes in Health, End-Stage Disease, Signs, and Symptoms Scale or CHESS²⁴) that have been used for palliative care assessment and planning. We also aim to present the survival curves for key patient populations (eg, people with and without cancer, dementia and with varying levels of functional limitations). Seventh, we aim to include risk factors that can be easily reported by the patient and their family—that is, no physical measures (eg, blood pressures and calf circumference) will be used. Self-reportable measures will allow the algorithm to be implemented as a web tool for the public that can be completed by individuals and their informal caregivers in the community. Furthermore, this may reduce erroneous interpretations of the questions by the patient and, consequently, an inaccurate estimate. Prognostication tools that are based on self-reported functional status are easy to administer and have been shown to have good predictive power for mortality and functional decline.¹⁰ Finally, we will use all RAI-HC assessments (ie, admission and follow-up assessments), such that the algorithm will reflect how the RAI-HC instrument is used in the real-life setting.

Method and analysis

Study design

RESPECT will be derived and validated using population-based data in Ontario, Canada—a multicultural province with 13.6 million residents. The study population of interest is people who are eligible for long-term (also known as ‘long-stay’) government-funded home care and have received at least one RAI-HC assessment between 1 January 2007 and 31 December 2013.

Eligibility criteria

In Ontario, the RAI-HC is used by service coordinators to determine the needs of clients receiving publicly funded home care and residential long-term care services, and has been a mandatory assessment since 2002.²⁹ This routinely administered assessment instrument contains information on the clients' demographic characteristics (eg, age, sex, marital status and level of education), living arrangement, presence of informal caregivers, functional status (eg, IADLs and ADLs), physical health (eg, chronic health conditions), cognitive capacity and prior health service use (eg, number of hospital admissions in the last 90 days).

According to the provincial policy, home care clients must undergo an RAI-HC assessment if they are expected to be a long-term recipient of home care (ie, require at least 60 consecutive days of services without being discharged from the home care programme) or if they wish to apply for admission to a long-term care facility (eg, nursing homes).²⁹ However, not everyone who has an RAI-HC assessment receives home care; individuals can decline the provision of home care or be admitted to other care settings (eg, hospitals or nursing homes) before home care can be initiated. Existing home care policy stipulates that a reassessment using the RAI-HC should be completed at least once every 6 months or when significant changes in the client's situation have been observed (eg, new disease diagnosis, postdischarge following a hospital stay and functional and/or health declines). However, adherence to this policy varies among care providers, resulting in irregular follow-up time between assessments for the clients in our sample. For this analysis, respondents will be excluded if they are not eligible for Ontario's universal health insurance programme (ie, the Ontario Health Insurance Plan or OHIP), younger than age 50 years at the time of the study or did not receive a structured RAI-HC assessment.

Sample size

The derivation cohort consists of ∼437 000 community-dwelling older adults who had at least one RAI-HC assessment between 1 January 2007 and 31 December 2012. These individuals contributed to over 2 435 000 person-years of follow-up, with a median duration of 1.79 years (IQR: 0.73–3.31 years) from their RAI-HC assessment to death or the end of the study (31 December 2012). During this period, there were ∼227 000 deaths or 52% of individuals in our derivation cohort.

All RAI-HC assessments will be included in our analysis. Initial assessments for those who were newly admitted to home care constitute ∼40% (n≈455 000) of our data set. Routinely conducted reassessments, usually ∼6 months following admission, make up 53% of all assessments (n≈604 000). An additional 7% (n≈80 000) of the total were non-routine assessments often conducted following a change in the client's health status or service plan.

Existing guidelines on sample size requirements for the derivation of prediction models^{30 31} stipulates that the number of participants experiencing the event should exceed 10 times the number of df (ie, 10 events per variable). Given the large number of events (∼227 000 deaths) in our derivation cohort, we expect that the RESPECT algorithm will have an adequate sample size for the number of predictors being considered. Additionally, Vergouwe et al³² recommends a minimum of 100 events and 100 non-events to obtain adequate power in external validation studies. Since our validation cohort will include ∼18 000 deaths from a sample of 122 000 home care recipients, our sample size provides sufficient power to conduct a temporal split sample validation using RAI-HC data collected between 1 January and 31 December 2013.

Outcomes

Death will be ascertained through linkage to a population-based registry—the Registered Persons Database (RPDB)—maintained by the Ontario Ministry of Health and Long-Term Care. The RPDB contains a historical listing of all unique healthcare numbers that have been issued under OHIP and where information on the deceased (ie, name, age, sex and date of death) can be obtained. The data will be modelled as time to death from the RAI-HC assessments. Results will be presented for the overall sample as well as for important subgroups (eg, individuals with dementia, cancer or varying levels of function limitation).

We will also present the probabilities of death at 6 and 12 months to allow comparison against other published algorithms (eg, CHESS). We will evaluate the acceptability, perceived usefulness and value of these metrics to home care clients and their care providers in a subsequent qualitative study to be conducted by our research team.

Analysis plan

We closely followed guidelines by Harrell³⁰ and Steyerberg³³ in the development of our analysis plan, which was constructed after accessing the study data set, but before any model fitting or any descriptive analyses involving the exposure–outcome associations. Key considerations in our approach include full prespecification of the predictor variables, use of flexible functions for continuous predictors and preserving statistical properties by avoiding data-driven variable selection procedures. All analyses will be performed in SAS Enterprise Guide (V.6.1).³⁴

Identification of predictors

Identification of predictor variables was based on data in the RAI-HC, the extant theoretical and empirical literature, clinical experience and consultation with subject-matter experts. We began by compiling predictors identified in prior studies and systematic reviews of mortality prediction models for an older, community-dwelling adult population.^{8 10–13} Then, we reviewed several frailty indices that were developed from mortality prediction models for additional factors that were associated health declines leading to mortality.^35–38 We adopted a multidimensional framework by considering overlapping domains of physical functioning (eg, difficulties with ADLs and reduced mobility), cognitive impairment (eg, memory decline and psychosis), sociodemographic factors (eg, marital status and level of education) and biological diseases (eg, diabetes, heart disease and cancer).^39–41 We included additional measures of prior healthcare use (eg, number of hospital admissions or emergency room visits in the last 90 days), dependency on life-sustaining therapies (eg, dialysis and respirator) and symptoms of reduced health and physiological reserve (eg, weight loss, oedema and vomiting) to improve the discriminatory power of our algorithm, especially for people nearing the end of life. We also incorporated cohort characteristics (eg, year of the RAI-HC assessment, the type of and reason for assessment) that may account for remaining heterogeneity in the estimated risks. We will select variables based on their clinical relevance and may exclude some that are difficult to understand by the general user. A list of predictors—comprising of 93 df from 47 variables—that will be operationalised in our development model is presented in table 1. Figure 1 presents a conceptual map that groups predictors based on evidence-supported associations to mortality found in the existing literature and clinical experience among members of the research team.

Table 1.

Prespecification of predictor variables taken from the RAI-HC, with initial df allocation

Variable	Scale	Codes/values	df
Sociodemographic factors
Age	Continuous	50–110	4
Sex	Dichotomous	0=Male 1=Female	1
Highest level of education	Categorical	0=No schooling; 8th grade or less; Grades 9–11 1=(Completed) high school 2=Technical or trade school 3=Some college/university; Diploma/Bachelor's degree; Graduate degree 4=Unknown/missing	4
Social support
Marital status	Categorical	0=Married 1=Widowed 2=Divorced; separated 3=Never married 4=Other	4
Primary informal caregiver lives with client	Dichotomous	0=No; no such helper 1=Yes	1
Health conditions
Hip fracture	Dichotomous	0=Not present 1=Present	1
Diabetes	Dichotomous	0=Not present 1=Present	1
Cerebrovascular accident (stroke)	Dichotomous	0=Not present 1=Present	1
Congestive heart failure	Dichotomous	0=Not present 1=Present	1
Coronary heart disease	Dichotomous	0=Not present 1=Present	1
Hypertension	Dichotomous	0=Not present 1=Present	1
Peripheral vascular disease	Dichotomous	0=Not present 1=Present	1
Dementia (including Alzheimer's disease)	Dichotomous	0=Not present 1=Present	1
Multiple sclerosis	Dichotomous	0=Not present 1=Present	1
Parkinson's disease	Dichotomous	0=Not present 1=Present	1
Cancer (any)	Dichotomous	0=Not present 1=Present	1
COPD/emphysema/asthma	Dichotomous	0=Not present 1=Present	1
Renal failure	Dichotomous	0=Not present 1=Present	1
Irregularly irregular pulse (atrial fibrillation)	Dichotomous	0=Not present 1=Present	1
Any psychiatric diagnosis	Dichotomous	0=Not present 1=Present	1
End-stage disease (with prognosis of <6 months to live)	Dichotomous	0=No 1=Yes	1
Functional status and cognition
Instrumental ADLs
Ordinary housework	Categorical	0=No difficulty 1=Some difficulty 2=Great difficulty	2
Meal preparation	Categorical	0=No difficulty 1=Some difficulty 2=Great difficulty	2
Using the telephone	Categorical	0=No difficulty 1=Some difficulty 2=Great difficulty	2
ADLs
Personal hygiene	Categorical	0=Independent 1=Setup help only; supervision; 2=Limited assistance 3=Extensive assistance 4=Maximum dependence; total dependence 5=Activity did not occur	5
Toilet use	Categorical	0=Independent 1=Setup help only; supervision; 2=Limited assistance 3=Extensive assistance 4=Maximum dependence; total dependence 5=Activity did not occur	5
Locomotion (in home)	Categorical	0=Independent 1=Setup help only; supervision; 2=Limited assistance 3=Extensive assistance 4=Maximum dependence; total dependence 5=Activity did not occur	5
Eating	Categorical	0=Independent 1=Setup help only; supervision; 2=Limited assistance 3=Extensive assistance 4=Maximum dependence; total dependence 5=Activity did not occur	5
ADL decline	Dichotomous	0=No 1=Yes	1
Cognitive function
Expression/making self-understood	Categorical	0=Understood 1=Usually understood 2=Often understood; sometimes understood 3=Rarely/never understood	3
Decision-making capacity	Categorical	0=Independent 1=Modified independence; minimally impaired 2=Moderately impaired 3=Severely impaired	3
Short-term memory	Dichotomous	0=Not impaired 1=Impaired	1
Decline in decision-making capacity	Dichotomous	0=No 1=Yes	1
Signs and symptoms of health decline
Vomiting	Dichotomous	0=No 1=Yes	1
Peripheral oedema	Dichotomous	0=No 1=Yes	1
Dyspnoea	Dichotomous	0=No 1=Yes	1
Weight loss	Dichotomous	0=No 1=Yes	1
Insufficient fluid	Dichotomous	0=No 1=Yes	1
Decrease in food or fluid intake	Dichotomous	0=No 1=Yes	1
Daily pain	Dichotomous	0=No 1=Yes	1
Healthcare use
Number of hospital admissions without overnight stays in last 90 days (or since last assessment)	Categorical	0, 1, 2, 3+	3
Number of emergency room visits without overnight in last 90 days (or since last assessments)	Categorical	0, 1, 2, 3+	3
Special therapy: respirator/oxygen therapy	Categorical	0=Not applicable 1=Scheduled, not received 2=Scheduled, partial/full adherence	2
Special therapy: chemotherapy	Categorical	0=Not applicable 1=Scheduled, not received 2=Scheduled, partial/full adherence	2
Special therapy: dialysis	Categorical	0=Not applicable 1=Scheduled, not received 2=Scheduled, partial/full adherence	2
Cohort characteristics
Year of RAI-HC assessment	Categorical	2008, 2009, 2010, 2011, 2012	5
Reason for assessment	Categorical	0=Initial (admission) assessment 1=Follow-up assessment 2=Routine (fixed interval) assessment 3=Significant change in status assessment 4=Review at the return from hospital	4

ADLs, activities of daily living; COPD, chronic obstructive pulmonary disease; IADLs, instrumental activities of daily living; RAI-HC, Resident Assessment Instrument for Home Care.

Figure 1.

Concept framework of predictors in RESPECT, grouped by evidence-supported contribution to mortality risk found in the existing literature. RESPECT, Risk Evaluation for Support: Predictions for Elder-life in the Community Tool.

Functional capacity will be represented by three commonly used functional and cognitive scales. The IADL Difficulty Scale is a hierarchical index reflective of difficulties with performing housework, preparing meals and using the telephone.⁶ The ADL Hierarchy Scale includes difficulties with personal hygiene, toileting, locomotion and eating. The extent of a person's limitations in performing these tasks will capture the progressive nature of senescence and is representative of early, middle and late loss functions. Moreover, the ADL Hierarchy Scale is suitable when significant functional deterioration is expected in the short term.⁴² The Cognitive Performance Scale (CPS) is constructed from items that assess home care clients' short-term memory, procedural memory, the capacity to communicate and be understood and their decision-making capacity. The CPS has been shown to correspond closely with scores generated by the Mini-Mental State Examination.⁴³ We will, first, consider specifying each of the tasks that form the scales as separate and independent categorical variables in our model; this will enable us to examine their independent effects on mortality. In application, this approach will offer clearer directions for intervention and care planning. However, as the variables are conceptually related, they are susceptible to multicollinearity. If multicollinearity among the scale items is evident, we will correct it by replacing the individual items with the scale scores.

Data cleaning and coding of predictors

Data cleaning and coding will proceed without examining predictor–outcome associations. Continuous variables will be inspected using boxplots and descriptive statistics to identify values outside of plausible ranges. Values that are clearly erroneous will be corrected, where possible, or otherwise set to missing. To avoid instability in the regression analyses, frequency distributions for categorical predictors will be examined, and categories with small numbers of respondents will be combined. Variables with narrow distributions or insufficient variation will also be excluded. Missing data are rare and does not present a significant risk to selection bias; only 184 of nearly 1 138 000 assessments (0.02%) from our derivation cohort contained missing data. Therefore, assessments with missing data will be excluded from our analysis.

Model specification

Age is the only continuous candidate variable and will be flexibly modelled using restricted cubic splines (given its known association with mortality). Following Harrell's³⁰ recommendation, we will consider the placement of knots at fixed quantiles of the distribution (eg, at the 5th, 27.5th, 50th, 72.5th and 95th centiles for a five-knot spline function). All other predictors will be modelled as linear terms. Ordinal variables with few levels will be specified as individual categories (ie, as multiple indicator variables), or appropriately collapsed based on sample size and interpretability.

Prespecification of predictors has advantages in limiting the risks of overfitting and spurious statistical significance, but may result in a final model that is overly complex and difficult to interpret. Unnecessary variables will also distort the estimated effects of other predictors and, given the size of our sample, may make the model more computationally intensive. It may be possible to derive a more parsimonious model that retains most of the prognostic information and performs as well or better than the full model, without increasing the type I error rate.^{30 44} We will use the stepdown procedure described by Ambler et al⁴⁴ to identify a more parsimonious model. This procedure involves deleting variables to a desired degree of accuracy based on contribution to model R². We may consider further reductions, beyond the stepdown procedure, if the completion time for our web calculator exceeds our goal of 5–10 min, or if the questions are found to be abstruse in our pilot study with community care providers. We will evaluate the reduced model against the full model using Akaike's Information Criterion and by examining the effect on discrimination and calibration.

Model estimation

Our main estimation model will be proportional hazards regression with death as the outcome of interest. We will use centred values of all predictors; this will allow us to generate a predicted value in the event of incomplete or partially completed web questionnaires. We will conduct a formal check for multicollinearity using the VARCLUS procedure in SAS.⁴⁵ Although the risk of overfitting will be minimal, due to prespecification of our model and the large sample size, we will nevertheless assess if this is a concern. The degree of overfitting in the model will be estimated using the heuristic shrinkage estimator, based on the log likelihood ratio χ² statistic for the full model.⁴⁶ If shrinkage is <0.90, adjustment for overfitting will be required.

Assessment of model performance

A key assumption underlying the proportional hazards model is that the relative risk of the outcome between strata of exposure and the baseline risk must be constant over time. Violation of this assumption has been shown to produce biased results,⁴⁷ although Allison⁴⁸ has argued that estimated coefficients of time-varying variables can simply be interpreted as an ‘average’ rather than instantaneous hazard. We will test this assumption by visually examining the plots of raw and smoothed-scaled Schoenfeld residuals against time to identify non-proportionality. We will further examine the existence of non-proportionality by including time-interacted variables in our regression model. If a violation of this assumption is identified—by non-zero correlations between the Schoenfeld residuals and time, or statistical significance of the time-interacted terms—we will implement corrective measures, such as the inclusion of time-interacted predictors or stratification, that are appropriate for proportional hazards regression. We may consider alternate estimation approaches, such as estimating an accelerated failure time model, to account for time-varying effects if these methods are unsuccessful.^{49 50}

Predictive performance will be assessed and reported using overall measures of predictive accuracy (eg, Nagelkerke's R² and the Brier score), discrimination (eg, the c-statistics) and calibration (eg, comparing the observed to predicted deaths by vigintiles of predicted probability). Steyerberg³³ and Cook^{51 52} suggest that calibration should receive more attention when evaluating prediction models and that assessment of recalibration tests and calibration slopes should be performed routinely. We will emphasise visualisation of model performance using plots, rather than formal statistical testing, since the significance of traditional Hosmer-Lemeshow goodness-of-fit tests, for example, may be an artefact of large sample sizes rather than true miscalibration.³⁰ We will generate the calibration slope by regressing the outcome in the validation cohort on the predicted mortality risk. This approach reflects the combined effect of overfitting to the derivation data as well as true differences in effects of predictors.⁴⁶ A well-calibrated model should have a slope of ‘1’.^{30 53} Deviations from this will be tested using a Wald or likelihood ratio test. Additionally, we will create calibration plots comparing mean predicted probabilities with Kaplan-Meier estimates of observed rates across intervals of predicted risk. Subgroup validation will also be implemented as a conceptually easy check of calibration. This entails comparing observed and predicted risks across vigintiles and within predefined subgroups of importance to clinicians and policymakers (eg, age groups, sex, diseases as well as physical and cognitive functioning scale scores). The clinical or policy relevant standard of calibration is defined as <20% difference between observed and predicted estimates for categories with prevalence higher than 5%.⁵⁴

Model presentation

First, we will present the final regression model—that is, a model estimated from the combined sample of the derivation and validation cohorts—showing estimated HRs and 95% CIs. However, given the anticipated complexity of the final regression model, the usual presentation may be less meaningful. We will complement the presentation of the regression models with plots of the survival functions for the entire sample and by subgroups of interest (eg, age groups, sociodemographic groups, diseases, levels of functional and cognitive impairment). Second, in addition to the predicted median survival time at 6 and 12 months, we will aim to estimate and report the survival times at other risk percentiles (eg, 25th and 75th). Third, we will present measures of functional performance (eg, scores on the ADL Hierarchy Scale or the CPS) in our web report on http://www.projectbiglife.ca to aid the interpretation of the predicted mortality risk, since these are commonly used indicators for determining the need for support in the community.

Additional considerations

As with all regression models, the omission of key predictors and misspecification of non-linear relationships can bias the model estimates. Potentially influential predictors that are not captured by the RAI-HC include earned income and rurality, which are indicators of access to healthcare and may serve as indirect proxies for health status. We will consider examining the effects of omitted variable by re-estimating the models with inputs from other health administrative data sources that contain these variables. In addition to the RAI-HC, ICES houses population-level data from physician billings, hospital discharge records and records from individuals receiving care in nursing homes. However, our primary model will only consider RAI-HC data; this is to support external validations and the replication of our study in provinces or countries that do not have linked RA-HC and other health administrative data.

Most existing predictive algorithms in older people have not accounted for non-linear relationships or interactive effects among predictors. For example, to account for individuals with varying severity of chronic obstructive pulmonary disease (COPD), we may interact COPD with receipt of oxygen therapy. We will consider the inclusion of interaction terms if the algorithm performs poorly for subgroups of interest. This process will be guided by relevant literature, such as the interactive effect of age with cancer diagnosis⁹ or disease by functional status.⁵⁵

Following the development of our full algorithm, we aim to create a set of screening questions to reduce response burden for people who are at a low risk of death. RESPECT-lite will be an abridged version of the full algorithm, created by recursive partitioning techniques. We will identify main branches within the classification tree that separates low-risk, medium-risk and high-risk individuals. These nodes will then serve as the screening questions. For example, we expect that people who are at a low risk of dying will have few restrictions in ADLs, whereas people who are at a high risk will have multiple functional limitations and symptoms that are worsening (with possible cumulative effects involving multiple predictors). The ADL items or scale could then be used to classify respondents into the risk groups. We anticipate that RESPECT-lite will take <1 min to complete, based on our experience with other calculators implemented on http://www.projectbiglife.ca. We will assess the validity of RESPECT-lite by comparing it against the full model.

The proportional hazards regression was developed for estimating relative risks associated with a set of covariates, and generating survival time post hoc may be computationally challenging.⁴⁸ We will consider taking a parametric approach to modelling (eg, estimating an exponential or Weibull model) if we are unable to derive survival estimates using proportional hazards regression.

Finally, since the RAI-HC assessments are usually completed by or in the presence of a trained healthcare professional (eg, nurse), self-reporting or the lay interpretation of the items may result in inaccurate estimates for the individual. For example, it has been shown that dementia patients often underestimate their own level of cognitive deficit, in comparison to their caregivers; though this is not a concern for self-reported functional status.^{10 56} In this example, an underestimation of their cognitive deficits may lead to an lower risk of mortality (ie, longer survival) than an accurate assessment. At the time of our algorithm development, the magnitude of the deviation between professional and lay responses is unknown. Nonetheless, we recognise this limitation and the implications of making this tool publicly accessible. We are conducting on-going assessments of the usability and impact of using the RESPECT web calculator through focus groups involving healthcare professionals, patients and their families or caregivers. Our future research involving RESPECT will also include a validation study comparing responses provided by healthcare professionals to those provided by patients.

Conclusion

The development of a communication aid using routinely collected health information suggests high potential for care planning. Risk calculation can be automated at the time of data collection (ie, as the case manager is performing the RAI-HC assessment with the potential home care recipient), and it would be possible to integrate RESPECT into the provincial home care programme to improve the delivery of palliative and end-of-life care. To the best of our knowledge, RESPECT will be the first population-based risk prediction algorithm of mortality among community-dwelling older adults across varying levels of mortality risks. Although a rigorous approach will be used to develop the model, stronger forms of validation may be required (eg, application in other countries and healthcare systems), to assess its generalisability and implementation in other populations.