Mastering Prognostic Tools: An Opportunity to Enhance Personalized Care and to Optimize Clinical Outcomes in Physical Therapy

Abstract

 

In health care, clinical decision making is typically based on diagnostic findings. Rehabilitation clinicians commonly rely on pathoanatomical diagnoses to guide treatment and define prognosis. Targeting prognostic factors is a promising way for rehabilitation clinicians to enhance treatment decision-making processes, personalize rehabilitation approaches, and ultimately improve patient outcomes. This can be achieved by using prognostic tools that provide accurate estimates of the probability of future outcomes for a patient in clinical practice. Most literature reviews of prognostic tools in rehabilitation have focused on prescriptive clinical prediction rules. These studies highlight notable methodological issues and conclude that these tools are neither valid nor useful for clinical practice. This has raised the need to open the scope of research to understand what makes a quality prognostic tool that can be used in clinical practice. Methodological guidance in prognosis research has emerged in the last decade, encompassing exploratory studies on the development of prognosis and prognostic models. Methodological rigor is essential to develop prognostic tools, because only prognostic models developed and validated through a rigorous methodological process should guide clinical decision making. This Perspective argues that rehabilitation clinicians need to master the identification and use of prognostic tools to enhance their capacity to provide personalized rehabilitation. It is time for prognosis research to look for prognostic models that were developed and validated following a comprehensive process before being simplified into suitable tools for clinical practice. New models, or rigorous validation of current models, are needed. The approach discussed in this Perspective offers a promising way to overcome the limitations of most models and provide clinicians with quality tools for personalized rehabilitation approaches.

Impact

Prognostic research can be applied to clinical rehabilitation; this Perspective proposes solutions to develop high-quality prognostic models to optimize patient outcomes.

Introduction

Understanding and predicting which patient will have the best outcomes or respond to a particular treatment has been the priority of many researchers in the physical rehabilitation profession.^1–6 However, these actions have not always translated into clinical settings. Typically, rehabilitation clinicians rely on pathoanatomical diagnoses to guide treatment and define prognosis.⁷ Yet, a diagnosis is a description aggregated from disease characteristics at a single point in time, whereas prognosis is a longitudinal concept regarding patient characteristics that will have an effect on the course of the disease or its outcomes.⁸ Many fields of medicine propose that prognosis should be incorporated into the treatment planning process.⁹ Over the last 15 years, the idea that prognostic-related findings must inform clinical decision making to enhance patient health outcomes has emerged in the literature.^10,11

Targeting prognostic factors is a promising way for rehabilitation clinicians to improve treatment decision-making processes, personalize rehabilitation approaches, and ultimately enhance patient outcomes.¹¹ Integrating prognosis into clinical decision making through prognostic modeling⁸ is the underpinning of personalized care in medicine. This approach funnels care toward those who do not have a favorable prognosis.¹¹ Moreover, a prognostic framework for decision making is advised to better align clinical practice with value-based care principles.^7,12

Rehabilitation clinicians would benefit from quality prognostic tools that provide accurate estimates of the risk of future outcomes for their patients and even predict treatment responsiveness. Accordingly, clinicians also need to be able to identify valuable prognostic tools that are ready for use in clinical practice. Only prognostic tools developed and validated through a rigorous methodological process should guide clinical decision making. This article expands on the work of previous studies and offers a perspective on prognosis in rehabilitation. We argue that mastering prognostic tools is essential to predict outcomes and reach personalized care in rehabilitation. Specifically, we will:

• (1) discuss the potential benefits of prognosis-based decision making;

• (2) outline the most relevant prognosis-related factors for rehabilitation practice;

• (3) explain how prognostic evidence is best integrated into prognostic tools; highlight key elements to consider when selecting a prognostic tool for clinical practice.

From Diagnosis to Prognosis-Informed Clinical Decision Making

Prognosis-related findings that will influence the course of the disease must inform clinical decision making to enhance patient health outcomes.¹¹

Diagnostic findings have been the cornerstone of clinical decision making for disease management by health care professionals.⁹ Diagnostic labeling is mostly based on a pathoanatomical model, focusing on deficits of body structures, using clinical examination and imaging techniques to support clinical reasoning. It is a statement regarding the pathological cause of the patient’s current health condition. This rules on the presence or absence of a pathology at a certain point in time.¹⁰ Patients and their health care professionals are accustomed to defining and managing their problem according to such a diagnostic label. Medical diagnosis primarily classifies the causes of health conditions, whereas physical therapists’ diagnosis primarily classifies the consequences that result from it. Thus, physical therapists use a similar process, but our diagnostic label relates not only to deficits, but also to activity limitations and restrictions in participation.¹³ This is an important advantage that facilitates the integration of the biopsychosocial model into the care we provide.

Yet, despite acknowledgments that a biopsychosocial model is an essential framework to better understand health conditions, the basic concept of diagnosis has not significantly evolved over the past several decades.^14,15 Researchers have only recently identified different groups of individuals with the same diagnostic label (subphenotypes in diagnoses).¹⁶ There are commonly several profiles of patients and trajectories of presentation for a single diagnostic label.¹⁶ Individuals who share the same diagnostic label, but present with different phenotypes, might have different responses to the same treatment. As an example, a recent study identified 5 distinct musculoskeletal phenotypes associated to 11 key biopsychosocial prognostic factors in patients seeking care for neck, shoulder, low back, or multisite/complex pain.¹⁷ These phenotypes at baseline were associated with function and health-related quality of life at 3-months follow-up, irrespective of primary pain location. These findings suggest that, independent of a single diagnostic label, clinical decision making should be informed by phenotype to personalize care and enhance patient outcomes.

Still, not only has diagnosis driven treatment decision making, it is also used to define prognosis.⁷ Diagnostic procedures are generally useful and essential for many health conditions. Yet, their overtly sensitive capacities often suggest pathological findings that are unlikely to cause symptoms, or are a natural process of aging.¹⁸ This phenomenon is known as overdiagnosis, that is, the identification of an asymptomatic pathology not causing any problem or harm to a person.^11,18 Studies assessing the prevalence of findings in diagnostic imaging of asymptomatic persons show surprising results. A study of Girish et al¹⁹ demonstrates the presence of abnormal findings in the shoulders of 96% of asymptomatic subjects who underwent ultrasound assessment. As health care professionals, why would we suggest any treatment for people with nonthreatening asymptomatic findings? And what assures us that diagnostic imaging findings are the real cause of symptoms for our patients? This calls into question the value of clinical decision making that is exclusively guided by diagnostic findings.

Because diagnosis is dichotomous in nature, it can be misleading to guide clinical decision making when used alone.²⁰ Diagnosis for diagnosis sake does not automatically lead to more adequate treatment—when patients with the same diagnosis have different risks of future outcomes, prognosis can be more useful to determine the most appropriate treatment. As suggested by Croft et al,¹¹ prognostic evidence can distinguish people in whom diagnosis and diagnostic testing informs management that improves outcomes, from people in whom those procedures can lead to unnecessary treatment, and even undesirable effects.

The biopsychosocial characteristics of the patient and clinical findings are prognosis-related when they influence the course of the disease.¹¹ These prognosis-related findings advise clinicians on the risk of poor/good outcomes for a patient, which is useful to guide management options. For a given health condition, patients at greater risk of poor outcomes will benefit from treatment strategies that are not needed by patients who will recover promptly. In the context of rehabilitation, this means giving the right treatment to the right patient at the right time—we need to consider that in some cases the right treatment could be the absence of treatment.

Prognosis, a longitudinal concept, is highly relevant to guide rehabilitation, which is a process that takes times. Prognosis goes beyond diagnosis, because it predicts the patient’s trajectory and outcomes (poor/good outcome).²⁰ Hence, considering prognostic variables to guide clinical decision making is a promising way to improve patient outcomes.^11,21

Prognostic Factors, Treatment-Effect Modifiers, and Treatment-Effect Moderators

Prognosis is a complex concept in medicine and its different dimensions and definitions are still progressing.^11,22

Prognosis is a rapidly expanding field of physical rehabilitation medicine.²³ Prognostic research intends to determine the likelihood of future outcomes in individuals with a given health status with the aim to improve health.²⁰ It requires elucidating the factors that predict different outcomes and the best way to assess individual risk to target treatments and improve outcomes.¹¹ Prescriptive research, which is a distinct field but also related to prognostic research, aims to guide decision making in health care by investigating the factors predicting response to a specific intervention.

The main dimensions of prognosis are risk factors, prognostic factors, treatment-effect modifiers (or moderators), and treatment-effect mediators—these are outlined in Table 1.

Table 1.

Types of Prognostic-Related Factors

Population	Factors	Physical Rehabilitation Examples
Individuals who are healthy	Risk factors are characteristics that increase the risk of developing a health condition in an individual without this condition (ie, who is still at risk of developing it).56 Risk factors are of interest in the field of disease prevention and screening for diseases.	Clinicians’ roles in preventive medicine might need to tackle screening tools designed with risk factors. Yet, in most circumstances, physical rehabilitation aims to improve the outcomes for patients with a health condition, meaning that they need to leverage prognostic factors instead.
People with a health condition	Prognostic factors are the characteristics of an individual that can potentially influence the course of a disease.25,57–59 Prognostic factors are associated with subsequent clinical outcomes (with varying time frames) and, if they are modifiable, might be targeted by an intervention to optimize patient outcomes.25	A recent Cochrane review on prognostic factors in low back pain found that patients who have positive recovery expectations are more likely to return to work.60 In the field of physical rehabilitation, yellow flags serve as examples of prognostic factors used to identify patients at high risk of persistent pain and long-term disability.61 Clinicians might adapt their interventions according to the presence of these factors to improve clinical outcomes.61,62
	Treatment-effect modifiers (or moderators) are the characteristics of an individual with a given health condition that predict a response to a treatment.26,57–59,63 They influence the relationship between a specific intervention and an outcome. Treatment-effect modifiers often inform the development of clinical decision rules and classification systems to guide clinical decision making for disease management.	A patient with low back pain who has an occupation with low physical demands and uses pain medication is expected to have a better response to an exercise therapy program compared with a patient who does not exhibit these characteristics (positive treatment-effect modifiers).64
	Treatment-effect mediators are the mechanisms by which the intervention produces the outcome.59,63 They are factors that change in response to the intervention and are correlated with a specific outcome. Mediators are not used to develop clinical decision rules but can inform on the treatment components that contributed the most to the successful result.59,63	When an individual’s physical function improves following exercises, the treatment mediator might be the decrease of fear-avoidance behaviors or kinesiophobia.

Risk factors can help in disease prevention for healthy individuals, which might be of interest as the role of rehabilitation clinicians in preventive medicine continues to develop. Prognostic factors mainly explain the variation in outcomes between individuals with a health condition, whereas predictors of treatment-effect relate to findings that predict if the patient will respond to a specific approach or not. Finally, treatment-effect mediators describe the mechanism of an effect.

Regarding patient care, modifiable prognostic factors and treatment-effect modifiers (Fig. 1) are of great value for rehabilitation clinicians to improve clinical decision making, because they can be targeted in rehabilitation to improve clinical outcomes. Integrating prognosis-related factors into practice could allow for personalized care and foster a better allocation of resources in health care systems, while improving patient outcomes, by giving the right level of care to the right patient.

Figure 1.

Prognostic factors and treatment-effect modifiers can be integrated into prognostic tools that can be used to personalize rehabilitation approaches.

Combining several known prognostic factors or treatment-effect modifiers in a single prognostic tool could improve the accuracy of outcome prediction and foster clinical applicability.²⁴

The “Ontogenesis” of Prognostic Tools

Prognostic tools are useful only if they can accurately predict an outcome for a patient. This requires a comprehensive development and validation process before being simplified into tools that are suitable for clinical practice.

The PROGnosis RESearch Strategy (PROGRESS) working group details the prognosis research rationale for fundamental prognostic research, prognostic factors, prognostic models and stratified medicine research (Fig. 2).^20,24–26 The process underlying the development of prognostic models first requires the investigation of prognostic factors via prospective observational studies. These are performed to determine the characteristics that might predict the outcomes between 2 groups of patients sharing the same health condition, but with different outcomes.^25,27 To improve on the accuracy of probability estimates, single factors must be integrated into prognostic models. These models are built by combining the effect of each known predictor according to their relative weight to obtain a probability.²⁸ When clinicians are interested in a patient’s prognosis, the use of clinical tools derived from prognostic models allows more accurate and individual prediction, compared with using only isolated prognostic factors.²⁴

Figure 2.

Representation of the PROGnosis RESearch Strategy (PROGRESS) framework.^20,24–26

We detail the entire process of prognostic model development, validation, and clinical impact assessment guided by a conceptual framework underlying the development of prognostic models, as recently outlined by Kent et al.²²

Prognostic Model Development

Prognostic model studies encompass development and validation studies. The development phase includes finding important predictors for the outcome of interest, and developing the model using multivariate analysis in a data set. The predictive performance of the model in the development sample is essential and must include measures of:

• Calibration (reliability): Agreement between observed and predicted outcomes for a specific period of time.^29–31

• Discrimination (accuracy): Ability of the model to distinguish people who will develop the outcome of interest from those who will not.^29–31

• Predictive values: Probability of developing the outcome of interest if the patient’s status on the model is positive (positive predictive value) or not developing the outcome of interest if the patient’s status on the model is negative (negative predictive value).^22,32

• Variance (R²): Amount of variance explained by the model in the outcome variable.²²

Prognostic Model Internal Validation

Internal validation is often completed with data from the development sample; this step serves to estimate the potential of overfitting and optimism in a model’s predictive performance (Tab. 2). This process is commonly made using bootstrapping to adjust for the natural optimism regarding performance measures in the sample that was used for model development.³⁰ (Bootstrapping is a statistical method that randomly samples with replacement a large number of study samples from the original study sample used for model development in which every step of the model development is repeated. This process leads to the development of a different model for each bootstrap sample that can be applied to the original study sample. An average of all the c-index differences observed between the models in a bootstrap sample and in the original study sample indicates the optimism in the initial c-index of the prediction model.)³⁰

Table 2.

Main Characteristics of the Development and Validation Processes of Prognostic Models

Model Development and Validation Steps	Study Design; Specific Analysis	Relevance
1. Model development	Prospective cohort (longitudinal); multivariate analysis of predictors	Provide an accurate estimate of risk of a future outcome for a patient
2. Model internal validation	Prospective cohort (longitudinal); usually performed with data from the development sample	Quantify predictive performance of the model Estimate overfitting and adjust for optimism
3. External validation	Prospective cohort (longitudinal); performed with data from a different sample	Allow adjusting for optimistic predictive performance found in internal validation (ie, overfitting) Assess the generalizability of the model (ie, reproducibility, transportability) Might include updating of the model, which increases its generalizability
4. Development of prognostic tool	If not pragmatic, the prognostic model is translated into simple clinical prediction rules (CPRs) and clinical decision rules (CDRs) intended for clinical use (ie, prognostic tool); typically involves choosing a relevant threshold for clinical practice	Makes it possible to provide a simple tool for clinicians while taking advantage of the precision of statistical models for prediction in the development phase
5. Impact analysis	Randomized trial in a clinical setting; performed with data from a different sample	Assess the impact of the implementation of the model on clinical outcomes or health care costs

Prognostic Model External Validation

Before being featured in a clinical format (ie, prognostic tool), a model needs to be externally validated with “new” data (from different samples) through prospective cohorts.²² This type of validation estimates the predictive performance of the model in other settings or populations (with varying case mix) and determines its appropriateness for individual prediction and clinical use.^22,33,34 This step is essential to ensure that the model is of clinical value.²⁴ External validation allows adjustment of the often optimistic predictive performance found during internal validation (ie, overfitting), which improves clinical applicability.²⁴ This process enables the assessment of the generalizability of the prediction.^35,36 The model might be updated and adjusted to patient characteristics in the new validation sample, which might increase its generalizability.³⁷ A relatively low proportion of prognostic models have undergone this third step of external validation.^24,38,39 Moreover, careful interpretation is warranted, because external validation studies are often secondary analyses of other study data, which might increase the risk of bias.³³

Development of a Prognostic Tool

Although the concept of prognostic models is well defined, the terminology used in prognosis research does not provide a clear distinction about what can be called a prognostic tool. We define a “rehabilitation prognostic tool” as a tool that operationalizes prognostic models in a pragmatic way, by giving a course of action to guide clinical decision-making processes and personalize rehabilitation approaches. To be implemented into clinical practice, externally validated prognostic models often need to be simplified into a suitable format for clinical practice. As an example, a regression formula combining multiple mathematical terms can be simplified into a score chart (see “Types of Prognostic Tools” below) that can be easily used by clinicians. They become prognostic tools that predict the probability of an outcome (ie, clinical prediction rules) or assist decision making by predicting response to treatment (ie, clinical decision rules).²² This process gives the precision of statistical models for prediction in the first stages of model development, while providing a simplified tool for clinical practice at a later stage. If transforming the prognostic model into a tool implicates the loss of information (not all predictors are included in the tool), this will implicitly result in a loss of predictive performance. If such tools are developed/proposed, their predictive performance measure needs to be assessed and described separately.^40,41 Moreover, considering that most prognostic models implicate the “dichotomization” of continuous measures, readers need to be aware that this simplification process is a limitation that can lead to loss of information and increase the risk of misclassification.

Impact Analysis

The impact of the implementation of a prognostic tool on clinical outcomes or health care costs needs to be assessed to demonstrate its relevance for clinical practice. For example, an impact study was done regarding the STarT Back Screening Tool, which assesses the level of risk of poor outcomes in patients with low back pain and stratifies treatments accordingly. It showed that the tool lowers disability and work absenteeism, without increasing health care costs in primary care practice.⁴² A recent systematic review investigated the effectiveness of stratified care using the STarT Back Screening Tool compared with standard physical therapy for the treatment of low back pain. This review demonstrated that strong evidence supports clinical, economical, and health-related cost benefits of stratified care for medium- and high-risk subgroups of patients, versus usual care.²¹ Yet, only a few externally validated prognostic models have gone through such impact analysis.^24,38,39

Consistent with the conceptual framework of Kent et al,²² to be accurate for individual prediction and useful for clinical practice, prognostic tools need to undergo this entire process of development, validation, and clinical impact assessment (Tab. 2).²⁴

Types of Prognostic Tools

To be implemented into clinical practice, prognostic tools need to be quick and easy to use and require information that is readily available by clinicians.

Steyerberg⁴³ proposes 2 broad categories of format to present prognostic models: clinical prediction models and clinical decision rules. A similar description was recently proposed by Kent et al,²² who also describe clinical prediction rules (versus model) and clinical decision rules.

Prognostic Model Presentation Formats

Clinical Prediction Rules, or Clinical Prediction Models

Clinical prediction rules (CPRs) estimate the probability of future outcomes for an individual, based on patient characteristics.^22,43 They combine the effect of several prognostic factors. CPRs can be presented as regression formula (with or without a calculator), nomogram, table, and score chart.⁴³ Because CPRs need to be suitable for clinical practice, they should be consistent with the prediction model clinical format.²² A regression formula with complex coefficients and mathematical operations should be simplified using a web-based or app-based calculator to be considered as a suitable clinical prediction rule (ie, ready for clinical practice). In score charts, each predictor is allocated a score according to its relative weight in outcome determination and the total score is used to predict the future outcome.⁴³ As an example, the STarT Back Tool is a questionnaire from which a total score is used to determine the level of risk of poor outcomes for patients with low back pain.⁴⁴ Score chart is the prediction system underlying many forms of prognostic tools found in rehabilitation, such as questionnaires, algorithms, or clinical prediction rules that use a score to estimate the risk of a future outcome.

Clinical Decision Rules, or Prescriptive Clinical Prediction Rules

Clinical decision rules,^22,43 also labeled as prescriptive clinical prediction rules (pCPRs),^45–47  suggest a course of action based on a combination of patient characteristics. They often combine the effects of treatment-effect modifiers. This type of presentation format guides clinicians in their decision-making and care pathways by predicting treatment response.^22,26  Regression trees, score chart rules, survival groups, and meta-models are the main subtypes of clinical decision rules.⁴³ In physical rehabilitation, several types can be found:

• Regression/classification tree: Individuals are classified according to precise characteristics displayed in a tree schema. Treatment recommendations are then linked to each group of classification to inform clinical decision making. As an example, the Treatment-Based Classification System for Low Back Pain updated in 2016 by Alrwaily et al³ is based on a 3-arm tree schema where a rehabilitation approach is suggested for each group of classification.

• Score chart rule: A model in which each predictor is allocated a score, contributing to the total score. When the total score exceeds a certain threshold, it indicates an action determined by the rule. The treatment system associated with the StarT Back Screening tool is a clinical prediction rule (CPR) but is also an example of a score chart rule. This is because it comes with treatment recommendations, and the score on the tool is associated with specific management strategies in rehabilitation to guide clinicians’ decision making.^42,48 Thus, the StarT Back Tool has both a CPR component (it estimates the risk of poor outcomes) and a pCPR component (it suggests specific management according to the score on the tool).

Methodological Contrast Between CPRs and pCPRs

CPRs and pCPRs have different foundations. CPRs use prognostic factors to estimate the probability of a future endpoint, whereas pCPRs combine multiple treatment-effect modifiers to suggest a course of action according to an expected treatment response. Consequently, the methodology underlying their development process differs.

Methodological features presented in The “Ontogenesis” of Prognostic Tools section, suit CPR development, because prognostic factor research requires a prospective longitudinal cohort design. Methodological considerations for the development and validation of clinical decision rules, or pCPRs, which use treatment-effect modifiers, are provided in Table 3. In the process of developing prognostic models, a prospective cohort design is preferred for development and internal and external validation, whereas a randomized controlled trial design with a control group (double-arm trial) is needed when treatment-effect modifiers are involved.⁴⁹ Using a randomized controlled trial design for the development and validation phases makes it difficult to distinguish between external validation and impact studies. In both cases, the same study design might be used, and the same outcome might be assessed, that is, patient response to treatment compared with the model’s prediction. Yet, specific analyses (test predictors by treatment interaction) are required. Accordingly, readers need to be aware that different study designs are required to assess treatment-effect modifiers from impact on decisions. Moreover, impact studies investigate the influence (on practice, costs, clinical outcomes) of the use of the tools versus providing an estimation of treatment effect.

Table 3.

Methodological Characteristics of Clinical Decision Rule or Prescriptive Clinical Prediction Rule^a

CDR/pCPR Development and Validation Steps	Study Design; Specific Analysis for Prognostic Models	Methodological Specifications for CDR/pCPR
1. Development	Prospective cohort (longitudinal); multivariate analysis of predictors	Randomized controlled trial design: exploratory studies of association and confirmatory studies of validation of models predicting response to treatment might use through randomized controlled trial design. This will confirm that the model is predictive of response to treatment instead of predictive of the natural course of the disease. A control group is required to verify the link between the status of the patient according to the model (combining multiple treatment-effect modifiers) and patient’s response to treatment.26
2. Internal validation	Prospective cohort (longitudinal); usually performed with data from the development sample
3. External validation	Prospective cohort (longitudinal); performed with data of a different sample

^a CDR = clinical decision rule; pCPR = prescriptive clinical prediction rule.

Types of Prognostic Tools in Rehabilitation: Where Do We Stand?

Most reviews of pCPRs in rehabilitation have found that only a few have undergone external validation and that most of them are not valid for a clinical application,^45,46,50,51 leading to methodological concerns about pCPRs.^47,52 Because most reviews of prognostic research in physical rehabilitation have focused on pCPRs, results were not indicative of the benefits for clinical practice. Hence, it is time to focus on the development and validation of prognostic tools through a high-quality methodological process. Systematic reviews should search for quality prognostic model (or tools). Because clinical practice implies time constraints, and prognostic models (CPRs or clinical decision rules) can take on several forms, ease and speed of use are important assets to look for.

How to Select a Prognostic Tool

To be suitable for clinical use, prognostic tools need to be externally validated and must show acceptable performance measures.

Most prognostic model studies were developed and published before guidance for reporting was available.^53,54 Therefore, each study needs to be appraised in detail to determine what could bias the results and their interpretation. A number of methodological problems or lack of information in previously published prognostic model studies have been reported.^22,24,54 In 2015, the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) was published to address this finding.^28,53

To narrow the gap between research evidence and clinical practice in physical rehabilitation, the results of prognosis research studies need to be easily understandable by their users. Clinicians should be able to identify quality models that are ready for clinical use. Besides the need for clinicians to look for prognostic models that have gone through an external validation step, statistical analysis measures and quality of studies must be considered.

Statistical Analysis

Key elements to consider when selecting a prognostic tool for clinical practice are presented in Table 4.

Table 4.

Key Elements of Statistical Analysis to Consider When Selecting a Prognostic Tool^a

Measures	Elements	Interpretation	Example
Performance	Calibration (reliability): What is the agreement between observed and expected event rate?29,31	Calibration slope65: 45° line slope = perfect calibration29 If calibration is poor, the model has limited usefulness for clinicians66 Recalibration can be done in external validation studies if calibration is poor in the external population65	The external validation of the WORRK model showed that its calibration was good, demonstrated by CIs of the observed probabilities, which covered the line of ideal calibration67
	Discrimination (accuracy): What is the ability of the model to discriminate between people who will develop the outcome of interest or not?29,31	C-statistics or c-index: 0.5: model unable to discriminate; 1: perfect discrimination29,36 In external validation studies, discrimination must be good because it is not possible to improve it, unlike calibration by recalibrating the model	The WORRK model external validation showed an area under the receiver operating characteristic (ROC) curve (c-index) of 0.73 (95% CI = 0.70–0.77), which was considered sufficient by the authors67
Classification	Sensitivity Specificity Negative predictive value Positive predictive value	Classification measures are calculated when 1 or more probability thresholds have been set, leading to a dichotomization of scores that can negatively affect model performance. It is important to ensure that the chosen threshold is relevant for clinical practice53	The WORRK model external validation showed a sensitivity of 72.4% (95% CI = 69.3–75.4) and a specificity of 61.2% (95% CI = 57.9–64.6) to predict the risk of not returning to work when the threshold was set to 0.5.67 Thus, this model is relevant only for patients who are on sick leave

^a WORRK = Wallis Occupational Rehabilitation RisK.

Performance Measures

Clinicians must look for prognostic models that have demonstrated acceptable calibration (reliability) and discrimination (accuracy) measures when externally validated to consider using them for their patients (Tab. 4).³¹ Otherwise, the prediction given by the model might be inaccurate for a patient and can mislead the clinician’s decision-making process. This situation can have a negative effect on patient outcomes and should be avoided. Performance measures (ie, calibration and discrimination) do not seem to be comprehensively reported in many prognostic model studies. However, they are essential questions that need to be answered when assessing the performance of a model according to available prognosis research guidelines.^22,33,53

Classification Measures

Classification measures (ie, sensitivity, specificity, and predictive values), when available, are provided when a probability threshold has been set. Clinicians must verify if the chosen threshold is relevant for their clinical setting and patient to determine if they can rely on these measures.

Quality and Risk of Bias

Assessment of quality and risk of bias of validated prognostic model studies requires time and an excellent understanding of prognosis research—2 important barriers to the clinical application of prognostic models.⁵⁵ Systematic reviews might help to overcome these issues. In systematic reviews of prognostic model studies, risk of bias is assessed using a PROBAST tool, which includes items in 4 domains (participants, predictors, outcomes, and analysis) that are rated as high, low, or unclear. The same rating is done for applicability to the systematic review question.³³ When available, systematic reviews might facilitate the translation of evidence into clinical practice. Systematic reviews appraise quality of evidence by assessing a study’s risk of bias, which can be useful for an inexperienced clinician. Therefore, systematic reviews of prognostic models are needed to inform clinicians.

Prognostic Tools in Physical Rehabilitation

Prognostic tools arising from prognostic models are proposed for most health conditions requiring physical rehabilitation, but only a few have reached satisfying validation levels for use in clinical practice. The Supplementary Table summarizes a few examples of prognostic tools that are relevant to physical rehabilitation practice.

Conclusion

In this Perspective article, we suggest that integrating validated prognostic tools to clinical practice is a promising way for rehabilitation clinicians to enhance their clinical decision making, personalize rehabilitation approaches, and ultimately optimize clinical outcomes for patients. To foster this change of practice, rehabilitation clinicians must not only master these tools, but most importantly must also refrain from using prognosis apart from diagnosis.

The field of prognostic tools is a dynamic one, and growing evidence shows that their use in clinical practice will likely have positive effects for rehabilitation clinicians, as well as their patients. Prognosis research in physical rehabilitation needs to improve on the methodological quality of the prognostic tools intended for clinical practice—we need prognostic models that have undergone a comprehensive development and validation process. Finally, future research should also focus on the dissemination and implementation strategies to facilitate the use of prognostic models in clinical practice. These strategies will enable exploration of the effects of using prognostic models on the choice of rehabilitation modalities as well as on patient outcomes and specific education strategies.

Author Contributions

Concept/idea/research design: Y. Tousignant-Laflamme, C. Houle, C. Cook, A. LeBlanc, S. Décary

Writing: Y. Tousignant-Laflamme, C. Houle, C. Cook, F. Naye, A. LeBlanc, S. Décary

Data analysis: Y. Tousignant-Laflamme

Project management: Y. Tousignant-Laflamme, S. Décary

Fund procurement: Y. Tousignant-Laflamme, S. Décary

Providing facilities/equipment: Y. Tousignant-Laflamme, S. Décary

Providing institutional liaisons: Y. Tousignant-Laflamme

Consultation (including review of manuscript before submitting): A. LeBlanc

Funding

This work was made possible by an unrestricted grant from the Ordre Professionnel de la Physiothérapie du Québec. The funder did not participate in the writing of this Perspective.

Disclosures

The authors completed the ICMJE Form for Disclosure of Potential Conflicts of Interest and reported no conflicts of interest.