Abstract
Background
Frailty is a robust predictor of poor outcomes among patients with chronic obstructive pulmonary disease yet is not measured in routine practice. We determined barriers and facilitators to measuring frailty in a hospital setting, designed and implemented a frailty-focused education intervention, and measured accuracy of frailty screening before and after education.
Methods
We conducted a pilot cross-sectional mixed-methods study on an inpatient respiratory ward over 6 months. We recruited registered nurses (RNs) with experience using the Clinical Frailty Scale (CFS). RNs evaluated 10 clinical vignettes and assigned a frailty score using the CFS. A structured frailty-focused education intervention was delivered to small groups. RNs reassigned frailty scores to vignettes 1 week after education. Outcomes included barriers and facilitators to assessing frailty in hospital, and percent agreement of CFS scores between RNs and a gold standard (determined by geriatricians) before and after education.
Results
Among 26 RNs, the median (IQR) duration of experience using the CFS was 1.5 (1–4) months. Barriers to assessing frailty included the lack of clinical directives to measure frailty and large acute workloads. Having collateral history from family members was the strongest perceived facilitator for frailty assessment. The median (IQR) percent agreement with the gold-standard frailty score across all cases was 55.8% (47.2%–60.6%) prior to the educational intervention, and 57.2% (44.1%–70.2%) afterwards. The largest increase in agreement occurred in the ‘mildly frail’ category, 65.4%–81% agreement.
Conclusions
Barriers to assessing frailty in the hospital setting are external to the measurement tool itself. Accuracy of frailty assessment among acute care RNs was low, and frailty-focused rater training may improve accuracy. Subsequent work should focus on health system approaches to empower health providers to assess frailty, and on testing the effectiveness of frailty-focused education in large real-world settings.
Keywords: Health professions education, Hospital medicine, Chronic disease management
What is already known on this topic
Frailty is a robust predictor of poor clinical outcomes among patients with chronic obstructive pulmonary disease yet is not measured in routine clinical practice.
What this study adds
This study demonstrates that barriers to assessing frailty, such as large acute workloads and lack of widespread clinical directives in hospital, are external to the measurement tool itself, and the accuracy of frailty assessments is low. A structured frailty-focused educational intervention has the potential to improve the accuracy of frailty assessments among hospitalised patients but requires testing in larger real-world samples.
How this study might affect research, practice or policy
Introducing routine patient frailty assessments to improve patient-centred care models for hospitalised patients first requires system-wide approaches to empower frontline staff with resources including clinical directives and targeted frailty-focused education to ensure accurate measurements.
Introduction
Chronic obstructive pulmonary disease (COPD) is a progressive, incurable, chronic lung disease caused by tobacco smoke exposure.1 2 Approximately 25%–50% of individuals with COPD live with some degree of frailty, defined as a multidimensional state of increased vulnerability to health stressors that is caused by accumulation of health deficits across multiple domains.3–6 Frailty among individuals with COPD is associated with frequent hospitalisation, longer hospital stays, increased costs, increased mortality and poor quality of life.4 5 7–10
Multiple validated instruments can reliably identify an individual’s degree of frailty.3 The Clinical Frailty Scale (CFS) is a practical, validated bedside tool for clinicians, which relies on the clinical history from patients, and is scored from 1 (very fit) to 9 (terminally ill).6 Compared with the 80-item Frailty Index, the CFS explains 80% of the variation in frailty index scores while taking less than a minute to administer.6 11 Increasing degrees of frailty on the CFS correlate with increasing risk of death and future need for long-term care among adults over 65 years of age.6 Progressive degrees of frailty are defined by increasing limitations in performing activities of daily living (ADLs) and Instrumental ADLs (online supplemental appendix B).6 Given the prevalence and prognostic value of frailty among patients with chronic lung disease, it is important for healthcare providers to accurately recognise and assess the degree of frailty, to facilitate appropriate treatments, medical decisions and care plans to meet individual patient needs.12 13
bmjoq-2022-001935supp001.pdf (459.3KB, pdf)
Previous studies have evaluated the accuracy of frailty assessments among junior medical residents as well as multidisciplinary clinical staff who are experienced in the practice of geriatric medicine.14 However, there is no evidence regarding the reliability of frailty assessments when performed by respiratory clinicians with little prior experience performing frailty assessment, especially using instruments that rely on subjective clinical data. Experts advocate that frailty assessments be incorporated into clinical pathways and programmes to provide risk stratification, yet there is currently no standardised educational process among respiratory healthcare staff to facilitate this change in practice.12 15
The aims of this study were to identify perceived barriers and facilitators to assessing frailty for respiratory patients in the acute care setting, and to design and implement a frailty-focused education intervention centred on common rater biases to educate acute care respiratory registered nurses (RNs). We sought to determine the degree of agreement between RNs and a gold-standard frailty measurement before and after education. If implementation of this educational intervention is successful, we hypothesise that frailty-specific rater training will improve the accuracy of frailty assessments between RNs and a gold-standard frailty assessment performed by geriatricians in the future.
Some of the preliminary results of this study have been previously reported in the form of an abstract.16
Methods
Study design
We conducted a pilot cross-sectional mixed-methods study on an inpatient respiratory hospital ward over 6 months in 2017. Figure 1 describes the study process.
Figure 1.
Participant recruitment and study procedures. *Intervention delivered to small groups (six to eight participants) in 1-hour sessions. RN, registered nurse.
The study consisted of: (1) a focus group among experienced RNs to identify barriers and facilitators that influence the current practices of RNs when rating frailty in hospital; (2) development and implementation of a new education intervention designed to address rater biases; and (3) measurements of agreement between RN and gold-standard frailty ratings before and after the educational intervention.
Setting
The study was conducted at The Ottawa Hospital (TOH) on the acute inpatient respiratory ward. TOH is a quaternary care academic hospital with approximately 1200 inpatient beds, serving a local population of >1 million individuals.17
Participants
We recruited RNs working on the acute inpatient respiratory ward at TOH who had experience using a new hospital-based COPD care model that incorporated the CFS instrument to assess frailty within a clinical pathway. RN participants were selected by purposive sampling based on guidance from the nursing clinical leads.
Focus group: barriers and facilitators to rating clinical frailty
We first conducted a focus group with six of the most experienced (>5 years in practice) RNs on the inpatient respiratory ward to identify nursing perceptions of the CFS instrument, perceived barriers and facilitators to using the CFS in a busy clinical setting, and to determine whether a rater training course would be perceived as useful (online supplemental appendix A). Verbal consent was obtained prior to participation in the focus group. A recording of the focus group discussion was independently reviewed by two study authors (AL, SM), to identify the emerging themes. The final themes were determined by consensus.
Educational intervention design
We developed a novel, structured, rater training, frailty-focused education intervention to review and overcome known rater biases for frontline healthcare workers.18–20 The course content was adapted from previous rater training concepts and included four key components: (1) introduction to the frailty concept, (2) frame of reference training, (3) rater error training and (4) performance dimension training (table 1).18–24 Frame of reference training was presented using a clinical example of an older patient with comorbidities who was hospitalised with acute exacerbation of COPD. Following independent frailty ratings, participants openly discussed specific patient factors that determined their rating. Rater error training was presented with an explanation of four common rating biases (leniency effect, halo effect, central tendency effect and primacy effect). Four clinical vignettes were presented along with a corresponding frailty rating and reasoning given by a fictitious rater to highlight each rating bias. In a facilitated group discussion, participants were asked to identify the rater bias present, and tips to avoid common rating biases were discussed. Performance dimension training involved a review of the CFS instrument degrees of frailty (online supplemental appendix B). Participants were separated into small groups (two to three members) and asked to rate the degree of frailty in two clinical vignettes of older patients admitted for acute exacerbation of COPD. After discussion, one spokesperson from each group presented a summary of the case as well as their suggested frailty rating. Questions and disagreement were encouraged and facilitated, and each rating was challenged by the facilitator to probe the participants’ underlying reasoning for the rating assigned. The goal was to reach a group consensus for each of the four vignettes discussed.
Table 1.
Components of the rater training education intervention to assist with recognition and assessment of frailty
| Educational section | Delivery method | Components |
| Concept of frailty | Lecture | The concept of frailty Use of the Clinical Frailty Scale Prevalence of frailty in COPD Impact of frailty on COPD outcomes (mortality, exacerbation, hospitalisation, etc) |
| Frame of reference training | Facilitated group discussion | Independent frailty ratings by participants of a clinical example Participants supported their ratings by citing specific patient factors |
| Rater error training | Facilitated group discussion | Examples of common rating errors (leniency, halo, central tendency, primacy) were provided Four clinical vignettes with rating errors were then presented Participants were encouraged to identify and discuss the rating errors present in the four clinical vignettes |
| Performance dimension training | Small and facilitated group discussion | Groups of 2–3 discussed and provided a rating for 1–2 clinical vignettes Each group was provided different vignettes A summary and rating were presented by a spokesperson of each group Group discussion to probe for supporting evidence and develop consensus was encouraged by the facilitator |
COPD, chronic obstructive pulmonary disease.
Education intervention delivery
The education intervention was delivered to groups of six to eight RN participants over four separate 1-hour small group sessions, facilitated by the study team members with training and experience in medical education and rater training (AL, SM).
Assessing clinical frailty ratings
Based on the authors’ clinical experience with patients admitted to the respiratory medicine ward, the CFS category ‘very fit’ (score=1) was not included in the study, as hospitalised patients with COPD at our centre generally did not meet criteria for ‘very fit’.25 For the purposes of this study, frailty categories studied ranged from ‘well’ (rating of 1) and final category was ‘terminally ill’ (rating of 8) (online supplemental appendix B).
Pre-intervention
One week prior to the rater training course, RN participants completed a structured questionnaire of baseline demographic questions, number of months of experience on the acute respiratory ward and the number of times the participant had used the CFS in clinical practice. Questions regarding barriers and facilitators to using the CFS were based on the responses provided by the initial focus group of experienced RNs (n=6), in addition to factors the study team deemed relevant. Participants were asked to rank eight potential barriers to using the CFS: from 1 (most important barrier) to 8 (least important barrier) (online supplemental appendix C). Participants were asked to select all items from a standardised list that they felt would facilitate the use of the CFS.
The second portion of the structured questionnaire presented 10 fictional clinical vignettes describing hospitalised patients with acute exacerbation of COPD (online supplemental appendix D). The clinical vignettes were developed, edited and reviewed by study authors (physicians and nurses) with experience in assessing and treating hospitalised patients with COPD (AL, ND, CB, SM). The information presented in each vignette simulated the degree of information received upon a standard admission history and chart review by an RN. All cases were reviewed by two of the study authors (AL, SM) to standardise the presentation of the cases. There was one case for each CFS category, with two cases each for the ‘vulnerable’ (rating of 3) and ‘moderately frail’ (rating of 5) categories. RN participants were asked to provide a frailty rating using the CFS for the patient described in each clinical vignette. Ratings were based on the description of the baseline functional status of the patient (at 2 weeks prior to hospitalisation). Participants were required to complete the questionnaire and frailty rating exercise to participate in the rater training educational course.
Post-intervention
One week following the rater training course, all participants were asked to complete a second questionnaire. RNs were asked to assign frailty ratings to the same 10 clinical vignettes, presented in random order to reduce the potential for recall bias.
Gold-standard frailty assessments
The clinical vignettes used in the questionnaires were distributed to a geriatrician (SH) and advance practice geriatrics nurse (TP) who each assigned a frailty assessment to each vignette. This geriatrics team had extensive experience in using the CFS instrument in routine clinical settings. Any disagreements in the ratings between the geriatrics team and authors of the vignettes were resolved by consensus.
Outcomes
The primary outcomes were the identification of barriers and facilitators to assessing frailty in the acute care setting, and the degree of agreement between RNs and the gold-standard frailty rating prior to the education intervention. Secondary outcomes included the per cent agreement between RNs and the gold-standard frailty after the educational intervention.
Statistical analysis
Demographic data of the RN participants were described using means (±SD) and proportions. Barriers to using the CFS were ordered from most important (lowest score) to least important (highest score), by calculating the sum of each participant’s ranking, where the lowest sum indicated the most important perceived barrier. Facilitators were described by the proportion of RNs who indicated their importance.
The agreement between RNs and the gold-standard frailty rating was assessed for each case using the percentage of agreement (number of RNs who agreed with the gold standard/total number of RNs), before and after the education intervention. McNemar’s test was used to compare the degrees of agreement before and after the education intervention.
Results
Initial focus group with experienced RNs to explore frailty assessment in acute care
Two main themes emerged from the focus group of six experienced acute care RNs; (1) the RNs found that the CFS instrument was straightforward to use, and (2) the barriers to using the frailty scale are external to the frailty scale itself, with time to assess and perform the rating being the most significant perceived barrier. Other barriers identified were included in the standardised questionnaire for RN study participants (online supplemental appendix C). The CFS was recognised as being important for patient care; however, RNs found their opportunity to perform frailty assessments was infrequent.
RN participant demographics
Among 26 participating RNs, a range of nursing experience was observed with a median (IQR) of 15 (15.5–105) months. The median (IQR) duration of prior use of the CFS instrument was minimal at 1.5 (1–4) months. A total of five RNs were lost to follow-up at the end of the study.
Barriers and facilitators to using the CFS in a clinical setting
Table 2 describes the perceived barriers and facilitators to using the CFS instrument at the time of hospitalisation. The most important barriers to using the CFS on an acute care respiratory ward were having a lack of clinical directive, lack of collateral patient history to assign the frailty score and high inpatient workload at the time that frailty rating was required. Understanding the CFS and timely completion of the CFS were not highly ranked barriers to use. The most frequently identified facilitators to using the CFS included the presence of family members at the bedside to provide additional history at the time of patient admission (n=21, 78%), increasing the flexibility in the time window to complete the CFS (ie, to complete within first 24 hours of admission) (n=19, 70%) and more education on how to use the CFS (n=14, 52%).
Table 2.
Barriers and facilitators to using the CFS in hospital (most to least important) among 26 acute care respiratory RNs
| Barriers* | Median rank† (IQR) |
| No clinical directive to use CFS | 1 (1–2.5) |
| High workload at time when CFS assessment required | 3 (3–4.5) |
| Lack of collateral history from caregivers/family members | 3 (3–4) |
| Patient too ill to provide information | 4 (2.5–5.5) |
| Communication barrier with patient | 4 (2.5–6.5) |
| Takes too long to use the CFS | 6 (5–7) |
| CFS is difficult to understand | 6 (5–7) |
| CFS is not practical | 6 (5–8) |
| Facilitators‡ | Number (%) |
| Family present at bedside | 21 (77.8) |
| More flexible time window to complete rating | 19 (70.0) |
| More education on how to use the scale | 14 (51.9) |
| Providing the rating at time of admission to hospital | 12 (44.4) |
| Discuss the rating with clinical colleagues | 9 (33.3) |
| Other§ | 5 (18.5) |
*Data missing from four participants. Barriers ranked from 1 to 8; 1 indicating most important barrier, 8 indicating least important barrier.
†Median score represents the median rank, with lower scores being the most important barriers.
‡Facilitators were selected from a standardised list and presented as the number (%) of RNs who selected the item.
§Other (open-ended) items participants felt would facilitate the use of the CFS.
CFS, Clinical Frailty Scale; RNs, registered nurses.
RNs’ agreement with gold-standard frailty assessment
The median (IQR) overall per cent agreement across all cases was 55.8% (47.2%–60.6%) prior to the education intervention, and 57.2% (44.1%–70.2%) afterwards.
Following the education intervention, there was an increased degree of agreement with gold-standard frailty ratings for all frailty categories (table 3), although none were statistically significant by the McNemar’s test p value cut-off of <0.05.
Table 3.
Per cent agreement between RN and gold-standard frailty ratings before and after the frailty education intervention
| Gold-standard CFS rating | Prior to education (n=26) % agreement |
Post-education (n=21)* % agreement |
P value† |
| Well and managing well (case 1, case 4) |
63.5 | 71.4 | 0.55 |
| Vulnerable (case 6, case 10) |
51.9 | 52.4 | 0.61 |
| Mildly frail (case 2) |
65.4 | 81.0 | 0.34 |
| Moderately frail, severely frail, very severely frail and terminally ill (cases 3, 7, 9, 8, 5) |
45.4 | 50.5 | 0.88 |
*After the educational intervention, five RNs were lost to follow-up and did not rate clinical vignettes.
†McNemar’s test used to compare % agreement before and after educational intervention with n=21 subjects. Categories of ‘vulnerable’ and ‘moderately frail, severely frail, very severely frail and terminally ill’ compared 20 subjects due to missing data points from one subject.
CFS, Clinical Frailty Scale; RNs, registered nurses.
The largest increase in agreement occurred within the ‘mildly frail’ category, 65.4%–81% agreement after the educational intervention.
Discussion
In this pilot study, the most important barriers to assessing patient frailty in the acute care respiratory setting were external to the CFS instrument itself and included a perceived lack of time due to large workloads, lack of collateral clinical history from caregivers and a lack of clinical directives to routinely perform frailty assessments on hospitalised patients. Further, half (51%) of our sample of RNs felt that frailty-focused education would facilitate the application of the CFS instrument in the hospital setting. At baseline, there was low agreement (56%) between RNs and a gold-standard frailty rating and following a structured rater training education intervention, there was a non-significant increase in agreement in all categories of frailty.
Advocates for person-centred care emphasise the benefits of using frailty assessments as a risk stratification tool to treat, prognosticate and support patients with chronic disease, calling frailty ‘a new vital sign’.12 13 15 Our study demonstrates that real barriers do exist to measuring frailty in the acute care setting and these must be addressed at a health system level before frailty assessments can be integrated into standard care processes for patients with chronic lung disease. Our findings among respiratory RNs are echoed in qualitative interviews with emergency, anaesthesia and surgery providers in Singapore, which identified the need for frailty-focused educational programmes for both patients and providers, an integrated approach to frailty screening, and hospital-wide adoption of a common frailty screening tool.26 Data from European health policy providers identified that widespread implementation of frailty screening and management requires a ‘culture shift’ and redeployment of health resources to facilitate an integrated and multidisciplinary care approach.27 Empowering other members of the healthcare team (allied health professionals) and patients and caregivers to participate in the frailty assessment in early stages of the hospitalisation may address important barriers to performing frailty assessments in acute care settings. Further, targeted education on potential interventions that can be applied for progressive degrees of frailty may increase the drive to perform frailty assessments in acute care.
While systemic change in health systems is needed to incorporate patient frailty assessment, prevention and supportive intervention, accurate identification of frailty is also essential. In our small pilot study, only half of participating RNs agreed with the gold-standard frailty assessment, and modest non-significant improvement occurred after further education. This is in contrast to data from the critical care literature which suggest that agreements in frailty assessment using the CFS instrument were ‘good’ with a kappa score of 0.64 (0.4–0.87, p<0.0001) between medical students and critical care attendings.28 A larger critical care study also found adequate reliability of frailty measurements between research coordinators, occupational therapists and geriatric medicine trainees.29 Robust data supporting the reliability of frailty assessments among lung health practitioners are lacking, and our pilot study identified a clear desire among RNs for frailty-focused training. The increase in cognitive load of a busy medical inpatient unit may increase the risk of rating error and bias.30 31 In the context of frailty assessment in hospitalised patients, there is a risk of both halo and primacy effects, where the rater may be biased by the patient’s clinical or functional state in the moment of their acute illness. Targeting such common and context-specific rating errors may decrease rater bias and improve accuracy of assessments. Given the small sample sizes and observational nature of this study, the observed increases in agreement of frailty scores between participants and the gold standard cannot be solely attributed to the educational intervention and must be further studied with larger samples and with real-world patients.
To our knowledge, this pilot study is the first to describe barriers to frailty assessment in the acute care respiratory setting and to design and implement a structured rater training educational intervention targeted at improving frailty assessments among frontline nursing staff in an acute care hospital. Strengths of this study include the components of the educational intervention, which were developed and rooted in standardised rater training literature and the qualitative feedback elicited from frontline healthcare staff in an acute care clinical setting. Limitations of our study include the small sample size and the use of 10 standardised clinical vignettes as opposed to an authentic clinical setting with real-world patients. Other limitations include the potential for recall bias among RNs, given the same clinical vignettes were used before and after the intervention (in scrambled order). However, we attempted to minimise the effect of recall bias by allowing a 1-week interval delay before RNs rescored the vignettes in sporadic order.
Conclusions and lessons learnt
This study demonstrates that barriers to assessing patient frailty, such as large acute workloads and lack of widespread clinical directives in hospital, are external to the measurement tool itself. Accuracy of frailty assessment among acute care RNs was low. Frailty-focused rater training was successfully implemented but did not statistically improve accuracy in this small pilot study. Our results support (1) the need for high-level health system planning and hospital-wide approaches to address barriers to incorporating patient frailty assessments in a meaningful way, and (2) expanding the rater training educational intervention to an iterative training process with larger sample sizes of real-world patients and scalable training formats to test its effectiveness in the acute care respiratory setting.
Acknowledgments
We gratefully acknowledge Jan Leahy, Sonia Joanise and Ley-Ann Mondor for their nursing leadership, assistance with study coordination and collaboration. We are highly appreciative of the nursing staff on the respiratory ward at The Ottawa Hospital who kindly agreed to participate in this study.
Footnotes
Contributors: AL, ND, CB, SH, TP and SM conceived the initial study design and collected data. AL, SM and ND conducted the analysis. All authors (AL, ND, CB, SH, TP, MC, DK, DM, JL, AF, SM) contributed to the analysis or interpretation of the data/analysis and critical review and revision of the manuscript for intellectual content. SM accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: The authors declare no financial or personal conflicts of interest related to this work.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Patient consent for publication
Not required.
Ethics approval
This study involves human participants and was approved by The Ottawa Health Sciences Research Ethics Board (OHSN-REB) ID: 20170676-01. Participants gave informed consent to participate in the study before taking part.
References
- 1.Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease, 2019
- 2.O'Donnell DE, Aaron S, Bourbeau J, et al. Canadian thoracic Society recommendations for management of chronic obstructive pulmonary disease - 2007 update. Can Respir J 2007;14 Suppl B:5B–32. 10.1155/2007/830570 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Clegg A, Young J, Iliffe S, et al. Frailty in elderly people. Lancet 2013;381:752–62. 10.1016/S0140-6736(12)62167-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Lahousse L, Ziere G, Verlinden VJA, et al. Risk of frailty in elderly with COPD: a population-based study. J Gerontol A Biol Sci Med Sci 2016;71:689–95. 10.1093/gerona/glv154 [DOI] [PubMed] [Google Scholar]
- 5.Marengoni A, Vetrano DL, Manes-Gravina E, et al. The relationship between COPD and frailty: a systematic review and meta-analysis of observational studies. Chest 2018;154:21–40. 10.1016/j.chest.2018.02.014 [DOI] [PubMed] [Google Scholar]
- 6.Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ 2005;173:489–95. 10.1503/cmaj.050051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bernabeu-Mora R, García-Guillamón G, Valera-Novella E, et al. Frailty is a predictive factor of readmission within 90 days of hospitalization for acute exacerbations of chronic obstructive pulmonary disease: a longitudinal study. Ther Adv Respir Dis 2017;11:383–92. 10.1177/1753465817726314 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kennedy CC, Novotny PJ, LeBrasseur NK. Frailty and clinical outcomes in chronic obstructive pulmonary disease. Ann Am Thorac Soc 2018. 10.1513/AnnalsATS.201803-175OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Maddocks M, Kon SSC, Canavan JL, et al. Physical frailty and pulmonary rehabilitation in COPD: a prospective cohort study. Thorax 2016;71:988–95. 10.1136/thoraxjnl-2016-208460 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Medina-Mirapeix F, Bernabeu-Mora R, García-Guillamón G, et al. Patterns, trajectories, and predictors of functional decline after hospitalization for acute exacerbations in men with moderate to severe chronic obstructive pulmonary disease: a longitudinal study. PLoS One 2016;11:e0157377. 10.1371/journal.pone.0157377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.McIsaac DI, Taljaard M, Bryson GL, et al. Frailty as a predictor of death or new disability after surgery: a prospective cohort study. Ann Surg 2020;271:283–9. 10.1097/SLA.0000000000002967 [DOI] [PubMed] [Google Scholar]
- 12.Sinha S, Mckee A, Dunning J. We Can’t Address What We Don’t Measure Consistently: Building Consensus on Frailty in Canada. Toronto, Ontario, 2018. https://www.ryerson.ca/nia/white-papers/frailty-paper.pdf [Google Scholar]
- 13.Cicutto LC. Frailty: is this a new vital sign? Chest 2018;154:1–2. 10.1016/j.chest.2018.03.041 [DOI] [PubMed] [Google Scholar]
- 14.Gregorevic KJ, Hubbard RE, Lim WK, Katz B, et al. The clinical frailty scale predicts functional decline and mortality when used by junior medical staff: a prospective cohort study. BMC Geriatr 2016;16:117. 10.1186/s12877-016-0292-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Muscedere J, Andrew MK, Bagshaw SM, et al. Screening for frailty in Canada's health care system: a time for action. Can J Aging 2016;35:281–97. 10.1017/S0714980816000301 [DOI] [PubMed] [Google Scholar]
- 16.Diab N, Leblanc A, Backman C. Assessing an educational intervention to improve recognition of frailty in hospitalized patients with COPD. European Respiratory Society 2019:PA1476. 10.1183/13993003.congress-2019.PA1476 [DOI] [Google Scholar]
- 17.The Ottawa Hospital annual report, 2021. Ottawa, Ontario, Canada
- 18.Chafouleas SM, Kilgus SP, Riley-Tillman TC, et al. Preliminary evaluation of various training components on accuracy of direct behavior ratings. J Sch Psychol 2012;50:317–34. 10.1016/j.jsp.2011.11.007 [DOI] [PubMed] [Google Scholar]
- 19.Cook DA, Dupras DM, Beckman TJ, et al. Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial. J Gen Intern Med 2009;24:74–9. 10.1007/s11606-008-0842-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence: a randomized trial. Ann Intern Med 2004;140:874–81. 10.7326/0003-4819-140-11-200406010-00008 [DOI] [PubMed] [Google Scholar]
- 21.Holmboe ES. Direct Observation by Faculty. In: Holmboe ES, Hawkins RE, eds. Practical guide to the evaluation of clinical competence. Philadelphia: Mosby Elsevier, 2008: 119–29. [Google Scholar]
- 22.Hansen T, Elholm Madsen E, Sørensen A. The effect of rater training on scoring performance and scale-specific expertise amongst occupational therapists participating in a multicentre study: a single-group pre-post-test study. Disabil Rehabil 2016;38:1216–26. 10.3109/09638288.2015.1076069 [DOI] [PubMed] [Google Scholar]
- 23.Raj JM, Thorn PM. A faculty development program to reduce rater error on milestone-based assessments. J Grad Med Educ 2014;6:680–5. 10.4300/JGME-D-14-00161.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Swanwick T, Forrest K, O’Brien BC. Understanding Medical Education: Evidence, Theory, and Practice. 3rd ed. Hoboken, MJ: Wiley-Blackwell, 2018. https://onlinelibrary.wiley.com/doi/pdf/ [Google Scholar]
- 25.Chin M, Voduc N, Huang S, et al. Practical lessons in implementing frailty assessments for hospitalised patients with COPD. BMJ Open Qual 2020;9:e000782. 10.1136/bmjoq-2019-000782 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Liu X, Le MK, Lim AYC, et al. Perspectives on frailty screening, management and its implementation among acute care providers in Singapore: a qualitative study. BMC Geriatr 2022;22:58. 10.1186/s12877-021-02686-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Gwyther H, Shaw R, Jaime Dauden E-A, et al. Understanding frailty: a qualitative study of European healthcare policy-makers' approaches to frailty screening and management. BMJ Open 2018;8:e018653. 10.1136/bmjopen-2017-018653 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pugh RJ, Thorpe CM, Subbe CP. A critical age: can we reliably measure frailty in critical care? Crit Care 2017;21:121. 10.1186/s13054-017-1704-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Shears M, Takaoka A, Rochwerg B, et al. Assessing frailty in the intensive care unit: a reliability and validity study. J Crit Care 2018;45:197–203. 10.1016/j.jcrc.2018.02.004 [DOI] [PubMed] [Google Scholar]
- 30.Royce CS, Hayes MM, Schwartzstein RM. Teaching critical thinking: a case for instruction in cognitive biases to reduce diagnostic errors and improve patient safety. Acad Med 2019;94:187–94. 10.1097/ACM.0000000000002518 [DOI] [PubMed] [Google Scholar]
- 31.O'Sullivan ED, Schofield SJ. Cognitive bias in clinical medicine. J R Coll Physicians Edinb 2018;48:225–32. 10.4997/jrcpe.2018.306 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjoq-2022-001935supp001.pdf (459.3KB, pdf)
Data Availability Statement
All data relevant to the study are included in the article or uploaded as supplemental information.