Validation of the Pictorial Fit-Frail Scale in a Thoracic Surgery clinic

Abstract

Objective:

Examine feasibility and construct validity of Pictorial fit frail scale (PFFS) for the first time in older surgical patients.

Background:

The PFFS uses visual images to measure health state in 14 domains and has been previously validated in outpatient geriatric clinics.

Methods:

Patients ≥65 year-old who were evaluated in a multidisciplinary thoracic surgery clinic from November 2020 to May 2021 were prospectively included. Patients completed an in-person PFFS and Vulnerable Elders Survey (VES-13) during their visit, and a frailty index was calculated from the PFFS (PFFS_trans). A geriatrician performed a comprehensive geriatric assessment (CGA) either in-person or virtually, from which a Frailty Index (FI-CGA) and FRAIL scale were obtained. To assess the validity of the PFFS in this population, the Spearman’s rank correlations (r_spearman) between PFFS_trans and VES-13, FI-CGA, FRAIL were calculated.

Results:

All 49 patients invited to participate agreed, of which 46/49 (94%) completed the PFFS so a score could be calculated. The majority of patients (59%) underwent an in-person CGA and the reminder (41%) a virtual CGA. The cohort was mainly female (59.0%), with a median age of 77 (range: 67–90). The median PFFS_trans was 0.27 (IQR 0.12–0.34), PFFS was 11 (IQR 5–14) and 0.24 (IQR 0.13–0.32) for FI-CGA. We observed a strong correlation between the PFFS_trans and FI-CGA (r_spearman=0.81, p<0.001) and a moderate correlation between PFFS_trans and VES-13 and FRAIL score (r_spearman 0.68 and 0.64 respectively, p<0.001).

Conclusions:

PFFS had good feasibility and construct validity among older surgical patients when compared to previously validated frailty measurements.

Mini-Abstract

In this study, we examine the construct validity of the Pictorial fit frail scale in older surgical patients. We show that it correlates well with other validated frailty measures, including a frailty index that is based on a geriatric assessment. Lastly, we demonstrate good feasibility in a busy surgery clinic.

Introduction:

With the growing number of older patients undergoing surgery, as well as the new standards set by the American College of Surgeons’ Geriatric Surgery Verification standards program^1,2, clinicians at many institutions are searching for validated tools to measure frailty in older adults. In order to implement these measurements and develop downstream treatment pathways, it is important to maintain feasibility across diverse surgical settings.² This process, among others, has made our team in the Geriatric-Thoracic clinic search for a self-reported frailty tool that is both detailed and comprehensive yet can be integrated into a busy clinic workflow. Since a full geriatric assessment might not be available to all older adults presenting to clinic, we sought a tool that required minimal resources and time to identify patients who would benefit from a comprehensive geriatric assessment (CGA) and interventions.

There are more than 35 instruments currently available to assess frailty.³ There are many ways to categorize these tools, one being by method of administration: self-reported,^4,5 or frailty scores derived from electronic medical records.^6–10 We considered as the “gold standard” a frailty tool that is based on clinical evaluation of the patient while performing a comprehensive geriatric assessment (CGA) and constructing a frailty index (FI-CGA).^11,12 The Pictorial Fit Frail Scale (PFFS) is an assessment of frailty that can be self-reported, and uses visual images to measure health state in 14 domains.¹³ The feasibility and reliability of the PFFS was previously studied in a general geriatric clinic.¹⁴ It was found to be quick to administer, taking less than 2 minutes when assessed by a clinician and less than 5 minutes for self and proxy completion. It was also shown to have good inter-rater reliability for patients and clinicians. In addition, it was tested in a memory clinic setting to further assess its feasibility, reliability, and validity.¹⁵ Here it was again found to be quick, taking less than 5 minutes for patients to complete, while inter-rater reliability remained high for clinicians but lower amongst patients with cognitive impairment. Furthermore, a comparison of the PFFS to FI-CGA found a moderately- high correlation between these two measures, suggesting that the PFFS is comprehensive and captures the multidimensionality of frailty.¹⁶ Recently, the PFFS was also studied in an outpatient geriatric clinic in which the majority of patients had lower health literacy and high rates of cognitive impairment, and was found to be both valid and feasible to administer.¹⁷ It was also studied in cultures and languages other than English, where it demonstrated high agreement among patients, clinicians and caregivers as well as high satisfaction from using this tool.¹⁸

In this study, we examined the feasibility of using the PFFS in a population of older surgical patients presenting to a Thoracic Surgery clinic in a high-volume, academic center. We also examined the construct validity of the PFFS by assessing its correlation to the self-reported Vulnerable Elders-13 Survey (VES-13), the FI-CGA, and FRAIL scale.

Methods:

Design and setting

The study was approved by the Institutional Review Board (IRB# 2018P000385) and requirement for written informed consent was waived by the IRB. It is a cross-sectional study in which patients were enrolled during a visit in a Thoracic Surgery clinic at the Brigham and Women’s Hospital in Boston, Massachusetts from November 2020 to May 2021. Exclusion criteria included patients who are non-English speaking and/or have severe visual impairment/blind without a willing caregiver to administer the questionnaires.

This outpatient clinic provides initial consults and follow-up visits to patients with lung and esophageal conditions necessitating surgical intervention. The patients know when coming to the clinic they have a mass or nodule that could be cancer. All new patients aged 70 and older with lung nodules or lung cancer, patients aged 65 and older with mesothelioma or gastroesophageal cancer, and any other patient referred by the surgeons for geriatric concerns receive a full geriatric assessment by a board-certified geriatrician in clinic. Due to the aggressive nature and expected treatment of mesothelioma and gastroesophageal cancer, with expected vulnerabilities in these cancers such as low nutritional status and reduced function, we found in our experience a greater need to refer to a geriatrician at a younger age than lung cancer or lung nodules. Patients present alone or with caregivers to the clinic. During the study period, due to the COVID-19 pandemic, our geriatric visits became virtual, so the CGA, from which the FI-CGA and FRAIL scale were derived, was done virtually while we continued to provide the PFFS and VES-13 during the in-person visit at the surgeon’s clinic. This allowed us to examine the correlation of PFFS and VES-13 with a virtual FI-CGA.

Participants and procedure

Consecutive patients presenting to the clinic and scheduled to be evaluated by the geriatrician were asked to complete 2 questionnaires (the PFFS and the VES-13) while in the waiting room or the exam room without any assistance from the clinical team. Since we wanted to test feasibility in routine practice, patients were allowed to complete the form with the assistance of their caregiver or individual accompanying them to the visit. To avoid any biases while performing the CGA, the geriatrician was not provided the results from the PFFS and VES-13.

Measures:

1. VES13-^19,20 A self-reported questionnaire, used in different populations including cancer patients²¹, to identify increased risk of death or functional decline. It includes 13 domains in physical and daily function as well as perception of health status. A score greater than or equal to 3 is regarded as vulnerable.

2. PFFS-^13–15,17 This scale uses visual images in 14 diverse health domains in order to assess and grade frailty levels and can be completed by the patient or caregiver. The tool was previously validated in memory, geriatric, and primary care clinics. It was designed to be simple, easy and feasible to administer while sensitive to cultural, educational, and language differences. In order to compare PFFS to FI-CGA we calculated a frailty index from the PFFS (PFFS_trans). The standard procedure to do this involves first summarizing the scores of all filled domains (scored 0–2/4/6 according to the level of each domain) for a total score between 0–43. Then, this summative score is divided by the total number of levels with a response to generate a frailty index (PFFS_trans) that ranges from 0–1. Patients who were missing responses in more than 3 domains (>20% of total domains) were omitted from the analysis since a stable PFFS_trans could not be calculated. The full PFFS is available at: https://www.dal.ca/sites/gmr/our-tools/pictorial-fit-frail-scale.html. (Figure 1).

3. FI-CGA-^11,22–24 By using the accumulation of health deficits to define frailty and aging, this tool was constructed and validated in large scale population studies as a tool to assess and grade frailty in older adults. It was shown to be predictive of outcomes in diverse clinical settings^25–27, including cancer patients²⁸. To construct the FI-CGA from a CGA, there are rigorous rules that need to be followed²⁹, all of which were applied in our FI-CGA tool that is embedded in the electronic health record (EHR) and easily accessible to all geriatricians. FI-CGA scores range from 0–1 with a score >0.2 categorized as frail, and higher values indicate greater frailty^23,30

4. FRAIL scale-^31,32 A quick frailty screen consisting of 5 items on fatigue, resistance, ambulation, weight loss, and illness with scores ranging from 0 to 5: 0- robust; 1 or 2 pre-frail; greater than 2 frail. In our institution FRAIL is part of the FI-CGA so it is similarly derived from the CGA conducted by the geriatrician. It was shown to predict mortality in community dwelling adults and post-surgical outcomes, especially in orthopedic and trauma patients.^33,34

Figure 1.

A sample (first page only) of the visual scale from the PFFS (adapted with permission, Theou et al.)

Statistical analysis:

Descriptive statistics (frequencies for categorical variables, and means, standard deviations, ranges and interquartile ranges for continuous variables) were used to summarize patient demographics and baseline characteristics such as age, gender, race, and level of education. For feasibility we measured how many of the patients were able to complete both scales while waiting in the clinic. For construct validity, Spearman’s rank correlation coefficients³⁵ were calculated to determine the strength of the relationship between the PFFS_trans score and VES-13, FI-CGA and FRAIL scale. All analyses were performed using R 4.0.3³⁶ and significance was determined by p<0.05.

Results:

A total of 49 patients were eligible to participate. Of those, 29/49 patients (59%) had an in-person CGA in clinic and 20/49 patients (41%) had a virtual video CGA scheduled after the in-person clinic visit with the thoracic surgeon, and thus a FI-CGA and FRAIL scale were calculated virtually. The cohort was mainly female (59%), white (92%), had an education level of college or higher (59%), and a median age of 77 (range: 67–90, IQR 73–79). Furthermore, 31/49 patients (63%) had ≥ 5 comorbidities and 10 patients (20%) were taking 10 or more medications at baseline. Patients commonly presented to the clinic with lung conditions, with lung cancer being the most common diagnosis (47%). Consequently, most prevalent surgeries in this cohort were wedge resections followed by lobectomies and segmentectomies (30%, 28%, and 6% respectively). The median PFFS score was 11 (IQR 5–14), median PFFS_trans score 0.27 (IQR 0.12–0.34), median FI-CGA score 0.24 (IQR 0.13–0.32), median VES-13 3 (IQR 3–5) and the median FRAIL score 1 (IQR 1–4) (Table 1).

Table 1-.

Population characteristics (n=49)

Variable
Virtual visit- n (%)	20 (41%)
Age- median (IQR)	77 (73–79)
Gender- Female n (%)	29 (59%)
Race- white n (%)	44 (92%)
Ethnicity- non-Hispanic n (%)	46 (98%)
BMI- median (IQR)	26 (22–29)
Education level
High school or less n (%)	16 (41%)
College n (%)	16 (41%)
Advanced degree n (%)	7 (18%)
Comorbidities
1–4 comorbidities	17 (35%)
5–9 comorbidities	29 (59%)
10 and above comorbidities	2 (4%)
Number of medications
0–4	4 (8%)
5–9	35 (72%)
10 and above	10 (20%)
Main diagnosis
Lung cancer	23 (47%)
Lung nodule	8 (16%)
Metastasis to lung	3 (6%)
Esophageal/gastric cancer	6 (12%)
Mesothelioma	3 (6%)
Other*	6 (12%)
Surgical interventions n (%)	36 (74%)
Wedge resection	11 (30%)
Segmentectomy	2 (6%)
Lobectomy	10 (28%)
Pleurectomy	3 (8%)
Esophagectomy	4 (11%)
Other**	6 (17%)
Discharge disposition- inpatient rehabilitation	7 (21%)
PFFS (mean, IQR) (n=46)	11, 5–14
PFFStrans (mean, IQR) (n=46)	0.27, 0.12–0.34
FI-CGA (mean, IQR) (n=49)	0.24, 0.13–0.32
VES-13 (mean, IQR) (n=41)	3, 3–5
FRAIL (mean, IQR) (n=49)	1, 1–4

^*Includes: rib/sternum fracture, carcinoid, thymus cancer, granulomatous disease, mycobacterial infection, liposarcoma

^**includes: pleural biopsy, thymectomy, sternal fracture fixation, gastrectomy

PFFS- Pictorial Fit-Frail scale; FI-CGA- Frailty Index based on Comprehensive Geriatric Assessment; VES-13- Vulnerable Elders Survey; FRAIL- Frailty Questionnaire

Almost half of the patients (49%) required some assistance or were dependent in instrumental activities of daily living (IADL’s). Most patients (66%) reported fatigue on a daily basis, but overall health attitude and motivation were good/high in most patients (54% and 96% respectively). Although the median BMI for the entire cohort was 26 (IQR 22–29), 11 patients (33%) experienced weight loss in the past year and 18 patients (39%) reported a poor or fair appetite. In addition, 10 patients (24%) had varying degrees of cognitive impairment (Mild Cognitive Impairment (MCI) or Dementia) as reported by the geriatrician performing the CGA. (Table 2)

Table 2.

Geriatric variables from the Geriatric Comprehensive Assessment (CGA) (n=49)

Variable	N (%)
Assistance in any ADL*	9 (17%)
Use of a mobility aid-cane/walker/wheelchair	14 (29%)
Assistance in any IADL**	24 (49%)
Fatigue- yes	31 (66%)
Health attitude- excellent	26 (54%)
Motivation- high/usual	46 (96%)
Impaired vision	10 (22%)
Impaired hearing	10 (22%)
Dentures	6 (13%)
Falls in the past 12 months	14 (30%)
Impaired balance	8 (17%)
Weight loss in the past year	11 (31%)
Appetite- poor/fair	18 (39%)
Able to climb a flight of stairs	28 (60%)
Able to walk a distance of 100 yards	30 (64%)
Cognitive status- MCI+/Dementia	10 (24%)

^*ADL- activities of daily living, including: feeding, dressing, bathing, grooming, toileting, bladder control, transferring

^**IADL- instrumental activities of daily living, including: cooking, cleaning, shopping, medication management, cleaning, shopping, driving

⁺MCI- mild cognitive impairment

Feasibility

During the study period, all 49 patients invited to participate agreed. Of those, 46/49 (94%) completed the PFFS tool so a score could be calculated, while 41/49 (84%) fully completed the VES-13. In 11 (22%) patients we observed missing domains in the PFFS of which 3 had >3 missing domains, so they were omitted from the analysis. In the remaining 8 patients (missing 1–3 domains) a stable PFFS_trans score could be calculated while in 8 patients (16%) a VES-13 score could not be calculated due to missing responses. No patient was reported to ask for additional assistance from the staff in completing the scales and scales were collected by one of the co-investigators at the end of the clinic visit.

Construct validity

The median PFFS_trans was 0.27 (IQR 0.12–0.34) and 0.24 (IQR 0.13–0.32) for FI-CGA, while the median score for VES-13 was 3 (IQR 3–5) and 1 (IQR 1–4) for FRAIL scale. There was a moderate correlation between the PFFS_trans and VES-13 (r_spearman=0.68, p<0.001), a strong correlation between the PFFS_trans and FI-CGA (r_spearman 0.81, p<0.001), and for those with cognitive impairment (MCI or Dementia) demonstrating a moderate correlation between PFFS_trans and FI-CGA (r_spearman 0.64, p=0.064) as well as a moderate correlation between PFFS_trans and FRAIL scale (r_spearman 0.62, p<0.001). VES-13 was also strongly correlated to FI-CGA (r_spearman= 0.79, p<0.001). When correlating the in-person PFFS_trans to a virtual FI-CGA, the correlation was moderate (r_spearman 0.69, p<0.001). Similarly, the correlation between the VES-13 and a virtual FI-CGA was also moderate (r_spearman =0.69, p=0.006) (Table 3). Histograms of frailty levels with relative frequencies of the 4 different scales are shown in Figure 2.

Table 3.

Spearman’s Correlation with Pictorial Fit-Frail Scale (PFFS_trans) (n=46)

	Correlation with PFFStrans	p-value
FI-CGA	0.81	<0.001
VES-13	0.68	<0.001
FRAIL	0.64	<0.001
Virtual FI-CGA (n=20)	0.69	0.006

Figure 2.

Frailty levels with relative frequencies of the four different scales.

Discussion:

This cross-sectional study of older patients in a Thoracic Surgery clinic demonstrated good feasibility of the PFFS tool. All patients agreed to complete this tool while waiting in clinic, and the majority were able to fully complete it without additional assistance other than their caregiver if needed. We were also able to demonstrate good construct validity of the PFFS tool as shown by a strong correlation to routine frailty measures used in this clinic by a board-certified geriatrician: the frailty index based on CGA (FI-CGA). During the 2020–2021 COVID pandemic geriatric assessments became virtual and yet, the PFFS maintained a strong correlation to the virtual FI-CGA. The geriatrician was blinded to the results of the PFFS score prior to completing his own assessment further reducing investigator bias. Most patients in our cohort had an education level of college, and mild to moderate frailty level as measured by both the PFFS and the FI-CGA.

Demonstrating good feasibility and construct validity of the PFFS in surgical patients is both novel and important. As mentioned, there are many tools to measure frailty³ but few tools can be self-reported while remaining detailed and comprehensive. The VES-13, which was used in this study as another validated, widely accepted self-reported tool to measure frailty, was found to strongly correlate to the FI-CGA but only moderately to the PFFS. We believe that this difference is probably due to the fact that the VES-13 focuses mainly on limitations in physical function and disability while the PFFS evaluates many other domains of health including mood, social connections, memory, and aggression to name a few. Another point to consider is that while a similar number of patients omitted some responses in these scales, for the VES-13 this meant that a final score could not be calculated, while this was not true for the PFFS, and a score was calculated for the majority of patients with missing responses (8/11, 73%) by using the domains available (as long as <20% missing). This is a valuable aspect to consider when implementing a new tool in a “real life” setting, to assure all patients have a score that can be used for future considerations such as selecting patients who would require a geriatric assessment prior to surgery. This study was not constructed to assess the correlation of PFFS scores to surgical outcomes, however, it would be interesting for future studies to investigate this as the VES-13 has shown mixed results in the surgical setting.³⁷ Another important aspect to consider is that the scale was not developed with the idea that each domain will be used individually, but rather that all domains together represent the health state of the individual.³⁸ None of the domain pictures will provide a comprehensive assessment of the domain but some may work for domain screening. It is possible that some domains are not very accurate in measuring the level of impairment in that domain but overall they accurately measure frailty. Also it is possible that we could have other domains and result in similar frailty scores. The deficit accumulation approach is based on this idea.

Another advantage of the PFFS is the ease of completing this questionnaire quickly while waiting in clinic. This supports previous findings in a memory clinic, where it demonstrated good feasibility and validity.¹⁵ In addition, although our population was quite homogenous, we strongly believe that this tool has the advantage of being generalizable across different languages, cultures and education levels since it is based on simple images that depict health- related domains not based on language, and was shown to be used across diverse levels of health literacy.^13,17 One future aim is to demonstrate these advantages in diverse surgical settings and populations.

There are additional tools that have been developed to assess frailty in the perioperative period, that aim to be both comprehensive and easy to administer. One noted example is the electronic rapid fitness assessment (eRFA) developed at Memorial Sloan Kettering Cancer Center (MSKCC). This tool was shown to be quick, taking approximately 11 minutes to fill by patients.³⁹ The main disadvantage of this and other electronic-based tools is that technology needs to be accessible to the patient either in the surgical clinic or at home. Shahrokni et al. showed that the majority of their patients filled the eRFA at home. This may add another barrier for older patients, as many may not have access or the ability to complete a web-based tool.⁴⁰ Other screening tools for frailty and vulnerabilities that aim to be quick and practical, such as the Sinai Abbreviated Geriatric Evaluation (SAGE) rely on an in-person geriatric evaluation of the patient to complete screening tests [Mini-Cog, gait speed and Time Up and Go (TUG)].⁴¹ The PFFS includes multiple domains as required for constructing a valid frailty index,²⁹ while including categories that are not often included in other self-reported measures. For example, in the PFFS the patient is asked to grade challenges in memory and thinking (scale 0–5) which is more detailed and patient-oriented than asking about “the presence of any neuropsychological problem such as dementia, mild cognitive impairment or depression”, as present in the Geriatric-8 tool.⁴ In the neuropsychological domain of the PFFS, patients are also asked about aggression, social connections and mood, allowing a multidimensional neuropsychological initial screen. Additionally, if a patient omits one of these questions, information can still be derived from other categories. The comprehensiveness of the PFFS, while remaining easy and quick to complete by patients (less than 5 minutes), is another component to consider when implementing the use of vulnerability screens in the perioperative clinic.¹⁴ For this reason, some advocate for using the FRAIL scale to screen for frailty in the preoperative setting.⁴² This tool, however, is less comprehensive than other frailty scales, was mostly validated in trauma and orthopedic surgery settings,^33,43 and is not currently listed as a recommended tool for CGA by the Society of Geriatric Oncology (https://www.siog.org/content/comprehensive-geriatric-assessment-cga-older-patient-cancer). We hope that by conducting future studies with the PFFS in the surgical-oncology setting, this tool could be more widely used for frailty screening and assessment.

Lastly, we were able to demonstrate good correlation between the PFFS to FI-CGA even in patients with cognitive impairment. This differs from previous data from a memory clinic which demonstrated lower correlation between PFFS and FI-CGA in patients with low Mini-Mental State Exam scores.¹⁵ We believe these differences might be due to several factors. First, our patients were free to have an accompanying caregiver or family member assist in completing the form. Second, we analyzed patients with MCI and Dementia in one group even though there is a wide range of cognitive abilities in patients with these diagnoses. Nevertheless, this finding importantly demonstrates that patients with cognitive impairment can also complete this frailty tool in the clinic setting.

Our study has several limitations. First, this study was conducted in a single thoracic surgery clinic, at a large academic center in a relatively homogenous patient population with higher levels of education, mild to moderate frailty level and overall low rates of cognitive impairment. Generalizing these results should be done with caution. Second, we did not record whether the patient completed the questionnaires alone or with assistance of a caregiver. However, since we wanted to demonstrate feasibility in “real world” setting of a busy surgical clinic, one will expect that if a patient comes with a caregiver, they might assist in completing these questionnaires and thus this reflects realistic practices. Third, we did not limit our participating patients to those who had an initial consult and subsequently underwent a surgical intervention, so validation and correlation of the PFFS to surgical outcomes is not possible in this cohort. Now that we have shown the feasibility and construct validity of this tool, we believe that it can be incorporated as a vulnerability screen for older surgical patients, eliminating a number of barriers presented by other frailty assessment tools.⁴⁴ Importantly this tool is not based on comorbidities and thus adds valuable information that is not readily available from past medical health records. In addition, the PFFS can be used to define a positive “vulnerability screen” which would trigger a pathway that includes a CGA by a geriatrician prior to surgery⁴⁵. This can also be used directly by the surgical team to address vulnerabilities in a specific domain such as mobility/function, for example: if a patient indicates some degree of functional impairment on PFFS, then the surgeon can immediately refer to physical therapy or occupational therapy for further evaluation and management. Using a frailty assessment or screen can assure more resources and attention will be directed to patients who are frail and at high risk for a complex post-operative course. Further work is needed to determine the clinical utility of this tool, optimal cut-offs for sensitivity and specificity as a screening instrument and the correlation of frailty as measured by this tool to surgical outcomes, especially if considering wide implementation processes across busy surgical services. We aim to examine these potential pathways, as well as the association of PFFS to post-operative outcomes and patient-reported outcomes. Lastly, in our study, 20 patients underwent a virtual CGA by the geriatrician (from which a FI-CGA was calculated) but the PFFS was completed in person while the patient was in clinic. During the COVID-19 pandemic, our services moved to a virtual delivery format, and found it feasible to conduct a CGA virtually while adapting the domains to a tele-health visit assessment.⁴⁶ As most of the assessments were conducted by video in this study, the geriatrician was able to assess domains such as function, mobility and cognition in an objective way and not only based on the patient’s report. Nevertheless, further larger studies are needed to validate a virtual frailty assessment to an in-person one.

In conclusion, it is feasible to administer the PFFS to older adults in a busy Thoracic Surgery clinic. In addition, the PFFS demonstrated good construct validity when correlated to the FI-CGA. Future studies should focus on more diverse populations across different surgical settings and the association of this novel frailty measure to surgical outcomes.