Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jun 1.
Published in final edited form as: Perioper Care Oper Room Manag. 2020 Jan 25;19:100089. doi: 10.1016/j.pcorm.2020.100089

Rapid In-Person Cognitive Screening in the Preoperative Setting: Test Considerations and Recommendations from the Society for Perioperative Assessment and Quality Improvement (SPAQI)

Franchesca Arias a,b,c,d, Margaret Wiggins b,c, Richard D Urman e, Rebecca Armstrong b,c, Kurt Pfeifer f, Angela M Bader e, David J Libon g, Anita Chopra g, Catherine C Price a,b,c,d
PMCID: PMC7185236  NIHMSID: NIHMS1575423  PMID: 32342018

Abstract

There are few cognitive screening tools appropriate for fast-paced settings with limited staffing, and particularly in preoperative evaluation clinics. The Society for Perioperative Assessment and Quality Improvement (SPAQI) convened experts in neuropsychology, geriatric medicine, and anesthesiology to conduct a review of the literature and compile a comprehensive list of cognitive screening tools used within primary care and preoperative settings. This Recommendations Statement: 1. summarizes a review of the literature on existing cognitive screening tools used within preoperative settings; 2. discusses factors to consider when selecting cognitive screening tools in a preoperative environment; and 3. includes a work flow diagram to guide use of these screening measures. Methodology involved searching peer-reviewed literature for 29 cognitive screening tools which were identified from the literature that fit inclusion criteria. Of these 29, seven tests have been used in preoperative settings and are discussed. These seven had an average administration time ranging from one to ten minutes. Memory, language, and attention were the most commonly evaluated cognitive domains. Most had adequate sensitivity and specificity to detect cognitive impairment/dementia. While information on the psychometric properties of these tools is limited, the tools discussed are appropriate for lay examiners, are short in duration, and accessible for free or at a low cost. We describe factors that must be considered prior to instrument selection.

Keywords: dementia; cognitive; screening, delirium; instruments; impairment; preoperative; assessment; tools

1. Introduction

The preoperative workup routinely involves an evaluation of risk across major organ systems such as cardiac, respiratory, hematological, neurologic, and gastrointestinal [1, 2], but has yet to include routine screening for cognitive functioning [3]. Briefly, preoperative cognitive impairment is associated with failure to arrive at scheduled procedures [4], increased use of emergency and rehabilitation services [5], prolonged hospitalization and complications [6, 7], and elevated incidence of delirium [810]. Failure to identify cognitive vulnerabilities in older adults presenting for surgery may increase costs and negatively impact benefits associated with necessary procedures [11]. Insight into a patient’s cognitive status in medical settings can identify need for prehabilitation, additional preparatory planning support, and increase perioperative monitoring [1113]. Discussions on pre-existing cognitive vulnerabilities help patient-caregiver dyads make informed healthcare decisions [14].

Such publications have increased clinician awareness regarding the value of cognitive assessment in the preoperative setting. Although in 2013 the United States Preventive Services Task Force (USPSTF) refrained from recommending routine screening for cognitive impairment (citing stress related to misdiagnosis and the absence of efficacious treatment to mitigate cognitive decline may reduce the potential benefits associated with early detection of dementia) [15], the new 2019 American College of Surgeons Geriatric Surgery Verification (ACS GSV) Program [16] strongly recommends preoperative and postoperative cognitive and functional status screening for older adults electing surgery. The ACS GSV Program also recommends that the cognitive screener completed preoperatively should also be administered post-operatively. Cognitive screening documents should then be attached to medical records for geriatric interdisicplinary care teams. Based on these recommenations and the plethora of growing data showing the value of cognitive screening in older adults, medical teams are now faced with the dilemma of which measure to choose and how to appropriately use them in the clincial setting.

The Society for Perioperative Assessment and Quality Improvement (SPAQI), in collaboration with experts in preoperative neuropsychology, anesthesiology, and geriatric medicine, sought to address this issue. The authors reviewed the published literature for instruments applied in fast-paced clinical settings. The current Recommendations Statement that was produced by the aforementioned expert group: 1. Summarizes a review of the published literature on existing cognitive screening tools used to date in preoperative settings; 2. Discusses factors to consider when selecting cognitive screening tools; and 3. Provides a basic work flow diagram to assist with preoperative implementation.

1.1. Clarifying Neuropsychological Evaluation versus Cognitive Screening Instruments

The terms “neuropsychological assessment” and “cognitive screening” are often used interchangeably. This is misleading. They are two distinct evaluations that differ in terms of their duration, clinical indications, contributions, and the level of professional training necessary for proper test administration and interpretation [17]. Formal neuropsychological assessments are comprehensive, administered in a standardized manner, either administered and/or supervised by clinicians with extensive training in brain-behavior relations, and interpreted in the context of clinical, demographic, and medical factors [17]. Neuropsychological evaluations provide wide breadth and depth of detail about each patient’s neurocognitive strengths and weaknesses that are useful when developing individualized rehabilitation plans and detecting even subtle changes in cognition[18]. Despite these benefits, limited time and reduced institutional resources make the incorporation of formal neuropsychological assessments into rapid settings, such as preoperative centers, extremely challenging.

Cognitive screening tests, by contrast, are focused on assessing selected neurocognitive domains and interpreted using a pre-specified cut-off score [17]. While the administration of cognitive screening tools does not substitute for a neuropsychological assessment, this approach is highly desirable in clinical environments with high throughput. The goal for the administration of cognitive screening measures is to identify at-risk patients who may require an extended evaluation by neurology and/or neuropsychology [17, 18]. Providers eager to systematically assess for cognitive impairment in medical and preoperative settings face a number of challenges. Most notable is the paucity of cognitive screening tools appropriate for fast-paced settings with limited resources and staff [19]. Although abbreviated cognitive screening instruments exist (e.g., the Mini Mental Status Exam [MMSE; [20]], Mini-Cog©[21], Montreal Cognitive Assessment [MoCA; [22]]), few have been validated to detect different levels of cognitive impairment in preoperative settings [15]. Users are encouraged to be aware that cognitive screening tools often use cut-off scores which can limit outcome analyses as patient range of cognitive strengths and weaknesses is not inherently considered with cut-off scores. Integrating screening tools with interdisciplinary team engagement and consultation is therefore encouraged.

2. Methods

2.1. Literature search

To identify screening tools for review, we searched terms “dementia, cognitive” and “screener, screening tools, screening instruments” combined with “surgery, primary care, and preoperative setting” across three major bibliographical databases (PubMed, PsycINFO, and CINAHL). Cognitive and dementia screening tools identified via manual review of the literature (i.e., using published books and earlier literature reviews) were used to complement our initial search [18, 23, 24].

2.2. Selection criteria

Primary care and preoperative clinics have limited time and resources and are staffed by interdisciplinary teams with diverse training backgrounds. This review required that the tools be accessible (i.e., available online or included in a published manuscript), affordable (i.e., free or available for a small fee), brief (i.e., < 10 minutes), minimal regarding props (e.g., only paper and pencil), and useful for clinicians with limited training in psychological assessment. We also required the test have clear and succinct administration and scoring instructions, assess multiple cognitive domains, and show evidence of sensitivity and specificity in medically-compromised adults. In settings where brevity is necessary, an ideal measure would assess memory and one other cognitive function such as attention or inhibitory function – domains commonly altered first in early dementia phases [25] and which are also susceptible to cognitive change after surgery [26].

2.3. Data extraction

Data were extracted by one author (FA) and corroborated on a separate occasion by a second author (RA). We acquired data using a structured data summary sheet with a SPICE (setting, population, intervention, comparison, evaluation) framework and details regarding the cognitive domains and psychometric properties [27]. We excluded abstracts written in a language other than English and all editorials. Duplicate publications, abstracts that did not include objective measures of cognition, or publications that referred to cognitive screening tools validated on adults younger than 45 years of age were excluded. A licensed psychologist and board-certified neuropsychologist (CP) corroborated the information regarding cognitive domains and psychometric properties.

3. Results

Initially, 787 publications were identified. Remaining abstracts (n= 674) were reviewed twice by a postdoctoral fellow in neuropsychology (FA) and a specialist in psychometry (RA). A total of 103 full-text articles were retrieved, reviewed by two of the authors (FA and RA), and used to extract information about cognitive screening instruments. Each abstract was reviewed twice (Figure 1).

Figure 1:

Figure 1:

Summary of manuscript identification process. A total of 674 records were screened, 103 full-text articles were assessed for eligibility, and 74 studies were ultimately included in the synthesis.

A total of 27 potential cognitive screening instruments were identified. Table 1 describes how each cognitive screener performed with respect to the selection criteria (i.e., accessibility, includes standardized administration and scoring instructions), as well as total administration time. At the time of this publication, over 80% of the cognitive screening tools included in this list were accessible to all clinicians for free (i.e., included in published manuscripts or accessible on the internet). Approximately 48% of the instruments identified provided administration instructions and approximately 88% included scoring instructions and a potential cut-off sensitive to cognitive impairment (dementia). Two cognitive screening tools were described as “abbreviated” but did not report total administration time. For those remaining, the average administration time was 4.5 minutes, and administration time ranged from 1 to 10 minutes (Table 1). In terms of type of setting used, most cognitive screening tools have demonstrated utility within primary care (>80%) and community settings (>50%).

Table 1.

Abbreviated Cognitive Screeners Used within Primary, Preoperative, or Community-Based Settings

5-Item Recall and Fluency (5-IRF) [21] F NR NR >1 P/C Kilada et al., 2005
6-Item Cognitive Impairment Test (6-CIT) [22] F NR Y 3 P Brooke & Bullock, 1999
AB Cognitive Screen 135 (ABCS) [23, 24] F NR NR 5 P Molloy et al., 2005; Standish, Molloy, Cunje, & Lewis, 2007
Abbreviated Mental Test (AMT) [25, 26] F NR NR 5 P Hodkinson, 1972; Swain, Nightingale, 1997
Brief Alzheimer’s Screener (BAS) *MMSE Selected Items+ Semantic Fluency [27] F NR Y <5 C Mendiondo et al., 2003
Brief Cognitive Scale (BCS) C Y Y NR C Krishnan et al., 2001
Clock Drawing Test (CDT) [28, 29] F Y Y 2–5 P/Pre/C Sunderland et al., 1989; Libon et al., 1996; Amini et al., 2019
DemTect (DemTect) [30] C NR Y 10 P/C Kalbe et al., 2004
Eight Item Screener (8-IS) [31] F NR Y NR P Chen, et al., 2011
General Practitioner Assessment of Cognition (GPCOG) *Cambridge Cognitive Examination [32] F Y Y 5 P Brodaty et al., 2002
Memory Impairment Screen (MIS)[33, 34] F NR Y <5 P Buschke et al., 1999; Verghese et al., 2012
Mini-Cog [35] F Y Y 4 Pre/C Borson et al., 2000
Mini-mental State Examination (MMSE) [3638] F Y Y 10 P/Pre/C Folstein et al., 1983; Browndyke et al., 2017; Kukull et al., 1994
Montreal Cognitive Assessment (MoCA) [39] C Y Y 10 P/C Nasreddine et al., 2005
Modified mini-mental state (3MS) [40] *MMSE Selected Items C NR Y 3 P Teng & Chui, 1987
Modified WORLD Test (MWorldT) [41] F NR Y 1 P Leopold & Borson, 1997
Months Forwards and Backwards Test (MBT) [42, 43] F NR Y 2 P/Pre Lamar et al., 2002; Meagher et al., 2015
Mental Alternation Test (MAT) [44] F NR Y <5 P Salib & McCarthy, 2002
Rapid Dementia Screening Test (RDST) F NR Y <5 P/C Kalbe, Kessler, & Calabrese, 2002
Rotterdam Version of the Cambridge Cognitive Examination (R-CAMCOG) **Cambridge Cognitive Exam C Y Y 10 P/C De Koning et al., 2000
Seven-Item Screener (7-IS) C NR NR 7 NR Chumbler & Zhang, 1998; Chumbler et al., 2001
Six Item Screener (SIS) [45] F NR Y 5 P/C Callahan et al., 2002
Short Orientation Memory Concentration Test (S-OMC) also known as the Short-Blessed Test (SBT) [46] F Y Y 5 P/Pre Katzman et al., 1983
Short Portable Mental Status Questionnaire (SPMSQ) [47] F Y Y 5 C Pfeiffer, 1975
Short Test of Mental Status (STMS) [48, 49] F Y Y 5 C Kokmen, Naessens, & Offord, 1987; Tang-Wai, et al., 2003
Time and Change (T&C) [50] F Y Y 1 P/Pre Inouye, Robison, Froehlich, & Richardson, 1998
The Telephone Interview of Cognitive Status (TICS) [51, 52] C Y Y NR Pre/C Brandt, Spence, & Folstein, 1988; Leung, Tsai, & Sands, 2011
The Sweet 16 (S-16) [53] F Y Y 2 P/C Fong et al., 2011
Three Word Recall (3WR) [54] F Y Y 4 P/Pre/C Kuslansky et al., 2002
Trail Making Test (TMT) [55] F Y Y 5 P/C Reitan, 1958
Verbal Fluency Categories (VF) [56] F Y Y 3 P Isaacs & Kennie, 1973

Note.

*

- Indicate tests that were derived from other more established cognitive screening instruments; Y- Yes; N- No; F- Free; C- There are costs associated with the test; NR-Not reported; Admin- Administration; Time is presented in minutes; P- primary; Pre- Preoperative; C-community-based.

A handful of cognitive screening instruments have been used in preoperative settings (29%). The average administration time for the cognitive screening tools previously used in preoperative settings was 5 minutes and administration time ranged from 1 to 10 minutes. Approximately 71% of the identified cognitive screening tools used in preoperative settings evaluated multiple cognitive domains. Memory, orientation, and attention were most commonly evaluated. Sensitivity, an instrument’s ability to accurately identify patients with a condition (e.g., cognitive impairment) and specificity, an instrument’s ability to correctly identify patients without a condition (e.g., cognitive impairment), were seldom reported [57]. When available, sensitivity and specificity ranged from 0.63 to 0.99 and 0.54 to 0.99, respectively. These tests are discussed below.

4. Recommendations for Cognitive Screening Tools with Utility within Preoperative Settings. Tests are presented in alphabetical order.

4.1. Clock Drawing Test Command and Copy (CDT)

The Clock Drawing Test (CDT) with command and copy conditions is a brief, paper-and-pencil screening tool that can be administered in approximately 2–5 minutes [28]. In the most widely used adaptation, the patient is asked to: “Draw the face of a clock, put in all of the numbers, and set the hands to ten after 11”. This instruction constitutes the command condition of the test. Next, the patient is asked to copy the face of the clock, which is presented to them on a piece of paper, representing the copy condition. Each component of the instrument assesses different thinking abilities. The linguistic proposition “ten after eleven”, requires the patient to disambiguate the complex linguistic proposition in addition to inhibiting a tendency to draw hands to the ‘10’ and ‘11’. Accurate representation for time setting requires an awareness of the sematic features associated with attributes designating the hour and the minutes [29]. For all of these reasons, the command condition is known to assesses many cognitive domains including language, semantic memory, visuoconstruction, inhibition, and planning [30, 31], and to associate highly with scores on other general cognitive screeners such as the MMSE [30, 32]. By contrast, the copy condition requires less cognitive load and is more dependent on visuoconstruction and executive functions[30, 33]. Comparing the command and copy condition can therefore assist with differential dementia diagnosis [33, 34].

Clock drawing to command and copy can be scored with a variety of scales. For example, it can be scored using the 0 to 3 scoring rules of the MoCA [22], 0–2 scoring of the Mini-Cog© [21], or it can be scored based on error type [30] (for more discussion see [35]). Selection of a scoring systems must be made early-on in the process and ought to be guided by the nature of the setting (e.g., training level of the staff and time constraints [35]. Clock drawing is a well-established cognitive screener for dementia, is integrated into many cognitive screening tools, and has moderate to high sensitivity for dementia in older adults [32].

Advantages for using the CDT include its ease of administration and that clock drawing can be conducted on a blank sheet of paper without a template. Researchers document that preoperative nursing staff can administer the CDT in rapid-paced environments [36], and that patient errors on the copy condition predict length of stay over and above comorbidity alone [28]. Limitations include little research on the value of preoperative command relative to copy conditions on post-operative outcome. Note that some studies using the clock drawing to command condition only do not help predict outcomes like delirium [37, 38]. Also, like most tests, clock drawing should be used cautiously with non-native English speakers, those with low education, or individuals with language comprehension difficulties. Available data suggest that consistent scoring and administration are instrumental to the CDT’s utility in preoperative settings [28].

4.2. Mini-Cog©

The Mini-Cog© is a screening tool that includes a clock drawing task to command with the clock face already provided to the patient, and a 3-word recall. The three-word memory recall adds a delay memory recall component. Test time is approximately 1.5 to 5 minutes. The clock is scored on a 2-point scale (2 points for a completely correct drawing and 0 points for anything less), and a point is given for each recalled word. Scores ranging from 3 to 5 suggest intact cognition. A score of 2 or less is used as a cut-off indicative of cognitive impairment. The Mini-Cog© has adequate sensitivity and specificity for detecting dementia within a variety of settings and values range from 0.76 to 1.00 and from 0.27 to 0.86, respectively [21, 39]. In preoperative settings, the Mini-Cog© has been used to detect probable cognitive impairment [8, 40].

The Mini-Cog© is available online. In addition to the tools needed for administration, information about scoring as well as cut-offs indicative of dementia are also provided. The Mini-Cog© has been described as “less stressful” for patients than longer cognitive instruments [69]. In terms of its limitations, the Mini-Cog© has been criticized for being simplistic and using a qualitative (e.g., normal versus abnormal) scoring of the clock drawing [41] without allowing for partial credit. Moreover, cognitive profile interpretation based on the clock is limited as it does not provide “spontaneous” clock drawing (i.e., the patient is anchored and given a circle to place the numbers and hand) and it does not provide a copy condition [33]. The Mini-Cog© should be used cautiously with non-native English speakers, those with low education, or individuals with language comprehension difficulties. Consistent scoring and administration are instrumental to the test’s use in preoperative settings

4.3. The Mini Mental Status Exam (MMSE)

The MMSE[20] is a screening tool that can be administered in approximately 10 minutes. It screens multiple domains (e.g., orientation, learning and memory, attention, language, and construction) and is one of the most widely used cognitive screening instruments available. Scores range from 0 to 30, with a score of 24 traditionally considered the cut-off indicative of cognitive impairment [20]. For the cut-off score of 24, specificity ranges from 0.90 to 0.96 and sensitivity ranges from 0.63 to 0.69, depending on the sample [42, 43]. Cut-off scores are, however, controversial; experts contend that raising the MMSE cut-off to 26 would increase sensitivity, which is particularly important for screening measures [18], with a recent study supporting a cut-off of 26 until age 93 [44].

Advantages include the number of cognitive domains available for screening and predictive value for post-operative outcome. In preoperative settings, the MMSE has been associated with poor postoperative outcomes and incident delirium [10, 45]. Research also shows that 3-word delayed recall and the attention/concentration items which include spelling WORLD backwards and serial sevens (i.e. counting backwards from 100 by 7) predict delirium after cardiothoracic surgery [10].

There are limitations. The MMSE scores are affected by age, education, and cultural background [46, 47], and the test should be used cautiously for individuals with less than 8 years of formal education [46]. The test length can take time despite some adaptations to incorporate it into a clinical interview [48]. Despite the many free versions of the test that are available on the internet, Psychological Assessment Resources claims that the official version is copyrighted and must be ordered only through the company[49].

4.4. Months Backward Test (MBT) or Months of the Year in Reverse Order (MOYR)

The MBT task is a 1- to 2-minute task designed to assess an individual’s capacity to maintain mental sets [5052]. During the MBT, the examiner requires a patient to recite the months of the year forward as quickly as possible (Trial 1; Months Fwd). This is recorded and timed. Next, the patient is required to recite the months of the year backward as quickly as possible (Trial 2; Months Bckwd). Total completion time per trial is often used as a measure of processing speed, and cognitively intact adults are likely to complete the task in 17+10 seconds[51]. The number and type of errors produced per trial are also recorded, and provide information about mental flexibility, semantic knowledge, and inhibitory control [50, 52].

The MBT has been used to detect dementia, and patients with Alzheimer’s disease (AD) score markedly below (37% correct) their counterparts with mild cognitive impairment (87% correct) and subjective complaints of cognitive difficulties (94%)[52]. Clinical research shows that individuals with more cerebrovascular disease have more difficulty with months backwards and particularly towards the end of the task[50]. Preoperative performance on the MBT predicts postoperative delirium in older adults undergoing cardiac surgery [53].

Advantages include the test’s brevity, that it does not need a test booklet (you can record a person’s responses on a sheet of paper, and record time with your watch. Limitations include its isolated assessment of “subcortical-frontal functions” (i.e., processing speed, working memory, inhibitory functions) and the fact that there is no incorporated screening of learning and memory, or other cognitive domains. The test is also vulnerable to educational level and cultural factors, and individuals with learning difficulties as well as those from diverse groups may underperform when compared to their counterparts [52].

4.5. Short-Blessed Test (SBT) and the Short Orientation Memory Concentration Test (S-OMCT)

The SBT test has an administration time of 2 minutes. While it has several iterations, in the most common version, the patient is given a 5-item address (John, Smith, 52, West Street, Boston) and asked to produce the address spontaneously at a later time. A score of 3 or more recall errors is associated with dementia. When using the above-mentioned cut-offs, the test showed a sensitivity of 0.71 and specificity of 0.56 [54]. Another widely used version, the S-OMCT has an administration time of up to 5 minutes. It includes a 5-item address recall as well as questions of orientation (e.g., What year is it now? What month is it now? What time is it now?) and working memory (e.g., Count backwards from 20 to 1; Say the months of the year in reverse). The S-OMCT has demonstrated utility detecting cognitive impairment with sensitivity of 0.88 and specificity of 0.94 [55, 56].

The SBT can be administered by novice clinicians and requires minimal props for completion. The S-OMCT can also be administered by novice clinicians but does require more props than its shorter version, the SBT. Inter-rater reliability for the SBT can be high even in rapid paced environments such as emergency department [54, 57]. The test is also freely available via the internet and has been translated into different languages.

SBT is used less often [58] and relies heavily on a single domain (memory) at the expense of other cognitive abilities (e.g., language deficits and executive dysfunction) [59]. Additionally, the SBT has exhibited low reliability and validity when used to assess cognitive impairment in primary care settings and may assign a cognitive impairment status to patients who are experiencing reversible causes of dementia [60].

4.6. Telephone Interview of Cognitive Status (TICS)

The TICS is a standardized screening instrument that can be administered over the phone or in person and takes approximately 10 minutes [51]. It contains 11 items, for a maximum of 41 points. It assesses orientation, learning and memory, attention, and language. It is one of the most popular telephone-based screening instruments and has been incorporated into multiple epidemiological trials. It was modeled after existing screening instruments and is highly correlated with the MMSE. While cut-off scores for the TICS vary depending on the characteristics of the population being assessed, 28 has been reported as the cut-off indicative of cognitive impairment for adults with vascular dementia [61], and 34 for individuals with amnestic mild cognitive impairment[62]. FOverall, the TICS demonstrated high sensitivity (~0.94) and specificity (~1.00) for detecting AD dementia and differentiating it from normal cognitive aging [61, 63].

In preoperative settings, the TICS has been associated with poor postoperative outcomes and incident delirium [64]. The TICS is available for purchase via the internet. Limitations associated with the TICS relate to the logistics associated with administration. Test developers recommend that prior to administration a caregiver must be identified and assigned as “proctor” for the evaluation. Additionally, language comprehension difficulties related to limited hearing are also a concern.

4.7. Time & Change (T&C)

Introduced by Inouye et al in 1998, the T&C consists of two tasks [65]. During the time task, the patient is shown the face of a clock with the hands at ten past eleven (11:10) and asked to report the time. The patient’s response, as well as the time it takes for the patient to provide the answer, are recorded. The patient has two attempts to provide a correct response within 60 seconds. An error is assigned if the patient fails to provide the accurate response across two trials. In the second part of the task, the patient’s ability to manipulate change is evaluated. At the beginning of the task, the patient receives three quarters, seven dimes, and seven nickels. Next, the patient is asked to provide one dollar in change to the examiner. The patient has two attempts to provide a correct response within 120 seconds. An error is assigned if the patient fails to provide the accurate response after two trials. A correct response on both tasks is negative for cognitive impairment.

In an acutely ill cohort of 776 adults >70 years, the T&C was useful in detecting cognitive impairment with sensitivity of 0.86 and specificity of 0.71 [65]. Other scoring criteria have also been reported [66]. In outpatient settings, T&C demonstrated adequate sensitivity (0.63) and excellent specificity (0.96) to detect cognitive impairment. After cut-off points were added, sensitivity ranged from 0.94 to 1.0 and specificity ranged from 0.37 to 0.46. Education influenced overall performance, and about 3% of the variance in T&C score was explained by number of years of education.

Advantages of this test include is novel face validity and incorporation of reasoning and calculations. Limitations include the need for props and potential cultural barriers to accurate test performance.

5. Other Considerations for Selecting a Cognitive Screening Instrument for Preoperative Settings

The patient’s functional status for instrumental and activities of daily living is also essential to consider. Impaired functional status as well as declining cognition should raise concerns about judgment/problem solving [67] and need for intensive discharge planning. The two most common approaches for assessment of functional status in older adults is by acquiring information on basic activities of daily living (ADLs; bathing, dressing, toileting, transferring, continence, and feeding), and by also addressing instrumental activities of daily living (IADLs; reflecting more adaptive and independent behaviors such as telephone use, medication management, financial management, driving, food preparation, etc.) [6871]. Acquiring documentation of preadmission status will help discuss preoperative outcomes, perioperative care needs, and potential discharge location. Incorporating functional status assessment into the preoperative screening is a highly recommended component for the 2019 standards of the Geriatric Surgery Verification Quality Improvement Program [16].

Prior to selecting a cognitive screening instrument and incorporating functional status measures into the preoperative environment, clinicians should consider demographic, cultural, and logistical factors unique to their settings. With regard to patient demographics, consider level of education, non-Native English speaker status, and frequency of sensory-perceptual limitations (e.g., hearing impairment) on cognitive instrument administration reliability and validity [18]. In a rapid paced setting, we encourage patient care teams to consider rate of staff rotations (e.g., residents) and demand needs for routine staff training. Intermittent reliability checks for administration and scoring will be necessary for accurate interpretation [18, 28] and accurate preoperative-postoperative comparison. It is also important to consider how to rapidly score and upload the information into a patient’s electronic medical file for later review and comparison relative to post-operative performance. In an ideal setting, the perioperative team will be comprised of experts in anesthesiology and surgery but also experts in brain-behavioral patterns for at-risk adults (neuropsychology) to assist with follow-up neurobehavioral screening (when needed), consultation opportunities with geriatric medicine, and consultation with other allied health professionals for prehabilitation purposes (nutritionist, physical therapists). Figure 2 provides a tentative workflow that could be implemented within preoperative settings.

Figure 2:

Figure 2:

Tentative work flow for implementing cognitive screening in the perioperative setting. The work flow includes a multidisciplinary team approach, identification of cognitive impairment, a workup that assesses cognitive strengths and weaknesses, communicating with the patient’s care team, and various stabilization techniques.

6. A Summary of Key Recommendations for Cognitive Screening in the Preoperative Setting

Perioperative professionals are in a unique position to identify cognitive vulnerabilities that may interfere with medical recommendations and hinder postoperative outcomes. This Recommendations Statement from SPAQI identified a handful of cognitive screening tools appropriate for identifying cognitive impairment in outpatient clinics and perioperative settings. While information on the psychometric properties of these tools is limited, all of them were appropriate for lay examiners, brief in duration, and accessible for free or at a low cost. The instruments listed in this document demonstrated adequate sensitivity for detecting cognitive impairment (dementia) in older adults. In order to select the most appropriate screening instrument for a given practice, clinicians must consider the needs of the surgical population, resources available at the facility, and ways to communicate the results to the patient’s care team.

Key Recommendations:

  1. Acquire education as part of the assessment so that education years is appreciated relative to the context of screener performance.

  2. Understand cognitive screener limitations. Screeners have limited utility for differential diagnosis (e.g., Alzheimer’s disease versus vascular dementia) and identification of cognitive strengths and weaknesses (information that may be pertinent to patient discharge planning).

  3. If using the Mini-Cog© or the CDT, ensure scoring is completed accurately, as rater reliability for clock drawing can vary[35].

  4. For predicting delirium risk, avoid using cognitive screening instruments that have no documented use within preoperative settings.

  5. Use administration and scoring instructions included with the screener to train staff. Standardized administration and consistent scoring are instrumental to the sensitivity of the instrument to detect/identify true impairment. If possible, consult with professionals from disciplines (e.g., neuropsychology) to assist with staff training.

  6. Incorporate functional status questions into the evaluation. Establishing level of independence will help with discharge planning.

  7. Develop pathways and ways to communicate the results of screening to the perioperative care team. Consider co-management options by incorporating geriatricians, occupational therapists, physical therapists, and neuropsychologists.

  8. Scan the test sheet and performance information into the medical chart for later post-operative comparison.

Acknowledgment:

This article was co-published in the Journal of Clinical Anesthesia

Funding: This work was supported by the National Institute on Aging (grant no. T32-AG04963, FA) and the National Institutes of Health (grant no. R01 AG055337, CP). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Aging or the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Individual Contribution/Conflict of Interest/Financial Disclosures:

Franchesca Arias, Ph.D.-Study concept, data acquisition, data analyses, manuscript drafts- No conflict of interest to disclose.

Margaret Wiggins, M.S.- Manuscript preparation and review- No conflict of interest to disclose.

Richard D. Urman, M.D., M.B.A.- Study concept & manuscript draft- No conflict of interest to disclose.

Rebecca Armstrong, B.A.- Data collection and manuscript review- No conflict of interest to disclose.

Kurt Pfeifer M.D., F.A.C.P.- Manuscript draft- No conflict of interest to disclose.

Angela M. Bader M.D., MPH.- Manuscript drafts- No conflict of interest to disclose.

David J. Libon, Ph.D., ABN, FACPN – Manuscript drafts – No conflicts of interest to disclose.

Anita Chopra, M.D. - Manuscript draft- No conflict of interest to disclose.

Catherine C. Price, Ph.D. ABPP-CN - Study concept and manuscript drafts- No conflict of interest to disclose.

References:

  • [1].Arnold MJ, Beer J. Preoperative evaluation: A time-saving algorithm. J Fam Pract 2016;65:702–10. [PubMed] [Google Scholar]
  • [2].Zambouri A Preoperative evaluation and preparation for anesthesia and surgery. Hippokratia 2007;11:13–21. [PMC free article] [PubMed] [Google Scholar]
  • [3].Fleisher LA, Beckman JA, Brown KA, Calkins H, Chaikof EL, Fleischmann KE, et al. ACC/AHA 2007 Guidelines on Perioperative Cardiovascular Evaluation and Care for Noncardiac Surgery: Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines on Perioperative Cardiovascular Evaluation for Noncardiac Surgery) Developed in Collaboration With the American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Rhythm Society, Society of Cardiovascular Anesthesiologists, Society for Cardiovascular Angiography and Interventions, Society for Vascular Medicine and Biology, and Society for Vascular Surgery. J Am Coll Cardiol 2007;50:1707–32. [DOI] [PubMed] [Google Scholar]
  • [4].Arias F, Riverso M, Levy SA, Armstrong R, Estores DS, Tighe P, et al. Pilot Study: Neurocognitive Disorders and Colonoscopy in Older Adults. Anesth Analg 2019. [DOI] [PMC free article] [PubMed]
  • [5].de Gelder J, Lucke JA, de Groot B, Fogteloo AJ, Anten S, Heringhaus C, et al. Predictors and Outcomes of Revisits in Older Adults Discharged from the Emergency Department. J Am Geriatr Soc 2018;66:735–41. [DOI] [PubMed] [Google Scholar]
  • [6].Fick DM, Kolanowski AM, Waller JL, Inouye SK. Delirium superimposed on dementia in a community-dwelling managed care population: a 3-year retrospective study of occurrence, costs, and utilization. J Gerontol A Biol Sci Med Sci 2005;60:748–53. [DOI] [PubMed] [Google Scholar]
  • [7].Culley DJ, Flaherty D, Fahey MC, Rudolph JL, Javedan H, Huang CC, et al. Poor Performance on a Preoperative Cognitive Screening Test Predicts Postoperative Complications in Older Orthopedic Surgical Patients. Anesthesiology. 2017;127:765–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Robinson TN, Wu DS, Pointer LF, Dunn CL, Moss M. Preoperative cognitive dysfunction is related to adverse postoperative outcomes in the elderly. J Am Coll Surg 2012;215:12–7; discussion 7–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [9].Rudolph JL, Marcantonio ER, Culley DJ, Silverstein JH, Rasmussen LS, Crosby GJ, et al. Delirium is associated with early postoperative cognitive dysfunction. Anaesthesia 2008;63:941–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [10].Price CC, Garvan C, Hizel LP, Lopez MG, Billings FTt. Delayed Recall and Working Memory MMSE Domains Predict Delirium following Cardiac Surgery. J Alzheimers Dis 2017;59:1027–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [11].Calkins MP. From Research to Application: Supportive and Therapeutic Environments for People Living With Dementia. Gerontologist. 2018;58:S114–S28. [DOI] [PubMed] [Google Scholar]
  • [12].Mohanty S, Rosenthal RA, Russell MM, Neuman MD, Ko CY, Esnaola NF. Optimal Perioperative Management of the Geriatric Patient: A Best Practices Guideline from the American College of Surgeons NSQIP and the American Geriatrics Society. Journal of the American College of Surgeons. 2016;222:930–47. [DOI] [PubMed] [Google Scholar]
  • [13].Prizer LP, Zimmerman S. Progressive Support for Activities of Daily Living for Persons Living With Dementia. Gerontologist. 2018;58:S74–S87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Arias F, Bursian AC, Sappenfield JW, Price CC. Delirium History and Preoperative Mild Neurocognitive Disorder: An Opportunity for Multidisciplinary Patient-Centered Care. American Journal of Case Reports. 2018;19:1324–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [15].Lin JS, O’Connor E, Rossom RC, Perdue LA, Eckstrom E. Screening for cognitive impairment in older adults: A systematic review for the U.S. Preventive Services Task Force. Ann Intern Med 2013;159:601–12. [DOI] [PubMed] [Google Scholar]
  • [16].Surgeons ACo. Optimal Resources for Geriatric Surgery: 2019 Standards. Chicago, IL: 2019. [Google Scholar]
  • [17].Block CK, Johnson-Greene D, Pliskin N, Boake C. Discriminating cognitive screening and cognitive testing from neuropsychological assessment: implications for professional practice. Clinical Neuropsychologist. 2017;31:487–500. [DOI] [PubMed] [Google Scholar]
  • [18].Lezak MD. Neuropsychological assessment. 5th ed. Oxford; New York: Oxford University Press; 2012. [Google Scholar]
  • [19].Solomon PR, Murphy CA. Should we screen for Alzheimer’s disease? A review of the evidence for and against screening for Alzheimer’s disease in primary care practice. Geriatrics. 2005;60:26–31. [PubMed] [Google Scholar]
  • [20].Folstein MF, Robins LN, Helzer JE. The Mini-Mental State Examination. Arch Gen Psychiatry. 1983;40:812. [DOI] [PubMed] [Google Scholar]
  • [21].Borson S, Scanlan JM, Chen P, Ganguli M. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc 2003;51:1451–4. [DOI] [PubMed] [Google Scholar]
  • [22].Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–9. [DOI] [PubMed] [Google Scholar]
  • [23].Cullen B, O’Neill B, Evans JJ, Coen RF, Lawlor BA. A review of screening tests for cognitive impairment. J Neurol Neurosurg Psychiatry. 2007;78:790–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Long LS, Shapiro WA, Leung JM. A brief review of practical preoperative cognitive screening tools. Can J Anaesth 2012;59:798–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [25].Jak AJ, Bangen KJ, Wierenga CE, Delano-Wood L, Corey-Bloom J, Bondi MW. Contributions of neuropsychology and neuroimaging to understanding clinical subtypes of mild cognitive impairment. Int Rev Neurobiol 2009;84:81–103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [26].Price CC, Garvan CW, Monk TG. Type and severity of cognitive decline in older adults after noncardiac surgery. Anesthesiology. 2008;108:8–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [27].Booth A. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech 2006;24(3):355–368. [Google Scholar]
  • [28].Amini S, Crowley S, Hizel L, Arias F, Libon DJ, Tighe P, et al. Feasibility and Rationale for Incorporating Frailty and Cognitive Screening Protocols in a Preoperative Anesthesia Clinic. Anesth Analg 2019. [DOI] [PMC free article] [PubMed]
  • [29].Freedman M, Leach L, Kaplan E. Clock drawing: A neuropsychological analysis. USA: Oxford University Press; 1994. [Google Scholar]
  • [30].Libon DJ, Malamut BL, Swenson R, Sands LP, Cloud BS. Further analyses of clock drawings among demented and nondemented older subjects. Arch Clin Neuropsychol 1996;11:193–205. [PubMed] [Google Scholar]
  • [31].Libon DJ, Swenson RA, Barnoski EJ, Sands LP. Clock drawing as an assessment tool for dementia. Arch Clin Neuropsychol 1993;8:405–15. [PubMed] [Google Scholar]
  • [32].Shulman KI. Clock-drawing: is it the ideal cognitive screening test? Int J Geriatr Psychiatry. 2000;15:548–61. [DOI] [PubMed] [Google Scholar]
  • [33].Price CC, Cunningham H, Coronado N, Freedland A, Cosentino S, Penney DL, et al. Clock drawing in the Montreal Cognitive Assessment: recommendations for dementia assessment. Dement Geriatr Cogn Disord 2011;31:179–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [34].Cosentino S, Jefferson A, Chute DL, Kaplan E, Libon DJ. Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. Cogn Behav Neurol 2004;17:74–84. [DOI] [PubMed] [Google Scholar]
  • [35].Frei BW, Woodward KT, Zhang MY, Amini S, Tighe P, Garvan CW, et al. Considerations for Clock Drawing Scoring Systems in Perioperative Anesthesia Settings. Anesth Analg 2019;128:e61–e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [36].Sherman JB, Chatterjee A, Urman RD, Culley DJ, Crosby GJ, Cooper Z, et al. Implementation of Routine Cognitive Screening in the Preoperative Assessment Clinic. A A Pract 2019;12:125–7. [DOI] [PubMed] [Google Scholar]
  • [37].Bryson GL, Wyand A, Wozny D, Rees L, Taljaard M, Nathan H. The clock drawing test is a poor screening tool for postoperative delirium and cognitive dysfunction after aortic repair. Can J Anaesth 2011;58:267–74. [DOI] [PubMed] [Google Scholar]
  • [38].Adamis D, Meagher D, O’Neill D, McCarthy G. The utility of the clock drawing test in detection of delirium in elderly hospitalised patients. Aging Ment Health. 2016;20:981–6. [DOI] [PubMed] [Google Scholar]
  • [39].McCarten JR, Anderson P, Kuskowski MA, McPherson SE, Borson S. Screening for cognitive impairment in an elderly veteran population: acceptability and results using different versions of the Mini-Cog. J Am Geriatr Soc 2011;59:309–13. [DOI] [PubMed] [Google Scholar]
  • [40].Culley DJ, Flaherty D, Reddy S, Fahey MC, Rudolph J, Huang CC, et al. Preoperative Cognitive Stratification of Older Elective Surgical Patients: A Cross-Sectional Study. Anesth Analg 2016;123:186–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [41].Doerflinger DMC. How to try this: the mini-cog. The American Journal of Nursing. 2007;107:62–71. [DOI] [PubMed] [Google Scholar]
  • [42].Kukull WA, Larson EB, Teri L, Bowen J, McCormick W, Pfanschmidt ML. The Mini-Mental State Examination score and the clinical diagnosis of dementia. J Clin Epidemiol 1994;47:1061–7. [DOI] [PubMed] [Google Scholar]
  • [43].Feher EP, Mahurin RK, Doody RS, Cooke N, Sims J, Pirozzolo FJ. Establishing the limits of the Mini-Mental State. Examination of ‘subtests’. Arch Neurol 1992;49:87–92. [DOI] [PubMed] [Google Scholar]
  • [44].Kvitting AS, Fallman K, Wressle E, Marcusson J. Age-Normative MMSE Data for Older Persons Aged 85 to 93 in a Longitudinal Swedish Cohort. J Am Geriatr Soc 2019;67:534–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [45].Saczynski JS, Marcantonio ER, Quach L, Fong TG, Gross A, Inouye SK, et al. Cognitive trajectories after postoperative delirium. N Engl J Med 2012;367:30–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [46].Tombaugh TN, Mcintyre NJ. The Mini-Mental-State-Examination - a Comprehensive Review. Journal of the American Geriatrics Society. 1992;40:922–35. [DOI] [PubMed] [Google Scholar]
  • [47].Freitas S, Simoes MR, Alves L, Santana I. The Relevance of Sociodemographic and Health Variables on MMSE Normative Data. Appl Neuropsychol Adult. 2015;22:311–9. [DOI] [PubMed] [Google Scholar]
  • [48].Goldberg RJ, Faust D, Novack D. Integrating the Cognitive Mental Status Examination into the Medical Interview. Southern Medical Journal. 1992;85:491–7. [DOI] [PubMed] [Google Scholar]
  • [49].Folstein MF, Folstein SE, Fanjiang G, Psychological Assessment Resources Inc. Mini-mental state examination : clinical guide. Lutz, FL: Psychological Assessment Resources; 2001. [Google Scholar]
  • [50].Lamar M, Price CC, Davis KL, Kaplan E, Libon DJ. Capacity to maintain mental set in dementia. Neuropsychologia. 2002;40:435–45. [DOI] [PubMed] [Google Scholar]
  • [51].Ball LJ, Bisher GB, Birge SJ. A simple test of central processing speed: an extension of the Short Blessed Test. J Am Geriatr Soc 1999;47:1359–63. [DOI] [PubMed] [Google Scholar]
  • [52].Meagher J, Leonard M, Donoghue L, O’Regan N, Timmons S, Exton C, et al. Months backward test: A review of its use in clinical studies. World J Psychiatry. 2015;5:305–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [53].Rudolph JL, Jones RN, Grande LJ, Milberg WP, King EG, Lipsitz LA, et al. Impaired executive function is associated with delirium after coronary artery bypass graft surgery. J Am Geriatr Soc 2006;54:937–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [54].Barbic D, Kim B, Salehmohamed Q, Kemplin K, Carpenter CR, Barbic SP. Diagnostic accuracy of the Ottawa 3DY and Short Blessed Test to detect cognitive dysfunction in geriatric patients presenting to the emergency department. BMJ Open. 2018;8:e019652. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [55].Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. Am J Psychiatry. 1983;140:734–9. [DOI] [PubMed] [Google Scholar]
  • [56].Davous P, Lamour Y, Debrand E, Rondot P. A comparative evaluation of the short orientation memory concentration test of cognitive impairment. J Neurol Neurosurg Psychiatry. 1987;50:1312–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [57].Carpenter CR, Bassett ER, Fischer GM, Shirshekan J, Galvin JE, Morris JC. Four sensitive screening tools to detect cognitive dysfunction in geriatric emergency department patients: brief Alzheimer’s Screen, Short Blessed Test, Ottawa 3DY, and the caregiver-completed AD8. Acad Emerg Med 2011;18:374–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [58].O’Sullivan D, O’Regan NA, Timmons S. Validity and Reliability of the 6-Item Cognitive Impairment Test for Screening Cognitive Impairment: A Review. Dement Geriatr Cogn Disord 2016;42:42–9. [DOI] [PubMed] [Google Scholar]
  • [59].Hessler J, Bronner M, Etgen T, Ander KH, Forstl H, Poppert H, et al. Suitability of the 6CIT as a screening test for dementia in primary care patients. Aging & Mental Health. 2014;18:515–20. [DOI] [PubMed] [Google Scholar]
  • [60].Galvin JE. Dementia Screening. Grand Round at the Alzheimer’s Disease Research Center at Washington University in Saint Louis. St. Louis, MO; 2010. [Google Scholar]
  • [61].Welsh KA, Breitner JC, Magruder-Habib KM. Detection of dementia in the elderly using telephone screening of cognitive status. Neuropsychiatry, Neuropsychology, & Behavioral Neurology. 1993;6:103–10. [Google Scholar]
  • [62].Cook SE, Marsiske M, McCoy KJ. The use of the Modified Telephone Interview for Cognitive Status (TICS-M) in the detection of amnestic mild cognitive impairment. J Geriatr Psychiatry Neurol 2009;22:103–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [63].Brandt J, Spencer M, Folstein MF. The telephone interview for cognitive status. Neuropsychiatry Neuropsychology Behavioral Neurology. 1988;1:111–7. [Google Scholar]
  • [64].Leung JM, Tsai TL, Sands LP. Preoperative frailty in older surgical patients is associated with early postoperative delirium. Anesth Analg 2011;112:1199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [65].Inouye SK, Robison JT, Froehlich TE, Richardson ED. The time and change test: a simple screening test for dementia. J Gerontol A Biol Sci Med Sci 1998;53:M281–6. [DOI] [PubMed] [Google Scholar]
  • [66].Froehlich TE, Robison JT, Inouye SK. Screening for dementia in the outpatient setting: the time and change test. J Am Geriatr Soc 1998;46:1506–11. [DOI] [PubMed] [Google Scholar]
  • [67].Mayo AM, Wallhagen M, Cooper BA, Mehta K, Ross L, Miller B. The relationship between functional status and judgment/problem solving among individuals with dementia. Int J Geriatr Psychiatry. 2013;28:514–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [68].Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9:179–86. [PubMed] [Google Scholar]
  • [69].Katz S, Akpom CA. 12. Index of ADL. Med Care. 1976;14:116–8. [DOI] [PubMed] [Google Scholar]
  • [70].Katz S, Downs TD, Cash HR, Grotz RC. Progress in development of the index of ADL. Gerontologist. 1970;10:20–30. [DOI] [PubMed] [Google Scholar]
  • [71].Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of Illness in the Aged. The Index of Adl: A Standardized Measure of Biological and Psychosocial Function. JAMA 1963;185:914–9. [DOI] [PubMed] [Google Scholar]

RESOURCES