Abstract
BACKGROUND
Executive function deficits are a main cause of participation restrictions post-traumatic brain injury (TBI). Assessing executive functions through actual daily participation may provide valuable information for treatment planning and progress.
AIM
This study aimed to validate the Assessment of Participation and Executive Functions (A-PEX), a tool for evaluating executive function deficits through actual participation in the inpatient rehabilitation context during the subacute phase following TBI.
DESIGN
A cross-sectional with a longitudinal component.
SETTING
Inpatient rehabilitation facility.
POPULATION
This study included 56 participants divided into two groups: 30 with orthopedic or spinal cord injuries and 26 with TBI.
METHODS
Internal consistency was evaluated by Cronbach’s alpha, and test-retest reliability was assessed using interclass correlation coefficients. Known-group construct validity was examined by comparing the A-PEX scores between the two groups, and A-PEX convergent construct validity for patients with TBI was examined using correlations between scores on the A-PEX, Multiple Errands Test-hospital version (MET-HV), and Color Trail Test (CTT).
RESULTS
Cronbach’s alpha coefficients for the A-PEX domains ranged between 0.83 and 0.96, indicating good-to-excellent internal consistency. Interclass correlations calculated for the control group indicated moderate test-retest reliability for most A-PEX components. Participants with TBI scored significantly lower than those with orthopedic or spinal cord injury for all A-PEX components (P<0.001). Within the TBI group, significant moderate-to-strong correlations were found between all A-PEX components and the MET-HV (0.52<r<0.73, P<0.05) and between the A-PEX executive function scales and the CTT (r=0.52, P<0.05).
CONCLUSIONS
The findings provide evidence for the initial reliability and validity of the A-PEX as a measure of post-TBI executive functions.
CLINICAL REHABILITATION IMPACT
By evaluating actual everyday participation, the A-PEX provides valuable clinical insight into the interrelationship between executive functions and participation in the post-TBI subacute phase.
Key words: Executive function, Traumatic brain injury, Rehabilitation, Inpatients
Traumatic brain injury (TBI) is a public health problem commonly resulting in lifelong disability.1-7 Executive function (EF) deficits are a main symptom and a reason for long-term disability following TBI. The principal EF deficits following TBI include problems with monitoring, initiation, planning, organization, cognitive flexibility, and working memory.8, 9 These deficits affect a broader range of everyday activities than other cognitive deficits10 and can restrict participation long after onset, profoundly reducing quality of life.2, 11 Effective, accurate assessment of EF deficits, as reflected in their role in participation restrictions, can play a significant part in achieving successful outcomes of rehabilitation following TBI.
The traditionally gold-standard neuropsychological assessment tools for evaluating EF deficits12 have been criticized for their limited ecological validity.13-15 In response, performance-based assessments, such as the Multiple Errands Test (MET), its variants,15, 16 and the Executive Function Performance Test,17 were developed. These assessments use tasks simulating everyday functions, such as shopping, bill-paying, and cooking. Several shared properties in neuropsychological and performance-based assessments, such as standardized administration, clear and well-defined instructions, and patient awareness of being evaluated, limit their ability to capture the full effect of EF deficits on participation in everyday life.18 Because changing contexts affect EFs significantly, an assessment in a natural environment with minimal clinician control would better measure the manifestation of EF deficits in everyday performance and their impact on participation.19 The trend toward formal and structured EF assessment in a real-world context, where multiple variables are free to co-occur, is an important development.2, 20, 21
The post-TBI subacute phase usually occurs in a specialized inpatient rehabilitation facility,22 where patients might remain for several weeks to months.23-25 The rehabilitation facility becomes the patients’ primary environment, and rehabilitation their central participation domain. During inpatient rehabilitation, patients engage in numerous leisure, social, and instrumental activities of daily living (IADL) during and outside treatment sessions. These activities are similar to community-based activities, albeit with different scales and complexities (e.g., managing treatment schedules, making appointments with the physician, maintaining relationships with staff and fellow patients). We introduce the term “inpatient participation” to describe this multidimensional participation comprised of open-ended, meaningful activities in the rehabilitation facility and argue for an interrelationship between EFs and inpatient participation. Inpatient participation could play a meaningful role in rehabilitation, measure EF deficits, indicate patients’ progress, and predict their community-based participation level post-discharge.
This paper presents a new assessment tool, the Assessment of Participation and Executive Functions (A-PEX). We designed the A-PEX to comprehensively evaluate EF components through participation, specifically in leisure, social, and IADL in an inpatient rehabilitation context. The A-PEX evaluates EF manifested in actual participation in everyday life situations relevant to each patient’s condition and context. This tool may provide new opportunities for planning treatment, measuring progress, and predicting rehabilitation outcomes.21, 26
This study’s overall objective is to examine the A-PEX psychometric properties in adults in an inpatient rehabilitation facility during the subacute phase following TBI, specifically: 1) internal consistency; 2) test-retest reliability; 3) known-group construct validity; 4) convergent construct validity. We hypothesized that performance on the A-PEX would differ significantly between inpatients following TBI and inpatients with no documented brain injury and that convergent validity with validated EF measures would achieve moderate-to-high correlations.
Materials and methods
Study design
This cross-sectional with a longitudinal component study was conducted in an inpatient rehabilitation medical center. Data collection took place between March 2020 and August 2022. The Loewenstein Rehabilitation Center Review Board (#0028-18-LOE) and the University of Haifa Ethics Committee (#161/19) approved the study protocol. All participants provided written consent prior to participation.
Participants
The study sample comprised 56 participants hospitalized in an inpatient rehabilitation medical center, divided into two groups. The TBI group included 26 participants; the control group included 30 participants, none with evidence of brain injury (20 had orthopedic injuries, and 10 had spinal cord injuries). Inclusion criteria for both groups were ages 18 to 70 years, able to understand the assessment tool instructions, and had at least one functional upper extremity, as determined by their occupational therapist, and intact or corrected vision. Further, control group participants were included if they scored 21/30 or more on the Montreal Cognitive Assessment (MoCA).27 The TBI group participants were included if a physician had diagnosed them with moderate-to-severe TBI confirmed by imaging (per their medical records) and they had preserved basic cognitive abilities, as determined by three Loewenstein Occupational Therapy Cognitive Assessment domains: orientation (scoring 6/8 or above), visual perception (3/4 or above), and spatial perception (2/4 or above).28 Participants in either group with a history of epilepsy, drug use, or psychiatric or neurologic disorders affecting cognitive function were excluded.
Procedure
We administered the A-PEX (twice, one month apart) and MoCA to all participants and the MET-hospital version (MET-HV) and Color Trail Test (CTT) to the TBI group. Although the A-PEX was readministered one month later, we analyzed only the TBI group’s first assessment in this study. Demographic information, medical data, and functional measure scores for the Functional Independence Measure (FIM) and Spinal Cord Independence Measure (SCIM) were obtained from medical records.
Measures
Assessment of participation and executive functions
The A-PEX was designed to comprehensively and systematically evaluate EF deficits reflected in inpatient participation, mainly leisure, social, and IADL for adults following TBI hospitalized in an inpatient rehabilitation facility. Scoring is based on clinical observations over a span of up to 2 weeks of the patient’s actual participation in the rehabilitation facility and semistructured interviews with the patient or primary caregiver. The A-PEX includes six functional domains: 1) treatment-schedule management (Supplementary Digital Material 1: Supplementary Text File 1); 2) medical and social-status management; 3) financial management; 4) cellphone use; 5) social interaction; 6) leisure. Each functional domain comprises several items for a total of 67 in the whole assessment. Fifty-one of these are relevant during the early stages of inpatient rehabilitation and were analyzed for this study. Each item is scored for consistency and efficiency on a scale from 1 (does not perform/ineffective performance) to 3 (performs consistently/effective performance). For example, a participant who arrived at treatment on time every day but used trial and error (went to the gym, realized he or she was in the wrong place, and then went to the therapy room) would receive a score of 3 for consistency and 2 for efficiency. Consistency and efficiency scores are then multiplied for each item, and a mean score of all items per functional domain is calculated. Final scores range from 1 to 9. Higher scores indicate better performance.
The A-PEX includes two overall EF scales, scored separately, for key EF components influencing participation: the EF-cognitive scale (e.g., monitoring, planning, initiation) (Supplementary Digital Material 1) and EF-behavioral scale (e.g., impulsivity, apathy, aggressiveness). Each component is rated on a scale between 1 (always) and 5 (never) based on its effect on inpatient participation. A mean score is then calculated for each EF scale, rendering a final score between 1 (severe EF deficits) and 5 (no EF deficits).
The occupational therapists who scored the A-PEX were experienced clinicians in cognitive rehabilitation during inpatient rehabilitation. In addition to receiving training on administration and scoring, clinicians were encouraged to consult the first author with any questions they might have during scoring. Furthermore, the A-PEX manual, which includes detailed administration instructions, scoring criteria, descriptions, and examples for each item, was provided. The A-PEX scoring process is usually completed within 30 to 45 min; the time gradually decreases as the clinician becomes more familiar with the patient.
The A-PEX content validity was established based on the judgment of 12 occupational therapists, all active researchers or clinicians specializing in cognitive rehabilitation. High agreement levels (75.0-91.7%) were obtained regarding the A-PEX functional domains and EF scales. Initial interrater reliability was established with two experienced occupational therapists assessing four patients at two time points, two weeks apart, for a total of eight assessments. Assessors alternated in administering the A-PEX so that each administered it four times, while the other scored according to observations. The percentage of overall agreement ranged between 83.6% (κ=0.715, P<0.05) and 93.2% (κ=0.882, P<0.05), indicating moderate-to-strong agreement between the raters.
Multiple errands test-hospital version
The MET-HV is a performance-based assessment to evaluate the effect of EF deficits on everyday functioning in a hospital environment. It consists of different tasks, such as running errands or obtaining information in a hospital setting. Scores range between 0 and 36; lower scores indicate better performance. Strategy use during the assessment is scored separately and ranges from 0 to 17; higher scores indicate more strategies used. Interrater reliability was good, and internal consistency was satisfactory. Criterion validity was established, and significant differences were found between healthy adults and adults with acquired brain injury.29
Color Trail Test
Designed to allow broader cross-cultural application, the CTT is a language-free version of the Trail-Making Test (TMT). The TMT was developed to test processing speed, sequence alternation, cognitive flexibility, visual search, and executive functioning.30 The CTT comprises two tasks. In CTT1, the participant connects circles in an ascending numbered sequence (1 to 25); in CTT2, the participant connects those numbers while alternating between two colors. Scores are generated by converting the time required to complete the task to a standard t score adjusted for age and education. Higher scores indicate better performance. Test-retest reliability was adequate for the CTT1 and excellent for the CTT2.31 The CTT’s concurrent validity with the TMT was excellent.32
Functional measures to describe sample characteristics
Functional independence measure
The FIM is commonly used in TBI and orthopedic departments to assess basic activities of daily living in inpatient rehabilitation programs. It comprises 18 items (13 motor and five cognitive). Scores range from 1 to 7 for each item; final (sum of items) scores range from 18 to 126; higher scores indicate more independence.33 Research has found the FIM to be a valid measure of functional independence for individuals with TBI.34
Spinal cord independence measure
The SCIM was designed to assess patients’ daily function in self-care, respiration and sphincter management, and mobility following spinal cord injury. Final scores range between 0 and 100; higher scores indicate better daily function.35 The SCIM has been found reliable and valid in assessing daily function following spinal cord injury.35, 36
Screening measures
Montreal Cognitive Assessment
The MoCA was designed as a cognitive screening test to detect mild cognitive impairment. It evaluates eight cognitive domains, such as attention, memory, orientation, and EFs. Scores range from 0 to 30; higher scores indicate better performance. The MoCA has discriminant validity between healthy adults and adults with mild cognitive impairment and has good internal consistency.27 Cut-off scores of 23/30 or below may optimize specificity and sensitivity in detecting cognitive impairment.37
Loewenstein Occupational Therapy Cognitive Assessment
The Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) was designed to provide a preliminary cognitive profile for patients with acquired brain injury.38 It comprises six cognitive domains: 1) orientation; 2) visual perception; 3) spatial perception; 4) praxis; 5) visuomotor construction; 6) thinking operations. We used orientation and visual and spatial perception to screen for the TBI group. The LOTCA’s psychometric properties have been established in various populations.28
Statistical analysis
We evaluated the cognitive assessment distribution for normality using the Shapiro-Wilk Test.39 Because the scores were not normally distributed, we used nonparametric tests for statistical analysis. Participants’ characteristics in both groups were summarized using median, interquartile range, or percentages. Sample size was calculated using G*Power software for between groups comparisons; one tail, with medium-large effect size (d=0.65), α=0.05 and power = 0.80. The sample size needed was n=30 in each group. Due to COVID-19 the sample size of the TBI group was smaller. Therefore, we calculated the study’s power post hoc, based on the between groups effect size, rendering an acceptable power of 0.82.
To assess the A-PEX’s internal consistency, we used Cronbach’s alpha for the functional domains and EF scales separately, and associations between the six functional domains and the two EF scales were examined using Spearman correlation coefficients. To assess the A-PEX test-retest reliability, we used interclass correlations (ICC) for each domain and scale. Correlation values were considered excellent (greater than 0.90), good (0.75-0.90), moderate (0.50-0.75), and poor (less than 0.50).40
The A-PEX known-group construct validity was examined using Mann-Whitney U and Pearson’s Chi-Square tests. To understand the extent of the between-group differences, we calculated effect size using Cohen’s r=Z/√N for nonparametric tests (with Z from the Mann-Whitney U-Test). We interpreted the Cohen’s r values as large (if over 0.50), medium (0.50-0.30), or small (<0.30) effects.41 The A-PEX’s convergent construct validity for the TBI group was evaluated by Spearman correlation coefficients between the A-PEX, MET-HV, and CTT scores. Statistically significant strong (r≥0.70) to moderate (r≥0.40) correlations were considered evidence of convergent construct validity of the A-PEX.42
Data availability
Data associated with the paper are available from the first author on reasonable request.
Results
Participant characteristics
Table I presents demographic and injury-related characteristics for both groups. In total, 33 patients following TBI were approached to participate in this study, seven declined, leaving 26 eligible participants. In the control group, 31 were approached, one of them declined to participate, leaving 30 eligible participants. Three participants in the control group did not complete the second assessment, because they were discharged early without notice. Significant between-group differences were found for the number of days from admission to rehabilitation, number of days since admission to time of evaluation, mobility, FIM cognitive scale, and MoCA score, but none in demographic data. The TBI group’s median MoCA score indicated that participants were exactly at the cut-off for cognitive impairment, whereas the median of the control group was in the normal range. More of the control group (83.3%) than the TBI group (61.5%) used wheelchairs for mobility. In the control group, 10 participants had spinal cord injuries and a median SCIM score of 56.5 (interquartile range 44.7-68.5).
Table I. —Participant demographics and characterization.
| Demographics | Median (interquartile range) | P value | |
|---|---|---|---|
| TBI group (N.=26) |
Control group (N.=30) |
||
| Age (years) | 33.0 (24.0-50.5) | 38.5 (30.5-50.2) | 0.320 |
| Education (years) | 12.0 (12.0-16.0) | 12.0 (12.0-16.0) | 0.965 |
| FIM* | |||
| FIM motor (13-91) | 61.0 (51.2-72.2) | 61.0 (57.0-65.5) | 1.00 |
| FIM cognitive (5-35) | 25.5 (22.7-28.2) | 35.0 (35.0-35.0) | >0.001 |
| FIM total (18-126) | 84.5 (74.0-97.5) | 95.5 (92.0-100.5) | 0.010 |
| MoCA | 23.0 (21.0-25.25) | 26.0 (23.75-28.0) | 0.007 |
| Days since injury | 38.0 (29.0-55.8) | 28.5 (19.0-55.5) | 0.200 |
| Days since admission | 12.0 (8.0-17.3) | 7.5 (5.8-11.3) | 0.005 |
| Sex | 0.305 | ||
| Male | 22 (84.6%) | 22 (73.3%) | |
| Female | 4 (15.4%) | 8 (26.7%) | |
| Marital status | 0.990 | ||
| Single | 11 (42.3%) | 12 (40.0%) | |
| Married | 12 (46.2%) | 14 (46.7%) | |
| Divorced | 2 (7.7%) | 3 (10.0%) | |
| Separated | 1 (3.8%) | 1 (3.3%) | |
| Cause of injury | 0.274 | ||
| Traffic accident | 16 (61.5%) | 14 (46.7%) | |
| Fall | 2 (7.7%) | 5 (16.7%) | |
| Gunshot | 1 (3.8%) | - | |
| Other† | 7 (26.9%) | 11 (36.7%) | |
| Mobility | 0.034 | ||
| Wheelchair | 16 (61.5%) | 25 (83.3%) | |
| Walking aid | 5 (19.2%) | 4 (10.3%) | |
| Walking independently | 5 (19.2%) | 1 (3.3%) | |
Median and interquartile range for continuous values; number of cases and percentage for nominal variables. FIM: Functional Independence Measure; MoCA: Montreal Cognitive Assessment. *Data were not available for 10 participants with spinal cord injuries since the Spinal Cord Independence Measure was administered; † includes injuries due to violence, sports, work, or illness.
Internal consistency
Cronbach’s alpha coefficients for the functional domains and EF scales ranged between 0.83 and 0.96, indicating excellent internal consistency (Table II). In general, removing an individual item did not improve the internal consistency in any domain. For both EF scales, internal consistency was excellent, with no improvement after removing any item. Spearman correlations among the six functional domains ranged from 0.56 to 0.82 (P<0.001), indicating moderate-to-strong correlations. A strong correlation was found between the EF scales (r=0.90, P<0.001). Correlations between the functional domains and EF scales ranged between 0.53 and 0.86 (P<0.001), indicating moderate-to-strong correlations (Supplementary Digital Material 2: Supplementary Table I).
Table II. —Internal consistency for the Assessment of Participation and Executive Functions (A-PEX) functional domains and EF scales.
| A-PEX | N. of items | α |
|---|---|---|
| Treatment-schedule management | 9 | 0.96 |
| Medical and social-status management | 5 | 0.83 |
| Financial management | 5 | 0.92 |
| Cell phone use | 15 | 0.93 |
| Social interaction | 12 | 0.90 |
| Leisure | 5 | 0.87 |
| EF-cognitive scale | 10 | 0.98 |
| EF-behavioral scale | 14 | 0.93 |
EF: executive function.
Test-retest reliability
The ICC for the control group indicated moderate test-retest reliability for most functional domains (Table III). Treatment-schedule and financial management received the highest ICC values (0.71 and 0.66, respectively), whereas leisure received the lowest ICC value (0.20). The ICC values of 0.87 for the EF-behavioral scale and 0.71 for the EF-cognitive scale indicated moderate-to-good test-retest reliability.
Table III. —Test-retest reliability for the Assessment of Participation and Executive Functions (A-PEX) functional domains and EF scales.
| A-PEX | ICC | 95% CI |
|---|---|---|
| Treatment-schedule management | 0.71 | 0.356-0.866 |
| Medical and social-status management | 0.56 | 0.039-0.800 |
| Financial management | 0.66 | 0.257-0.846 |
| Cell phone use | 0.61 | 0.144-0.822 |
| Social interaction | 0.49 | -0.131-0.765 |
| Leisure | 0.20 | -0.747-0.637 |
| EF-cognitive scale | 0.71 | 0.359-0.867 |
| EF-behavioral scale | 0.87 | 0.710-0.940 |
EF: executive function; ICC: interclass correlation; CI: confidence interval.
Known-group validity
Table IV presents the Mann-Whitney U-Test results indicating that participants following TBI scored significantly lower than those with orthopedic or spinal cord injury in all functional domains and both A-PEX EF scales (P<0.001). The effect sizes were large for the treatment-schedule management, financial management, social interaction, and leisure domains and medium for medical and social-status management and cell phone use. The EF scales’ effect sizes were large.
Table IV. —Mann-Whitney U-Test comparing the Assessment of Participation and Executive Functions (A-PEX) median scores by group.
| A-PEX | Mean (range) | z | P value | Cohen’s r | |
|---|---|---|---|---|---|
| TBI group (N.=26) | Control group (N.=30) | ||||
| Treatment-schedule management | 6.4 (2.4-8.6) | 9.0 (9.0-9.0) | -4.9 | <0.001 | 0.65 |
| Medical and social-status management | 4.2 (3.1-8.1) | 8.1 (6.2-9.0) | -3.4 | 0.001 | 0.45 |
| Financial management | 2.7 (1.0-6.7) | 9.0 (7.5-9.0) | -4.4 | <0.001 | 0.58 |
| Cell phone use | 7.8 (4.8-9.0) | 9.0 (8.2-9.0) | -3.5 | <0.001 | 0.46 |
| Social interaction | 6.0 (4.4-7.3) | 9.0 (8.1-9.0) | -5.9 | <0.001 | 0.78 |
| Leisure | 3.7 (2.5-6.7) | 9.0 (9.0-9.0) | -5.6 | <0.001 | 0.71 |
| EF-cognitive scale | 2.8 (2.1-3.5) | 5.0 (5.0-5.0) | -6.4 | <0.001 | 0.85 |
| EF-behavioral scale | 4.3 (3.5-4.6) | 5.0 (4.9-5.0) | -5.9 | <0.001 | 0.78 |
Data are presented as median and interquartile range. EF: executive function.
Convergent construct validity (TBI group)
The median MET-HV score was 11.25 (IQR: 8.0-15.1). Participants used a median of eight strategies while performing the assessment (IQR: 4.7-10.0). Correlations between the A-PEX and MET-HV were moderate-to-strong, ranging from -0.52 to -0.73 (lower MET-HV scores indicate better performance). The median t score for the CTT1 was 34 (IQR: 20-47) and 37 (IQR: 20-50) for the CTT2. Moderate correlations between the A-PEX and CTT were found in the treatment-schedule management and cell phone use domains and the EF-cognitive scale (Table V).
Table V. —Spearman correlations between the Assessment of Participation and Executive Functions (A-PEX), Multiple Errands Test-Hospital Version (MET-HV), and Color Trail Test (CTT) for the group with traumatic brain injury (N.=26).
| A-PEX | MET-HV Final score |
MET-HV Strategy use |
CTT1 T score |
CTT2 T score |
||||
|---|---|---|---|---|---|---|---|---|
| r | P value | r | P value | r | P value | r | P value | |
| Treatment-schedule management | -0.65** | <0.001 | 0.73** | <0.001 | 0.43* | 0.026 | 0.46* | 0.017 |
| Medical and social-status management | -0.57** | 0.002 | 0.48** | <0.012 | 0.09 | 0.650 | 0.08 | 0.664 |
| Financial management | -0.56** | 0.003 | 0.58** | 0.002 | 0.11 | 0.581 | 0.13 | 0.497 |
| Cell phone use | -0.69** | <0.001 | 0.63** | 0.002 | 0.39* | 0.046 | 0.40* | 0.040 |
| Social interaction | -0.71** | <0.001 | 0.67** | <0.001 | 0.18 | 0.370 | 0.21 | 0.291 |
| Leisure | -0.52** | 0.006 | 0.51** | 0.007 | 0.06 | 0.758 | 0.00 | 0.987 |
| EF-cognitive scale | -0.73** | <0.001 | 0.73** | <0.001 | 0.42* | 0.030 | 0.52** | 0.006 |
| EF-behavioral scale | -0.54** | 0.004 | 0.57** | 0.002 | 0.15 | 0.455 | 0.29 | 0.139 |
EF: executive function. *P<0.05; **P<0.01.
Additional analysis revealed statistically significant moderate correlations in the TBI group between the A-PEX functional domains and the MoCA, ranging between 0.57 and 0.69 (P<0.01). Better participation (higher scores) on the A-PEX was associated with higher scores on the MoCA. Moderate correlations were found between the A-PEX EF-cognitive scale and the MoCA (r=0.63, P<0.01). No correlations were found between the EF-behavioral scale and the MoCA (Table VI).
Table VI. —Spearman correlations between the Assessment of Participation and Executive Functions (A-PEX) and the Montreal Cognitive Assessment (MoCA) for the group with traumatic brain injury (N.=26).
| A-PEX | MoCA Final score |
|
|---|---|---|
| r | P value | |
| Treatment schedule management | 0.69* | <0.001 |
| Medical and social status management | 0.64* | <0.001 |
| Money management | 0.57* | 0.002 |
| Cell phone use | 0.69* | <0.001 |
| Social interaction | 0.67* | <0.001 |
| Leisure | 0.67* | <0.001 |
| EF-cognitive scale | 0.63* | 0.001 |
| EF-behavioral scale | 0.33 | 0.094 |
EF: executive function. *P<0.01.
Discussion
The A-PEX was developed to evaluate EF deficits in adults following TBI through leisure, social, and IADL participation in the inpatient rehabilitation setting. The study’s findings indicate that the A-PEX has excellent internal consistency and moderate test-retest reliability, known-group validity, and convergent validity with a well-established neuropsychological assessment as well as a performance-based EF assessment.
The A-PEX’s internal consistency was excellent, suggesting that the items that compose the functional domains and EF scales contribute to each domain’s or scale’s final score. Additionally, the correlations between the functional domains and EF scales suggest an interrelationship between EF and participation in complex daily activities. These findings align with previous literature on the role of EFs in complex daily activities.8, 43 The two EF scales correlated at r=0.9, which raises the question of whether the two scales measure different constructs. Possibly, this high correlation is because we did not include TBI patients with behavioral deficits because some refused to participate and others were unable to cooperate with the study’s demands (i.e., an hour-long cognitive assessment with the measures used to validate the A-PEX). Clinically, the A-PEX could be used with patients with behavioral deficits, who may score differently on each EF scale.
The A-PEX test-retest scores for most functional domains and the two EF scales suggest stability over time, indicating that the clinical observations and semistructured interviews carried out with the A-PEX were designed satisfactorily. Accordingly, treatment-schedule management, medical and social-status management, financial management, and cell phone use, as well as the two EF scales, may be considered reliable for evaluating the efficacy of EF deficit interventions during inpatient rehabilitation in clinical settings and research contexts.
Treatment-schedule management is the most stable of the functional domains, and social interaction and participation in leisure activities are the most unstable. The reason may be that inpatient rehabilitation requires adapting to a new environment, expectations, and routines, especially during the first few weeks.44 Consequently, participation in leisure activities and social interaction, which initially may seem unimportant, is postponed. Conversely, participation in essential functional domains, such as daily schedule management, occurs as soon as possible in patients without EF deficits. Once patients adjust, they may be more capable and emotionally ready to partake in leisure activities. This distinction is valuable for clinicians’ considerations when constructing a treatment plan focused first on promoting treatment management and then on leisure activities.
The A-PEX discriminated between patients with and without brain injury in all functional domains and both EF scales. As the literature suggested, IADL require EF, such as planning and problem-solving, to adapt to novelty and changing environmental demands.43 Based on the control group’s shorter time since admission and greater mobility impairment compared to the TBI group — and similar environmental demands for both groups — the between-group differences most likely resulted from the TBI group’s EF deficits. The discrimination between patients with and without brain injury aligns with the literature on performance-based assessments.29, 45-47 Notably, those studies were conducted in the post-injury chronic phase, whereas this study provides evidence-based knowledge regarding EF deficits during the sub-acute phase.
Convergent validity was established by correlations between the A-PEX EF-cognitive scale and the CTT, indicating that the A-PEX is valid for assessing EF deficits. In contrast, most functional domains did not correlate with the CTT. This finding supports previous studies that found a limited connection between neuropsychological assessments and actual everyday function.13-15 Correlations between the A-PEX and MET-HV, which has evidence of ecological validity,29, 45-48 validate the A-PEX as an ecological assessment tool of EF and their manifestation in inpatient participation. In addition, the correlation between the A-PEX and the MoCA, a measure of global cognition, indicates that the A-PEX assesses not only EF but also other cognitive components. These results indicate that the A-PEX meets the literature’s previously suggested need for an EF assessment tool administered in a natural environment where multiple variables are free to co-occur.19, 21
An additional advantage of the A-PEX is how it is administered. Participants experienced fatigue and frustration during the formal administration of existing assessment tools. This noticeable strain, common during the early rehabilitation phases following a TBI, potentially interferes with the patients’ performance. In contrast, the A-PEX primarily quantifies the clinician’s knowledge based on clinical observations and a semistructured interview. Patients experienced no discomfort during its administration, making the A-PEX a suitable assessment tool in the early rehabilitation phases.
The A-PEX administration is conceptually different from existing EF measures in that it encourages the clinician to address and quantify the patient’s actual everyday participation systematically. Clinicians must adjust their thinking process regarding EF assessment procedures and become aware of subtleties in the patient’s inpatient participation. The clinician scoring the A-PEX spends about the same time as administering other EF measures. As the clinician becomes more familiar with the patient’s inpatient participation, the A-PEX scoring time may decrease. Interpreting the A-PEX results may be useful in tailoring treatments to each patient’s needs (e.g., a low efficiency score should be addressed differently in treatment than a low consistency score). The A-PEX also can facilitate structured treatment planning within clinical practice because its items are essentially inpatient participation activities graded in increasing complexity, which can be set as treatment goals.
Limitations of the study
The A-PEX is currently limited in terms of interrater reliability because COVID-19 restrictions at the time of the study prevented including more clinicians and patients as the process requires. It would benefit future research to establish interrater reliability more broadly. The results of the current study should be interpreted cautiously due to the small sample size in the TBI group that was recruited from a single rehabilitation center. Because the A-PEX evaluates actual participation in everyday life situations in the patient’s current context, it depends heavily on the setting; therefore, future research should recruit a larger number of participants from several facilities to establish validity across settings assuring generalizability of results. In addition, due to ethical guidelines, this study did not include low-functioning patients or patients with behavioral deficits following TBI; broadening the inclusion criteria to capture the heterogeneity of patients following TBI may benefit future research. This study did not address the clinicians’ experiences with the A-PEX. Future research could examine the clinicians’ experiences in administering and scoring the A-PEX and the possible effects on treatment planning. Finally, the test-retest interval was 1 month, longer than previously suggested for outcome measures.49, 50 Future research should examine the A-PEX’s test-retest reliability using a shorter interval.
Conclusions
This preliminary study’s results provide initial evidence of initial reliability, known-group validity, and convergent validity for the A-PEX as an assessment for EF deficits through the participation of inpatients following a TBI. Future studies should be conducted to prove reproducibility of the current results in various rehabilitation centers. The A-PEX does not unduly burden patients and its’ administration is feasible during early rehabilitation phases post-TBI. It provides clinically relevant information regarding EF that may support clinical considerations and help to define, individually tailored treatment goals and planning during inpatient rehabilitation. The information it generates regarding EF deficits and participation during the subacute phase may predict community and at-home participation, an aspect examined in a separate, ongoing study.
Supplementary Digital Material 1
Supplementary Text File 1
An example from the Assessment of Participation and Executive Functions (A-PEX)
Supplementary Digital Material 2
Supplementary Table I
Spearman correlations between the Assessment of Participation and Executive Functions (A-PEX) six functional domains and EF scales.
Footnotes
Conflicts of interest: The authors certify that there is no conflict of interest with any financial organization regarding the material discussed in the manuscript.
Funding: This work was supported by the Loewenstein Rehabilitation Center under grant number KM600010303. The authors report no involvement in the research by the sponsor that could have influenced the outcome of this work.
Congresses: Parts of this work were presented as posters at the Virtual International Society of Physical and Rehabilitation Medicine (ISPRM) on 12th-15th June 2021 and at the Virtual World Congress on Brain Injury on 5th-6th July 2021.
References
- 1.Brazinova A, Rehorcikova V, Taylor MS, Buckova V, Majdan M, Psota M, et al. Epidemiology of traumatic brain injury in Europe: a living systematic review. J Neurotrauma 2021;38:1411–40. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26537996&dopt=Abstract 10.1089/neu.2015.4126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.U.S. Centers for Disease Control and Prevention. Report to Congress on traumatic brain injury in the United States: epidemiology and rehabilitation. Atlanta: National Center for Injury Prevention and Control; Division of Unintentional Injury Prevention; 2015. [Google Scholar]
- 3.Jourdan C, Bayen E, Pradat-Diehl P, Ghout I, Darnoux E, Azerad S, et al. A comprehensive picture of 4-year outcome of severe brain injuries. Results from the PariS-TBI study. Ann Phys Rehabil Med 2016;59:100–6. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26704071&dopt=Abstract 10.1016/j.rehab.2015.10.009 [DOI] [PubMed] [Google Scholar]
- 4.Ma VY, Chan L, Carruthers KJ. Incidence, prevalence, costs, and impact on disability of common conditions requiring rehabilitation in the United States: stroke, spinal cord injury, traumatic brain injury, multiple sclerosis, osteoarthritis, rheumatoid arthritis, limb loss, and back pain. Arch Phys Med Rehabil 2014;95:986–995.e1. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=24462839&dopt=Abstract 10.1016/j.apmr.2013.10.032 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Maas AI, Stocchetti N, Bullock R. Moderate and severe traumatic brain injury in adults. Lancet Neurol 2008;7:728–41. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18635021&dopt=Abstract 10.1016/S1474-4422(08)70164-9 [DOI] [PubMed] [Google Scholar]
- 6.Peeters W, van den Brande R, Polinder S, Brazinova A, Steyerberg EW, Lingsma HF, et al. Epidemiology of traumatic brain injury in Europe. Acta Neurochir (Wien) 2015;157:1683–96. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26269030&dopt=Abstract 10.1007/s00701-015-2512-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Roozenbeek B, Maas AI, Menon DK. Changing patterns in the epidemiology of traumatic brain injury. Nat Rev Neurol 2013;9:231–6. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=23443846&dopt=Abstract 10.1038/nrneurol.2013.22 [DOI] [PubMed] [Google Scholar]
- 8.Rabinowitz AR, Levin HS. Cognitive sequelae of traumatic brain injury. Psychiatr Clin North Am 2014;37:1–11. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=24529420&dopt=Abstract 10.1016/j.psc.2013.11.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wood RL, Worthington A. Neurobehavioral abnormalities associated with executive dysfunction after traumatic brain injury. Front Behav Neurosci 2017;11:195. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=29123473&dopt=Abstract 10.3389/fnbeh.2017.00195 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Spitz G, Ponsford JL, Rudzki D, Maller JJ. Association between cognitive performance and functional outcome following traumatic brain injury: a longitudinal multilevel examination. Neuropsychology 2012;26:604–12. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=22799747&dopt=Abstract 10.1037/a0029239 [DOI] [PubMed] [Google Scholar]
- 11.Riggio S, Wong M. Neurobehavioral sequelae of traumatic brain injury. Mt Sinai J Med 2009;76:163–72. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19306386&dopt=Abstract https://doi.org/ 10.1002/msj.20097 [DOI] [PubMed] [Google Scholar]
- 12.Hahn B, Baum C, Moore J, Ehrlich-Jones L, Spoeri S, Doherty M, et al. Development of additional tasks for the executive function performance test. Am J Occup Ther 2014;68:e241–6. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25397771&dopt=Abstract 10.5014/ajot.2014.008565 [DOI] [PubMed] [Google Scholar]
- 13.Burgess PW, Alderman N, Forbes C, Costello A, Coates LM, Dawson DR, et al. The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology. J Int Neuropsychol Soc 2006;12:194–209. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=16573854&dopt=Abstract 10.1017/S1355617706060310 [DOI] [PubMed] [Google Scholar]
- 14.Dawson DR, Marcotte TD. Special issue on ecological validity and cognitive assessment. Neuropsychol Rehabil 2017;27:599–602. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28480821&dopt=Abstract 10.1080/09602011.2017.1313379 [DOI] [PubMed] [Google Scholar]
- 15.Shallice T, Burgess PW. Deficits in strategy application following frontal lobe damage in man. Brain 1991;114:727–41. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=2043945&dopt=Abstract 10.1093/brain/114.2.727 [DOI] [PubMed] [Google Scholar]
- 16.Alderman N, Burgess PW, Knight C, Henman C. Ecological validity of a simplified version of the multiple errands shopping test. J Int Neuropsychol Soc 2003;9:31–44. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12570356&dopt=Abstract 10.1017/S1355617703910046 [DOI] [PubMed] [Google Scholar]
- 17.Baum C, Morrison T, Hahn M, Edwards D. Executive function performance test: test protocol booklet. St. Louis: Washington University School of Medicine; 2003. [Google Scholar]
- 18.Chaytor N, Schmitter-Edgecombe M. The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol Rev 2003;13:181–97. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=15000225&dopt=Abstract 10.1023/B:NERV.0000009483.91468.fb [DOI] [PubMed] [Google Scholar]
- 19.Poncet F, Swaine B, Dutil E, Chevignard M, Pradat-Diehl P. How do assessments of activities of daily living address executive functions: A scoping review. Neuropsychol Rehabil 2017;27:618–66. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=28075219&dopt=Abstract 10.1080/09602011.2016.1268171 [DOI] [PubMed] [Google Scholar]
- 20.Chan RC, Shum D, Toulopoulou T, Chen EY. Assessment of executive functions: review of instruments and identification of critical issues. Arch Clin Neuropsychol 2008;23:201–16. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18096360&dopt=Abstract 10.1016/j.acn.2007.08.010 [DOI] [PubMed] [Google Scholar]
- 21.Lewis MW, Babbage DR, Leathem JM. Assessing executive performance during cognitive rehabilitation. Neuropsychol Rehabil 2011;21:145–63. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21229458&dopt=Abstract 10.1080/09602011.2010.543867 [DOI] [PubMed] [Google Scholar]
- 22.Silver JM, McAllister TW, Arciniegas DB, editors. Textbook of traumatic brain injury. Washington DC: American Psychiatric; 2018. [Google Scholar]
- 23.Andelic N, Bautz-Holter E, Ronning P, Olafsen K, Sigurdardottir S, Schanke AK, et al. Does an early onset and continuous chain of rehabilitation improve the long-term functional outcome of patients with severe traumatic brain injury? J Neurotrauma 2012;29:66–74. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=21864138&dopt=Abstract 10.1089/neu.2011.1811 [DOI] [PubMed] [Google Scholar]
- 24.Dahdah MN, Barisa MT, Schmidt K, Barnes SA, Dubiel R, Dunklin C, et al. Comparative effectiveness of traumatic brain injury rehabilitation: differential outcomes across TBI model systems centers. J Head Trauma Rehabil 2014;29:451–9. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=24052093&dopt=Abstract 10.1097/HTR.0b013e3182a61983 [DOI] [PubMed] [Google Scholar]
- 25.Seel RT, Barrett RS, Beaulieu CL, Ryser DK, Hammond FM, Cullen N, et al. Institutional variation in traumatic brain injury acute rehabilitation practice. Arch Phys Med Rehabil 2015;96(Suppl):S197–208. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26212397&dopt=Abstract 10.1016/j.apmr.2015.02.034 [DOI] [PubMed] [Google Scholar]
- 26.Flanagan SR, Cantor JB, Ashman TA. Traumatic brain injury: future assessment tools and treatment prospects. Neuropsychiatr Dis Treat 2008;4:877–92. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19183780&dopt=Abstract 10.2147/NDT.S1985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–9. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=15817019&dopt=Abstract 10.1111/j.1532-5415.2005.53221.x [DOI] [PubMed] [Google Scholar]
- 28.Katz N, Itzkovich M, Averbuch S, Elazar B. Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery for brain-injured patients: reliability and validity. Am J Occup Ther 1989;43:184–92. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=2735378&dopt=Abstract 10.5014/ajot.43.3.184 [DOI] [PubMed] [Google Scholar]
- 29.Knight C, Alderman N, Burgess PW. Development of a simplified version of the multiple errands test for use in hospital settings. Neuropsychol Rehabil 2002;12:231–55. 10.1080/09602010244000039 [DOI] [Google Scholar]
- 30.Lezak MD, Howieson DB, Loring DW, Fischer JS. Neuropsychological assessment. 3rd ed. New York: Oxford University Press; 2004. [Google Scholar]
- 31.D’Elia L, Satz P, Uchiyama CL, White T. Color Trails Test: CTT. Odessa FL: Psychological Assessment Resources; 1996. [Google Scholar]
- 32.Elkin-Frankston S, Lebowitz BK, Kapust LR, Hollis AM, O’Connor MG. The use of the Color Trails Test in the assessment of driver competence: preliminary report of a culture-fair instrument. Arch Clin Neuropsychol 2007;22:631–5. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=17481851&dopt=Abstract 10.1016/j.acn.2007.04.004 [DOI] [PubMed] [Google Scholar]
- 33.Donaghy S, Wass PJ. Interrater reliability of the Functional Assessment Measure in a brain injury rehabilitation program. Arch Phys Med Rehabil 1998;79:1231–6. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9779676&dopt=Abstract 10.1016/S0003-9993(98)90267-2 [DOI] [PubMed] [Google Scholar]
- 34.Corrigan JD, Smith-Knapp K, Granger CV. Validity of the functional independence measure for persons with traumatic brain injury. Arch Phys Med Rehabil 1997;78:828–34. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9344301&dopt=Abstract 10.1016/S0003-9993(97)90195-7 [DOI] [PubMed] [Google Scholar]
- 35.Catz A, Itzkovich M, Agranov E, Ring H, Tamir A. SCIM—spinal cord independence measure: a new disability scale for patients with spinal cord lesions. Spinal Cord 1997;35:850–6. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=9429264&dopt=Abstract 10.1038/sj.sc.3100504 [DOI] [PubMed] [Google Scholar]
- 36.Itzkovich M, Gelernter I, Biering-Sorensen F, Weeks C, Laramee MT, Craven BC, et al. The Spinal Cord Independence Measure (SCIM) version III: reliability and validity in a multi-center international study. Disabil Rehabil 2007;29:1926–33. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=17852230&dopt=Abstract 10.1080/09638280601046302 [DOI] [PubMed] [Google Scholar]
- 37.Luis CA, Keegan AP, Mullan M. Cross validation of the Montreal Cognitive Assessment in community dwelling older adults residing in the Southeastern US. Int J Geriatr Psychiatry 2009;24:197–201. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=18850670&dopt=Abstract 10.1002/gps.2101 [DOI] [PubMed] [Google Scholar]
- 38.Averbuch S, Katz N. Cognitive rehabilitation: A retraining model for clients with neurological disabilities. In: Katz N, editor. Cognition and occupation across the life span. 2nd ed. Bethesda MD: AOTA Press; 2005. p. 144–62. [Google Scholar]
- 39.Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples). Biometrika 1965;52:591–611. 10.1093/biomet/52.3-4.591 [DOI] [Google Scholar]
- 40.Portney LG, Watkins MP. Foundations of clinical research: applications to practice. Upper Saddle River, NJ: Pearson/Prentice Hall; 2009. [Google Scholar]
- 41.Coolican H. Research methods and statistics in psychology. London: Psychology Press; 2017. [Google Scholar]
- 42.Schober P, Boer C, Schwarte LA. Correlation coefficients: appropriate use and interpretation. Anesth Analg 2018;126:1763–8. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=29481436&dopt=Abstract 10.1213/ANE.0000000000002864 [DOI] [PubMed] [Google Scholar]
- 43.Giles GM, Clark-Wilson J, Baxter DM, Tasker R, Holloway M, Seymour S. The interrelationship of functional skills in individuals living in the community, following moderate to severe traumatic brain injury. Brain Inj 2019;33:129–36. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=30424682&dopt=Abstract 10.1080/02699052.2018.1539762 [DOI] [PubMed] [Google Scholar]
- 44.Merbitz NH, Westie K, Dammeyer JA, Butt L, Schneider J. After critical care: challenges in the transition to inpatient rehabilitation. Rehabil Psychol 2016;61:186–200. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=27196861&dopt=Abstract 10.1037/rep0000072 [DOI] [PubMed] [Google Scholar]
- 45.Clark AJ, Anderson ND, Nalder E, Arshad S, Dawson DR. Reliability and construct validity of a revised Baycrest Multiple Errands Test. Neuropsychol Rehabil 2017;27:667–84. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=26670475&dopt=Abstract 10.1080/09602011.2015.1117981 [DOI] [PubMed] [Google Scholar]
- 46.Dawson DR, Anderson ND, Burgess P, Cooper E, Krpan KM, Stuss DT. Further development of the Multiple Errands Test: standardized scoring, reliability, and ecological validity for the Baycrest version. Arch Phys Med Rehabil 2009;90(Suppl):S41–51. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=19892074&dopt=Abstract 10.1016/j.apmr.2009.07.012 [DOI] [PubMed] [Google Scholar]
- 47.Goverover Y, DeLuca J. Actual reality: using the Internet to assess everyday functioning after traumatic brain injury. Brain Inj 2015;29:715–21. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=25688468&dopt=Abstract 10.3109/02699052.2015.1004744 [DOI] [PubMed] [Google Scholar]
- 48.Maeir A, Krauss S, Katz N. Ecological validity of the Multiple Errands Test (MET) on discharge from neurorehabilitation hospital. OTJR (Thorofare, NJ) 2011;31:S38–46. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=24650263&dopt=Abstract https://doi.org/ 10.3928/15394492-20101108-07 [DOI] [PubMed] [Google Scholar]
- 49.Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol 2003;56:730–5. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=12954464&dopt=Abstract 10.1016/S0895-4356(03)00084-2 [DOI] [PubMed] [Google Scholar]
- 50.Park MS, Kang KJ, Jang SJ, Lee JY, Chang SJ. Evaluating test-retest reliability in patient-reported outcome measures for older people: A systematic review. Int J Nurs Stud 2018;79:58–69. https://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=29178977&dopt=Abstract 10.1016/j.ijnurstu.2017.11.003 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Text File 1
An example from the Assessment of Participation and Executive Functions (A-PEX)
Supplementary Table I
Spearman correlations between the Assessment of Participation and Executive Functions (A-PEX) six functional domains and EF scales.
Data Availability Statement
Data associated with the paper are available from the first author on reasonable request.