Questionnaire-Based Assessment of Executive Functioning: Part 2. Case Studies

Abstract

Delays in the development of executive functioning skills are frequently observed in pediatric neuropsychology populations and can have a broad and significant impact on quality of life. As a result, assessment of executive functioning is often relevant for the development of formulations and recommendations in pediatric neuropsychology clinical work. Questionnaire-based measures of executive functioning behaviors in everyday life have unique advantages and complement traditional neuropsychological measures of executive functioning. Two case studies of children with spina bifida are presented to illustrate the clinical use of a new questionnaire measure of executive and learning-related functioning, the Learning, Executive, and Attention Functioning Scale (LEAF). The LEAF emphasizes clinical utility in assessment by incorporating four characteristics: brevity in administration, breadth of additional relevant content, efficiency of scoring and interpretation, and ease of availability for use. LEAF results were consistent with another executive functioning checklist in documenting everyday behavior problems related to working memory, planning, and organization while offering additional breadth of assessment of domains such as attention, processing speed, and novel problem-solving. These case study results demonstrate the clinical utility of questionnaire-based measurement of executive functioning in pediatric neuropsychology and provide a new measure for accomplishing this goal.

Executive function (EF) is an umbrella construct describing a group of related neuropsychological processes responsible for self-regulation in the service of focused and goal-directed thinking and behavior (Barkley, 2011; Gioia, Isquith, Guy, & Kenworthy, 2000; Kronenberger, Pisoni, Henning, & Colson, 2013). Core EF processes include self-directed attention, working memory, response inhibition, cognitive flexibility, controlled attention under processing speed demands, and concentration, and are mediated by the prefrontal cortex, striatum, and other interconnected brain regions (Barkley, 2012; McAuley & White, 2011). Multiple factors influence the development of executive functioning skills during childhood, including genetics (Goldberg & Weinberger, 2004), biological influences such as injury and illness (Horton, Soper, & Reynolds, 2010), family environment (Holt, Beer, Kronenberger, Pisoni, & Lalonde, 2012), personal experience (Bernier, Carlson, & Whipple, 2010), and direct instruction (Diamond & Lee, 2011).

Delays and disturbances in EF have been reported in many pediatric conditions such as spina bifida (O’Hara & Holmbeck, 2013), cerebral palsy (Weierink, Vermeulen & Boyd, 2013), epilepsy (Parrish, Geary, Jones, Seth, Hermann, & Seidenberg, 2007), traumatic brain injury (Horton et al., 2010), cancer (Daly & Brown, 2007), hearing loss (Kronenberger et al., 2013), and attention deficit/hyperactivity disorder (ADHD; Barkley, 2012). Furthermore, executive functioning skills are often important in adaptive behavior and coping in response to pediatric conditions (Miller et al., 2013). Because of its central role in self-regulation and the direction and execution of planned behavior, executive functioning has a broad impact on learning, adjustment, and quality of life, and deficits in executive functioning can have pervasive and significant negative consequences (Barkley, 2012; Gathercole, Pickering, Knight, & Stegmann, 2003). Identifying EF delays and disturbances is therefore of considerable importance in clinical neuropsychology settings.

Executive functioning is most commonly assessed with office-based, neuropsychological tests of controlled attention, shifting, flexibility, inhibition, and planning (Barkley, 2012). While these tests have strong reliability and validity, they are related only modestly to behavioral measures of executive functioning in everyday life and can be costly and time-consuming to obtain and administer (Barkley, 2012). As a result, clinical assessment of executive functioning is common in more traditional neuropsychological settings and for more significant or severe neuropsychological conditions (e.g., traumatic brain injury) but may be neglected (or applied much more variably across programs and settings) in other pediatric populations (e.g., children with cancer, diabetes, etc.) or when the primary goal of assessment involves academic learning deficits or delays. Thus, a clinical need exists for measures of EF that are ecologically valid and relatively inexpensive, so that they can be applied in a larger and wider range of pediatric neuropsychology settings.

In order to address the need for brief EF measures based on everyday behavior, behavior checklists have been developed to specifically evaluate EF as rated by parents or teachers (Barkley, 2011; Gioia et al., 2000; Naglieri & Goldstein, 2013). These EF behavior checklists have shown good psychometrics as well as clinical utility in assessment of psychiatric and neuropsychological disorders (Gioia et al., 2000). Furthermore, use of EF behavior checklists in combination with traditional neuropsychological interview and assessments can provide a broader, multisource-multimethod assessment of EF encompassing both ability and behavioral adjustment in the environment.

In this paper, we present two case studies to illustrate the clinical application of a new behavior checklist measure of EF: the Learning, Executive, and Attention Functioning (LEAF) scale. The LEAF was designed to enhance the clinical assessment of EF in busy pediatric neuropsychology settings, particularly when learning and memory problems may be present. In addition, a secondary goal of this paper is to encourage and illustrate the use of EF behavior checklists in clinical practice as part of a multisource-multimethod evaluation of EF. In a companion paper in this journal (Castellanos, Pisoni, & Kronenberger, 2016), we provide psychometric results supporting the reliability and validity of the LEAF behavior checklist in a clinically-referred sample of children and adolescents. Other established behavioral checklists of EF are also very useful and are complementary to the LEAF, but differ from the LEAF in fundamental ways:

First, although the LEAF assesses core domains of EF (e.g., attention, working memory) that are also assessed by other EF behavioral checklists, the LEAF also measures some domains of EF that are not evaluated by other EF behavior checklists (e.g., processing speed under concentration-demanding conditions) as well as some closely related domains of cognitive-learning (e.g., concept formation, memory) and academic (e.g., mathematics, reading, writing) functioning that are dependent on but not central to EF. Second, the LEAF meets all of Levy, Kronenberger, & Dunn’s (2013) characteristics of a clinically useful behavior checklist: ease of administration, rapid scoring without special materials (templates, computer software), simple rules for interpretation (e.g., does not require reference to norms tables), and ease of access/availability for use (free from the authors). For example, each LEAF subscale has exactly five items grouped sequentially on the checklist, and items are rated on a 0 (never) – 3 (very often) severity scale; this allows for very rapid scoring by simply adding each sequential group of 5 items. Finally, the LEAF provides criterion-referenced scores, as opposed to norm-referenced scores. In contrast to norm-referenced scores, which provide information about the abnormality of rated behaviors (compared to the norm sample), criterion-referenced LEAF scores offer information about the degree to which EF-related behaviors cause problems in everyday functioning.

The case studies presented in this paper are the results of testing batteries for two children (a school-aged child and a teenager) with spina bifida. Our primary goal is to demonstrate the clinical use of the LEAF as an EF behavior checklist and not to characterize typical EF in all children with spina bifida. Spina bifida is a congenital malformation characterized by failure of the neural tube to close properly during fetal development, resulting in protrusion of spinal tissue at the area of the malformation. In the most common form of spina bifida, myelomeningocele, nervous tissue protrudes through the spine (Au, Ashley-Koch, & Northrup, 2010). As a result of this malformation, significant neurological conditions are often present in children with spina bifida myelomeningocele, including hydrocephalus, Chiari II malformation, midbrain abnormalities, dilated ventricles, and loss of cerebral gray matter (particularly in the posterior cerebral hemisphere) and white matter (particularly in the corpus callosum; Au et al., 2010; Fletcher & Dennis, 2010).

Spina bifida myelomeningocele is associated with a variety of cognitive risks (Fletcher & Dennis, 2010). A leading model of neurocognitive outcomes in children with spina bifida explains that deficits in assembled cognitive processes (organizing and integrating details of information to construct meaningful wholes) arise from the primary (and some secondary) neurological abnormalities caused by spina bifida, such as callosal dysgenesis and disruption of development of white matter (Fletcher & Dennis, 2010). Assembled cognitive processes include comprehension, inferring meaning from details, novel/nonverbal reasoning, and visual-spatial organization. Associated cognitive processes (formation of associations and fund of information), on the other hand, may or may not be affected in children with spina bifida, depending on environmental and learning influences (such as SES and parent-child interactions) and the severity and pervasiveness of secondary neurological effects (Fletcher & Dennis, 2010). Although some degree of deficits in assembled cognitive processes are commonly seen in children with spina bifida, considerable variability in assembled and associative process outcomes is present across the population of children with spina bifida, depending on neurological, environmental, and learning factors (Fletcher & Dennis, 2010).

The model of assembled and associated cognitive processing proposed by Fletcher and Dennis (2010) is one of the most comprehensive and well-supported models of neurocognitive outcomes associated with spina bifida. Other models of spina bifida neurocognitive outcomes overlap significantly with the model proposed by Fletcher and Dennis (2010), including nonverbal learning disability resulting from white matter deficiency (Rourke, 1995) and empirical findings of executive functioning and related deficits (especially deficits in the posterior attention networks; see Kelly et al., 2012), although there are some differences in these models (see Fletcher & Dennis, 2010).

Deficits in some domains of executive functioning are frequently reported in samples of children and adolescents with spina bifida, likely as a result of direct and indirect white and gray matter effects of the condition. Specifically, delays in controlled attention, working memory, focus, shifting, speed of processing concentration-based information, and other areas of executive functioning have been reported (Kelly et al., 2012; Rose & Holmbeck, 2007; Zabel et al., 2011). These deficits in executive functioning and related abilities, in turn, predict a variety of negative outcomes, including academic and social outcomes (Kelly et al., 2012; Rose & Holmbeck, 2007).

As a result of neurocognitive risks for executive functioning delays, as well as models suggesting deficits in areas that may influence or be influenced by executive functioning (such as assembled cognitive processing), assessment of executive functioning and related abilities is commonly warranted in children with spina bifida who are experiencing cognitive and academic challenges or delays. Therefore, spina bifida is an ideal condition to illustrate the potential utility of the LEAF in child neuropsychological assessment.

Methods

Participants

Two cases are presented of patients with spina bifida myelomeningocele: a 14 year old, home-schooled 7^th grader (Daniel) and a 7 year old, 1^st grader (Amy). Both children were referred because of concerns about delays in learning that might be related to deficits in EF. In order to address potential ethical concerns in the presentation of this case study data, some aspects of the cases (e.g., demographic information not relevant for interpretation) have been altered for confidentiality purposes. Additionally, the descriptions of clinical materials and results in this paper are sufficiently masked or global that neither protected test content nor identifiable patient-specific clinical results are divulged. Additional specific clinical characteristics of both cases are presented in the Results section.

Procedure

Daniel and Amy were seen for clinical interview and neuropsychological testing at our pediatric testing clinic. All assessments were completed in a single session and included tests of intelligence followed by tests of academic achievement. Amy’s mother and teacher completed questionnaire-based assessments of executive functioning. Because Daniel was homeschooled, only mother–completed questionnaire data was obtained. A licensed clinical psychologist (WGK) collected all data presented in these case studies.

Measures

In addition to the LEAF, other test scores are provided for the purpose of illustrating the integration of LEAF results with neuropsychology testing results. Descriptions of the cases focus primarily on clinical information relevant for understanding of LEAF and other EF behavior checklist results, as opposed to an in-depth review of all psychological testing results and formulations or a more general discussion of the neuropsychology or evaluation of executive functioning in children with spina bifida. Fletcher and Dennis (2010) provide an excellent description of a comprehensive model of neurocognitive development and outcomes in children and adolescents with spina bifida, and readers interested in a broader overview of the neuropsychology of spina bifida are referred to their work.

Learning, Executive, and Attention Functioning Scale (LEAF)

The LEAF is a 55 item parent- or teacher-completed rating scale of cognitive abilities broadly related to executive functioning that is applicable to children and adolescents between the ages of 6 – 17 years old. It consists of 2 Cognitive-Learning content areas (areas of learning and cognitive processing that are associated with executive functioning but that are not core EF domains: Comprehension and Conceptual Learning, and Factual Memory) and 6 Cognitive-EF content areas (core EF domains: Attention, Processing Speed, Visual-Spatial Organization, Sustained Sequential Processing, Working Memory, and Novel Problem Solving). Because children who display disturbances in executive functioning also frequently experience delays in academic functioning (Barkley, 2012; Gathercole, Pickering, Knight, & Stegmann, 2003), the LEAF scale also contains 3 Academic content areas (Mathematics Skills, Basic Reading Skills, and Written Expression Skills) to measure at-risk areas for poor academic achievement. Five test items were developed for each of the Cognitive-Learning, Cognitive-EF, and Academic content areas. See Table 1 for a description of the LEAF content areas.

Table 1.

Description of LEAF Content Areas

LEAF Subscale	Measures
Cognitive-Learning:
1. Comprehension and Conceptual Learning	Understanding, tracking, and comprehending new learning information and/or information in spoken or written form; getting the main idea of information
2. Factual Memory	Memorization and retention of facts and details, particularly in learning settings
Cognitive-EF:
3. Attention	Poor focus/attention; distractible
4. Processing Speed	Speed of completing work under concentration conditions; slow to start or finish tasks
5. Visual-Spatial Organization	Organization, messiness, visual-construction and visual-motor skills; attention to visual detail
6. Sustained Sequential Processing	Planning, following, and completing multistep sequences or directions
7. Working Memory	Overwhelmed by volume of information, can only do one thing at a time, forgets or loses track if required to remember something and engage in other mental activity simultaneously
8. Novel Problem-Solving	Difficulty independently learning or managing new, unfamiliar, or different material or situations.
Academic:
9. Mathematics Skills	Difficulty or dysfluency in mathematics; calculation weaknesses
10. Basic Reading Skills	Difficulty or dysfluency in reading, phonics, and word recognition
11. Written Expression Skills	Limited/impoverished written expression; errors in expressive writing; written expression is slow/effortful

Note. Content areas correspond to LEAF subscales sequentially, with five test items per subscale (e.g., Comprehension and Conceptual Learning = Items 1 – 5; Factual Memory = Items 6 – 10; Attention = Items 11 – 15, etc.). Higher scores on all LEAF subscales indicate greater problems.

Child behavior is rated on a 0 to 3 point scale ranging from 0 (never: not a problem; average for age) to 3 (very often: major daily problem). Scores of 0 and 1 are anchored to ratings indicating that they do not reflect problems, whereas scores of 2 and 3 are anchored to ratings descriptive of everyday functional problems. Higher raw scores on LEAF items and subscales (subscale raw scores are sums of the five item scores and range from 0 to 15) indicate greater problems in executive functioning and learning. Psychometric analyses of the LEAF in a large clinical population with a diverse set of medical and neuropsychological diagnoses (see ***removed for blind review***) have shown strong internal consistency (all Cronbach’s alpha values = 0.69 or higher; for 8 subscales, alpha > 0.85) and moderate test-retest reliability (r = 0.50 or greater for 7 subscales, 0.40 or greater for 3 subscales, and 0.34 for the Mathematics subscale) of all subscales. Subscales on the LEAF assessing core EF and related cognitive learning domains were compared with subscales on the Behavior Rating Inventory of Executive Functioning (BRIEF; Gioia et al., 2000), while subscales assessing academic domains were compared with performance-based scores on the Woodcock-Johnson III Tests of Achievement, Third Edition-Normative Update (WJ-III-NU; McGrew, Schrank, & Woodcock, 2007). LEAF subscale scores correlated strongly and significantly with corresponding BRIEF subscales (all r = 0.60 or greater) or WJ-III-NU scores (r = 0.48 or greater), and correlations with traditional individually-administered neuropsychological measures of EF were statistically significant in most cases. Importantly, LEAF subscales correlated more strongly with corresponding BRIEF or WJ-III-NU subscales/subtests (e.g., those reflecting the same domain of functioning; convergent validity) than with non-corresponding subscales/subtests. Furthermore, LEAF subscale scores with no corresponding subscale on the BRIEF were much less highly correlated with BRIEF scores. Additional psychometrics of the LEAF are reported in ***removed for blind review***.

For interpretation purposes, LEAF subscale raw scores are divided into three criterion-referenced ranges. Scores above 10 fall in the “Problem Range” because an average rating of “2” per item (“2” is anchored to a problem rating) is required to obtain a score of 10 or higher (e.g., 5 items per subscale yielding a total raw score of 10). Scores of 5 – 9 fall in the “Borderline Problem Range,” while scores below 5 fall in the “No Problem Range” because an average rating of less than “1” per item (“1” is anchored to a no problem rating) would yield a total raw score of 4 or less (see ***removed for blind review***, this issue, for an extensive discussion of and rationale for LEAF interpretive ranges). Support for these criterion-referenced ranges was demonstrated by ***removed for blind review***, who demonstrated that children scoring in the “No Problem Range” on the LEAF have mean BRIEF and WJ-III-NU scores in the middle of the Average range relative to representative normative samples, whereas those scoring in the “Problem Range” have scores 1 – 2 SD discrepant from the normative mean (in the direction of having problems; approximately 2 SD above the mean on corresponding BRIEF subscales and approximately 1 SD below the mean on corresponding WJ-III-NU subtests). Children scoring in the “Borderline Problem Range” scored between these extremes.

Neuropsychological Measures

Cognitive ability was assessed with the Kaufman Brief Intelligence Test, Second Edition (KBIT-2; Kaufman & Kaufman, 2004), and either the Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV; Wechsler et al., 2004) or the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999). The K-BIT-2 is a brief measure of intellectual ability that yields two scores: Verbal IQ (word knowledge and fund of information) and Nonverbal IQ (nonverbal reasoning and perceptual organization). The WISC-IV is a well-established, extensive measure of intellectual ability comprised of multiple subtests that yield four composite index scores: Verbal Comprehension (word knowledge, fund of information, verbal concept formation), Perceptual Reasoning (visual-organizational skills, nonverbal reasoning, and nonverbal concept formation), Working Memory (short-term verbal memory for rote-sequential information and short-term verbal memory for information concurrent with other mental operations), and Processing Speed (speed of completion of visual-perceptual and visual association/reproduction tasks). The WASI is a brief measure of intellectual ability comprised of four subtests that are identical (other than in item content) to subtests from the WISC-IV. WASI results provide Verbal (vocabulary and verbal concept formation) and Performance (visual-organizational skills and nonverbal reasoning) IQ scores.

Academic achievement was assessed with the Broad Reading, Broad Math, and Broad Written Language scores of the WJ-III-NU (McGrew, Schrank, & Woodcock, 2007). The WJ-III-NU is a widely used and well-validated broad-based academic achievement test that yields age-based standard scores in each composite area. The Broad Reading score is derived from subscales measuring word recognition, sentence reading fluency, and reading comprehension skills; the Broad Math score reflects arithmetic calculation, math fact fluency, and applied mathematics skills; and the Broad Written Language score includes components of spelling, writing fluency, and written expression. Standard scores based on nationally representative norm samples were used for all WISC-IV, WASI, K-BIT-2, and WJ-III-NU index and composite scores.

The BRIEF (Gioia et al., 2000) was also completed as an additional parent-rated questionnaire measure of EF problems in 8 domains/subscales: Inhibit, Shift, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials, and Monitor. Higher raw LEAF scores and BRIEF subscale T-scores (obtained based on comparison to a norm sample) indicate greater difficulty in EF and related learning skills.

Results

Case 1: Daniel

Daniel was a 14 year old, home-schooled 7^th grader who was referred because of academic and learning delays, problems with attention and concentration, and slow completion of academic work. Daniel often became overwhelmed when he was presented with long or complex information-processing tasks and activities in either a verbal or visual modality, leading to problems with organization, comprehension, reasoning, and novel problem solving. Parents reported that he frequently avoided academic effort as well as situations involving concentration or other types of sustained mental effort. On the other hand, Daniel reportedly did much better with shorter, less demanding tasks involving vocabulary, fund of information, and short-term, rote memory.

Because Daniel was home-schooled, he did not receive formal grades, but his mother stated that she believed that he would have significant difficulty doing grade-level work in most subjects. One of mother’s primary concerns was Daniel’s slow completion of academic work. Socially, Daniel was described by his mother as being shy, and he had few friends. In stimulating or novel settings, he was inhibited and withdrawn, but he also tended to be a follower even in familiar settings. Mother described the family environment as cohesive and supportive. Both parents had graduate-level degrees and worked in the medical field. Based on an evaluation with the developmental pediatrician, Daniel had been diagnosed with Attention-Deficit/Hyperactivity Disorder (ADHD) Predominantly Inattentive Type. The developmental pediatrician also had concerns about possible learning disorders, but Daniel had not received psychoeducational testing in the past. Past treatment had included a brief trial with methylphenidate, which was discontinued as a result of tics. There had been no past specialized tutoring or psychological interventions. Daniel’s medical history included myelomeningocele (L4 level), repaired shortly after birth, with a shunt placed at age 3 days. There had been no known shunt malfunctions and no other known Central Nervous System events (e.g., bleeds, infections). He was followed regularly in the spina bifida clinic at a large children’s hospital.

Daniel presented in interview as a shy, cooperative adolescent who ambulated by wheelchair. He denied most emotional and behavioral concerns and seemed to have low insight and primarily avoidant coping strategies. He denied problems with mood, anxiety, and social behavior. He admitted not liking to concentrate or to do academic work, and he expressed a clear preference for brief, information-based subjects such as vocabulary over more abstract, reasoning based subjects such as reading comprehension or mathematics.

Select results of Daniel’s neuropsychological testing and parent-reported LEAF scores are presented in Table 2. Consistent with interview reports, Daniel showed scores that fell within the “Problem Range” on LEAF Cognitive-Learning and Cognitive-EF subscales measuring memory (Factual Memory, Working Memory subscales), speed under concentration conditions (Processing Speed subscale), planning (Sustained Sequential Processing subscale), and novel problem-solving (Novel Problem-Solving subscale). Scores for Comprehension and Conceptual Learning and Visual-Spatial Organization were at the low end of the “Borderline Problem Range” and suggest few if any concerns in those areas; Attention subscale scores fell in the middle of the “Borderline Problem Range”. Elevated LEAF Academic subscales suggest that Mathematics and Written Expression were rated as significant problems, whereas Basic Reading skills were rated as less significant but possible problems (Table 2).

Table 2.

Select Test Results for Case Study 1 (Daniel) and Case Study 2 (Amy)

Test	Index/Subtest/Subscale	Daniel	Amy	Amy
		Parent		Teacher
LEAF	Comprehension and Conceptual Learning	5	3	0
	Factual Memory	10	5	0
	Attention	7	5	4
	Processing Speed	10	6	7
	Visual-Spatial Organization	5	7	3
	Sustained Sequential Processing	12	2	3
	Working Memory	10	5	2
	Novel Problem-Solving	14	3	0
	Mathematics Skills	15	10	4
	Basic Reading Skills	8	3	2
	Written Expression Skills	13	7	2
BRIEF	Inhibit	47	40
	Shift	57	53
	Emotional Control	51	43
	Initiate	77	65
	Working Memory	74	61
	Plan/Organize	65	59
	Organization of Materials	50	57
	Monitor	81	52
		Daniel	Amy
WISC-IV/ WASI	Verbal Comprehension	76	109
	Perceptual Reasoning	94	105
	Working Memory	80
	Processing Speed	65
KBIT-2	Verbal IQ	93	117
	Nonverbal IQ	99	114
WJ-III-NU	Broad Reading	75	100
	Broad Math	72	96
	Broad Written Language	76	97

Note. WISC-IV = Wechsler Intelligence Scale for Children. WASI = Wechsler Abbreviated Scale of Intelligence. KBIT-2 = Kaufman Brief Intelligence Test, Second Edition. WJ-III-NU = Woodcock-Johnson III Normative Update. Values for the LEAF are raw scores; values for the BRIEF are T-scores (M = 50, SD = 10); values for the WISC-IV, WASI, K-BIT-2 and WJ-III are standard scores (M = 100, SD = 15). Higher scores on the LEAF and BRIEF subscales indicate greater problems in specific domains of EF and related learning skills.

On the WISC-IV and K-BIT-2, Daniel scored highest on measures of perceptual-organizational and perceptual reasoning skills, consistent with the relative lack of difficulties reported by mother on LEAF subscales reflecting visual-organizational and conceptual processing. On the other hand, weaknesses in processing speed and working memory skills on the LEAF were mirrored by relative weaknesses in WISC-IV Working Memory and Processing Speed Index scores, demonstrating that these WISC-IV assessed weaknesses were also present in Daniel’s everyday behavior. On the WJ-III-NU, Daniel scored well below average on broad measures of reading, math, and writing skills, with no significant subtest scatter within those areas. His elevated LEAF Mathematics Skills and Written Expression Skills subscale scores are consistent with these deficits, although the LEAF Basic Reading Skills subscale showed a less severe problem than indicated by the WJ-III-NU. On the BRIEF, Daniel’s mother reported scores in the potentially clinically significant range (defined as T-score of 65 or higher; see Gioia et al, 2000) on the Initiate (difficulty starting or organizing things), Working Memory (forgets things, attention/concentration problems), Plan/Organize (does not plan ahead or finish things), and Monitor (lack of self-awareness, careless/sloppy), subscales.

Case 2: Amy

Amy was a 7 year old referred for evaluation in order to identify potential learning challenges as she completed first grade. Parents reported that she had strong verbal skills and above average fund of information, and that she already had acquired grade-level or better basic reading skills. Her organization skills were adequate most of the time, and she did not have significant or abnormal attention problems. However, she did not like mathematics, and she needed more time than the average student to understand mathematical concepts and to remember mathematical facts. She was noted to be over reliant on inefficient, concrete mathematical strategies such as use of manipulatives and slow counting on her fingers. Parents reported that they had to work extensively with Amy at home on mathematics and spelling/writing skills so that she could keep up at school. Her teachers reported that her fine-motor coordination skills were poor and that she made an above average amount of careless errors, although not to the extent that this created problems in daily functioning.

Amy’s psychosocial, behavioral, and emotional functioning was described by parents as good, with no notable difficulties. She was a cooperative and friendly girl with age-appropriate social skills. She had several friends and was well-integrated into the peer group. However, parents said that she sometimes had difficulty following rapidly changing, complex social interactions, and that she sometimes “missed the big picture.” The family reported no significant stresses and was of middle-class socioeconomic status. Father was a college graduate, and mother was a high-school graduate.

Amy’s medical history included premature birth (at approximately 35 weeks gestational age) and myelomeningocele (L3–4 level), repaired at birth. She had not received a shunt and was followed closely in the spina bifida clinic. She wore leg braces and ambulated sometimes with crutches and sometimes without assistance.

Amy presented in interview as a friendly, happy, and talkative girl who was very engaging and chatty. She was cooperative in answering questions and reported no concerns about psychological, behavioral, emotional, social, or academic functioning. She named several friends and said that she liked school. Her fund of information and vocabulary were advanced, but at times, she paused when speaking as if taking time to gather her thoughts. No problems with attention or comprehension were evident.

Select results of Amy’s neuropsychological testing and parent- and teacher-reported LEAF scores are presented in Table 2. Parent and teacher reported relatively low levels of concerns on the LEAF. Consistent with reports that she needed more time to understand some types of information (e.g., mathematics, social communication), both parent- and teacher-reported LEAF Processing Speed subscales fell in the “Borderline Problem Range”. Parents also endorsed mild difficulties on LEAF Cognitive-Learning and Cognitive-EF subscales reflecting memory/attention (Factual Memory, Attention, Working Memory) and organization (Visual-Spatial Organization), but the teacher reported fewer of these difficulties at school. On LEAF Academic subscales, parents endorsed moderate problems with Mathematics Skills and mild difficulty with Written Expression, whereas the teacher expressed no major concerns in those areas.

On the WASI and K-BIT-2, Amy scored in the upper end of the Average to the High Average range, as expected based on low LEAF scores on language and reasoning ability subscales (e.g., Comprehension and Conceptual Learning). Consistent with teacher-reported LEAF Academic subscales, her WJ-III-NU results indicated broad reading, math, and written expression functioning in the Average range. It is notable that parents reported in the clinical interview that Amy needed considerable extra effort at home in order to master written language skills and mathematical concepts/calculations. Parents’ endorsement of more difficulties in mathematics and writing compared to the teacher’s report of few problems in these areas may be explained by extra efforts at home. On the BRIEF, only the Initiate subscale was endorsed in the potentially clinically significant range.

Discussion

These two case examples illustrate the use of the LEAF in the clinical evaluation of two children referred for neuropsychological evaluations. We also demonstrate how the LEAF may be used as a part of a multisource-multimethod (e.g., collecting multiple sources of data across multiple methods involving behavioral checklists and performance-based assessments; Holmbeck, Li, Schurman, Friedman, & Coakley, 2002) evaluation of EF. In both cases, the LEAF provided results complementing data obtained from clinical interview and neuropsychological testing and showed that neuropsychological testing results extended into everyday behavior. Based on clinical interview and neuropsychological testing, Daniel experienced more severe difficulties in broad areas of cognitive, academic, and executive functioning, and his LEAF Cognitive-Learning, Cognitive-EF, and Academic subscale scores were consistently higher than those of Amy, who experienced fewer and milder problems. In both cases, parents identified difficulty with processing speed as a chief concern in the clinical interview, and LEAF scores were consistent with this concern.

Other behavior checklist measures that focus on more classic components of EF (Barkley, 2011; Gioia et al., 2000) do not include measures of less commonly assessed EF domains such as processing speed under concentration demands (McAuley & White, 2011) or EF-related cognitive learning processes such as concept formation (Castellanos et al., 2015). As a result, the LEAF provided clinical information about functioning in these areas that was not evaluated on the BRIEF. This information validated results of performance-based neuropsychological tests such as the WISC-IV and WJ-III-NU, providing information about parent-rated child behaviors in everyday life that was not available from other checklist measures such as the BRIEF. For example, Daniel’s low WISC-IV Verbal Comprehension score likely reflected long-term verbal memory weaknesses that were supported by a “Problem Range” score on the LEAF Factual Memory subscale, and his low WISC-IV Processing Speed score was similarly supported by a “Problem Range” score on the LEAF Processing Speed subscale. In both of these cases, LEAF results showed that WISC-IV findings corresponded to everyday behaviorally-based problems seen and reported by parents; no such corresponding scales are available on the BRIEF or other questionnaire measures of EF.

In the case of Amy, differences in interpretation between criterion-referenced interpretation of LEAF scores and norm-referenced interpretation of BRIEF scores were more evident. Amy’s parent-report BRIEF scores on Initiate and Working Memory were more than 1 SD above the normative mean, raising questions about the functional severity of problems in these areas based on abnormality relative to norms. However, LEAF Novel Problem Solving (which corresponds to BRIEF Initiate; *** reference removed for blind review ***) and LEAF Working Memory criterion-referenced scores indicate that parents saw those areas as sources of borderline problems at most, reducing the level of clinical concern about those domains. Hence, LEAF scores can provide criterion-referenced information that adds important novel or incremental information to results of existing EF questionnaires such as the BRIEF.

Conversely, BRIEF results provided evaluation of some areas of EF not covered by the LEAF, such as undercontrolled behaviors (e.g., disinhibition, poor emotional control) and poor self-awareness. Furthermore, LEAF and BRIEF results converged in several ways in these clinical cases, particularly for Daniel. Daniel’s elevated BRIEF subscales suggested difficulty starting or organizing things (Initiate subscale), consistent with problems in novel situations on the LEAF (Novel Problem-Solving subscale). Similarly, elevations on Daniel’s BRIEF Working Memory and Plan/Organize subscales mirror “Problem Range” scores on the LEAF Working Memory and Sustained Sequential Processing subscales, respectively.

LEAF results also suggested profiles of strengths and weaknesses in both clinical cases that were useful for formulations and recommendations. Academically, both children showed the highest elevated scores on the LEAF Mathematics Skills scale and the lowest scores on the LEAF Basic Reading Skills scale, consistent with the finding that children with spina bifida are at greater risk for weaknesses in math achievement (Dennis & Barnes, 2002). Supports in mathematics and writing were emphasized in both cases, with a reduction in volume and demands in these academic areas (especially for Amy and her parents at home).

Cognitively, Daniel’s LEAF scores indicated that problems with working memory, novel problem solving, sequential processing, and processing speed were clearly much greater than the negligible problems reported for visual-spatial organization and comprehension/conceptual learning. As a result, recommendations for Daniel included reducing the complexity of communication and learning, simplifying instructions and memory demands, providing previews for unfamiliar information, monitoring during long-sequential tasks, and providing extra time for learning and demonstrating knowledge. For Amy, on the other hand, mild cognitive challenges in LEAF subscales reflecting processing speed and (at home) visual-spatial organization were addressed with recommendations to reduce academic volume and provide extra time for learning.

Several characteristics of both cases were consistent with deficits in assembled cognitive processes commonly associated with spina bifida, as described by Fletcher & Dennis (2010). According to parents and teachers, and even by his own self-report, Daniel had difficulty with complex tasks requiring integration, organization, and comprehension (assembled cognitive processing) and did better with brief tasks involving vocabulary, fund of information, and rote memory (associated cognitive processing). Similarly, despite her generally good cognitive and academic functioning, Amy had more difficulty with concepts (assembled processing) and preferred concrete strategies and facts (associated processing). Both children struggled more with conceptual demands of math and written expression than with overlearned skills involved in basic reading. On the other hand, Daniel and Amy also showed unique strengths and weaknesses that did not follow a classic division of assembled and associated processing, consistent with individual differences in neurocognitive outcomes seen across the spina bifida population (Fletcher & Dennis, 2010).

The LEAF complements performance-based measures of EF, and taken together these instruments may be used by pediatric neuropsychologists to assist in early identification and intervention of EF delays and disturbances. The LEAF can be clinically useful for several reasons. First, based on its broad content and good psychometric properties in a diverse clinically-referred sample (***removed for blind review***), it appears likely that the LEAF can be used to evaluate EF delays and disturbances in children with a wide spectrum of pediatric conditions such as those resulting from central nervous system injury or disease, including spina bifida. EF delays are common in children with serious medical or neuropsychological conditions, making broad, multitrait-multisource assessment of EF delays important in neuropsychological practice. Second, compared to other EF scales, the LEAF is both comprehensive and relatively brief, and it includes subscales that provide insight into learning and memory, which are often compromised when significant EF delays are present. Evidence from a psychometric study (***removed for blind review***) suggests that the LEAF can provide information about a broad range of EF and related abilities, as well as information about deficits in academic areas commonly affected by EF deficits. Third, the LEAF meets existing recommendations to enhance the clinical utility of behavior checklists (Levy et al., 2013), such as having five or fewer items per scale, grouping items consecutively, and using simple rules for interpretation. Fourth, simple criterion-referenced rules for cutoff scores (e.g., 10 or higher = “Problem Range”) allow for rapid interpretation of whether a raw LEAF score is likely to suggest a problem in daily functioning. Because these interpretation ranges are criterion-referenced, they add additional information to the norm-based interpretation of existing EF questionnaires. Criterion-referenced scores differ from norm-referenced scores because criterion-referenced scores reflect the extent to which behaviors constitute a problem, as opposed to the extent to which problems are abnormal relative to norms. Using criterion- and norm-referenced questionnaires of EF such as the BRIEF and LEAF together allows for understanding the functional severity of problems (criterion-referenced) in addition to the abnormality (norm-referenced) of behaviors. For example, the widely-used Child Symptom Inventory-4 (Gadow & Sprafkin, 1998) provides information about the extent to which behavior problems correspond to DSM-IV symptoms, which can complement the results of conventional norm-based behavior checklists such as the Behavior Assessment System for Children-3 (Reynolds & Kamphaus, 2015). The LEAF may occupy a similar role relative to the BRIEF and other EF questionnaires. Finally, the LEAF may be obtained free of charge.

More globally, there are important reasons for using behavior checklist measures of EF in neuropsychological assessment. First, the brevity and ease of assessment and scoring of a behavior checklist measure such as the LEAF makes this method of assessment ideal for screening, either to identify at-risk children for more extensive assessment or to complement interview data. Second, behavior checklist data about EF can provide critical information about the behavior of the child in the everyday environment, which can be integrated with individually-administered neuropsychological tests. In the case of the LEAF, this information about everyday behavior can be obtained about a very broad range of EF and closely related cognitive domains, including some areas (e.g., Processing Speed, Factual Memory, Comprehension and Conceptual Learning) that are not covered by other EF questionnaires. Third, behavior checklists of EF can be used to monitor change in home and school settings, which may not be reflected on office-based performance ability tests of EF. Because of the high frequency of EF challenges in pediatric populations, EF behavior checklists such as the LEAF offer considerable clinical value to pediatric neuropsychologists.

Limitations of the LEAF include the lack of normative data, focus on the cognitive (as opposed to undercontrolled behavioral and emotional) components of EF, early stage of scale development and validation, and, similar to other scales of EF, vulnerability to rater effects such as bias or limited knowledge. Although the LEAF provides criterion-referenced ranges to interpret raw scores, the lack of norm-referenced scores on the LEAF restricts its ability to provide information about the abnormality of scores. Other EF-related behavior checklists such as the BRIEF provide norm-referenced scores and have been much more widely used and extensively validated. Furthermore, the LEAF focuses on cognitive domains of EF and does not cover behavioral or emotional EF domains such as impulsivity and emotional control. Therefore, at this stage of scale development, use of the LEAF in tandem with a well-established EF questionnaire such as the BRIEF may offer the best set of advantages in questionnaire-based EF assessment. Finally, like other EF behavior checklists, LEAF scores are influenced by rater characteristics. Raters may differ in their awareness of the child’s behavior, and rater bias to overreport or underreport problems may influence scale results. For these reasons, behavior questionnaires such as the LEAF should be used and interpreted in the context of a broader clinical evaluation which may include components such as review of records, background information, interview, and/or administration of neuropsychological instruments.

This case study report provided an illustrative description of how EF questionnaires might complement and extend traditional neuropsychological performance-based tests, but additional research using the LEAF with diverse neuropsychological samples and ranges of severity is needed. Our intent in this paper was to illustrate use of the LEAF in two clinical cases that had specific referral questions about both executive functioning and learning skills. Therefore, the assessment batteries focused on intellectual and learning abilities. For other clinical cases, it is likely that performance-based neuropsychological tests of executive functioning would also be used and would be integrated with LEAF results in a multisource-multimethod strategy of assessment. Additional research investigating integration of LEAF results with results of other neuropsychological tests is recommended to better inform the clinical use of the LEAF and other EF behavior checklists in the context of broad multisource-multimethod neuropsychological batteries. In a companion paper, we validated the LEAF with a broad clinically referred sample with a diverse set of pediatric and neuropsychological diagnoses, using traditional performance-based neuropsychological tests such as the Stroop Color-Word Test and Conners’ Continuous Performance Test (***removed for blind review***). Furthermore, we are investigating the psychometric properties of the LEAF with other samples, including children with hearing loss (***removed for blind review***). This additional research is intended both to validate use of the LEAF and to enhance the use of the LEAF and other EF behavior checklists in the context of broad neuropsychological test batteries.