Dynamic testing with graduated prompts training: Adolescents with executive function deficits

Abstract

Introduction

Adolescents with executive function deficits have difficulties performing complex daily tasks. Process-oriented dynamic testing with graded prompts training offers valuable insights to improve adolescents’ thinking and performance abilities in occupational therapy interventions. This study explores the effectiveness of graded prompts training in improving performance during the Rey-Osterrieth Complex Figure (ROCF) task.

Method

This secondary analysis included data from 41 adolescents with executive function deficit profiles who had participated in the therapeutic occupational therapy intervention program ‘Functional Individualized Therapy for Teenagers’. Dynamic pre- and post-test phases, focused on copying and immediate recall, were assessed using accuracy, placement, and organization strategy scores.

Results

Results indicate significant improvements following graded prompts training in copying and memory accuracy (67% and 180%), placement (49% and 237%), and organization strategy (66% and 99%). Positive correlations were observed between improved copying accuracy and organization, memory accuracy and organization, and copying and memory organization (p < .001).

Conclusion

Graded prompts training may improve the performance of adolescents with executive function deficits during complex tasks (e.g., ROCF). Future research should explore the training’s long-term effects and applications in diverse clinical settings to validate further and expand these promising results. This training could become integral to therapeutic interventions for enhancing this population’s daily functioning and overall well-being.

Introduction

Adolescence is a crucial developmental stage with significant cognitive, emotional, and social changes, where executive functions (EFs) like cognitive flexibility, working memory, planning, and inhibitory control mature, support goal-directed behavior and adaptive functioning (Goossens, 2020; Jansen & Kiefer, 2020). However, these processes can be significantly impaired in adolescents with EF deficits (EFDs), resulting in difficulty executing tasks that require advanced cognitive engagement. This impairment can lead to needing more assistance, riskier behaviors, and challenges in academics, social interactions, and daily living skills (Josman & Rosenblum, 2018).

Few studies have explored using dynamic testing with graded prompts to improve the processes of adolescents with EFDs, despite its potential to enhance cognitive strategies in occupational therapy (Fogel et al., 2021; Toglia & Foster, 2021). Occupational therapy (OT) recognizes the importance of addressing EFDs in adolescents. A promising approach is process-oriented dynamic testing with graded prompts, which assesses abilities and gradually reduces support as competence improves (Veerbeek et al., 2019). Metacognitive and cognitive approaches are promising OT interventions for children and adolescents with EFDs. They focus on teaching strategies for organizing, problem-solving, and managing EF skills in daily activities (Josman & Rosenblum, 2018).

Various OT intervention approaches underscore the use of graded prompts training to enhance occupational performance. These included the cognitive orientation to daily performance for children and adolescents with developmental coordination disorders (McEwen et al., 2018), the teen cognitive–functional intervention model for adolescents with attention-deficit/hyperactivity disorder (Maeir et al., 2018), and the ‘Functional Individualized Therapy for Teenagers' (FITTED) - an eight-session intervention designed to enhance executive functioning through personalized therapeutic activities and graded prompts training. Graded prompts training enables individuals to review, check, and compare outcomes with goals. It facilitates the correction, revision, and generation of new strategies for future use (Toglia & Foster, 2021).

Graded prompts training, part of process-oriented dynamic testing, provides structured scaffolding, gradually reducing support to promote independent problem-solving and strategy use. Unlike static testing, dynamic testing offers feedback and training to reveal cognitive functioning and learning potential by assessing both independent abilities and learning potential with support (Elliott et al., 2018; Resing & Elliott, 2011). Previous research has suggested that graded prompts training improves test accuracy (Stevenson et al., 2016) and task-solving processes (Resing et al., 2017).

The Rey-Osterrieth Complex Figure (ROCF) task assesses visuospatial ability and visual memory by having individuals copy and later reproduce a complex figure from memory. It is effective for evaluating EFs as it requires planning, organization, and memory, areas where adolescents with EFDs often struggle. Integrating graded prompts with the ROCF task allows therapists to understand better cognitive processes and tailor interventions (Fastame, 2020). The dynamic ROCF test helps therapists develop personalized treatment plans, enhancing motivation and rehabilitation outcomes (Feuerstein et al., 2002; Shefa et al., 2013). Originally designed for adults, the ROCF has been successfully utilized and normed with school-aged children (6 years or older). It is sensitive to developmental changes (Resch et al., 2019) and possesses good psychometric properties. It demonstrated ecological (Martens et al., 2014) and discriminative validity between children with and without acquired central nervous system disorders (Davies et al., 2011).

This study used a dynamic testing design (pre-test–training–post-test) with graded prompts to enhance self-awareness and cognitive strategy use in adolescents with EFDs during the ROCF test. The hypothesis was that graded prompts training would significantly improve accuracy, memory recall, and organizational strategies for the ROCF test in adolescents with EFDs.

Methods

This study conducted a secondary data analysis of 41 adolescents with EFDs who participated in the FITTED intervention. This analysis used a subset of participants from a larger study to develop and evaluate a conceptual program for adolescents with EFDs, focusing on the ROCF task. This activity, conducted in the second of eight sessions, illustrated how process-oriented dynamic testing could offer deeper insights into this population and improve their daily performance outcomes through a pre–post-test design involving copying and immediate recall phases. The mediation stage employed graded prompt training with two established scoring methods (see Scoring Method section). A qualified occupational therapist administered the dynamic ROCF, while two additional trained therapists, blinded to the study’s purpose, independently performed the scoring (Figure 1).

Figure 1.

Process-oriented dynamic testing using the Rey-Osterrieth complex Figure.

Procedure

Ethics

The Ethics Committee of the Faculty of Social Welfare and Health Sciences, University of Haifa Ethics Committee approved this study (253/13) on 17 October 2013. All adolescents and their parents signed written informed consent forms prior to participation and consented to the publication of their drawings.

Mediation and training for ROCF

This study used the dynamic version of the ROCF, part of the Learning Propensity Assessment Device battery (Feuerstein et al., 2002). Its administration includes five stages: initial copy, recall drawing, mediation, second copy, and second recall drawing.

Two training methods were used in the current study: (a) Level 2 mediation involving verbal analysis of figure components and (b) graded prompts training guiding participants through drawing steps with increasingly specific prompts.

Level 2 mediation

The Level 2 mediation included the participant naming the drawing’s components and focusing on planning and organizing the copying sequence (their exposure to the ROCF form during mediation was of unlimited duration; Shefa et al., 2013).

Graded prompts training

This training aimed to facilitate task-solving by dividing the figure into its basic structure and internal and external components. It began with general metacognitive prompts, progressing to specific cognitive prompts and, if needed, modeling the task-solving process. Participants were asked to verbalize their approach before drawing and received additional prompts based on their responses. The training involved guiding questions, starting with the base structure (e.g., “What would be useful to start with?”), followed by dividing lines (e.g., “Are there lines that divide the figure?”), and finally addressing internal and external elements (e.g., “Could you use the inside or outside to divide the elements?”; Resch et al., 2019; Veerbeek et al., 2019).

Participants

A total of 41 adolescents, aged 10–14 years, were recruited through community advertisements targeting those with and without daily functioning difficulties. The sample size was determined based on the scope of this pilot study and practical recruitment constraints. Volunteers with known psychiatric, emotional, or autistic spectrum disorders, physical disabilities, or neurological diseases were excluded.

Instruments

Participants were screened for the EFD group inclusion criteria using the Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al., 2000) and the Wechsler Intelligence Scale for Children (WISC-R; Wechsler, 1998).

BRIEF

The adolescents’ parents completed the BRIEF-parent version, a validated tool for assessing executive functioning in children and adolescents aged 5 to 18 (Gioia et al., 2000). The BRIEF consists of 86 items across two indexes—the Behavior Regulation Index (BRI) and the Metacognitive Index (MI)—and a Global Executive Composite (GEC). The BRI includes scales for inhibiting, shifting, and emotional control; the MI includes initiating, working memory, planning/organizing, organization of materials, and monitoring. The questionnaire takes 10–15 min to complete. Items are rated on a three-point scale (never, sometimes, often). A GEC t-test score of 65 or above indicates clinical-level executive function problems.

WISC-R

The WISC-R assesses children’s intellectual functioning through verbal and nonverbal capacities (Wechsler, 1998). The vocabulary and block design subtests were selected for their common use in evaluating intellectual function in children aged 10 years and older with neurodevelopmental disabilities and their strong psychometric properties. These subtests determined each participant’s suitability for the study, with a mean score of 10.

Outcome measure: ROCF

The ROCF is a widely used neuropsychological tool (Osterrieth, 1944). It involves a novel geometric figure comprising 18 shapes arranged around and within a central rectangle. Organizing these shapes into a meaningful perceptual unit poses a complex cognitive task (Wechsler, 1998). The original test requires subjects to (a) accurately copy the geometric shapes and (b) recall and draw the figure from memory after a specified interval (3 min in the current study based on established norms and existing research protocols) without warning (Anderson et al., 2001). This timeframe balances the need to engage working memory and organizational skills while minimizing potential distractions or fatigue.

The pre- and post-mediation copying tasks are performed within a single session as part of the dynamic testing design, the second session of an eight-session intervention series. The total duration varies depending on the individual participant’s needs but generally allows sufficient time for copying and mediation phases to ensure comprehensive assessment and skill reinforcement. Various scoring procedures exist for the ROCF to assess performance and differentiate specific cognitive skills’ influence (Resch et al., 2019).

Scoring Methods

This study used two scoring methods: accuracy and placement, reflecting the product, and organization strategy, reflecting the process.

Accuracy and placement

The accuracy and placement scoring method evaluates the participant’s copy and recall during their ROCF performance. It assigns 0 (inaccurate) or 1 (accurate) point for placement of the 18-line clusters regardless of drawing order, yielding scores ranging from 0 to 18 in each phase. Higher scores reflect a higher similarity to the drawn figure (Taylor, 1969).

Organization strategy

The Rey Complex Figure Organizational Strategy Score (RCF-OSS) assesses organizational strategy. As a process-oriented scoring approach, the RCF-OSS considers the sequence in which the participant draws various elements of the complex figure. The researcher who administers the task tracks the order in which the participant draws the lines and rates the drawings on a seven-point scale: 1 (unrecognizable drawing or substitution), 2 (poor organization), 3 (random organization), 4 (piecemeal/fragmented organization), 5 (part-configured organization), 6 (conceptual organization), or 7 (excellent organization). A higher score represents a higher level of organization (Anderson et al., 2001).

Interrater reliability for the current study was assessed with intraclass correlation coefficients, showing moderate to high agreement (0.50–0.75). The RCF-OSS is a valid and reliable method for measuring organizational skills in children (Anderson et al., 2001).

Results

Demographics

The participants (n = 41) included 29 (70.7%) boys and 12 (29.3%) girls with a mean age of 11.88 years (SD = 1.08). Their parents’ education levels were measured as academic (at least 2 years of higher education toward a degree or professional certification) or nonacademic; 34 (82.9%) mothers and 30 (73.2%) fathers reported academic education. The participants’ sociodemographic status by monthly income was divided into low (under 7,000 Israeli shekel [ILS]: two families; 5%), average (7,000–14,000 ILS: 20 families; 50%), or high (over 14,000 ILS: 18 families; 45%).

The mean WISC-R subtest scores were 9.63 (SD = 2.43) for vocabulary and 11.20 (SD = 2.96) for block design. The mean BRIEF scores were 69.09 (SD = 5.26) for the GEC, 67.70 (SD = 9.72) for the BRI, and 66.65 (SD = 6.34) for the MI.

Differences and improvement following the post-mediation stages

Table 1 compares the ROCF scores in the pre- and post-mediation stages and shows that all indices significantly increased following mediation. Specifically, copying accuracy increased 67%, t(40) = 10.85 (p < .001), copying placement increased 49%, t(40) = 7.60 (p < .001), memory accuracy increased 180%, t(40) = 19.61 (p < .001) and memory placement increased 237%, t(40) = 21.99 (p < .001). The copying organization strategy score increased 66%, t(40) = 15.55 (p < .001), and the memory organization strategy score increased 99%, t(40) = 19.19 (p < .001). Figures 2 and 3 present examples from two participants, both 12 years old.

Table 1.

Comparison of Indices by Stage (Pre- and Post-Mediation).

Index	Pre-mediation		Post-mediation		t(40)	p	d	Improvement (%)
	M	SD	M	SD
Copying: Accuracy	8.07	2.64	13.53	2.55	10.85	<.001	1.69	67.66
Copying: Placement	10.53	4.76	15.78	1.62	7.60	<.001	1.18	49.86
Memory: Accuracy	4.23	2.08	11.85	2.45	19.61	<.001	3.06	180.14
Memory: Placement	4.45	3.41	15.00	1.56	21.99	<.001	3.43	237.08
Copying: Organization strategy	3.50	0.96	5.82	0.69	15.55	<.001	2.42	66.29
Memory: Organization strategy	2.90	0.93	5.78	0.61	19.19	<.001	3.00	99.31

Figure 2.

Example 12-Year-Old Boy’s Dynamic Process for the Rey-Osterrieth Complex Figure. Source: Reproduced with permission of the drawer and his parents.

Figure 3.

Example 12-Year-Old Girl’s Dynamic Process for the Rey-Osterrieth Complex Figure. Source: Reproduced with permission of the drawer and her parents.

Relationships following the mediation stage

The results showed that improved copying accuracy positively correlated with copying organization (r = .48, p < .001), improved memory accuracy positively correlated with memory organization (r = .34, p < .05), and improved copying organization positively correlated with memory organization (r = .35, p < .05).

Discussion

This study explores the efficacy of graded prompts training in improving performance during the ROCF task among adolescents with EFDs. The results reveal significant improvements across various indices following the mediation phase, indicating the beneficial impact of the mediation. The significant increase in copying accuracy and organization suggests that the mediation stage facilitated participants’ ability to reproduce complex geometric figures accurately. Although the findings are consistent with previous literature, the absence of a control condition limits claims regarding verified effectiveness. Instead, they focus on observed associations and potential implications. The current findings support the concept that providing structured prompts and guidance can help adolescents with EFDs organize and execute cognitive tasks effectively (Veerbeek et al., 2019).

Graded prompts training was found to be an effective method for dynamic testing of analogical reasoning (Stevenson et al., 2016), series completion (Feuerstein et al., 2002), and the ROCF task (Davies et al., 2011). Stad et al. (2019) investigated the extent to which children’s potential for learning related to their cognitive flexibility level among 153 neurotypical 6- to 7-year-olds. Their findings suggested that dynamic training enhances children’s cognitive flexibility, especially the graded prompting procedure. By providing a structured yet adaptable approach, dynamic training reveals more of children’s cognitive potential, as measured through inductive reasoning.

Other researchers constructed a dynamic complex figure task using the graded prompts approach to investigate processes involved in complex figure task-solving and changes in these processes after training among 106 primary-school children. Their results indicated that graded prompts training could lead to significantly more progress between pre-test and post-test than unguided controls in complex figure drawing performance (Veerbeek et al., 2019).

Another study among 60 participants (23–69 years) from social clubs for adults with mental illnesses compared the dynamic (experimental group) and static (control group) ROCF assessments of cognitive functioning and examined the dynamic form’s efficacy. The findings revealed that the post-test copy and memory scores and degrees of improvement were significantly higher for the dynamic version than the static version. These findings can be explained by the mediation given to the dynamic (experimental) group. The mediation consisted of verbalizing the figures’ components by naming them and directing the participants’ focus toward consistently planning and organizing the copying task (Level 2 mediation). The mediation, a learning process, helped the participants improve their performance in the post-mediation copying stage and remember the figure better, resulting in better scores in the post-mediation memory stage (Shefa et al., 2013).

The most notable improvement observed in the current study is in memory accuracy. Copying tasks from memory inherently engages executive skills, such as working memory, visual-spatial perception, and mental representation, all of which play critical roles in drawing performance (Fastame, 2020). Specifically, mental representational processes are key cognitive skills contributing to drawing from memory (Stad et al., 2019). This study’s results suggest that adolescents with EFDs may exhibit enhanced abilities in mental representation processes following graded prompts training, leading to significant improvements in memory performance.

Adolescents with EFDs often face high distractions from various stimuli, including visual inputs. This potential sensory overload may require increased effort, potentially diminishing their performance and limiting their ability to improve following mediation (Senese et al., 2015) due to cognitive load. In essence, cognitive load refers to the total demands that performing a particular task imposes on the learner’s cognitive system (de Bruin et al., 2020).

Cognitive load theory addresses the cognitive processing induced by the design of educational tasks and its impact on students’ performance. Task design can hinder performance by overwhelming the cognitive system with unnecessary cognitive processing (extraneous cognitive load) or enhance performance by stimulating cognitive processes relevant to learning (germane cognitive load; Sepp et al., 2019). The intrinsic cognitive load stems from features inherent to the task (e.g., task complexity) and the individual working on it (e.g., prior knowledge). Consequently, cognitive load research examines how task characteristics relative to a person’s skills and knowledge level affect cognitive load.

The current study’s results may indicate that adolescents with EFDs often show difficulties, challenges, and problems that mask their true functioning and potential. Dynamic assessments can facilitate the adolescents’ ability to retain and reproduce complex visual information and help them see past the externalizing behavior, identify its effect, and label it accordingly (Sweller et al., 1998).

The positive correlations in this study between improvements in copying accuracy and organization, memory accuracy and organization, and copying and memory organization highlight the interconnected nature of cognitive skills. The findings suggest that enhancing one aspect of cognitive functioning may facilitate improvements in related domains, indicating the potential for generalized cognitive gains (Anderson et al., 2001).

Senese et al. (2015) investigated the ROCF among neurotypical children aged 7–10 years. They found strong correlations between copying and memory and cognitive abilities, such as visual perception. Spatial mental representation directly affected copying performance, the only specific predictor for drawing from memory. Similarly, Fastame (2020) found partial correlation patterns between the ROCF copy and memory tasks among children (r = .47, p < .001). This finding may imply that strategies applied in the copy condition could be adopted in the drawing-from-memory condition; that is, the representation of the ROCF task stored in memory may be actively integrated when processing spatial information relative to its elements.

These correlations support the dynamic systems perspective, emphasizing the dynamic interplay between cognitive processes and the adaptability of cognitive systems in response to learning experiences (Resing et al., 2016). The participants in this study possibly developed different cognitive strategies for execution in the mediation phase and used them in the memory phase.

Dynamic assessment assumes that cognition is modifiable, and performance and learning changes occur with task experience. Its core goal is to see how a participant will change if an opportunity is provided and reveal the participant’s potential for learning. From a therapy perspective, dynamic testing provides a snapshot of the person’s response to intervention and directly examines their ability to learn and apply strategies (Toglia & Foster, 2021). The dynamic approach of utilizing the ROCF measure holds significant value for understanding human performance. The approach is a tool to uncover discrepancies between the cognitive system and actual activity and between abilities and real-world performance, characteristic of adolescents with EFDs. Bridging this gap is crucial for objective examination and effective intervention.

The current study has several limitations. First, it involved only 41 participants, which may limit the generalizability of the findings. A larger sample size would better represent the adolescent population with EFDs and yield more robust results. Second, participants were recruited through community advertisements and included only those without known psychiatric, emotional, or autistic spectrum disorders, physical disabilities, or neurological diseases. This exclusion limits the applicability of the findings to adolescents with more complex profiles who also experience EFDs. Third, the study focused solely on immediate pre- and post-intervention outcomes without investigating long-term effects. To understand the durability of the improvements seen after the graded prompts training, follow-up assessments over a longer period would be necessary. Fourth, the study analyzed the effectiveness of only one task (the ROCF) within the broader FITTED intervention, meaning the results may not generalize to other activities or contexts within occupational therapy interventions for adolescents with EFDs. Finally, although two trained occupational therapists conducted the scoring, potential bias could still influence the results. Despite being blinded to the study’s purposes, the therapists might have had unconscious biases or variances in interpreting the scoring methods, potentially affecting the reliability of the outcome measures.

Addressing these limitations in future research could enhance the understanding and applicability of the findings, ensuring that interventions are effective and benefit a broader range of adolescents with EFDs. Future research should delve deeper into the long-term effectiveness of dynamic approach interventions and explore additional factors influencing cognitive outcomes in adolescents with EFDs. Qualitative inquiries into participants’ perceptions and experiences of the intervention could provide profound insights into the underlying mechanisms of cognitive skill development and pave the way for more comprehensive intervention approaches.

Conclusion

Continued research and clinical evidence are imperative for comprehending and devising strategies to overcome this gap. The potential to generate a repertoire of cognitive strategies relevant to specific tasks can markedly enhance the functioning of these adolescents, underscoring its importance in research and clinical practice.

Ethical statement

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. The Faculty of Social Welfare and Health Sciences, University of Haifa Ethics Committee approved this study (253/13) on 17 October 2013.

Informed consent

The author affirms that the participants provided informed consent for their participation in the study and the publication of the drawings in Figures 2 and 3.

ORCID iD

Yael Fogel https://orcid.org/0000-0002-7755-519X