Systematic Review: A Prevention-Based Model of Neuropsychological Assessment for Children With Medical Illness

Abstract

Objective

Many pediatric chronic illnesses have shown increased survival rates, leading to greater focus on cognitive and psychosocial issues. Neuropsychological services have traditionally been provided only after significant changes in the child’s cognitive or adaptive functioning have occurred. This model of care is at odds with preventative health practice, including early identification and intervention of neuropsychological changes related to medical illness. We propose a tiered model of neuropsychological evaluation aiming to provide a preventative, risk-adapted level of assessment service to individuals with medical conditions impacting the central nervous system based on public health and clinical decision-making care models.

Methods

Elements of the proposed model have been used successfully in various pediatric medical populations. We summarize these studies in association with the proposed evaluative tiers in our model.

Results and Conclusions

This model serves to inform interventions through the various levels of assessment, driven by evidence of need at the individual level in real time.

Advances in medical treatment and supportive care have resulted in dramatically increased survival rates in many pediatric chronic illnesses (Compas, Jaser, Dunn, & Rodriguez, 2012). Although children are living well into adulthood, managing diseases once considered fatal, survivorship is not without cost. Given increased survival, more attention has been paid to quality-of-life issues, including cognitive and psychosocial functioning. Neuropsychological deficits occur in association with many congenital and acquired pediatric diseases. Robust evidence exists for such deficits in children affected by central nervous system-impacting cancer (e.g., Campbell et al., 2007; Robinson et al., 2010), sickle cell disease (SCD; e.g., Hijmans et al., 2011), and epilepsy (e.g., Sherman et al., 2011). Cognitive difficulties have also been documented in children with systemic lupus erythematosus (e.g., Williams et al., 2011), multiple sclerosis (e.g., Charvet et al., 2014), human immunodeficiency virus (e.g., Le Doare, Bland, & Newell, 2012), and diabetes (e.g., Schwartz, Wasserman, Powell, & Axelrad, 2014). As such, recurrent neuropsychological assessments are standard of care for many children with chronic illness (Walsh et al., 2016).

Despite this, neuropsychological services have largely been unincorporated into a health prevention model. Many children are not referred for neuropsychological evaluation until significant functional impairments are evident. In response, a comprehensive approach to evaluation, typically requiring lengthy testing to adequately address differential diagnoses and provide treatment recommendations, has been the traditional approach to neuropsychological care. Despite established clinical utility, it is unclear whether traditional batteries predict individual outcomes and trajectories over time. It is important to examine other models of health care delivery to optimize neuropsychological care in pediatric illness populations, particularly given growing evidence of the efficacy of early intervention programs on cognitive outcomes for children with neurodevelopmental disorders such as autism (Estes et al., 2015).

Public Health-Based Models of Assessment and Treatment

Progressive models of health care delivery have been proposed for decades (Leavell & Clark, 1965). Most public health care models characterize preventative health care as primary, secondary, and tertiary. Primary services are basic measures provided to all individuals in a specific context or category (e.g., well-child checks). Secondary services are those intended for individuals with signs, symptoms, or identified risk for a specific condition (e.g., diabetes) to foster early diagnosis and prevention/intervention. Tertiary services refer to rehabilitation efforts intended to offset the consequences of an illness or condition (e.g., traumatic brain injury; Katz & Ali, 2009). This model of preventative care is intended to triage and treat individuals for specific conditions with minimal burden on community resources and the greatest efficiency for providers and patients, by adapting the intensity of services to individual risk level.

Youngstrom (2013) has advocated for the application of evidence-based medicine (EBM) approaches to psychological assessment, so that selected measures improve outcomes related to prediction, prescription, and process. That is, test results should be associated with some criterion of importance, guide the choice of treatment, or inform the clinician’s evaluation of treatment outcome or response for the individual patient, not just for groups of patients with similar characteristics. An EBM approach supports the development of specific, clinically relevant questions and application of the most efficient, focused means of assessment required to answer those questions. Using this approach, the clinician may decide that a diagnosis can effectively be ruled out, that continued assessment is warranted, or to proceed with necessary interventions (Youngstrom, 2014).

Applying ideas from both public health and assessment utility frameworks can guide a novel model of neuropsychological care—one that emphasizes targeted use of resources that are adapted to the individual child, with the aim of identifying emerging problems before they manifest as functional impairments. Models based on this approach have already successfully been utilized for general mental health care among children with medical illness, such as the Pediatric Psychosocial Preventative Health Model (PPPHM; Kazak, 2006). Kazak and colleagues (2015) have published extensively on the use of the Psychosocial Assessment Tool (PAT 2.0), a screening tool based on this model, which has been used with a variety of pediatric populations including pediatric cancer, SCD, Congenital Heart Defects (CHD), diabetes, and organ transplant. There is emerging evidence that mental health screening in primary care settings may improve rates of referral to, and subsequent attendance at, psychiatric care appointments (Jonovich & Alpert-Gillis, 2013). To our knowledge, there has not been any effort to translate this approach to neuropsychological service delivery. We propose a tiered model of neuropsychological assessment for use with pediatric populations most at risk for cognitive disruption over the course of development. In addition to detailing the model below, we also illustrate how elements of this model have been investigated through the use of abbreviated assessments with a variety of pediatric populations.

Universal Monitoring

The first level of assessment, Universal Monitoring, includes basic evaluation of specific neuropsychological processes (Figure 1). Assessment at this level is intended to occur frequently for all children with a medical condition that places them at elevated risk of neuropsychological impairment. Given that this level of service is intended for a large number of children at regular intervals, delivery by a specialist such as a neuropsychologist would not be required. Just as a child might receive a brief vision screening conducted by a nurse, a cognitive monitoring evaluation may be administered in conjunction with a medical visit by a trained member of the child’s medical team. Monitoring tools should be brief, repeatable, and sensitive to deficits and change over time. Measure selection should be informed by specific domains of functioning at the highest risk for disruption by a particular medical condition or treatment. For example, children with early-onset type 1 diabetes are at risk for cognitive difficulties in multiple domains, including learning and memory, executive dysfunction, and processing speed (Ryan, van Duinkerken, & Rosano, 2016). Because children with diabetes are seen routinely for follow-up by endocrinology specialists, a monitoring battery for this population might include a parent questionnaire of executive functioning and a computerized assessment of processing speed, working memory, and list learning. Children and families could complete these assessments as administered by trained personnel in clinic, and feedback could be given to families in real-time so that appropriate referrals can be made for further testing, if needed, during the same visit. A similar model is currently used in a research trial conducted by the Children’s Oncology Group (COG). Specifically, children aged 6–11 when diagnosed with B-cell acute lymphoblastic leukemia, treated on COG protocol AALL1131, are evaluated approximately every 6–12 months using a parent-completed questionnaire (the Behavioral Rating Inventory of Executive Function) and a computerized measure of processing speed, sustained attention, and executive functions (CogState). Since the trial began in 2012, >600 individuals (mostly nurses and clinical research assistants) at 170 hospitals have been trained to administer the battery to >400 enrolled participants in clinic. Feedback is given to providers of children who show deficits (i.e., >1.5 standard deviations beyond the mean in the direction indicating difficulties) so that providers can make referrals for additional testing, if warranted (Hardy et al., 2014). Although this is a research paradigm, it demonstrates that the model can be successfully implemented in a pediatric medical setting.

Figure 1.

Prevention-based model of neuropsychological service delivery.

Proposed components of the Universal Monitoring level of assessment include the following: (a) the battery is composed exclusively of questionnaire measures and computerized, performance-based tasks; (b) the measures are psychometrically sound and have criterion norms based on a standardization sample; (c) the included measure(s) are associated with a defined criteria or cutoff score that is indicative of impairment; (d) the battery is brief (no more than 30 min). In developing the Universal Monitoring criteria, we endeavored to identify tools that did not require doctoral-level training to administer or score, yet still provided information about established or emerging cognitive deficits. We identified seven published studies that developed and/or evaluated batteries meeting these criteria and specifically addressing the ability of the tests to identify children with cognitive deficits (see Table I). Although several investigators described their batteries as “screening,” we used the above criteria to instead identify them as Universal Monitoring approaches.

Table I.

Summary of Published Studies Examining Targeted, Evidence-Based Evaluative Procedures in At-Risk, Medically Involved Children

Level of assessment	Study authors	Sample population (age range)	Sample size	Measures	Domains assessed	Time to complete
Monitoring	Bull et al. (2015)	Cerebellar tumor (8–14) Comparison (8–14)	72 38	Questionnaire: BRIEF (parent; teacher) PedsQL (parent; child) SDQ (parent; child; teacher)	Executive functions Patient-reported outcome/QL Psychosocial functioning	Not reported
Monitoring	Hardy, Willard, Wigdor, Allen, & Bonner (2015)	Pediatric cancer survivors (5–18)	70	Questionnaire: Conners Rating Scale (parent)	Attention	10–15 min
Monitoring	Vega-Fernandez et al. (2015)	Systemic lupus erythematosus (9–17) Juvenile idiopathic arthritis (9–17) Controls (9–17)	108 18 40	Computerized: PedANAM	Attention Cognitive processing efficiency Executive functions Memory/Learning Spatial processing Working memory	33–55 min
Monitoring	Lai et al. (2014)	Pediatric cancer (7–17)	515	Questionnaire: pedsPCF (parent)	Global cognitive functioning	Not reported
Monitoring	Vega-Fernandez et al. (2014)	Systemic lupus erythematosus (10–16) Healthy controls (10–16)	40 24	Questionnaires: BRIEF (parent) CBCL (parent) SAND-C (child)	Global cognitive functioning Executive functioning Psychosocial functioning	Not reported
Monitoring	Brunner et al. (2013)	Systemic lupus erythematosus Healthy controls	40 40	Computerized: Ped-ANAM	Attention Cognitive processing efficiency Executive functions Memory/Learning Spatial processing Working memory	35–55 min
Monitoring	Brunner et al. (2007)	Systemic lupus erythematosus (10–21)	27	Computerized: Ped-ANAM	Attention Cognitive processing efficiency Executive functions Memory/Learning Spatial processing Working memory	30–40 min
Screening	Charvet et al. (2014)	Multiple sclerosis (6–24) Other neurological diagnoses (6–21) Healthy controls (8–19)	70 40 32	Paper and pencil: SDMT	Processing speed	5 min
Screening	Castellino, Tooze, Flowers, and Parsons (2011)	Pediatric brain tumor survivors (9–17)	13	Paper and pencil: PPVT	Language	20 min
Screening	Krull et al. (2008)	Pediatric cancer (6–18)	48	Paper and pencil: Grooved pegboard HRNB: Trail making MAE-3: Verbal fluency WISC/WAIS: DS Questionnaire: CSI (parent)	Global cognitive functioning Psychosocial functioning	30 min

Note: BASC-2 = Behavior Assessment System for Children, Second Edition; BRIEF = Brief Inventory of Executive Function; CMS = Children’s Memory Scale; COWAT = Controlled Oral Word Association Test; CVLT-C/II = California Verbal Learning Test for Children/California Verbal Learning Test, Second Edition; HRNB = Halstead-Reitan Battery; MAE-3 = Multilingual Aphasia Examination, Third Edition; PAT = Psychosocial Assessment Tool; Ped-ANAM = Pediatric Automated Neuropsychological Assessment Metrics; PEDS = Parents’ Evaluation of Developmental Status; pedsPCF = Pediatric Perceived Cognitive Function Item-Bank; PPVT = Peabody Picture Vocabulary Test; SAND-C = Subjective Awareness of Neuropsychological Deficits Questionnaire for Children; SDMT = Symbol Digit Modality Test; SDQ = Strengths & Difficulties Questionnaire; TEA-Ch = Test of Everyday Attention for Children; TMT = Trail Making Test; TOVA = Test of Variables of Attention; VMI = Beery Test of Visual Motor Integration; WAIS = Wechsler Adult Intelligence Scale; WASI = Wechsler Abbreviated Scale of Intelligence; WIAT = Wechsler Individual Achievement Test; WISC = Wechsler Intelligence Scale for Children.

Computerized performance-based tasks and symptom questionnaires are best-suited to this level of assessment, as they are brief and easily administered. Computerized batteries are particularly suitable, as they target cognitive domains most sensitive to disruption, are automated, and often manage practice effects with multiple forms. These features enable providers to re-assess patients more rapidly than is valid using traditional measures. We identified three studies utilizing a computerized measure, the Pediatric Automated Neuropsychological Assessment Metrics (PED-ANAM), within pediatric populations. The PED-ANAM was developed from conventional neuropsychological tests and designed for repeated testing, and shows good concurrent validity and sensitivity.

Similarly, symptom questionnaires often are brief, and require minimal training to administer and score. We identified four studies that used questionnaires exclusively to monitor for neuropsychological impairment. Most studies described a meaningful association between questionnaire ratings and cognitive functioning. However, consistent with previous research (Wochos, Semerjian, & Walsh, 2014), there was evidence that teacher ratings correlate more robustly with cognitive performance than parent ratings. It will be important to include some measure of functioning within the academic setting as part of screening (and monitoring) given the limitations of parent report in at least some pediatric medical groups.

Monitoring batteries investigated thus far have shown high sensitivity to cognitive impairment (Bull et al., 2015; Brunner, 2007; Brunner et al., 2013; Vega-Fernandez et al., 2015), which is critical for this level of evaluation. This approach has also shown good feasibility (Brunner, 2007; Brunner et al., 2013) and acceptance by patients and families (Triplett & Asato, 2015). It is notable that an approach fitting our description of the monitoring battery is already being used in adult primary care practices to track potential cognitive changes in elderly patients. Indeed, cognitive screening is a requirement of Medicare’s Annual Wellness Visit, and the Alzheimer’s Association recently recommended use of structured assessments tools, including computerized assessments and questionnaires (Cordell et al., 2013). This recommendation was based, in part, on evidence that structured cognitive assessments are more accurate than provider observation at detecting cognitive impairment (Borson, Scanlan, Watanabe, Tu, & Lessig, 2006). Children followed with monitoring batteries may be identified as needing more comprehensive testing based on the results of a single assessment or on a threshold of decline over time. Depending on the type and severity of difficulties identified and anticipated cognitive risks based on a child’s medical status, the testing clinician can make an appropriate referral for a focused battery of screening tests (Targeted Screening), advance to a comprehensive neuropsychological evaluation (Comprehensive Evaluation), and/or targeted treatment (Figures 1 and 2).

Figure 2.

Methodology of the prevention-based model of neuropsychological service delivery.

Targeted Screening

The second tier of the model, Targeted Screening¸ is intended to apply to children who either have a higher risk status based on unique features of their medical condition or treatment, or who have exceeded cutoffs indicating impairment on Universal Monitoring. Screening batteries are designed to be flexible and abbreviated (2–3 hr maximum), but to include traditional measures of neuropsychological functioning administered by a psychologist or neuropsychologist. As with monitoring batteries, screening tools should be selected on the basis of their appropriateness to a specific medical population. Results from Targeted Screening may be used as the basis for referral to a “full” neuropsychological assessment, justification for insurance coverage of further testing (Walsh et al., 2016), or to initiate the process of establishing therapeutic interventions.

Within chronic pediatric health conditions, neuropsychologists have begun to assess the utility and feasibility of administering a focused battery to detect individual cognitive deficits and identify those patients most in need of a more comprehensive evaluation. Three published studies have used assessments corresponding to our definition of a screening battery (Table I) and that provide information on the ability of the battery to identify deficits adequately. All of the studies used traditional, paper-and-pencil measures of neuropsychological functioning to evaluate broad cognitive domains, and one also included symptom questionnaires completed by a caregiver. The reported time to complete these batteries was an hour or less.

The utility of these abbreviated batteries has been promising as a means of detecting cognitive dysfunction, identifying impairments in specific functional domains, and discriminating between patient populations in an efficient, individualized manner. Targeted screening batteries may provide stand-alone information to inform interventions, or supplement data from psychoeducational assessments conducted through the public school systems with the use of measures that characterize disease-specific cognitive profiles.

Comprehensive Evaluation

Within pediatric medical populations, there will always be a need for some children to undergo comprehensive neuropsychological evaluation. An adaptive model of care allows children at greatest risk (e.g., brain tumors treated with radiation), those who have experienced medical events with established impact on the central nervous system (e.g., stroke), or those already demonstrating clear cognitive, academic, or other functional impairments to bypass monitoring and screening assessments and undergo thorough neuropsychological evaluation on initial referral. Results of comprehensive testing may identify emerging cognitive deficits or vulnerable domains that could be subsequently tracked by monitoring or screening assessments, utilizing the full spectrum of care options this model provides. This approach may also help providers track responsiveness to pharmacological, cognitive, or behavioral interventions, document the trajectory of emerging or worsening deficits, or reduce the need for full assessments by establishing stability of functioning over time. In this way, full assessments need not be scheduled as a matter of course, but rather conducted when there is evaluative or medical evidence to support it.

Clinical Implications and Future Directions

Applying an individualized, evidence-based approach to neuropsychological care will maximize the use of temporal, financial, and personnel resources and increase access to care for a greater number of at-risk patients. This prevention-based model is flexible, applicable to a wide variety of patient populations, and tailored to meet individual needs. This model requires an expansion beyond traditional neuropsychological service approaches, including consideration of, and development of, novel tools. Given advances in technology, development of such tools should include consideration of cost, ease of administration and scoring, and minimization of practice effects. Tablet-based tools and assessments that can be completed at home have the potential to be particularly beneficial in this regard. The psychometric properties of tests and batteries for this use need to be thoroughly investigated for sensitivity, specificity, practice effects, and reliability, ideally by utilizing analytic techniques prevalent in the precision-medicine literature (e.g., receiver operating curves).

As with many proposed changes to care, there are likely to be barriers to successful implementation of this model at both the monitoring and screening levels. Specialty care clinics may feel that they lack the time or resources to conduct brief cognitive assessments, or they may be pessimistic about receiving sufficient reimbursement for such services. Some clinics may be able to conduct monitoring evaluations, but be uncertain about when, how, and to whom they should refer their patients for further testing, particularly at institutions that lack robust pediatric psychology or neuropsychology presence. Finally, psychologists and neuropsychologists whose clinics are structured around full-day, comprehensive battery models may have difficulty adapting their schedules and personnel to accommodate shorter, screening batteries.

Despite these potential barriers, implementation of this model of care should lead to more prompt and precise identification of at-risk children and their neuropsychological needs, with the goal of implementing preventative, proactive therapeutic services. Because the model uses a multidisciplinary framework, it is also likely to result in improved communication between a child’s medical providers and psychologists or neuropsychologists involved in care, as well as improved documentation in the medical record of children’s cognitive difficulties and needed interventions. This model also serves to inform interventions through the various levels of assessment, driven by evidence of need at the individual level. In addition, the collection and use of monitoring and screening data should be appealing to third-party payors as providing evidence for additional assessment and intervention coverage, which ultimately would improve the utilization of fiscal resources. Importantly, intervention for cognitive difficulties in children with medical illness may include revising treatment of the disease in addition to more traditional school-based services and supports or pharmacotherapies. That is, if cognitive problems are thought to be driven, in part, by suboptimal disease management (e.g., sickle-cell disease, epilepsy, diabetes). The model can then be applied to determine effectiveness of medical modifications and interventions as well as the emergence of new or intensifying cognitive impairments, which then can be used to fine tune interventions in real time.