Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Dec 16.
Published in final edited form as: Contemp Clin Trials. 2018 Jul 11;72:8–15. doi: 10.1016/j.cct.2018.07.004

Design and rationale of the “Sedation strategy and cognitive outcome after critical illness in early childhood” study

Martha AQ Curley 1,#, R Scott Watson 2,#, Amy M Cassidy 3, Cheryl Burns 4, Rachel L Delinger 5, Derek C Angus 6, Lisa A Asaro 7, David Wypij 8, Sue R Beers 9, RESTORE-cognition Study Investigators
PMCID: PMC6914341  NIHMSID: NIHMS1545222  PMID: 30017814

Abstract

There is increasing concern that sedatives commonly used during critical illness may be neurotoxic during the period of early brain development. The Sedation strategy and cognitive outcome after critical illness in early childhood (RESTORE-cognition) study is a prospective cohort study designed to examine the relationships between sedative exposure during pediatric critical illness and long-term neurocognitive outcomes. We assess multiple domains of neurocognitive function 2.5-5 years post-hospital discharge, at a single time point and depending on participant and clinician availability, in up to 500 subjects who had normal baseline cognitive function, were aged 2 weeks to 8 years at pediatric intensive care unit admission, and were enrolled in a cluster randomized controlled trial of a sedation protocol (the RESTORE trial; U01 HL086622 and HL086649). In addition, to provide comparable data on an unexposed group with similar baseline biological characteristics and environment, we are studying matched, healthy siblings of RESTORE patients. Our goal is to increase understanding of the relationships between sedative exposure, critical illness, and long-term neurocognitive outcomes in infants and young children by studying these subjects 2.5 to 5 years after their index hospitalization. This paper highlights the design challenges in conducting comprehensive neurocognitive assessment procedures across a broad age span at multiple testing centers across the United States. Our approach, which includes building interprofessional teams and novel cohort retention strategies, may be of help in future longitudinal trials.

Keywords: healthcare outcomes, respiratory failure, pediatric, post-intensive care syndrome

INTRODUCTION

Ensuring the safety and comfort of the more than 100,000 critically ill infants and children supported on mechanical ventilation (MV) in the U.S. each year is integral to the practice of pediatric critical care.(1-3) Humane care of these young patients requires the use of sedating medications, most commonly combinations of opioids and benzodiazepines.(2, 3) Unfortunately, sedative use also carries risk. Animal studies found that even transient administration of benzodiazepines and other sedatives during periods of developmental synaptogenesis(4) caused widespread neuronal apoptosis and residual learning and memory deficits. (5-9) Sedation can be administered for days to weeks to >90% of acutely-ill, ventilated infants and children.(2, 3) Thus, a commonly used treatment in critically ill young children may itself have detrimental, age-dependent long-term effects.(10)

Most studies on this topic have analyzed large, existing databases to explore relationships between exposure to anesthesia in infancy or early childhood and subsequent school-related learning problems or academic achievement.(11-16) In a population-based retrospective birth cohort study in Olmsted County, MN, children younger than 4 years receiving multiple anesthestics had higher than baseline rates of learning disabilities, with the risk increasing with anesthetic duration.(14) Using the same database, but comparing outcomes after vaginal vs. cesarean delivery and exposure to general vs. regional anesthesia, no relationship was found between anesthesia exposure and subsequent learning disabilities.(13) To our knowledge, there are no studies investigating neurocognitive outcomes after prolonged exposure to sedatives as used in the pediatric intensive care unit (PICU) in older infants and children.

An opportunity to increase our understanding of the long-term neurocognitive effects of sedation during pediatric critical illness was provided by the cluster randomized controlled trial (RCT) of a sedation protocol, Randomized Evaluation of Sedation Titration for Respiratory Failure (RESTORE, U01 HL086622 and HL086649). This trial enrolled 2449 patients at 31 sites across the U.S. to determine whether a sedation protocol used at intervention sites decreased MV duration and sedative exposure among children with acute respiratory failure (vs. usual care at control sites).(17) The protocol did not change the duration of MV but did allow patients to be more awake while intubated and exposed to fewer sedative and analgesic medications. Detailed data were collected on doses and durations of sedative medications and in-hospital course.

In this study, Sedation strategy and cognitive outcome after critical illness in early childhood (RESTORE-cognition), we are assessing multiple domains of neurocognitive function 2.5 to 5 years post-hospital discharge in a subset of RESTORE subjects and matched, healthy siblings. The purpose of RESTORE-cognition is to determine the relationships between sedative exposure during pediatric critical illness and long-term neurocognitive outcomes. The study is funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01 HD074757; Multiple Principal Investigators: Curley and Watson) and is being coordinated at the University of Pennsylvania (Philadelphia, PA) and Seattle Children’s Research Institute (Seattle, WA).

MATERIALS AND METHODS

Objectives and hypotheses

RESTORE-cognition has two primary objectives/aims:

  • To determine if the magnitude of exposure to specific sedative medications is associated with neurocognitive outcomes when controlling for severity of illness, hospital course, and baseline factors.

    We are testing hypotheses that IQ will be lower in subjects exposed to 1) Any sedative medication (vs. no exposure in sibling controls), 2) Higher cumulative/peak/average daily doses of sedative medications (vs. lower doses, to determine dose-response), and 3) Specific classes of medications. We are examining similar hypotheses related to secondary outcomes of eight neurocognitive domains. We are also testing specific hypotheses, motivated by results of animal studies,(9) that benzodiazepines and ketamine will be associated with greater impairment in memory and learning and visual-spatial skills than in other domains.

  • To determine if exposure to sedative medications at earlier developmental stages is associated with worse neurocognitive outcomes when controlling for medication, dose, and severity of illness.

    We are testinghypotheses that younger children exposed to sedative medications, specifically benzodiazepines and/or ketamine, have worse neurocognitive outcomes than older children with similar sedative exposure.

Design, participants, and timeframe of enrollment and visits

RESTORE-cogtiition is a 5-year prospective cohort study enrolling up to 500 RESTORE subjects and a control group of matched siblings for neurocognitive outcome assessment. As part of the RESTORE trial, we obtained consent for a telephone follow-up interview 6 months after PICU discharge from 87% of patients’ parents/guardians.(18) From these consented subjects, we identify potentially eligible subjects for RESTORE-cognition and contact their parents/guardians by mail and/or phone to determine eligibility and obtain consent for RESTORE-cognition. Data collection then begins with a brief telephone interview followed by an in-person neurocognitive assessment 2.5 to 5 years after hospital discharge starting at 4 years of age (see Figure 1). For Spanish-only speaking families, we conduct the interview and assessment in Spanish.

Figure 1. Study recruitment and flow.

Figure 1.

HRQL – Health-related quality of life; PCPC – Pediatric Cerebral Performance Category

The study was approved for human subjects at the University of Pennsylvania (PENN). Local sites can opt to enter into a reliance agreement with PENN, engage in an individual investigator’s agreement with PENN, or obtain local institutional review board (IRB) approval. Parents/guardians are consented in person at the time of neurocognitive testing. Children of the appropriate age, according to IRB requirements at each site, are asked to provide assent.

Participants

We are focusing on young subjects who were deemed to be generally within normal limits cognitively at RESTORE study discharge (Table 1). From the RESTORE study database, we identified subjects meeting age criteria at PICU admission and PCPC criteria at PICU admission and hospital discharge. These subjects were considered as potentially eligible for RESTORE-cognition. Sibling controls are selected to be as similar as possible to the RESTORE subject in age and environmental exposure. If more than one sibling meets enrollment criteria, the sibling closest in age is selected.

Table 1.

RESTORE-cognition participant eligibility

graphic file with name nihms-1545222-t0002.jpg

Definitions of abbreviations: MV = mechanical ventilation; PCPC = Pediatric Cerebral Performance Category; PICU = pediatric intensive care unit; TBI = traumatic brain injury.

Recruitment

At the conclusion of subject participation in RESTORE, we sent letters of invitation to all potentially eligible families explaining RESTORE-cognition. Interested parents/guardians were invited to contact study personnel to complete a more detailed screening questionnaire via a telephone call during which enrollment criteria were assessed. We attempted to contact via phone families whose letters were returned as “undeliverable.” During the first year of enrollment, we relied on phone and address information from the RESTORE database. In the second year of enrollment, we attempted to contact previously unreachable families by using secondary contacts. Subsequently, we subscribed to an online database (LexisNexis®) to obtain updated contact information on families who had new addresses or phone numbers. Recruitment flyers and data-limited voicemails were sent to new addresses, emails, and phone numbers found using this service. We also returned to the original RESTORE sites to search for updated contact information and leveraged our East and West Coast coordination centers to make phone calls in the evening hours. With approval from our IRB, we searched Facebook and sent messages when we could confirm that a parent post included a RESTORE subject. We also built a website and study Facebook page so that parents themselves could search for and find the study. Although the most reliable way of maintaining contact with families has been through their mobile devices, phones may be turned off to save minutes, connections lost due to provider coverage, or range limited by relying on Wi-Fi signals. We protocolized our process for finding difficult-to-reach families, exhausting all efforts of contact before classifying a family as unable to be contacted.

Eligible and verbally consented parent/guardians are provided with information regarding the neurocognitive testing visit and participate in a 15-20 minute interview to determine their child’s health and functional status, heath care-related resource use, and health-related quality of life.

Exam/visit components

Neurocognitive testing is conducted in-person by either a licensed psychologist with specialized training in neuropsychology or a supervised neuropsychology technician. Appointments for the neurocognitive testing are made per preference of the local testing site (centrally by a coordinating center or locally by the testing site). Appointments made centrally are optimal because that method allows scheduling during the enrollment call, a consistent point of contact for families, and relationship-building with coordinating center study staff.

Long time intervals between enrollment and testing hinder subject retention. Many families prefer to delay testing until school holidays or over the summer, in which case study staff send appointment reminders and attempt to maintain family contact during the interim. In addition, personnel at testing sites are commonly busy conducting clinical assessments, and some have limited availability to conduct research assessments. This has required addition of more than one testing site in some locations, commonly identified via referrals from academic centers and via professional societies and networks. Larger academic sites frequently have more flexibility in subject scheduling, and private practices often have weekend options important to families with school-aged children.

When a cluster of potential subjects is identified in a location without testing facilities or capacity, we send a member of our core outcome team to the area to conduct the assessments. The logistics of these visits are coordinated centrally and involve the creation of an appropriate, quiet testing environment, most often using a hotel conference room.

Primary outcome:

The primary outcome is IQ estimated by the age-appropriate Wechsler Scale of Intelligence Vocabulary and Block Design subtests.(19, 20) Secondary outcomes include attention, processing speed, learning and memory, visual-spatial skills, motor skills, language, executive function, and behavior as assessed using standardized tests (Table 2). These domains have been found to be relevant in preclinical studies of sedation, studies of adults with acute lung injury/acute respiratory distress syndrome, and pediatric studies of critically ill infants (hypoxemia, shock, prematurity).(21-33) We begin testing at 4 years of age to provide a comparable testing battery thoughout the subject age range.

Table 2.

Testing battery

CHILD NEUROPSYCHOLOGICAL ASSESSMENT
Domain Instrument Time
Estimated
(minutes)
Age (years:months)
4:0 –
4:11
5:0 –
5:11
6:0 –
16:11
Intellectual Ability WPPSI-IV Vocabulary & Block Design 20 X X
WISC-IV Vocabulary & Block Design 20 X
Language WPPSI-IV/WISC-IV Vocabulary As above X X X
Processing Speed WPPSI-IV Coding 5 X X
WISC-IV Symbol Search & Coding 5 X
Memory and Learning TOMAL-2 Memory for Stories, Facial Memory, & Memory for Location 20 X X
WPPSI-IV Zoo Locations 8 X
CVLT-C 30 X X
Visual-spatial Skills VMI – 6th Ed. 5 X X X
WPPSI-IV/WISC-IV Block Design As above X X X
Attention TOMAL-2 Digits Forward, Digits Backward 5 X X
Motor and Psychomotor Skills BOT-2 Fine Motor 5 X X X
PARENT/CHILD REPORT
Attention and Executive Function BRIEF Preschool & Child Versions 10 X X X
Behavior and Emotion BASC-II* 20 X X X
Functional Status and Adaptive Behavior ABAS-II Parent Report Form 15 X X X
PCPC/POPC 2 X X X
Quality of Life PedsQL – Version 4.0 5 X X X
Developmental History / Medical Confounders Detailed Medical & Developmental History Questionnaire 5 X X X
Potential Social / Environmental Confounders MOS Social Support Survey 3 X X X
SES Questionnaire 4 X X X

Definitions of abbreviations: ABAS-II = Adaptive Behavior Assessment System – 2nd Ed.; BASC-II = Behavior Assessment System for Children – 2nd Ed.; BOT-2 = Bruininks-Oseretsky Test of Motor Proficiency – 2nd Ed.; BRIEF = Behavior Rating Inventory of Executive Functioning; CVLT-C = California Verbal Learning Test-Child; MOS = Medical Outcomes Study; PCPC/POPC = Pediatric Cerebral Performance Category/Pediatric Overall Performance Category; PedsQL = Pediatric Quality of Life Inventory – Version 4.0; SES = Socioeconomic status; TOMAL-2 = Test of Memory and Learning – 2nd Ed.; VMI = Test of Visual-Motor Integration – 6th Ed.; WISC-IV = Wechsler Intelligence Scale for Children – 4th Ed.; WPPSI-IV = Wechsler Preschool and Primary Scale of Intelligence – 4th Ed.

*

The BASC-II includes a structured developmental history.

Includes birth, developmental, chronic illness, and anesthesia history.

Exposures of interest:

As part of the RESTORE trial, we collected extensive data on exposure to all sedative medications:

  • Classes of medications: Benzodiazepines, opioids, barbiturates, N-methyl-D-aspartate (NMDA) antagonists, alpha-2 adrenergic agonists, volatile anesthetic agents, other.

  • Amount of medication, in mg/kg: a) Cumulative dose received during study period (in midazolam equivalents for benzodiazepines, morphine equivalents for opioids), b) highest (peak) dose during a 24-hour period (midnight-midnight), c) average dose during a 24-hour period, and d) serum concentration profile (when available from a subset of subjects enrolled in RESTORE ancillary study; R01 HL098087).

  • Duration of exposure: Total number of days exposed.

  • Development of tolerance: Decreasing clinical effects of a drug after prolonged exposure,(34) operationalized as a doubling (or more) of the dose provided on the first full day of drug exposure.

Potential confounders or effect modifiers:

1) Baseline health (medical history [birth history, chronic conditions], developmental history, nutrition history, and environment [socioeconomic status (SES), maternal education level]; 2) Presenting complaint (e.g., pneumonia, bronchiolitis, sepsis, asthma); 3) Severity of illness (Pediatric Risk of Mortality [PRISM III-12] score(35), number of organ dysfunctions, daily oxygenation index, iatrogenic withdrawal syndrome); and 4) Hospital course (duration of MV, PICU stay, and hospitalization). Baseline health data not collected as part of the RESTORE trial were collected during the testing visit.

Quality assurance and control procedures

We designed a broad-based assessment that balances the testing of multiple domains with the feasibility of performing evaluations on hundreds of subjects at multiple centers. From the many available standardized instruments, our choices are widely used and familiar to most experienced pediatric neuropsychologists. These instruments are psychometrically sound, have representative normative data from large samples of healthy children, and generate a focused assessment of critical at-risk domains. Evaluations require approximately 2 hours to conduct. A description of assessment instruments is provided in the Box 1.

Box 1. Description of testing battery.

Intelligence, language, processing speed, and visual-spatial skills

Wechsler Preschool and Primary Scale of Intelligence, 4th ed. (WPPSI-IV)(20); Wechsler Intelligence Scales for Children, 4th ed. (WISC-IV).(19) The WPPSI-IV is a measure of cognitive functioning of children 2 years, 6 months to <6 years of age; the WISC-IV is for those 6 to 16 years of age. Four subtests are used: 1) Vocabulary – to assess language skills, 2) Block Design – to assess nonverbal intellectual function, 3) Symbol Search – to assess processing speed, and 4) Coding – to assess processing speed. The Vocabulary and Block Design subtests will provide an estimate of Full Scale IQ.

Memory and learning

Test of Memory and Learning, 2nd ed. (TOMAL-2).(56) Instrument used in subjects over age 5 to assess verbal, non-verbal, and composite memory through the following subtests: Memory for Stories, Facial Memory, Memory for Location, and Memory for Stories Delayed.(20)

Wechsler Preschool and Primary Scale of Intelligence, 4th ed. – Zoo Location subtest. Measure of cognitive development for preschoolers and young children that emphasizes developmentally appropriate features including processing speed and working memory.

California Verbal Learning Test – Child (CVLT-C).(57) Assesses multiple strategies and processes in learning and recalling verbal material in children and adolescents 5 to 16 years of age. It tests 8 recall and 4 recognition measures associated with verbal learning and provides data on encoding strategies and errors.

Visual-spatial skills

Test of Visual-Motor Integration, 6th ed. (VMI).(58) Identifies problems with motor control and visual perception. Helpful in identifying children who may need special educational services and learning assistance.

Attention and executive function

Test of Memory and Learning, 2nd ed. (TOMAL-2).(56) Provides a supplementary index of attention and concentration, as measured through the digits forward and digits backward subtests.

Behavior Rating Inventory of Executive Function (BRIEF).(59, 60) Used to assess executive functioning (e.g., ability to regulate behavior, working memory, ability to plan, organize, and initiate). The 63-item preschool version is completed by parents of subjects 3 years to <6 years of age. The 86-item version is used for subjects 6 years and older.

Motor and psychomotor skills

Bruininks-Oseretsky Test of Motor Proficiency, 2nd Ed. (BOT-2).(61) Fine Motor Precision subtest used in subjects aged 3.5 years and older.

Behavior and emotion

Behavior Assessment System for Children, 2nd ed. (BASC-II).(62) Multi-dimensional assessment of behaviors, thoughts, and emotions. Includes an assessment of externalizing, internalizing, school, and other problems as well as adaptive skills, behavioral symptoms, hyperactivity, and attention problems.

Functional status and adaptive behavior

Adaptive Behavior Assessment System, 2nd ed. (ABAS-II), Parent Form.(63) Measure of daily living skills evaluating 3 domains (conceptual, social, and practical) including the 10 adaptive skills specified in the DSM-IV.(64)

Pediatric Cerebral Performance Category (PCPC) and Pediatric Overall Performance Category (POPC).(55) Classifies a child’s cerebral and overall funcation status in broad categories (ranging from normal to brain dead).

Health-related quality of life (QOL)

Pediatric Quality of Life Inventory, Version 4.0 (PedsQL).(65). Evaluates QOL in four areas: physical, social, emotional, and school-based functioning.

Family and environment

Data on SES, parental education level, and family characteristics including the Medical Outcomes Study (MOS) Social Support Survey(66) that consists of four separate social support subscales and an overall functional social support index.

Measures to ensure protocol fidelity across multiple testing sites:

Trained neuropsychologists or supervised neuropsychology technicians perform each neurocognitive assessment. We created a detailed study manual of operations that includes an assessment battery manual and data codebook, which is available on the study website. We held an in-person start-up workshop for the site testers and supervisors, chaired by our neuropsychology lead (SRB). Her team also conducts one-on-one training sessions with those joining the study after the initial start-up meeting. After evaluation, the lead neuropsychology team at the University of Pittsburgh reviews each test protocol to ensure that each data packet is complete and absent of errors. If either nonstandard or inaccurate test administration is noted, further training is provided as required for ongoing quality control of tester performance. Finally, the principal investigators and project managers conduct routine calls to review procedural issues, progress, and problems. The training session and conference calls help problem solve general and specific subject issues, lessen examiner drift, and foster a sense of community among study personnel.

Measures to retain subjects and minimize missing data:

We have frequent communication with testing sites to discuss subjects entering the testing window and provide logistical, troubleshooting, and problem-solving resources. In addition, successful follow-up is facilitated by the following strategies:

  • Collection of extensive family contact information.

  • Telephone interviews with a limited number of interviewers.

  • Regular contact with families by mail, phone, and email.

  • Development of study website and Facebook page to enhance study enthusiasm.

  • Flexibility in accommodating family schedules with a wide window around the clinic visit and offering weekend/evening testing whenever possible.

  • Child care and transportation support as needed for neurodevelopmental clinic visit appointments.

  • Reimbursement for subject and family’s time.

  • Provision of verbal feedback of performance at the time of appointment. If requested, a written summary describing the child's cognitive and behavioral profile is also provided. Treatment recommendations, if any, are noted.

  • Assistance in referrals for medical and psychiatric services that directly benefit the child and enhance participation are provided.

  • Troubleshooting challenges of individual families, such as providing additional transportation funds, and arranging lodging for families living a long distance from study centers.

Data management plan

Once all testing forms are scored, they are scanned and emailed to the neuropsychology team at the University of Pittsburgh within 5 working days of the assessment. After testing data is reviewed by the central neuropsychology team, data are coded for validity and entered into a web-based data entry interface programmed in DATSTAT at the Center for Child Health, Behavior, and Development at the Seattle Children’s Research Institute.

Sample size and power

We estimated that approximately 500 RESTORE subjects would be eligible for RESTORE-cognition of whom at least 80% would consent and undergo testing (n=400 primary subjects), and of whom 62% would have eligible siblings (n=248 sibling controls). All power calculations assume a two-sided significance level of 0.05 and power of 0.8. With 400 primary subjects, the study is powered to detect correlation coefficients of 0.14 or greater between IQ and sedative doses. For multivariable modeling, assuming five covariates already included in a regression model with an R2 value of 0.20, we have sufficient power to detect a new predictor variable that improves the R2 value by 0.016. When the total sample is split into subgroups, even for comparing two groups of 100 subjects each, the power is sufficient to detect differences of 0.4 SD or larger (e.g., 6 IQ points). Assuming sibling correlations between 0 and 0.5,(36) the study is powered to detect mean differences of 0.2 SD (e.g., 3 IQ points) between 248 sibling pairs. Thus, for any correlation or regression analyses or comparisons of subgroups or sibling pairs, the study is sufficiently powered.

Analysis plan

Confounding will be an important issue when delineating the relationship between sedation and neurocognitive outcomes. Therefore, we will perform extensive regression modeling, including formal variable selection methods, model assessment techniques, and partial or sparse partial least squares methods to evaluate the effects of specific sedative medications on neurocognitive outcomes. Models including interaction terms will be used to assess whether age is an effect modifier of the sedative mediation effect. Center effects will be addressed using generalized estimating equations or hierarchical models with center included as a random effect. As secondary analyses, we will also test the effect of RESTORE treatment group (sedation protocol vs usual care) on neurocognitive outcomes using regression methods.

DISCUSSION

There is increasing concern that commonly used sedatives in critically ill children may have detrimental long-term effects. This study leverages the opportunity of a large clinical trial to enhance the understanding of the prolonged neurocognitive effects of sedation in young children. We are determiningif specific medications or combinations of medications, amounts of medications, or durations of exposure to medications increase the risk of subsequent neurocognitive impairment and if that risk changes with age. We are taking advantage of rich clinical data from a pediatric sedation trial, enrollment of sibling controls, and sophisticated statistical analyses to assess the influence of multiple potential confounding factors. Given the extensive use of sedation in children, this information has great importance not only to patients and families, but also to society in general.

A dose-response relationship between medications and neurocognitive impairment, when controlling for baseline and quantifiable insult from critical illness, would suggest that the medications themselves may contribute to the mechanistic pathway of impairment. Even more compelling would be if an effect found in multivariable analyses were limited to specific classes of medication (e.g., benzodiazepines, but not opioids), or if different classes were associated with different domains of impairment (e.g., in visual-spatial skills but not memory).

Similarly, finding that subjects exposed to certain medications at a young age are at greater risk of neurocognitive impairment than older children when controlling for illness and sedative exposure would be consistent with many preclinical studies and would also implicate the medications themselves as contributing to the adverse outcomes. Any of these findings would have the potential to markedly improve the critical care for infants and young children. Conversely, in light of increasing concern about the neurocognitive effects of sedation on the developing brain, it would be extremely reassuring to find that subjects exposed to sedative medications had similar neurocognitive function as their sibling controls and that there was no dose-response, age-dependent, or neurocognitive domain-specific relationship discernible in this population.

This study design does not eliminate the difficulty in untangling the neurocognitive effects of sedation vs. the effects of critical illness itself, as more severely ill children are likely to have greater exposures to sedative medications. Aspects of critical illness (e.g., severe hypoxemia) have been implicated as leading to impaired neurocognitive function in neonates and adults.(3, 21-29, 37-51) Severity of illness itself will increase the exposure to sedative medications (both in terms of duration, via longer durations of MV, PICU stay, and hospitalization; and potentially in terms of dose and types of medication). With longer exposure, tolerance develops, higher doses are needed to achieve the same effect, and different classes of medications are often added to the sedation regimen. We therefore are focusing on subjects without known significant impairment at baseline and will perform multivariable analyses to control for and assess the extent to which other baseline and illness factors are related to neurocognitive outcomes.

Variation in sedation exposure will also be related to patient-to-patient differences in the clinical effects of equivalent doses of medication. Sedation scores collected on all subjects in the RESTORE study will help us understand if these inter-individual differences affect the relationship between sedation exposure and neurocognitive outcomes, as will the subset of subjects on whom we have serum drug and metabolite levels.

Neuropsychology has gained a developmental perspective and benefits from tests specifically constructed to measure brain function and development during childhood. (52) Test selection, however, continues to present a particular challenge and requires expertise and training in pediatric (i.e., developmental) neuropsychology. Unlike instruments designed for adults, pediatric instruments differ depending on the developmental epoch, even though they may be measuring the same cognitive construct. The RESTORE-cognition battery is planned to assess a range of patients, from older toddlers to adolescents.

While it would be ideal to have multiple control groups varying only in a single aspect of illness or sedation exposure, it would be unethical to deny critically ill children sedation for the purposes of a study. Similarly, a non-critically ill cohort of children receiving sedation for procedures would have much different levels of exposure to sedative medications than the days-to-weeks of sedation that children receive in the PICU. We are taking advantage of 1) natural practice variation among sites and practitioners and 2) randomization to the use of a sedation protocol, both of which will create variability in the relationship between sedation exposure and illness severity/duration. This variability will increase our ability to estimate the independent neurocognitive effects of classes of sedative medications themselves.

By comparing primary subjects to biological sibling controls, we will gain additional information about the effects of sedation on neurocognitive outcomes due to the similar long-term environmental exposures and biological similarities between the two groups. However, little is known about sibling outcomes after critical illness in childhood, and those outcomes could affect the control group findings in this study. For example, parental stress and family financial strain could influence sibling development; and answers to questions in the parent-report measures may also be affected. If siblings suffer negative effects related to the critical illness, differences between critically ill children and the control siblings will be blunted. We will attempt to account for these factors as much as possible in analyses looking for common patterns of problems across the sibling control group and comparison of results to published norms for primary subjects and siblings for each measure. While it would have been ideal to have multiple control groups, their addition would have added logistical and cost burdens that would have risked making the study infeasible.

In summary, RESTORE-cognition will provide important insight into understanding the effects of sedative medications, as currently used in the PICU, on neurocognitive outcomes of infants and young children. Children with post-discharge neurocognitive impairment, even if mild, are at risk of poor school performance and may need special educational placement and assistance.(53, 54) In addition to informing other research being conducted in this area, this line of inquiry will enhance the design and feasibility of future clinical trials determining which sedative regimens are safest, thus improving the health and quality of life of children and their families after critical illness.

Acknowledgements:

The authors would like to acknowledge Dr. Bob Noll for sharing his measurement expertise, Dr. Jane Newburger for her expertise in successful cohort retention, Dr. Lisa Weissfeld for providing initial drafts of the data analysis plan, the RESTORE study investigators who participated in the parent study, and the Clinical Psychologists and Psychometrists who are providing their time and expertise in support of this study.

Supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health (R01 HD074757 to M.A.Q. Curley and R.S. Watson).

Appendix

The RESTORE-cognition study investigators include:

Martha A.Q. Curley (Principal Investigator; School of Nursing and the Perelman School of Medicine, University of Pennsylvania; The Children’s Hospital of Philadelphia Research Institute, Philadelphia, PA); R. Scott Watson (Principal Investigator; Department of Pediatrics, University of Washington, Seattle, WA; Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, Seattle, WA); Joseph Ackerson (Ackerson and Associates, Vestavia, AL); Derek C. Angus (Clinical Research, Investigation, and Systems Modeling of Acute Illness [CRISMA] Center and Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA); Lisa A. Asaro (Department of Cardiology, Boston Children’s Hospital, Boston, MA); Sue R. Beers (Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA); Mary Best (Yale University School of Medicine, New Haven, CT); Cheryl Burns (Department of Psychiatry, University of Pittsburgh School of Medicine, Pittsburgh, PA); Alanna A. Conder (Carolina Neuropsychological Service, Raleigh, NC); Rachyll Dempsey (Psychological Assessments Inc., Oakland, CA); Monica D. Dowling (University of Miami); Susanne W. Duval (Oregon Health and Science University, Portland, OR); Chaya B. Gopin (New York-Presbyterian Hospital/Weill Cornell Medical College, New York, NY); Lana L. Harder (Children’s Medical Center Dallas); Nina Hattiangadi Thomas (Children’s Hospital of Philadelphia); Katie Herrington (Nashville Family Wellness); Scott J. Hunter (University of Chicago); Gad E. Klein (Rockville Center, NY); Roger E. Lauer (Center for Neuropsychology, Learning and Development, Ann Arbor, MI); Jonathan D. Lichtenstein (Dartmouth-Hitchcock Medical Center, Lebanon, NH); Margaret M. Manning (University of Massachusetts Medical School, Worcester, MA); Joan W. Mayfield (Children’s Medical Center Dallas); Megan M. Morse (Munroe-Meyer Institute, Omaha, NE); Edward M. Moss (Moss and Malamut, PC, Bryn Mawr, PA); Grace A. Mucci (Children’s Hospital of Orange County, Orange, CA); Nicolle Napier-Ionascu (USCF Benioff Children’s Hospital Oakland); Marivelisse Rodriquez-Rivera (Children’s Medical Center Dallas); Cynthia F. Salorio (Kennedy Krieger Institute, Baltimore, MD); Anabela D. Smith (Hartford, CT); Julien T. Smith (Wasatch Pediatric Neuropsychology Inc., Salt Lake City, UT); J. Robin Timm (University of California Davis Medical Center); Marion Wallace (University of Alabama Birmingham); William J. Warzak (University of Nebraska Medical Center, Omaha, NE); Desiree A. White (Washington University, St. Louis, MO); Elizabeth J. Willen (Children’s Mercy Hospital Kansas City); Lisa J. Woodcock-Burroughs (Center for Neuropsychology, Learning and Development, Ann Arbor, MI); David Wypij (Department of Biostatistics, Harvard T.H. Chan School of Public Health; Department of Pediatrics, Harvard Medical School; Department of Cardiology, Boston Children's Hospital, Boston, MA); Maya M. Zayat (Nemours/AI duPont Hospital for Children, Wilmington, DE).

Footnotes

Trial Registration:

ATS Subject Code: 4.11 Pediatric Critical Care

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Martha A.Q. Curley, School of Nursing, University of Pennsylvania, Philadelphia; Perelman School of Medicine, University of Pennsylvania, Philadelphia; The Children’s Hospital of Philadelphia Research Institute, Philadelphia.

R. Scott Watson, Department of Pediatrics, University of Washington, Seattle; Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, Seattle.

Amy M. Cassidy, School of Nursing, University of Pennsylvania, Philadelphia.

Cheryl Burns, Department of Psychiatry, University of Pittsburgh School of Medicine.

Rachel L. Delinger, Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, Seattle.

Derek C. Angus, Clinical Research, Investigation, and Systems Modeling of Acute Illness Center and Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh.

Lisa A. Asaro, Department of Cardiology, Boston Children's Hospital, Boston.

David Wypij, Department of Cardiology, Boston Children's Hospital, Boston; Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston; Department of Pediatrics, Harvard Medical School, Boston.

Sue R. Beers, Department of Psychiatry, University of Pittsburgh School of Medicine.

REFERENCES

  • 1.Marx CM, Rosenberg DI, Ambuel B, Hamlett KW, Blumer JL. Pediatric intensive care sedation: survey of fellowship training programs. Pediatrics 1993; 91: 369–378. [PubMed] [Google Scholar]
  • 2.Rhoney DH, Murry KR. National survey on the use of sedatives and neuromuscular blocking agents in the pediatric intensive care unit. Pediatr Crit Care Med 2002; 3: 129–133. [DOI] [PubMed] [Google Scholar]
  • 3.Twite MD, Rashid A, Zuk J, Friesen RH. Sedation, analgesia, and neuromuscular blockade in the pediatric intensive care unit: survey of fellowship training programs. Pediatr Crit Care Med 2004; 5: 521–532. [DOI] [PubMed] [Google Scholar]
  • 4.Dobbing J, Sands J. Comparative aspects of the brain growth spurt. Early Hum Dev 1979; 3: 79–83. [DOI] [PubMed] [Google Scholar]
  • 5.Fredriksson A, Ponten E, Gordh T, Eriksson P. Neonatal exposure to a combination of N-methyl-D-aspartate and gamma-aminobutyric acid type A receptor anesthetic agents potentiates apoptotic neurodegeneration and persistent behavioral deficits. Anesthesiology 2007; 107: 427–436. [DOI] [PubMed] [Google Scholar]
  • 6.Young C, Jevtovic-Todorovic V, Qin YQ, Tenkova T, Wang H, Labruyere J, Olney JW. Potential of ketamine and midazolam, individually or in combination, to induce apoptotic neurodegeneration in the infant mouse brain. British Journal of Pharmacology 2005; 146: 189–197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Jorda EG, Jimenez A, Verdaguer E, Canudas AM, Folch J, Sureda FX, Camins A, Pallas M. Evidence in favour of a role for peripheral-type benzodiazepine receptor ligands in amplification of neuronal apoptosis. Apoptosis 2005; 10: 91–104. [DOI] [PubMed] [Google Scholar]
  • 8.Fredriksson A, Archer T, Alm H, Gordh T, Eriksson P. Neurofunctional deficits and potentiated apoptosis by neonatal NMDA antagonist administration. Behavioural Brain Research 2004; 153: 367–376. [DOI] [PubMed] [Google Scholar]
  • 9.Jevtovic-Todorovic V, Hartman RE, Izumi Y, Benshoff ND, Dikranian K, Zorumski CF, Olney JW, Wozniak DF. Early exposure to common anesthetic agents causes widespread neurodegeneration in the developing rat brain and persistent learning deficits. Journal of Neuroscience 2003; 23: 876–882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Rappaport B, Mellon RD, Simone A, Woodcock J. Defining safe use of anesthesia in children. N Engl J Med 2011; 364: 1387–1390. [DOI] [PubMed] [Google Scholar]
  • 11.Flick RP, Lee K, Hofer RE, Beinborn CW, Hambel EM, Klein MK, Gunn PW, Wilder RT, Katusic SK, Schroeder DR, Warner DO, Sprung J. Neuraxial Labor Analgesia for Vaginal Delivery and Its Effects on Childhood Learning Disabilities. Anesth Analg 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Flick RP, Wilder RT, Sprung J, Katusic SK, Voigt R, Colligan R, Schroeder DR, Weaver AL, Warner DO. Anesthesia and cognitive performance in children: no evidence for a causal relationship. Are the conclusions justified by the data? Response to Bartels et al, 2009. Twin Res Hum Genet 2009; 12: 611–612. [DOI] [PubMed] [Google Scholar]
  • 13.Sprung J, Flick RP, Wilder RT, Katusic SK, Pike TL, Dingli M, Gleich SJ, Schroeder DR, Barbaresi WJ, Hanson AC, Warner DO. Anesthesia for cesarean delivery and learning disabilities in a population-based birth cohort. Anesthesiology 2009; 111: 302–310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Wilder RT, Flick RP, Sprung J, Katusic SK, Barbaresi WJ, Mickelson C, Gleich SJ, Schroeder DR, Weaver AL, Warner DO. Early exposure to anesthesia and learning disabilities in a population-based birth cohort. Anesthesiology 2009; 110: 796–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Bartels M, Althoff RR, Boomsma DI. Anesthesia and cognitive performance in children: no evidence for a causal relationship. Twin Res Hum Genet 2009; 12: 246–253. [DOI] [PubMed] [Google Scholar]
  • 16.DiMaggio C, Sun LS, Kakavouli A, Byrne MW, Li G. A retrospective cohort study of the association of anesthesia and hernia repair surgery with behavioral and developmental disorders in young children. J Neurosurg Anesthesiol 2009; 21: 286–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Curley MAQ, Wypij D, Watson RS, Grant MJC, Asaro LA, Cheifetz IM, Dodson BL, Franck LS, Gedeit RG, Angus DC, Matthay MA for the RESTORE Study Investigators and the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Protocolized sedation vs usual care in pediatric patients mechanically ventilated for acute respiratory failure: A randomized clinical trial. JAMA 2015; 313: 379–389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Watson RS, Asaro LA, Hertzog JH, Sorce LR, Kachmar AG, Dervan LA, Angus DC, Wypij D, Curley MAQ, for the RESTORE Study Investigators and the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Long-term Outcomes After Protocolized Sedation vs Usual Care in Ventilated Pediatric Patients. Am J Respir Crit Care Med 2018. January 9. [Epub ahead of print]. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Wechsler D Wechsler Intelligence Scales for Children-4th Edition. San Antonio, TX: Psychological Corporation; 2003. [Google Scholar]
  • 20.Wechsler D Wechsler Preschool and Primary Scale of Intelligence, 4th Ed (WPPSI-IV). San Antonio, TX: Pearson; 2012. [Google Scholar]
  • 21.Hopkins RO, Weaver LK, Pope D, Orme JF, Bigler ED, Larson-Lohr V. Neuropsychological sequelae and impaired health status in survivors of severe acute respiratory distress syndrome. American Journal of Respiratory & Crit Care Med 1999; 160: 50–56. [DOI] [PubMed] [Google Scholar]
  • 22.Hopkins RO, Weaver LK, Collingridge D, Parkinson RB, Chan KJ, Orme JF Jr. Two-year cognitive, emotional, and quality-of-life outcomes in acute respiratory distress syndrome. Am J Respir Crit Care Med 2005; 171: 340–347. [DOI] [PubMed] [Google Scholar]
  • 23.Rothenhausler HB, Ehrentraut S, Stoll C, Schelling G, Kapfhammer HP. The relationship between cognitive performance and employment and health status in long-term survivors of the acute respiratory distress syndrome: results of an exploratory study. Gen Hosp Psychiatry 2001; 23: 90–96. [DOI] [PubMed] [Google Scholar]
  • 24.Hopkins RO, Weaver LK, Chan KJ, Orme JF Jr. Quality of life, emotional, and cognitive function following acute respiratory distress syndrome. Journal of the International Neuropsychological Society 2004; 10: 1005–1017. [DOI] [PubMed] [Google Scholar]
  • 25.Katz-Salamon M, Gerner EM, Jonsson B, Lagercrantz H. Early motor and mental development in very preterm infants with chronic lung disease. Archives of Disease in Childhood Fetal & Neonatal Edition 2000; 83: F1–F6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Singer L, Yamashita T, Lilien L, Collin M, Baley J. A longitudinal study of developmental outcome of infants with bronchopulmonary dysplasia and very low birth weight. Pediatrics 1997; 100: 987–993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hack M, Taylor HG, Klein N, Eiben R, Schatschneider C, Mercuri-Minich N. School-age outcomes in children with birth weights under 750 g. N Engl J Med 1994; 331: 753–759. [DOI] [PubMed] [Google Scholar]
  • 28.Hack M, Flannery DJ, Schluchter M, Cartar L, Borawski E, Klein N. Outcomes in young adulthood for very-low-birth-weight infants. N Engl J Med 2002; 346: 149–157. [DOI] [PubMed] [Google Scholar]
  • 29.Klein NK, Hack M, Breslau N. Children who were very low birth weight: development and academic achievement at nine years of age. Journal of Developmental & Behavioral Pediatrics 1989; 10: 32–37. [PubMed] [Google Scholar]
  • 30.Bronner MB, Knoester H, Sol JJ, Bos AP, Heymans HS, Grootenhuis MA. An explorative study on quality of life and psychological and cognitive function in pediatric survivors of septic shock. Pediatr Crit Care Med 2009; 10: 636–642. [DOI] [PubMed] [Google Scholar]
  • 31.Odd DE, Lewis G, Whitelaw A, Gunnell D. Resuscitation at birth and cognition at 8 years of age: a cohort study. Lancet 2009; 373: 1615–1622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Odd DE, Rasmussen F, Gunnell D, Lewis G, Whitelaw A. A cohort study of low Apgar scores and cognitive outcomes. Arch Dis Child Fetal Neonatal Ed 2008; 93: F115–F120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Elison S, Shears D, Nadel S, Sahakian B, Garralda ME. Neuropsychological function in children following admission to paediatric intensive care: a pilot investigation. Intensive Care Medicine 2008; 34: 1289–1293. [DOI] [PubMed] [Google Scholar]
  • 34.Anand KJ, Willson DF, Berger J, Harrison R, Meert KL, Zimmerman J, Carcillo J, Newth CJ, Prodhan P, Dean JM, Nicholson C. Tolerance and withdrawal from prolonged opioid use in critically ill children. Pediatrics 2010; 125: e1208–e1225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med 1996; 24: 743–752. [DOI] [PubMed] [Google Scholar]
  • 36.Devlin B, Daniels M, Roeder K. The heritability of IQ. Nature 1997; 388: 468–471. [DOI] [PubMed] [Google Scholar]
  • 37.Sukantarat KT, Burgess PW, Williamson RC, Brett SJ. Prolonged cognitive dysfunction in survivors of critical illness. Anaesthesia 2005; 60: 847–853. [DOI] [PubMed] [Google Scholar]
  • 38.Hopkins RO, Ely EW, Jackson JC. The role of future longitudinal studies in ICU survivors: understanding determinants and pathophysiology of brain dysfunction. Curr Opin Crit Care 2007; 13: 497–502. [DOI] [PubMed] [Google Scholar]
  • 39.Gunther ML, Jackson JC, Ely EW. Loss of IQ in the ICU brain injury without the insult. Med Hypotheses 2007; 69: 1179–1182. [DOI] [PubMed] [Google Scholar]
  • 40.Jackson JC, Hart RP, Gordon SM, Shintani A, Truman B, May L, Ely EW. Six-month neuropsychological outcome of medical intensive care unit patients. Crit Care Med 2003; 31: 1226–1234. [DOI] [PubMed] [Google Scholar]
  • 41.Hopkins RO, Jackson JC. Long-term neurocognitive function after critical illness. Chest 2006; 130: 869–878. [DOI] [PubMed] [Google Scholar]
  • 42.Ehlenbach WJ, Hough CL, Crane PK, Haneuse SJ, Carson SS, Curtis JR, Larson EB. Association between acute care and critical illness hospitalization and cognitive function in older adults. JAMA 2010; 303: 763–770. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Pustavoitau A, Stevens RD. Mechanisms of neurologic failure in critical illness. Critical Care Clinics; 24: 1–24. [DOI] [PubMed] [Google Scholar]
  • 44.Jones C, Griffiths RD, Slater T, Benjamin KS, Wilson S. Significant cognitive dysfunction in non-delirious patients identified during and persisting following critical illness. Intensive Care Med 2006; 32: 923–926. [DOI] [PubMed] [Google Scholar]
  • 45.Russell EW, Russell SL, Hill BD. The fundamental psychometric status of neuropsychological batteries. Arch Clin Neuropsychol 2005; 20: 785–794. [DOI] [PubMed] [Google Scholar]
  • 46.Stuss DT, Peterkin I, Guzman DA, Guzman C, Troyer AK. Chronic obstructive pulmonary disease: effects of hypoxia on neurological and neuropsychological measures. J Clin Exp Neuropsychol 1997; 19: 515–524. [DOI] [PubMed] [Google Scholar]
  • 47.Incalzi RA, Gemma A, Marra C, Muzzolon R, Capparella O, Carbonin P. Chronic obstructive pulmonary disease. An original model of cognitive decline. Am Rev Respir Dis 1993; 148: 418–424. [DOI] [PubMed] [Google Scholar]
  • 48.Bartholomew CJ, Jensen W, Petros TV, Ferraro FR, Fire KM, Biberdorf D, Fraley E, Schalk J, Blumkin D. The effect of moderate levels of simulated altitude on sustained cognitive performance. Int J Aviat Psychol 1999; 9: 351–359. [DOI] [PubMed] [Google Scholar]
  • 49.Chun MM, Phelps EA. Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage. Nat Neurosci 1999; 2: 844–847. [DOI] [PubMed] [Google Scholar]
  • 50.Litch JA, Bishop RA. Transient global amnesia at high altitude. N Engl J Med 1999; 340: 1444. [DOI] [PubMed] [Google Scholar]
  • 51.Hack M, Wilson-Costello D, Friedman H, Taylor GH, Schluchter M, Fanaroff AA. Neurodevelopment and predictors of outcomes of children with birth weights of less than 1000 g: 1992-1995. Archives of Pediatrics & Adolescent Medicine 2000; 154: 725–731. [DOI] [PubMed] [Google Scholar]
  • 52.Goldstein B, Obrzut JE, John C, Ledakis G, Armstrong CL. The impact of frontal and non-frontal brain tumor lesions on Wisconsin Card Sorting Test performance. Brain Cognition 2004; 54: 110–116. [DOI] [PubMed] [Google Scholar]
  • 53.Harbarth S, Holeckova K, Froidevaux C, Pittet D, Ricou B, Grau GE, Vadas L, Pugin J, Geneva SN. Diagnostic value of procalcitonin, interleukin-6, and interleukin-8 in critically ill patients admitted with suspected sepsis. Am J Respir Crit Care Med 2001; 164: 396–402. [DOI] [PubMed] [Google Scholar]
  • 54.Taylor HG, Klein N, Hack M. School-age consequences of birth weight less than 750 g: a review and update. Developmental Neuropsychology 2000; 17: 289–321. [DOI] [PubMed] [Google Scholar]
  • 55.Fiser DH. Assessing the outcome of pediatric intensive care. Journal of Pediatrics 1992; 121: 68–74. [DOI] [PubMed] [Google Scholar]
  • 56.Reynolds CR, Voress JK. Test of Memory and Learning, Second Edition (TOMAL-2). Austin, TX: PRO-ED, Inc; 2007. [Google Scholar]
  • 57.Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test- Children's Version. San Antonio, TX: Psychological Corporation; 1994. [Google Scholar]
  • 58.Beery KE, Buktenica NA, Beery NA. The Beery-Buktenica Developmental Test of Visual - Motor Integration, 6th Ed. San Antonio, TX: Pearson; 2010. [Google Scholar]
  • 59.Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior rating inventory of executive function. Neuropsychol Dev Cogn Sect C Child Neuropsychol 2000; 6: 235–238. [DOI] [PubMed] [Google Scholar]
  • 60.Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior Rating Inventory of Executive Function: Professional Manual. Odessa, FL: Psychological Assessment Resources; 2000. [Google Scholar]
  • 61.Bruininks RH, Bruininks BD. Bruininks-Oseretsky Test of Motor Proficiency- 2nd Edition. Minneapolis, MN: Pearson Assessment; 2005. [Google Scholar]
  • 62.Reynolds CR, Kamphaus RW. Behavior Assessment for Children Manual (2nd ed.). Circle Pines, MN: American Guidance Service, Inc.; 2004. [Google Scholar]
  • 63.Harrison PL, Oakland T. Adaptive Behavior Assessment System - Second Edition. San Antonio, TX: Harcourt Assessment; 2003. [Google Scholar]
  • 64.Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Washington, DC: American Psychiatric Association; 1994. [Google Scholar]
  • 65.Varni JW, Seid M, Kurtin PS. PedsQL 4.0: reliability and validity of the Pediatric Quality of Life Inventory version 4.0 generic core scales in healthy and patient populations. Medical Care 2001; 39: 800–812. [DOI] [PubMed] [Google Scholar]
  • 66.Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med 1991; 32: 705–714. [DOI] [PubMed] [Google Scholar]

RESOURCES