Skip to main content
BJA: British Journal of Anaesthesia logoLink to BJA: British Journal of Anaesthesia
. 2019 Feb 4;122(4):470–479. doi: 10.1016/j.bja.2018.12.020

Performance on the Operant Test Battery in young children exposed to procedures requiring general anaesthesia: the MASK study

David O Warner 1,, John J Chelonis 4, Merle G Paule 4, Ryan D Frank 2, Minji Lee 2, Michael J Zaccariello 3, Slavica K Katusic 2, Darrell R Schroeder 2, Andrew C Hanson 2, Phillip J Schulte 2, Robert T Wilder 1, Juraj Sprung 1, Randall P Flick 1
PMCID: PMC6435940  PMID: 30857603

Abstract

Background

It is not known whether the neurotoxicity produced by anaesthetics administered to young animals can also occur in children. Exposure of infant macaques to ketamine impairs performance in selected domains of the Operant Test Battery (OTB), which can also be administered to children. This study determined whether a similar pattern of results on the OTB is found in children exposed to procedures requiring general anaesthesia before age 3 yr.

Methods

We analysed data from the Mayo Anesthesia Safety in Kids (MASK) study, in which unexposed, singly-exposed, and multiply-exposed children born in Olmsted County, MN, USA, from 1994 to 2007 were sampled using a propensity-guided approach and prospectively underwent OTB testing at ages 8–12 or 15–20 yr, using five tasks that generated 15 OTB test scores.

Results

In primary analysis, none of the OTB test scores depended upon anaesthesia exposure status when corrected for multiple comparisons. Cluster analysis identified four clusters of subjects, with cluster membership determined by relative performance on the OTB tasks. There was no evidence of association between exposure status and cluster membership. Exploratory factor analysis showed that the OTB scores loaded onto four factors. The score for one factor was significantly less in multiply-exposed children (mean standardised difference –0.28 [95% confidence interval, –0.55 to –0.01; P=0.04]), but significance did not survive a sensitivity analysis accounting for outlying values.

Conclusions

These findings provide little evidence to support the hypothesis that children exposed to procedures requiring anaesthesia show deficits on OTB tasks that are similar to those observed in non-human primates.

Keywords: anaesthetic neurotoxicity, general anaesthesia, neuropsychological tests, neurodevelopment, Operant Test Battery, psychology, developmental


Editor's key points.

  • Exposure of many species, including non-human primates, to general anaesthetics can cause neurodegenerative changes with subsequent adverse effects on learning and behaviour.

  • The Operant Test Battery, which is impaired in infant macaques exposed to ketamine, was administered to children exposed to general anaesthesia before age 3 yr.

  • None of the test scores in 991 children depended upon anaesthesia exposure status when corrected for multiple comparisons.

  • Children exposed to general anaesthesia do not show demonstrable deficits on Operant Test Battery tasks that are similar to those observed in non-human primates.

Exposure of young animals, including non-human primates, to anaesthetics consistently causes brain neurodegenerative changes that are associated with later adverse effects on learning and behaviour.1, 2, 3 The potential clinical significance of these observations is a topic of debate and concern. Most studies in children find that multiple exposures to anaesthesia at a young age are associated with deficits in at least some cognitive domains and problems with learning and behaviour,4, 5, 6, 7, 8, 9, 10 whereas most (but not all) studies examining single exposures do not.11, 12 Thus, although some evidence supports the translational potential of the animal observations to humans, more data are needed, especially because there may be challenges in making direct comparisons using cognitive and behavioural domains that can be assessed in animals and humans.13

Investigators at the Food and Drug Administration's National Center for Toxicological Research described the effects on neonatal macaques of 24 h of ketamine exposure on subsequent performance on their Operant Test Battery (OTB), used to examine the effects of neuroactive drugs on primate neurodevelopment.14, 15, 16 Ketamine-impaired performance in several tasks assessing aspects of learning, motivation, impulse control, and short-term memory that persisted over almost 4 yr of repeated testing (approximately equivalent to human adolescence). The OTB has also been utilised in children, although there is less experience in using this assessment in older adolescents.15 OTB performance by children is not generally distinguishable from well-trained monkeys,17, 18, 19, 20, 21 and is correlated with general intelligence (correlations with other cognitive domains measured in standard neuropsychological testing have not been examined).18 Thus, the pattern of OTB results observed after exposure of primates to anaesthesia can be compared with the pattern of results observed in children exposed to anaesthesia.

The objective of the Mayo Anesthesia Safety in Kids (MASK) study was to examine the association between exposure to procedures requiring general anaesthesia before the third birthday and assessments of behaviour and neuropsychological function.22 We conducted prospective testing of a propensity-guided sample of a birth cohort, examining children unexposed, singly exposed, and multiply exposed to general anaesthesia before age 3 yr. An initial report described the primary analysis, finding that anaesthesia exposure was not associated with differences in the primary outcome of general intelligence.23 In secondary analysis of other domains, processing speed and fine motor abilities were modestly decreased in multiply-exposed but not singly-exposed children; other domains including memory were not different. As a part of the prospective testing, performance on five OTB tasks was also assessed.

This report presents the results of OTB testing in the MASK study. Two sets of hypotheses were tested. One set was generated from the prior study of rhesus macaques,14 positing that a similar pattern of results would be observed in children multiply exposed to general anaesthesia. Specifically, we hypothesised that compared with unexposed children, multiple, but not single, exposures would be associated with deficits in three Incremental Repeated Acquisition task variables (percent task completed, accuracy, and response rate) and a Progressive Ratio task variable (response rate). We further hypothesised that exposure status would not be associated with differences in two Conditioned Position Response task variables (percent task completed and overall accuracy), but would be associated with deficits in the Conditioned Position Response task variable of response rate. The second set of hypotheses was generated based on prior studies that examined the effects of the administration of stimulant medication in children with attention deficit hyperactivity disorder (ADHD).24, 25, 26, 27, 28 We have shown that the frequency of ADHD is higher in children multiply exposed to anaesthesia7, 29 and that parents of multiply-exposed children report more behavioural problems.23 Thus, the second set of hypotheses posited that children multiply exposed to anaesthesia would demonstrate deficits in those OTB variables that have been associated with ADHD and its treatment. Specifically, we hypothesised that compared with unexposed children, multiple, but not single, exposures would be associated with deficits in a Delayed-Match-To-Sample task variable (overall accuracy) and a Temporal Response Differentiation task parameter (timing accuracy).

Methods

This study was approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards, and written informed consent/assent was obtained. Study methods have been previously published22, 23 and are summarised here.

Summary of recruitment and testing procedures

Individuals born from January 1, 1994 to December 31, 2007 in Olmsted County, MN, USA, who resided within Olmsted County until their third birthday and who resided within 25 miles of Rochester, MN, according to available records at study onset were identified. Anaesthesia exposure status at 3 yr of age was classified as unexposed, singly exposed, or multiply exposed through review of their medical records. Each subject was tested once, either when 8–12 or 15–20 yr old. Subjects were selected for recruitment using a frequency-matched approach, with strata defined based on their propensity for receiving single and multiple exposures to general anaesthesia. Each subject was assessed by a trained psychometrist who evaluated domains typically measured in clinical practice.22, 23 Parent/guardian questionnaires assessed perceived behaviour and learning difficulties.23

Summary of Operant Test Battery testing

After the neuropsychological testing previously reported,23 five OTB tasks were administered to each subject using a test panel that contained response levers, press plates, various stimulus lights, and a tray in which nickels could be dispensed.19, 20 The entire testing session lasted approximately 1 h with short breaks between each task. Before each task an instructional video was shown that described how to perform that task. The order of the tasks were: Progressive Ratio (measuring aspects of motivation); Conditioned Position Response (measuring aspects of visual discrimination, attention, and response speed); Temporal Response Differentiation (measuring aspects of time perception); Delayed Matching-to-Sample (measuring working memory, attention, and response speed); and Incremental Response Acquisition (measuring learning and memory). Each task has been previously described in detail elsewhere15, 19, 20, 21, 30 and is summarised in Supplementary File 1.

Analyses

Details of the analysis plan were made available via a web-based repository before commencing analysis (https://osf.io/k93nb/).

After weighting to account for residual imbalances in subject characteristics between groups using inverse probability of treatment weighting (IPTW),23 effects of exposure status were calculated separately for each of 15 OTB variables (task scores) derived from the five OTB tasks as the primary analysis. However, these variables are likely not all independent (i.e. some may vary in a similar way). Thus, two secondary analyses were used to further account for the multiple variables analysed. To reduce these multiple variables to fewer dimensions, factor analysis was used to identify those OTB task scores that varied similarly, on average, across subjects, and thus loaded onto common underlying latent variables (factors). The association between exposure status and these factors was then determined. This approach could potentially increase the sensitivity for finding associations between anaesthesia exposure status and OTB performance if these common factors were affected. In cluster analysis, subjects were grouped such that the pattern of OTB results within a given group was more similar to each other than other groups, with groups denoted as ‘clusters’. For example, higher and lower performers on OTB tasks could fall into different clusters based on their overall performance. The association between exposure status and cluster membership was then determined. This approach could potentially increase the sensitivity for finding associations if they were present in only some subjects.

For all analyses, exposure status was categorised as 0, 1, and ≥2 exposures. Linear regressions including IPTW weights evaluated the relationship between exposure status and the appropriate dependent variable (individual OTB tests [primary analysis], cluster membership, or factor scores), using generalised estimating equations and a robust variance. All models including a factor to account for age at testing (by year), as many OTB variables were dependent upon age. Each model further tested interactions between exposure and age at testing (8–12 vs 15–20 yr) to assess whether the relationship between exposure and OTB tests was age at testing dependent. For both factor and cluster analyses, each OTB variable was transformed to have a mean=0 (standard deviation [sd]=1), with multiple imputations (50) used for missing data.

Primary analysis

An overall 2 degrees of freedom test (i.e. both single and multiple exposures) was performed across exposure categories for each individual OTB test score, and pairwise comparisons of single or multiple exposures vs no exposure. There was no a priori ordering of the individual OTB tests. The Hochberg procedure31 was used to correct for multiple comparisons of individual OTB tests to control the family-wise error rate at level 0.05. P-values are compared with adjusted thresholds under the Hochberg procedure.

Cluster analysis

Hierarchical agglomerative clustering using Ward's minimum variance method was performed.32 Clustering has been used previously to describe disease or patient phenotypes without making assumptions about the relationship of features.33 The number of clusters was chosen based on numeric (cubic clustering criterion and semi-partial R2 statistics) and subjective evaluation (number of subjects per cluster) before assessing associations with exposure status. After clusters were determined, baseline characteristics and parent report variables across cluster membership were compared using analysis of variance (anova) F-tests for continuous variables and Pearson χ2 or Fisher exact tests (where appropriate) for categorical variables. The association between exposure status and cluster membership was assessed with an IPTW multinomial regression model. The relationship between cluster membership and parent reports of learning and behaviour was also examined to explore whether these reports were associated with OTB performance.

Factor analysis

There were no theoretical a priori assumptions regarding the relationship between OTB variables. Thus, exploratory factor analysis (EFA) was conducted using R packages psych34 and lavaan.35 The number of factors was chosen based on inspection of scree plots and eigenvalues, with a preference for the most parsimonious solution which could be interpreted within the context of the underlying variables. Factors were extracted using promax rotation, which allows for correlation between factors. The fit index of the model was evaluated using the standardised root mean square residual.36, 37, 38 Once the factor structure was determined, factor scores derived from the EFA model were calculated using the Bartlett approach.39 The relationship between exposure status and factor scores was evaluated in weighted linear regression models for each factor.

Analyses were performed using SAS (SAS Institute Inc., Cary, NC, USA) version 9.4 and R for factor analyses (www.R-project.org). All tests were two-sided, and a P value <0.05 was considered significant.

Results

Of the 997 participants who completed neuropsychological testing, six declined OTB testing and are not reported. Supplementary Files 2–6 show the scores for the 15 OTB task variables analysed as a function of age and exposure status. In general, performance on tasks improved with age, illustrating the need for age adjustment in the analyses.

Primary analysis

There were no significant differences for any test scores between unexposed and singly-exposed subjects (Table 1). For multiply-exposed subjects, the only test score that differed significantly with individual testing was the Incremental Repeated Acquisition – Percent task completed. Only 22 children had less than 100% completion of this task; of these 13 were multiply exposed. When corrected for multiple comparisons using the Hochberg procedure, the P-value for this test score (P=0.019) was greater than the critical value required to conclude significance (P=0.003). The P-values for all other test scores also exceeded the critical P-values (data not shown). There was no significant interaction between exposure status and age at testing (8–12 vs 15–20 yr) for any score, indicating the lack of association did not depend on the age at testing (i.e. was present in both younger and older subjects). Thus, in primary analysis anaesthesia exposure was not associated with any significant difference in any OTB test score.

Table 1.

Primary analysis (inverse probability treatment weighting) for Operant Test Battery (OTB) scores chosen a priori as of primary interest. This primary analysis used inverse probability of treatment weighting to account for imbalances across exposure categories among children actually tested. Values are estimate (95% CI) of mean difference from children not exposed to anaesthesia from weighted analysis adjusted for age at testing. 2 degrees of freedom test for overall difference, significance indicated in bold for P<0.05. Pairwise comparisons of single or multiple exposures vs no exposure, significance indicated in bold for P<0.025 (Bonferroni adjustment). N, number of children completing a given test. CI, confidence interval; sd, standard deviation

Task parameter N P-value Single exposure, estimate (95% CI) P-value Multiple exposures, estimate (95% CI) P-value
Progressive ratio
 Response rate (responses s−1) 991 0.554 –0.10 (–0.29, 0.08) 0.282 –0.05 (–0.26, 0.16) 0.645
Conditioned position response
 Accuracy (%) 990 0.172 –0.15 (–0.64, 0.33) 0.541 –0.85 (–1.74, 0.04) 0.061
 Observing response latency (s) 990 0.855 0.00 (–0.10, 0.09) 0.925 0.03 (–0.12, 0.17) 0.727
 Choice response latency (s) 990 0.213 –0.01 (–0.05, 0.04) 0.744 0.05 (–0.02, 0.12) 0.158
 Response rate (responses s−1) 990 0.779 –0.02 (–0.10, 0.06) 0.612 –0.03 (–0.11, 0.05) 0.495
Temporal response differentiation
 Timing accuracy (%) 990 0.946 –0.67 (–5.52, 4.19) 0.787 –0.72 (–5.47, 4.03) 0.766
 Average hold times (s) 990 0.563 0.18 (–0.24, 0.61) 0.397 –0.05 (–0.53, 0.43) 0.840
 sd of hold times (s) 990 0.452 0.21 (–0.12, 0.54) 0.220 0.09 (–0.20, 0.38) 0.530
Delayed match to sample
 Percent task completed (%) 987 0.469 –0.47 (–2.32, 1.37) 0.615 –1.29 (–3.34, 0.76) 0.219
 Overall accuracy (%) 987 0.205 –0.21 (–1.06, 0.64) 0.628 –0.98 (–2.06, 0.11) 0.078
 Observing response latency (s) 987 0.552 0.02 (–0.12, 0.17) 0.771 0.20 (–0.17, 0.56) 0.289
 Choice response latency (s) 987 0.343 0.04 (–0.08, 0.15) 0.557 0.14 (–0.05, 0.34) 0.144
Incremental repeated acquisition
 Percent task completed (%) 986 0.019 –0.57 (–1.17, 0.04) 0.067 –1.63 (–2.87, –0.39) 0.010
 Overall accuracy (%) 986 0.099 –0.96 (–2.20, 0.28) 0.129 –1.45 (–2.89, –0.02) 0.047
 Effective response rate (responses s−1) 986 0.212 0.02 (–0.04, 0.08) 0.580 –0.04 (–0.10, 0.02) 0.231

Cluster analysis

Using cubic clustering criterion and semi-partial R2, the optimum number of clusters was 4 with 557 subjects (56%) in Cluster A, 332 (34%) in Cluster B, 57 (6%) in Cluster C, and 45 (5%) in Cluster D. In the determination of cluster membership, the Incremental Repeated Acquisition –Percent task completed variable was not included because there were only 22 individuals with values <100%, and this factor dominated the clustering (e.g. resulting in a cluster with only two individuals).

For subjects in Cluster D, the mean values for all test scores indicate lower performance on all tasks, and for subjects in Cluster B, mean values for all test scores indicate higher performance on all tasks (Table 2). For subjects in Cluster A, the mean values for all test scores were intermediate between the means for Clusters B and D. For subjects in Cluster C, the mean values for all test scores were intermediate between Clusters A and B, with the exception of the Temporal Response Differentiation variables, which indicated the lowest performance of any cluster.

Table 2.

Mean values of task parameters for each cluster. Values are mean (95% confidence interval) unless otherwise indicated. sd, standard deviation

Task parameter Cluster A (N=557) Cluster B (N=332) Cluster C (N=57) Cluster D (N=45)
Progressive ratio
 Response rate (responses per s) 3.88 (3.80, 3.96) 4.47 (4.38, 4.56) 4.13 (3.89, 4.37) 2.76 (2.32, 3.20)
Conditioned position response
 Accuracy
 Categories
 <91% 6.8 0.6 3.5 11.1
 91–99% 19.9 9.3 7.0 20.0
 100% 73.2 90.1 89.5 68.9
 Mean (%) 98.2 (97.9, 98.5) 99.5 (99.3, 99.7) 99.4 (98.8, 99.9) 95.7 (92.5, 98.9)
 Observing response latency (s) 0.97 (0.93, 1.01) 0.71 (0.68, 0.74) 0.97 (0.76, 1.18) 1.77 (1.39, 2.15)
 Choice response latency (s) 0.62 (0.60, 0.64) 0.48 (0.46, 0.50) 0.55 (0.50, 0.60) 1.06 (0.66, 1.46)
 Response rate (responses per s) 1.39 (1.36, 1.42) 1.81 (1.76, 1.86) 1.53 (1.40, 1.66) 0.95 (0.81, 1.09)
Temporal response differentiation
 Timing accuracy (%) 61.4 (59.1, 63.7) 69.9 (67.5, 72.3) 37.6 (31.9, 43.4) 64.4 (57.6, 71.1)
 Average duration of timing holds (s) 10.53 (10.37, 10.69) 10.95 (10.80, 11.10) 14.46 (13.67, 15.25) 11.05 (10.64, 11.46)
 sd of duration of timing holds (s) 2.02 (1.91, 2.13) 1.73 (1.62, 1.84) 5.25 (4.46, 6.04) 2.21 (1.77, 2.65)
Delayed match to sample
 Percent task completed
 Categories
 <80% 19.9 1.2 17.5 57.8
 81–89% 21.7 3.9 7.0 20.0
 90–99% 23.7 19.0 14.0 15.6
 100% 34.6 75.9 61.4 6.7
 Mean (%) 89.5 (88.6, 90.5) 98.1 (97.7, 98.6) 92.3 (88.9, 95.8) 75.1 (69.6, 80.6)
 Overall accuracy (%) 93.4 (92.9, 93.8) 97.6 (97.3, 97.9) 94.9 (93.1, 96.8) 89.6 (86.3, 93.0)
 Observing response latency (s) 1.52 (1.47, 1.57) 1.05 (1.00, 1.10) 1.38 (1.19, 1.57) 3.72 (1.94, 5.50)
 Choice response latency (s) 2.01 (1.95, 2.07) 1.47 (1.42, 1.52) 1.73 (1.55, 1.91) 2.99 (2.19, 3.79)
Incremental Repeated Acquisition
 Percent task completed
 <100% 2.9 0.9 1.8 8.9
 100% 97.1 99.1 98.2 91.1
 Overall accuracy (%) 81.9 (81.2, 82.6) 84.5 (83.8, 85.1) 81.9 (79.8, 84.0) 80.0 (76.4, 83.6)
 Effective response rate (responses s−1) 1.03 (1.00, 1.06) 1.31 (1.26, 1.36) 1.03 (0.95, 1.11) 0.92 (0.84, 1.00)

Subject characteristics did not differ according to cluster membership, including mother and father's age or education, sex, estimated gestational age at birth, Apgar scores, birth weight, and socio-economic status (Supplementary File 7). Parent reports of learning and behaviour significantly differed according to cluster membership (Table 3). Parents of children in Cluster D reported the most problems, and parents of subjects in Cluster B reported the least problems.

Table 3.

Values of parental reports according to Operant Test Battery cluster membership. For each characteristic, higher mean scores indicate more problems. For the BRIEF and CBCL, scores > 60 are considered to be clinically significant. Analysis of variance (anova) F-test. χ2 test. ADHD, attention deficit hyperactivity disorder; BRIEF, Behavior Rating Inventory of Executive Function; CBCL, Child Behavior Checklist; CLDQ, Colorado Learning Difficulties Questionnaire; sd, standard deviation

Characteristic Cluster A (N=557) Cluster B (N=332) Cluster C (N=57) Cluster D (N=45) P-value
CLDQ: Math Scale (ZS) <0.001
 Missing 60 33 4 5
 Mean (sd) 0.24 (1.21) –0.25 (0.96) 0.38 (1.28) 0.51 (1.30)
 Range –0.86, 3.59 –0.86, 3.59 –0.86, 3.37 –0.86, 3.59
CLDQ: Reading Scale (ZS) <0.001
 Missing 62 30 4 5
 Mean (sd) 0.01 (1.05) –0.36 (0.73) –0.22 (0.82) 0.03 (1.20)
 Range –0.81, 3.44 –0.81, 3.63 –0.81, 2.52 –0.81, 3.27
BRIEF: Global Executive Composite <0.001
 Missing 88 53 8 7
 Mean TS (sd) 49.12 (11.37) 46.03 (9.97) 48.63 (11.54) 51.71 (12.58)
 Range 31, 92 31, 85 33, 78 36, 79
 Proportion with score >60 74 (14.1%) 24 (7.5%) 8 (14.8%) 9 (20.9%) 0.008
CBCL category: Internalizing Problems 0.04
 Missing 94 55 9 9
 Mean TS (sd) 48.59 (10.70) 47.18 (9.84) 49.94 (10.65) 51.75 (12.21)
 Range 33, 87 33, 72 33, 72 33, 80
 Proportion with score >60 71 (13.7%) 26 (8.1%) 9 (17.0%) 7 (17.1%) 0.04
CBCL category: Externalizing Problems 0.005
 Missing 94 55 9 9
 Mean TS (sd) 45.56 (10.01) 43.02 (9.29) 45.50 (11.26) 46.50 (12.05)
 Range 31, 86 33, 78 33, 76 33, 76
 Proportion with score >60 35 (6.7%) 16 (5.0%) 7 (13.2%) 5 (12.2%) 0.07
CBCL category: Total Problems <0.001
 Missing 95 55 9 9
 Mean TS (sd) 46.20 (11.65) 42.97 (10.79) 46.50 (12.07) 48.78 (12.64)
 Range 24, 86 24, 73 24, 77 24, 74
 Proportion with score >60 54 (10.4%) 16 (5.0%) 5 (9.4%) 6 (14.6%) 0.03
CBCL category: ADHD Problems <0.001
 Missing 95 55 9 9
 Mean TS (sd) 53.76 (6.26) 51.99 (4.32) 53.71 (6.12) 55.08 (7.64)
 Range 50, 78 50, 75 50, 75 50, 75
 Proportion with score >60 61 (11.8%) 14 (4.4%) 6 (11.3%) 7 (17.1%) 0.001

There was no evidence of associations between anaesthesia exposure and cluster membership (P=0.91, Table 4), as the distribution of subjects among clusters was similar across exposure categories.

Table 4.

Anaesthesia exposure and cluster membership. Inverse probability of treatment weighting was used to account for imbalances across exposure categories. The numbers in the table list the proportion of patients in each cluster by exposure status after weighting. Overall P-value for the exposures variable, indicating that exposure status was not significantly associated with cluster membership. CI, confidence interval

Characteristic Weighted proportion within each exposure status (%)
Odds ratios (95% CI)
Cluster A Cluster B Cluster C Cluster D Clusters B vs A Clusters C vs A Clusters D vs A P-value
Anaesthesia exposures 0.910
 None 57.1 34.5 4.9 4.9 1.00 (reference) 1.00 (reference) 1.00 (reference)
 Single 57.6 30.2 5.9 5.9 0.88 (0.57, 1.36) 1.16 (0.51, 2.66) 1.62 (0.57, 4.60)
 Multiple 56.4 33.7 4.5 4.5 1.03 (0.68, 1.57) 0.96 (0.40, 2.33) 1.45 (0.59, 3.52)

Factor analysis

There were three factors with eigenvalues >1.5 and 5 with eigenvalues >1 in the initial EFA model (eigenvalues measure the amount of variation in the total sample accounted for by each factor and help assess which model may best fit the data). Based on the principle of parsimony (i.e. a preference for the simplest solution), a four-factor solution was selected (Table 5), which accounted for 59% of the variability. The standardised root mean square residual was 0.06 for this model. Factor 1 included all response rate variables and the response latency parameters from the Conditioned Position Response task. Factor 2 included the four variables derived from the Delayed Match to Sample task. Factor 3 included the three variables derived from the Temporal Response Differentiation task. Factor 4 included the two accuracy variables (for Conditioned Position Response and Incremental Repeated Acquisition tasks), and the Incremental Repeated Acquisition – Percent task completed variable.

Table 5.

Factor loadings for exploratory factor analysis. Values reflect the relationship of each variable to the underlying factor. They can be interpreted as standardised regression coefficients expressing the correlation between that variable and the underlying factor. sd, standard deviation

Test Factor 1 Factor 2 Factor 3 Factor 4
Progressive Ratio, Response Rate –0.38
Conditioned Position Response, Observing Response Latency 0.77
Conditioned Position Response, Choice Response Latency 0.68
Conditioned Position Response, Response Rate –1.07
Incremental Repeated Acquisition, Effective Response Rate –0.45
Delayed Match to Sample, Percent Task Completed 1.03
Delayed Match to Sample, Overall Accuracy 0.75
Delayed Match to Sample, Observing Response Latency –0.45
Delayed Match to Sample, Choice Response Latency –0.64
Temporal Response Differentiation, Timing Accuracy –0.49
Temporal Response Differentiation, Average Hold Times 0.36
Temporal Response Differentiation, sd of Hold Times 1.01
Conditioned Position Response, Accuracy 0.31
Incremental Repeated Acquisition, Overall Accuracy 0.70
Incremental Repeated Acquisition, Percent Task Completed 0.74

For factor scores calculated using this EFA model, Factor 4 standardised scores were significantly lower in multiply-exposed children (P=0.04); there were no other significant associations between exposure status and any factor (Table 6). However, this factor included the Incremental Repeated Acquisition – Percent task completed, which had outlying values (resulting from a skewed distribution, as only 22 subjects had values <100%). When winsorising was applied to account for these outlying values in a post hoc sensitivity analysis (i.e. limiting these extreme values without excluding them), this difference was no longer statistically significant (P=0.07).

Table 6.

Exposure status and Operant Test Battery (OTB) factor scores from exploratory factor analysis. Values are the estimate (95% CI) of the standardised difference from unexposed for all subjects, expressed as z scores for each factor (mean=0, sd=1). The analysis used inverse probability of treatment weighting (IPTW) to account for imbalances across exposure categories among children actually tested. A separate model was fit for each factor score. The reference group for anaesthesia exposure was no exposure. Pairwise comparisons of single or multiple exposures vs no exposure. Because there were significant outliers for some variables, in this post hoc sensitivity analysis, observations below the 2.5 percentile set to the 2.5th percentile and observations above the 97.5 percentile set to the 97.5 percentile. This approach limits these extreme values without excluding them. CI, confidence interval; sd, standard deviation

Characteristic Factor 1
Factor 2
Factor 3
Factor 4
Mean (95% CI) P-value Mean (95% CI) P-value Mean (95% CI) P-value Mean (95% CI) P-value
Primary analysis
 Single exposures 0.05 (–0.15, 0.24) 0.64 –0.03 (–0.25, 0.19) 0.82 0.07 (–0.13, 0.28) 0.48 –0.08 (–0.26, 0.10) 0.37
 Multiple exposures 0.06 (–0.14, 0.26) 0.57 –0.08 (–0.28, 0.12) 0.43 0.03 (–0.17, 0.24) 0.75 –0.28 (–0.55, –0.01) 0.04
Variables winsorised
 Single exposures 0.05 (–0.14, 0.24) 0.63 –0.02 (–0.22, 0.18) 0.82 0.06 (–0.10, 0.23) 0.46 –0.04 (–0.19, 0.10) 0.56
 Multiple exposures 0.04 (–0.14, 0.23) 0.66 –0.09 (–0.27, 0.09) 0.31 0.03 (–0.15, 0.21) 0.74 –0.14 (–0.29, 0.01) 0.07

Discussion

The OTB is sensitive to a variety of toxicological and pharmacological agents in both non-human primates14, 15, 16 and humans.24, 25, 26, 27, 28, 40 In children, performances on several OTB tasks are correlated with IQ; direct correlations with other neurocognitive domains assessed in standard neuropsychological testing have not been reported.18 The variables that are most affected by ketamine exposure in non-human primates are the ones that are typically more highly correlated with IQ in children. With the exception of Incremental Repeated Acquisition response rate (which has a correlation of 0.21 with IQ in children), all the other variables that are significantly affected by ketamine exposure in non-human primates have a correlation with IQ in children of 0.49–0.58.18 A detailed comparison of OTB and neuropsychological tests in the MASK subjects is beyond the scope of this report. However, the results of cluster analysis demonstrate that lower levels of OTB performance are associated with parent-reported problems in learning and behaviour, suggesting that OTB performance is associated with other meaningful functional developmental outcomes.

The first set of hypotheses posited parallels in OTB outcomes between a prior study of primates exposed to ketamine for 24 h at 1 week of life14 and MASK subjects. Rhesus macaques were trained in OTB tasks beginning at 7 months of age and assessed up to nearly 4 yr of age. During training, acquisition of proficiency in several tasks was impaired in ketamine-exposed animals. At the end of training (at a development stage approximating human adolescence), ketamine-exposed animals continued to have impaired performance in the Incremental Repeat Acquisition task (learning and memory) but not in most variables of the Conditioned Position Response task (visual discrimination and response) or the Delayed Match To Sample task (working memory and attention). Most response rate task variables were also impaired. In the MASK subjects, although the factor analysis identified an association between multiple exposures and a factor that included two OTB variables hypothesised to be affected (Incremental Repeated Acquisition – Overall accuracy and Percent task completed), significance did not survive a sensitivity analysis accounting for outlying variables and also depended upon a small number of multiply-exposed children, thus representing at best weak evidence.

Several factors may contribute to the differing findings between non-human primates and children. First, the methodology used in primates assesses training in OTB task performance via longitudinal assessments over many months, whereas our assessment in children is cross-sectional and does not reflect training. In primates, effects of exposure were observed in both the rate of task skill acquisition and steady state performance. We used the latter to formulate hypotheses in an attempt to increase comparability, but this still represents a significant difference in approach between species. Second, the OTB may not be sufficiently challenging in these children to detect small effects, especially in older children. For example, the potential for a ceiling effect was noted for two variables (values for the Conditioned Position Response – Overall accuracy and Incremental Repeated Acquisition – Percent task completed were 100% in many subjects), making it less likely that differences could be observed. However, the lack of significance for an interaction term between age at testing and exposure status suggests a lack of association with exposure even in younger children, in whom there is less potential for a ceiling effect. Finally, it is possible that the injury produced by ketamine in primates is simply not present in these children. The injury produced in the primate study is caused by higher doses and a more prolonged exposure time (24 h) than used in children (the total exposure time even in multiply exposed was almost always less than 24 h).23 Also, the primates were studied in the first week of life, whereas the age at exposure of these children was over a wider range (up to 3 yr). There may also be differences in effects between exposure to ketamine (in the primate study) and volatile anaesthetics (primarily sevoflurane in MASK subjects), or moderating effects of simultaneous exposure to surgical stimulation (present in most children). Subsequent work in primates has pointed to effects of anaesthesia exposure at a young age on behaviour,41, 42 consistent with the findings of the MASK study in regard to parental reports of behavioural difficulties, but these effects may not be captured by the OTB in children.

The second set of hypotheses was suggested by prior work showing (i) an association between multiple exposures and the incidence of ADHD,7, 29 and (ii) administration of stimulant medication to children with ADHD improved performance in some OTB tasks, including the Delayed Match to Sample and Temporal Response Differentiation tasks.24, 25, 26, 27, 28 However, evidence did not support these hypotheses. Several factors may be responsible. First, parent reports indicated that 8% of subjects had current or past use of medications to treat ADHD (data not shown). Because subjects were not specifically instructed to discontinue these medications, continued use could have biased against finding differences in the small proportion of subjects who were treated. Second, in the primary analysis of neuropsychological tests, tests of attention and executive function that are abnormal in many children with ADHD43, 44, 45 were also not different in children exposed to anaesthesia, consistent with this OTB finding. The lack of association between exposure and OTB performance on these tasks provides further evidence that behaviours diagnosed as ADHD that are associated with anaesthesia exposure may have features distinct from other aetiologies of ADHD.46, 47, 48

Are these OTB results consistent with the neuropsychological testing results previously reported?23 Both forms of testing found that anaesthesia exposure was not associated with test scores measuring learning, memory, and attention. Performances on neuropsychological tests that depend upon fine motor skills were consistently impaired in multiply-exposed children. All OTB tasks require motor skills (i.e. pressing a lever) to complete. However, such skills are measured and conceptualised in various ways, for example, gross vs fine motor skills. It is not clear that those skills needed to press a lever are comparable with those needed to complete the fine motor assessments in the neuropsychological testing (e.g. Grooved Pegboard or trail making). Indeed, the lack of association between exposure and OTB performance suggests that different skills are involved, and that anaesthesia exposure is not associated with deficits in all motor skills. The other neuropsychological domain found to be impaired in multiply-exposed children was reading processing speed, which was not assessed by the OTB. Thus, the OTB does not appear to measure domains that are altered by anaesthesia exposure, or at least not with sufficient sensitivity to detect in a study of this size. Finally, given the correlation between IQ and OTB performance demonstrated in prior studies,18 the prior finding that exposure does not affect IQ23 is consistent with a lack of effect on OTB performance.

In summary, exposure to volatile anaesthetics before age 3 yr did not consistently affect performance on the administered OTB tasks. These findings provide little support for the study hypothesis that children exposed to procedures requiring general anaesthesia show a pattern of effects on OTB tasks similar to that seen in non-human primates.

Authors' contributions

Study design: DOW, JJC, MGP, SKK, RTW, JS, MJZ, DRS, RPF.

Obtaining funding: DOW, RPF.

Data collection: DOW, MJZ, DRS, RPF.

Data analysis: DOW, RDF, ML, ACH, MJZ, DRS, PJS.

Data interpretation: DOW, JJC, MGP, SKK, RTW, JS, MJZ, PJS, RPF.

Drafting of the manuscript: DOW.

Revision of the manuscript: JJC, MGP, SKK, RTW, JS, RDF, ML, ACH, MJZ, DRS, PJS, RPF.

Approval of the final version of the manuscript and agreement to accountability: all authors.

Acknowledgements

The authors thank the staff at the Mayo Clinic Psychological Assessment Laboratory for their dedicated work in subject testing. The authors also recognise the extraordinary contributions of Robert Colligan, who was crucial in the design of the study and died during its conduct; he is sorely missed.

Editorial decision: 28 December 2018

Handling editor: H.C. Hemmings Jr

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.bja.2018.12.020.

Declaration of interest

The authors declare that they have no conflicts of interest.

Funding

Eunice Kennedy Shriver National Institute of Child Health and Human Development of the US National Institutes of Health (Bethesda, MD, USA) (grant number R01 HD071907), and resources of the Rochester Epidemiology Project supported by R01 AG034676 from the National Institute on Aging of the US National Institutes of Health. The funding sources had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Disclaimer

This document has been reviewed in accordance with US Food and Drug Administration (FDA) policy and approved for publication. Approval does not signify that the contents necessarily reflect the position or opinions of the FDA nor does mention of trade names or commercial products constitute endorsement or recommendation for use. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the FDA.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Multimedia component 1
mmc1.zip (267.6KB, zip)

References

  • 1.Jevtovic-Todorovic V., Hartman R.E., Izumi Y. Early exposure to common anesthetic agents causes widespread neurodegeneration in the developing rat brain and persistent learning deficits. J Neurosci. 2003;23:876–882. doi: 10.1523/JNEUROSCI.23-03-00876.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Brambrink A.M., Evers A.S., Avidan M.S. Isoflurane-induced neuroapoptosis in the neonatal rhesus macaque brain. Anesthesiology. 2010;112:834–841. doi: 10.1097/ALN.0b013e3181d049cd. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Disma N., Mondardini M.C., Terrando N., Absalom A.R., Bilotta F. A systematic review of methodology applied during preclinical anesthetic neurotoxicity studies: important issues and lessons relevant to the design of future clinical research. Paediatr Anaesth. 2016;26:6–36. doi: 10.1111/pan.12786. [DOI] [PubMed] [Google Scholar]
  • 4.DiMaggio C., Sun L.S., Li G. Early childhood exposure to anesthesia and risk of developmental and behavioral disorders in a sibling birth cohort. Anesth Analg. 2011;113:1143–1151. doi: 10.1213/ANE.0b013e3182147f42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Flick R.P., Katusic S.K., Colligan R.C. Cognitive and behavioral outcomes after early exposure to anesthesia and surgery. Pediatrics. 2012;128:e1053–e1061. doi: 10.1542/peds.2011-0351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Wilder R.T., Flick R.P., Sprung J. Early exposure to anesthesia and learning disabilities in a population-based birth cohort. Anesthesiology. 2009;110:796–804. doi: 10.1097/01.anes.0000344728.34332.5d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Sprung J., Flick R.P., Katusic S.K. Attention-deficit/hyperactivity disorder after early exposure to procedures requiring general anesthesia. Mayo Clin Proc. 2012;87:120–129. doi: 10.1016/j.mayocp.2011.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.O'Leary J.D., Janus M., Duku E. A population-based study evaluating the association between surgery in early life and child development at primary school entry. Anesthesiology. 2016;125:272–279. doi: 10.1097/ALN.0000000000001200. [DOI] [PubMed] [Google Scholar]
  • 9.Graham M.R., Brownell M., Chateau D.G., Dragan R.D., Burchill C., Fransoo R.R. Neurodevelopmental assessment in kindergarten in children exposed to general anesthesia before the age of 4 years: a retrospective matched cohort study. Anesthesiology. 2016;125:667–677. doi: 10.1097/ALN.0000000000001245. [DOI] [PubMed] [Google Scholar]
  • 10.Glatz P., Sandin R.H., Pedersen N.L., Bonamy A.K., Eriksson L.I., Granath F. Association of anesthesia and surgery during childhood with long-term academic performance. JAMA Pediatr. 2016:e163470. doi: 10.1001/jamapediatrics.2016.3470. [DOI] [PubMed] [Google Scholar]
  • 11.Davidson A.J., Disma N., de Graaff J.C. Neurodevelopmental outcome at 2 years of age after general anaesthesia and awake-regional anaesthesia in infancy (GAS): an international multicentre, randomised controlled trial. Lancet. 2015;387:239–250. doi: 10.1016/S0140-6736(15)00608-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Sun L.S., Li G., Miller T.L. Association between a single general anesthesia exposure before age 36 months and neurocognitive outcomes in later childhood. JAMA. 2016;315:2312–2320. doi: 10.1001/jama.2016.6967. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Premack D. Human and animal cognition: continuity and discontinuity. Proc Natl Acad Sci U S A. 2007;104:13861–13867. doi: 10.1073/pnas.0706147104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Paule M.G., Li M., Allen R.R. Ketamine anesthesia during the first week of life can cause long-lasting cognitive deficits in rhesus monkeys. Neurotoxicol Teratol. 2011;33:220–230. doi: 10.1016/j.ntt.2011.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Paule M.G., Cranmer J.M., Wilkins J.D., Stern H.P., Hoffman E.L. Quantitation of complex brain function in children: preliminary evaluation using a nonhuman primate behavioral test battery. Neurotoxicology. 1988;9:367–378. [PubMed] [Google Scholar]
  • 16.Paule M.G. Use of the NCTR operant test battery in nonhuman primates. Neurotoxicol Teratol. 1990;12:413–418. doi: 10.1016/0892-0362(90)90002-t. [DOI] [PubMed] [Google Scholar]
  • 17.Chelonis J.J., Cox A.R., Karr M.J., Prunty P.K., Baldwin R.L., Paule M.G. Comparison of delayed matching-to-sample performance in monkeys and children. Behav Process. 2014;103:261–268. doi: 10.1016/j.beproc.2014.01.002. [DOI] [PubMed] [Google Scholar]
  • 18.Paule M.G., Chelonis J.J., Buffalo E.A., Blake D.J., Casey P.H. Operant test battery performance in children: correlation with IQ. Neurotoxicol Teratol. 1999;21:223–230. doi: 10.1016/s0892-0362(98)00045-2. [DOI] [PubMed] [Google Scholar]
  • 19.Baldwin R.L., Chelonis J.J., Prunty P.K., Paule M.G. The use of an incremental repeated acquisition task to assess learning in children. Behav Process. 2012;91:103–114. doi: 10.1016/j.beproc.2012.06.004. [DOI] [PubMed] [Google Scholar]
  • 20.Chelonis J.J., Daniels-Shaw J.L., Blake D.J., Paule M.G. Developmental aspects of delayed matching-to-sample task performance in children. Neurotoxicol Teratol. 2000;22:683–694. doi: 10.1016/s0892-0362(00)00090-8. [DOI] [PubMed] [Google Scholar]
  • 21.Chelonis J.J., Flake R.A., Baldwin R.L., Blake D.J., Paule M.G. Developmental aspects of timing behavior in children. Neurotoxicol Teratol. 2004;26:461–476. doi: 10.1016/j.ntt.2004.01.004. [DOI] [PubMed] [Google Scholar]
  • 22.Gleich S.J., Flick R., Hu D. Neurodevelopment of children exposed to anesthesia: design of the Mayo Anesthesia Safety in Kids (MASK) study. Contemp Clin Trial. 2014;41:45–54. doi: 10.1016/j.cct.2014.12.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Warner D.O., Zaccariello M.J., Katusic S.K. Neuropsychological and behavioral outcomes after exposure of young children to procedures requiring general anesthesia: the Mayo Anesthesia Safety in Kids (MASK) study. Anesthesiology. 2018;129:89–105. doi: 10.1097/ALN.0000000000002232. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Baldwin R.L., Chelonis J.J., Flake R.A. Effect of methylphenidate on time perception in children with attention-deficit/hyperactivity disorder. Exp Clin Psychopharmacol. 2004;12:57–64. doi: 10.1037/1064-1297.12.1.57. [DOI] [PubMed] [Google Scholar]
  • 25.Chelonis J.J., Edwards M.C., Schulz E.G. Stimulant medication improves recognition memory in children diagnosed with attention-deficit/hyperactivity disorder. Exp Clin Psychopharmacol. 2002;10:400–407. doi: 10.1037//1064-1297.10.4.400. [DOI] [PubMed] [Google Scholar]
  • 26.Chelonis J.J., Johnson T.A., Ferguson S.A. Effect of methylphenidate on motivation in children with attention-deficit/hyperactivity disorder. Exp Clin Psychopharmacol. 2011;19:145–153. doi: 10.1037/a0022794. [DOI] [PubMed] [Google Scholar]
  • 27.Meaux J.B., Chelonis J.J. Time perception differences in children with and without ADHD. J Pediatr Health Care. 2003;17:64–71. doi: 10.1067/mph.2003.26. [DOI] [PubMed] [Google Scholar]
  • 28.Meaux J.B., Chelonis J.J. The relationship between behavioral inhibition and time perception in children. J Child Adolesc Psychiatr Nurs. 2005;18:148–160. doi: 10.1111/j.1744-6171.2005.00030.x. [DOI] [PubMed] [Google Scholar]
  • 29.Hu D., Flick R.P., Zaccariello M.J. Association between exposure of young children to procedures requiring general anesthesia and learning and behavioral outcomes in a population-based birth cohort. Anesthesiology. 2017;127:227–240. doi: 10.1097/ALN.0000000000001735. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Chelonis J.J., Gravelin C.R., Paule M.G. Assessing motivation in children using a progressive ratio task. Behav Process. 2011;87:203–209. doi: 10.1016/j.beproc.2011.03.008. [DOI] [PubMed] [Google Scholar]
  • 31.Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988;75:800–802. [Google Scholar]
  • 32.Ward J.H. Hierarchical grouping to optimize an objective function. J Am Stat Assoc. 1963;58:236–244. [Google Scholar]
  • 33.Hermens D.F., Redoblado Hodge M.A., Naismith S.L., Kaur M., Scott E., Hickie I.B. Neuropsychological clustering highlights cognitive differences in young people presenting with depressive symptoms. J Int Neuropsychol Soc. 2011;17:267–276. doi: 10.1017/S1355617710001566. [DOI] [PubMed] [Google Scholar]
  • 34.Revelle W. 2016. Psych: procedures for personality and psychological Research.http://CRAN.R-project/package=psych Available from: (accessed 30 May 2018) [Google Scholar]
  • 35.Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw. 2012;48:1–36. [Google Scholar]
  • 36.Mueller R.O., Hancock G.R. Best practices in structural equation modeling. In: Osborne J., editor. Best practices in quantitative methods. Sage Publications, Inc.; Thousand Oaks, CA: 2008. pp. 488–508. [Google Scholar]
  • 37.Yu C.Y. University of California; Los Angeles: 2002. Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes. PhD dissertation. [Google Scholar]
  • 38.Hu L.T., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6:1–55. [Google Scholar]
  • 39.DiStephano C., Zhu M., Mindrila D. Understanding and using factor scores: considerations for the applied researcher. Pract Assess Res Eval. 2009;14:1–11. [Google Scholar]
  • 40.Paule M.G., Chelonis J.J., Blake D.J., Dornhoffer J.L. Effects of drug countermeasures for space motion sickness on working memory in humans. Neurotoxicol Teratol. 2004;26:825–837. doi: 10.1016/j.ntt.2004.07.002. [DOI] [PubMed] [Google Scholar]
  • 41.Coleman K., Robertson N.D., Dissen G.A. Isoflurane anesthesia has long-term consequences on motor and behavioral development in infant rhesus macaques. Anesthesiology. 2016;127:74–94. doi: 10.1097/ALN.0000000000001383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Raper J., Alvarado M.C., Murphy K.L., Baxter M.G. Multiple anesthetic exposure in infant monkeys alters emotional reactivity to an acute stressor. Anesthesiology. 2015;123:1084–1092. doi: 10.1097/ALN.0000000000000851. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Pasini A., Paloscia C., Alessandrelli R., Porfirio M.C., Curatolo P. Attention and executive functions profile in drug naive ADHD subtypes. Brain Dev. 2007;29:400–408. doi: 10.1016/j.braindev.2006.11.010. [DOI] [PubMed] [Google Scholar]
  • 44.Nigg J.T., Blaskey L.G., Huang-Pollock C.L., Rappley M.D. Neuropsychological executive functions and DSM-IV ADHD subtypes. J Am Acad Child Adolesc Psychiatr. 2002;41:59–66. doi: 10.1097/00004583-200201000-00012. [DOI] [PubMed] [Google Scholar]
  • 45.Frazier T.W., Demaree H.A., Youngstrom E.A. Meta-analysis of intellectual and neuropsychological test performance in attention-deficit/hyperactivity disorder. Neuropsychology. 2004;18:543–555. doi: 10.1037/0894-4105.18.3.543. [DOI] [PubMed] [Google Scholar]
  • 46.Nigg J.T., Willcutt E.G., Doyle A.E., Sonuga-Barke E.J. Causal heterogeneity in attention-deficit/hyperactivity disorder: do we need neuropsychologically impaired subtypes? Biol Psychiatr. 2005;57:1224–1230. doi: 10.1016/j.biopsych.2004.08.025. [DOI] [PubMed] [Google Scholar]
  • 47.Willcutt E.G., Doyle A.E., Nigg J.T., Faraone S.V., Pennington B.F. Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review. Biol Psychiatr. 2005;57:1336–1346. doi: 10.1016/j.biopsych.2005.02.006. [DOI] [PubMed] [Google Scholar]
  • 48.Makris N., Biederman J., Monuteaux M.C., Seidman L.J. Towards conceptualizing a neural systems-based anatomy of attention-deficit/hyperactivity disorder. Dev Neurosci. 2009;31:36–49. doi: 10.1159/000207492. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia component 1
mmc1.zip (267.6KB, zip)

Articles from BJA: British Journal of Anaesthesia are provided here courtesy of Elsevier

RESOURCES