Utility of the N-back task in survivors of childhood acute lymphoblastic leukemia

Abstract

The N-back task is often used in functional brain imaging studies to activate working memory networks; however, limited information is available on its association to clinical outcomes in children or cancer survivors. One hundred thirty-seven survivors of ALL (mean [SD] current age = 14.3 [4.8] years; time since diagnosis = 7.6 [1.6] years) completed the N-back task and comprehensive neurocognitive testing, including standardized measures of attention, processing speed, and working memory. Results indicated females demonstrated significantly slower reaction times (0-back p=0.02; 1-back p=0.03) compared to males. Survivors < 15 years old at the time of testing demonstrated a significant decrease in accuracy as working memory load increased compared to survivors ≥ 15 years old (p<0.001). Performance on the N-back task was associated with nonverbal working memory (r_s=0.56, p<0.001) in survivors ≥ 15 years of age. For younger survivors, N-Back performance was more strongly associated with attention skills. Results suggest the N-back assesses different cognitive constructs at younger compared to older childhood ages. These age differences should be considered in interpreting functional brain imaging results.

INTRODUCTION

Survivors of childhood acute lymphoblastic leukemia (ALL) treated on chemotherapy only protocols demonstrate neurocognitive impairment at rates estimated between 20–40% (Krull et al., 2008; Moleski, 2000). Although changes can be detected by the end of cancer therapy (Krull et al., 2013), manifestation of clinical impairment commonly occurs 2 to 5 years post treatment completion (Bhatia, 2003; Montour-Proulx et al., 2005). Risk factors for impairment include age at diagnosis, time since completion of treatment, treatment intensity and female gender (Buizer, de Sonneville, & Veerman, 2009; Conklin et al., 2012; Jain, Brouwers, Okcu, Cirino, & Krull, 2009; Janzen & Spiegler, 2008; Montour-Proulx et al., 2005; Waber et al., 2012). Cognitive domains commonly impacted by therapy for ALL involve attention, processing speed and working memory (WM) (Anderson, Godber, Smibert, & Ekert, 1997; Jain et al., 2009; Moore, 2005; Mulhern & Butler, 2004). Cognitive functions are traditionally assessed through clinical neurocognitive evaluation. Neuroimaging of cortical activation during a skill specific task is also used to understand brain function. Although the task performed during imaging is typically selected from affected cognitive domains, the specific tasks are generally not normed and association with clinical assessment of survivors is often unknown.

The N-back is an experimental task commonly used to assess WM networks (Braver et al., 1997; Gevins, Smith, McEvoy, & Yu, 1997; Owen, McMillan, Laird, & Bullmore, 2005; Perlstein, Dixit, Carter, Noll, & Cohen, 2003). This task parametrically manipulates cognitive load by increasing the number of sequential stimuli that need to be held in WM until an action is performed. N-back task performance has been shown to differentiate between cognitively impaired and healthy subjects (Borkowska, Drozdz, Jurkowski, & Rybakowski, 2009; Dean & Sterr, 2013; Harvey et al., 2004). Furthermore, N-back performance appears to be particularly sensitive to frontal lobe integrity in a variety of populations including head injury (Dean & Sterr, 2013), ADHD (Karatekin, Bingham, & White, 2009; Massat et al., 2012), and schizophrenia (Elsabagh, Premkumar, Anilkumar, & Kumari, 2009; Meiron, Hermesh, Katz, & Weizman, 2012). Recent studies have found the N-back task to be associated with traditional clinical measures of processing speed (Miller, Price, Okun, Montijo, & Bowers, 2009), working memory (Shelton, Elliott, Hill, Calamia, & Gouvier, 2009), and global intelligence (Jaeggi, Buschkuehl, Perrig, & Meier, 2010; Kane, Conway, Miura, & Colflesh, 2007), though these studies focus on adults. Despite growing evidence for N-back’s association to clinical measures and sensitivity to cognitive impairment, few studies take in to account young age or gender differences in task performance.

When examining the cortical activation evoked by task performance, the N-back has been shown to activate regions of dorsolateral prefrontal cortex, anterior cingulate cortex, and regions of posterior parietal cortex (Braver et al., 1997; Gevins et al., 1997; Owen et al., 2005). Differential activation in these regions has been observed between healthy controls and mild traumatic brain injury (Chen et al., 2012; McAllister et al., 1999), multiple sclerosis (Sweet, Rao, Primeau, Durgerian, & Cohen, 2006), and survivors of childhood ALL (Robinson et al., 2010). Moreover, prefrontal regions activated by N-back performance are among the last to myelinate and demonstrate increased activation with age (Ciesielski, Lesnik, Savoy, Grant, & Ahlfors, 2006; Klingberg, Forssberg, & Westerberg, 2002; Thomason et al., 2009). Due to these physiological changes, adolescence is a time for rapid development of executive function skills (Kalkut, Han, Lansing, Holdnack, & Delis, 2009; Korkman, Lahti-Nuuttila, Laasonen, Kemp, & Holdnack, 2013). Although it is well established that WM skills are protracted through childhood and into adolescence, age at which working memory networks are fully developed has not been consistently defined - with some accounts of patients demonstrating adult like WM accuracy as early as 15 years of age (Huizinga & van der Molen, 2007; Luciana, Conklin, Hooper, & Yarger, 2005). Despite the N-back’s capacity to discern cognitive impairment, its utility may be limited by patient’s age.

In the present study, we evaluate the N-back’s utility in assessing WM in young survivors of childhood ALL. We examine the effects of gender and age on N-back performance, as well as the association between the N-back and standardized clinical measures of cognitive function. We predicted that performance on the N-back task would correspond to traditional measures of WM in ALL survivors, regardless of age. We also expected to find associations between performance on a Continuous Performance Test and the N-back, particularly the 0-back condition as it is essentially a quite similar task.

METHODS

Survivors

At the time of analyses, 178 long term survivors meeting eligibility criteria were approached for participation and 137 (76.0%) consented. Survivors were at least 5 years post-diagnosis and at least 8 years of age. Treatment intensity was represented by risk stratification according to established institutional guidelines (Pui et al., 2009), with survivors dichotomized into one of two risk groups defined by factors such as white blood cell counts, CNS infiltration, genetics and age at diagnosis. Of the 137 survivors, 70 were female. Average current age was 14.3 years (range: 8 to 26.5 years) with a mean education of 7.5 years. Mean time since diagnosis was 7.6 years and treatment risk stratum included 63 standard/high risk survivors. Additional survivor characteristics are provided in Table 1. Exclusion criteria included a history of relapse or bone marrow transplantation, treatment with cranial or total-body radiation therapy, and a history of head injury, neurological condition unrelated to ALL treatment, or diagnosis of a genetic disorder associated with neurocognitive impairment (e.g. Down syndrome). The study protocol was approved by the St. Jude Children’s Research Hospital Institutional Review Board and informed consent was obtained from all parents or survivors, where appropriate.

Table 1.

Survivor Characteristics

	Survivors (N=137)
	N	Mean (SD)	Range
Demographics
Sex
Female	70
Male	67
Current Age (years)	137	14.3 (4.8)	8.0–26.5
8 to 10	45
11 to 14	46
15 and older	46
Survivors’ Education (years)	137	7.5 (4.2)	2–17
Mothers’ Education (years)	134	13.9 (2.6)	4–20
Fathers’ Education (years)	131	13.8 (3.2)	0–22
Treatment Characteristics
Risk Stratification
Low Risk	74
Standard/High Risk	63
Age at Diagnosis (years)	137	6.6 (4.5)	1.0–18.5
Time Since Diagnosis (years)	137	7.6 (1.6)	5.1–11.2

Procedure

Survivors completed a battery of neurocognitive tasks, including the experimental N-back, during a single testing session. Specifically, tests administered during evaluation included: Delis Kaplan Executive Function System (Delis, 2001); Wisconsin Card Sort Test(Berg, 1948); Continuous Performance Test(Conners, 1995); Rey-Osterrieth Complex Figure Test(Rey, 1941); Wechsler Digit Span, Spatial Span, Coding and Symbol Search(Wechsler, 1989, 1991, 2008); Attention Network Test(Fan, McCandliss, Sommer, Raz, & Posner, 2002); Grooved Pegboard(Klove, 1963); and Wechsler Abbreviated Scale of Intelligence(Wechsler, 1999). The current study reports on measures directly related to the N-back performance. To minimize the effect of fatigue, survivors were offered breaks at fixed intervals during the evaluation. Assessment order was standardized with consideration for administration consistency, fatigue effects and order effects. The N-back was administered roughly two hours after the CPT-II task. Tests were administered by a licensed/certified psychological examiner, under the general supervision of a board certified clinical neuropsychologist. Computerized tasks, including the N-back, were performed on a Dell desktop PC with 19 inch monitor. The N-back, was run using E-Prime® 2.0 Professional software and survivors responded using a serial response box (see task description below).

Measures

The following standardized tasks were administered:

N-back task

Survivors viewed a continuous stream of sequential single random stimuli from a set of uppercase consonants. Each consonant was presented for 500 msec, with an inter-stimulus interval of 2,500 msec. Trials were broken up into three conditions, 0, 1, and 2-back, each divided into two blocks of 20 non-targets and 10 targets randomly presented. In 0-back condition, the target was an uppercase “X”. The 1-back target was any consonant that was identical to the consonant presented immediately before it in the series. In the 2-back condition, WM load was increased and the target was any consonant identical to the consonant two back in the series (See Figure 1 for depiction of experimental procedure). Breaks were given before blocks and each of the three N-back conditions was preceded by a practice session. Survivors could view their practice performance and were allowed to repeat this practice session up to three times to ensure task comprehension. Responses were made on a serial response box as either Y for targets or N for non-targets. For each condition, reaction time (RT) and accuracy was recorded for all targets and non-targets separately. Calculated scores do not include the first three consonant presentations of each block.

Figure 1.

N-back Experimental Procedure

Conners’ Continuous Performance Test II (CPT-II)

The CPT-II (Conners, 1995) is a computerized measure of sustained attention and cognitive control. Survivors were presented a series of letters in random order and were asked to press the space bar for the appearance of any letter except “X”. During the 14 minute task, indices of omissions, commissions, RT, variability, detectability and response style were assessed.

Processing Speed Index (PSI)

Survivors were administered age-appropriate measures of processing speed, including the Coding and Symbol Search, from the Wechsler Intelligence scales (survivors aged 8–16 completed the Wechsler Intelligence Scale for Children-IV (WISC-IV) (Wechsler, 1991); survivors 17 and older completed the Wechsler Adult Intelligence Scale-IV (WAIS-IV) (Wechsler, 2008). The Processing Speed Index (PSI) was calculated as the age-adjusted standard score for these combined tests.

Digit Span

The digit span subtest was administered from WISC-IV or WAIS-IV depending on survivor’s age. Survivors were presented a series of digits orally and asked to repeat the series either in the exact sequence (forward) or in reverse order (backward). The forward and backward conditions were considered separately and scaled scores were calculated based on the longest sequence of digits recalled.

Spatial Span

Survivors completed a measure of spatial span from the age-appropriate Wechsler scale (survivors age 8–16 years completed the WISC-IV-IN (Wechsler, 1989); survivors ≥17 years completed the Wechsler Memory Scale-III). Spatial span is a visual variation of the digit span subtest, requiring survivors to repeat a tapped sequence of cubes. Forward and backward conditions were again considered separately.

Statistical Analyses

Descriptive statistics were computed for demographics, treatment characteristics, and neurocognitive measures. Given the age range of survivors and potential for developmental differences on N-back performance, three age groups were identified: 8–10 years old (n=45), 11 to 14 years old (n=46), and ≥ 15 years old (n=46). Three age groups were selected to be consistent with existing literature and standards, to maintain relatively equal sample sizes and to eliminate within group age association with working memory performance (confirmed by correlation analysis). The upper age range of 15+ years was identified to be consistent with the National Cancer Institute’s definition of Adolescents and Young Adults (AYA; (http://www.cancer.gov/cancertopics/aya)); the lower age range of 8–10 years was defined as pre-adolescence, leaving the 11–14 year range as the early adolescence phase. Means were calculated for N-back reaction times (RT) per each task condition (0, 1, 2-back). N-back accuracy was calculated as [1 - ((number of target omissions + number of target misses)/total number of targets)] x 100. Patients with N-back performance indicating disengagement or random responding were removed from all analyses moving forward. Specifically, patients with target omission rates consistent with chance levels (≥ 50 percent of trials) or error rates across the entire condition (considering both targets and non-targets) consistent with chance levels were excluded. Three survivors demonstrated a response style on 2-back that warranted removal: 1 survivor omitted >50% of responses, while 2 survivors demonstrated error rates for both targets and non-targets indicative of random responding. Two survivors refused to continue the task after the 0-back condition. For three of the survivors the E-prime software malfunctioned and their reaction time data was lost. To examine the effects of age, gender and risk level on N-back performance, accuracies and RTs were examined across task conditions using repeated measures ANOVA. Age, gender and risk level were analyzed as the between subjects factors and WM load as the within subjects factor. Significant effects between classes with more than three levels were identified through Tukey post-hoc analyses. Correlation between N-back and neurocognitive test performance was examined using Spearman’s rho. Receiver operator characteristic (ROC) curves were calculated to determine the optimal cut points of N-back accuracy for the identification of both verbal and non-verbal WM impairment (defined as ≤ 1 SD below the age-adjusted mean). Area under the curve (AUC), sensitivity, specificity, and correct classification rate was calculated for 0-, 1- and 2-back load accuracy for every age group. The criterion for cutoff selection was the N-back accuracy score that maximized both sensitivity and specificity. SAS version 9.2 (SAS Institute, Cary, NC) was used for all statistical analyses.

RESULTS

N-back

Accuracy

Descriptive statistics for accuracy on all N-back conditions are provided in Table 2. Gender and treatment risk was not significantly related to N-back accuracy. Age was found to be significantly correlated with 0-back accuracy (r_s=0.50, p=0.001) 1-back accuracy (r_s=0.46, p=0.001) and 2-back accuracy (r_s=0.60, p=0.001) for the entire sample. Correlation analysis justified our age range grouping by demonstrating a non-significant association between N-back WM performance and age within each of the three defined age groups (8–10 group: age vs. 1-back accuracy [r_s=0.17, p=0.27]; 8–10 group: age vs. 2-back accuracy [r_s=0.13, p=0.40]; 11–14 group: age vs. 1-back accuracy [r_s=0.07, p=0.63]; 11 to 14 group: age vs. 2-back accuracy [r_s= −0.15, p=0.33]; ≥ 15 group: age vs. 1-back accuracy [r_s=0.17, p=0.26]; ≥ 15 group: age vs. 2-back accuracy [r_s=0.25, p=0.10]). Accuracy demonstrated a significant main effect of age group F(2, 129) = 43.18, p<0.0001 and a main effect of WM load F(2, 258) = 104.6, p<0.0001. Results also indicated a significant WM load by age group interaction F(4, 258) = 5.5, p<0.001 on N-back accuracy. As WM load increased, accuracy performance for survivors ≥ 15 years of age decreased less than the accuracy of younger survivors. Interaction effect and accuracy of each WM load by age group is displayed in Figure 2.

Table 2.

Tested measures

	N	Mean (SD)	95% CI	% Impaired
Clinical Neurocognitive Performance2
WASI3
Full Scale IQ	136	100.0 (14.4)	97.5–102.4	15.4
Conners’ Continuous Performance Test-II4
Omissions	135	51.8 (15.0)	49.2–54.3	14.8
Commissions	135	51.6 (10.5)	49.8–53.4	25.2
Hit Reaction Time	135	50.7 (10.3)	49.0–52.5	17.0
Variability	135	50.2 (10.4)	48.5–52.0	22.2
Detectability (d prime)	135	51.5 (10.5)	49.7–53.3	17.8
Beta	135	50.4 (9.0)	48.9–51.9	10.4
WISC-IV/ WAIS-IV 5
Processing Speed Index6	137	97.0 (14.8)	94.5–99.5	22.6
Digit Span7
Forward	136	9.0 (3.1)	8.5–9.5	35.3
Backward	136	9.2 (3.0)	8.7–9.7	27.2
WISC-IV-IN/WMS-III8
Spatial Span7
Forward	137	9.7 (3.0)	9.2–10.2	24.8
Backward	137	9.8 (3.1)	9.3–10.4	27.7
Research Measure Performance9
N-back10
0-Back
Accuracy	137	83.4 (16.1)	80.7–86.1
Reaction time	134	539.3 (159.1)	512.1–566.5
1-Back
Accuracy	135	73.6 (20.6)	70.0–77.1
Reaction time	132	601.8 (217.3)	564.4–639.2
2-Back
Accuracy	132	59.5 (24.9)	55.2–63.8
Reaction time	129	703.3 (299.7)	651.1–755.5

¹Impairment defined as performance ≤ 1 SD below the normative mean and ≥1SD above the normative mean for measures where higher scores equal worse performance.

²Measures are nationally standardized.

³Wechsler Abbreviated Scale of Intelligence, IQ reported as Standard Score

⁴A sustained attention and cognitive control task, CPT-II measures represent age adjusted T-Scores, with higher scores representing worse performance.

⁵Wechsler Intelligence Scale for Children-IV / Wechsler Adult Intelligence Scale-III; WISC-IV administered to survivors 16 and younger, WAIS-IV administered to survivors 17 and older.

⁶PSI is computed as a standard score from Coding/Symbol Search subtests of the WISC-IV/WAIS-IV.

⁷Scores listed are age adjusted scaled scores.

⁸Wechsler Intelligence Scale for Children-IV-IN / Wechsler Memory Scale-III; WISC-IV-IN administered to children 16 and younger; WMS-III administered to survivors 17 and older.

⁹Non-standardized measure used for research purposes only. Reaction times are in milliseconds and accuracy represents percent correct

¹⁰WM measure that varies WM load. Difficulty increases from 0 to 2-back. Two survivors refused to continue the task after the 0-back condition. Three survivors demonstrated a response style on 2-back that warranted removal: 1 survivor omitted >50% of responses, while 2 survivors demonstrated error rates for both targets and non-targets indicative of random responding. For three of the survivors the E-prime software malfunctioned and their reaction time data was lost.

Figure 2.

Reaction Time

RT data for 0, 1, and 2-back conditions are presented in Table 2. Results showed a significant main effect for gender F(1, 123) = 4.8, p<0.05, with females demonstrating slower RTs than males (Female mean=662.1 [255.6], Male mean=558.1 [205.1] milliseconds). There was also a significant main effect of age group F(2, 123) = 15.3, p<0.0001, and WM load F(2, 246) = 37.0, p<0.0001. Reaction time also demonstrated a significant age group by WM load interaction F(4, 246) = 2.5, p<0.05. The youngest age group demonstrated a larger increase in RTs as WM load increased compared to the older groups. RT performance by age group for each level of WM load is depicted in Figure 2.

Association with Standardized Neurocognitive Measures

Significant associations were demonstrated between N-back accuracy and neurocognitive performance. Specifically, 0-back accuracy correlated with measures of sustained attention (r_s=−0.43, p<0.001), whereas 1 and 2-back accuracy correlated with measures of sustained attention and WM (1-back: attention r_s=−0.38, p<0.001, nonverbal WM r_s=0.28, p<0.001; 2-back: attention r_s=−0.32, p<0.001, nonverbal WM r_s=0.25, p<0.01, verbal WM r_s=0.17, p<0.05). Verbal WM was not significantly correlated with 1-back accuracy when assessing the entire sample (r_s=0.09, p=0.27). Given the significant differences in N-back performance by age, correlations with standard measures were also examined by age group (see Table 3). Among younger survivors, performance on the 1-back condition was correlated with nonverbal WM (10 and younger: r_s =0.36, p<0.02; 11 to 14 years of age: r_s =0.35, p<0.02), while among survivors 15 and older it was correlated only with measures of sustained attention (r_s =−0.62, p<0.001). The more difficult 2-back condition was correlated with nonverbal and verbal WM measures, but only in survivors 15 and older (nonverbal r_s =0.56, p<0.01; verbal r_s =0.30, p=0.04).

Table 3.

Spearman’s rho between N-back accuracy and neurocognitive performance by age group

	0-back			1-back			2-back
	8–10	11 – 14	15+	8–10	11 – 14	15+	8–10	11 – 14	15 +
	rs (p)	rs (p)	rs (p)	rs (p)	rs (p)	rs (p)	rs (p)	rs (p)	rs (p)
CPT-II+
Omissions	−0.55 (<0.001)	−0.38 (0.01)	−0.45 (0.002)	−0.62 (<0.001)	−0.45 (0.002)	−0.30 (0.04)	−0.36 (0.02)	−0.33 (0.03)	−0.54 (<0.001)
Comissions	−0.34 (0.03)	−0.41 (0.004)	−0.59 (<0.001)	−0.06 (0.73)	−0.55 (<0.001)	−0.62(<0.001)	−0.10 (0.53)	−0.25 (0.10)	−0.62 (<0.001)
Hit Reaction Time	−0.12 (0.42)	0.25 (0.09)	0.23 (0.12)	−0.12 (0.44)	0.20 (0.18)	0.28 (0.06)	−0.09 (0.58)	0.09 (0.56)	0.12 (0.41)
Variability	−0.59 (<0.001)	−0.25 (0.10)	−0.25 (0.09)	−0.43 (0.005)	−0.18 (0.24)	−0.24 (0.12)	−0.25 (0.12)	−0.06 (0.69)	−0.47 (0.001)
d prime	−0.21 (0.08)	−0.39 (0.007)	−0.57 (<0.001)	0.01 (0.93)	−0.56(<0.001)	−0.56(<0.001)	−0.004 (0.98)	−0.12 (0.42)	−0.58 (<0.001)
Beta	−0.27 (0.17)	−0.35 (0.02)	0.10 (0.53)	−0.36 (0.02)	−0.33 (0.03)	0.11 (0.47)	−0.27 (0.09)	−0.25 (0.09)	0.07 (0.67)
Processing Speed
PSI	0.17 (0.28)	0.36 (0.01)	0.22 (0.14)	0.36 (0.02)	0.45 (0.002)	−0.004 (0.98)	0.27 (0.09)	0.28 (0.06)	0.25 (0.09)
Digit Span
Forward	0.38 (0.01)	0.06 (0.67)	0.02 (0.91)	0.25 (0.10)	0.04 (0.78)	−0.17 (0.26)	0.16 (0.31)	0.28 (0.07)	0.21 (0.17)
Backward	0.08 (0.60)	−0.08 (0.58)	0.19 (0.21)	0.20 (0.20)	0.18 (0.24)	−0.15 (0.33)	0.08 (0.62)	0.27 (0.07)	0.30 (0.04)
Spatial Span
Forward	0.09 (0.54)	0.07 (0.66)	0.27 (0.07)	0.33 (0.03)	0.15 (0.31)	0.40 (0.006)	−0.07 (0.67)	0.27 (0.07)	0.45 (0.002)
Backward	0.25 (0.09)	0.19 (0.22)	0.37 (0.01)	0.36 (0.02)	0.35 (0.02)	0.29 (0.05)	0.29 (0.06)	0.16 (0.28)	0.56 (<0.001)

Bolded values represent significant correlation at p < 0.05

⁺Higher scores represent poorer performance

Correlation analyses of N-back RT revealed an association with only sustained attention measures (0-back: r_s =0.57, p<0.001; 1-back: r_s =0.44, p<0.001; 2-back: r_s =0.54, p<0.001). Results did not change when correlations were broken down by either age groups or gender.

Receiver Operator Characteristics

As shown in Table 4, AUC of the ROCs identifies 2-back accuracy as differentiating between verbal WM impairment and non-impairment for all ages. An accuracy cut-off value of 0.80 yielded a correct classification rate for verbal WM of 73.9 %, with sensitivity of 90.0% and specificity of 69.4%. Regarding nonverbal WM, the AUC for 2-back accuracy among survivors 15 years of age and older was 0.86 (95% CI=0.75 – 0.96), indicating good diagnostic utility relative to standard clinical assessment. When a cut-off value of 0.80 was used for 2-back accuracy of survivors 15 and older, sensitivity and specificity was optimized (81.3% and 77.0% respectively) and correct classification rate was 78%. Although AUC is higher for 2-back accuracy (0.62 [95% CI= 0.35 – 0.89]) than 1-back accuracy (0.58 [95% CI= 0.30 – 0.85]) in survivors 11–14, nonverbal WM correct classification rate is higher for 1-back (69.6%) than 2-back accuracy (55.6%).

Table 4.

Sensitivity and specificity of N-back accuracy to distinguish between working memory impaired and non-impaired survivors

	Verbal Working Memory Impairment						Nonverbal Working Memory Impaired
Age (years)		ROC area under the curve	N-back Accuracy Cut-off	Correct classification rate (%)	Sensitivity (%)	Specificity (%)	ROC area under the curve	N-back Accuracy Cut-off	Correct classification rate (%)	Sensitivity (%)	Specificity (%)
8 – 10	0-back	0.55	≤0.70	54.55	46.15	58.07	0.63	≤0.70	62.22	57.14	64.52
	1-back	0.65	≤0.55	65.11	58.33	67.74	0.71	≤0.60	67.44	69.23	70.00
	2-back	0.67	≤0.35	68.29	66.67	65.52	0.61	≤0.40	56.10	61.54	53.57
11 – 14	0-back	0.50	≤0.85	47.83	71.43	37.50	0.49	≤0.85	52.17	50.00	52.63
	1-back	0.65	≤0.55	58.70	71.43	68.75	0.58	≤0.60	69.57	50.00	73.68
	2-back	0.67	≤0.55	64.44	76.92	59.36	0.62	≤0.55	55.56	71.43	52.63
15+	0-back	0.69	≤0.95	56.52	90.00	47.22	0.71	≤0.95	60.87	81.25	50.00
	1-back	0.59	≤0.85	65.22	60.00	66.67	0.64	≤0.85	65.22	56.25	70.00
	2-back	0.83	≤0.80	73.91	90.00	69.44	0.86	≤0.80	78.26	81.25	77.00

Verbal working memory impairment defined by a Digit Span Backward scaled score ≤ 7. Nonverbal working memory impairment defined by a Spatial Span Backward scale score ≤ 7. N-back accuracy represents percent correct.

DISCUSSION

We examined the influence of three factors on N-back performance in ALL survivors: current age, gender and treatment risk stratum. N-back performance differed significantly by survivor gender and age, with females demonstrating slower RTs and younger age groups demonstrating lower accuracy and slower RT. Conjointly, age at time of the evaluation interacted with N-back WM load for accuracy and RT measures. Considering these group differences when assessing convergent validity with standard clinical measures, we found that N-back accuracy was associated with different cognitive skills depending on survivor age and WM load combination. Classification accuracy calculations suggest that N-back utility in assessing WM in ALL survivors is limited to the accuracy score of the 2-back load in older age groups.

Gender group differences on N-back performance were reflected in RT but not accuracy; female survivors demonstrated slower overall N-back RT compared to males. This finding is contradictory to N-back literature where studies in healthy participants have reported no gender difference in N-back RT (Lejbak, Crossley, & Vrbancic, 2011; Schmidt et al., 2009). However, studies have reported differential brain activation by gender during N-back performance in populations with known risk of cognitive deficit (Phillips et al., 2011; Valera et al., 2010). Our results appear to be consistent with greater risk of cognitive impairment in female ALL survivors. Existing literature demonstrates females at risk for developing deficits in attention and processing speed, though this literature is predominately based on survivors who were treated with cranial radiation (Jain et al., 2009; Janzen & Spiegler, 2008). Gender effects are less consistent in survivors treated with chemotherapy only. It also appears that gender differences, when they exist, may be dependent on specific cortical networks. We previously demonstrated that female survivors of ALL demonstrated poorer performance on tasks commonly associated with frontomedial and subcortical attention systems, while boys demonstrated poorer performance on tasks commonly associated with an anterior cortical attention system (Jain et al., 2009). We hypothesized that these differences were associated with sex-based difference in brain morphology and maturation (i.e. girls show earlier development of white matter in frontal lobes, while boys show larger increases in white matter latter in development). In the current study we did not find significant gender differences on traditional neuropsychological measures. Our overall findings suggest that reaction time on the N-back task may be more sensitive to the subcortical attention system compared to the traditional neuropsychological measures.

Age differences in N-back performance have been previously reported in healthy subjects (Ciesielski et al., 2006; Thomason et al., 2009). Consistent with these past reports, our results demonstrate that compared to older survivors, younger survivors demonstrate greater decrease in accuracy and greater increase in RT as WM load increases. A closer look at each age group by WM load combination revealed slightly different results for RT compared to accuracy. RT in the 8–10 year old group significantly slowed as WM load increased when compared to the 11–14 and 15+ groups. In contrast, both the 8–10 and 11–14 year old groups demonstrate significantly lower accuracy as WM load increased compared to the 15+ group. This difference suggests that the WM skills needed to accurately perform the N-back may develop later than skills needed for response speed in our survivors. Additionally, the skills needed for survivors to meet the demands of larger WM loads may not develop until adolescence or later. Contradictory to our findings, past N-back studies reported healthy children completed increasing levels of WM loads without reaching levels of chance responding (≤50% accuracy). Specifically, children ages 7–12 demonstrated 71.9% accuracy for nonverbal WM and 75.7% accuracy for verbal WM(Thomason et al., 2009). This performance is comparable to accuracies of survivors 15 years of age and older in our sample. Survivors in age groups 8–10 and 11–14 in our sample demonstrate chance responding at the 2-back load for both nonverbal and verbal WM, suggesting that this level of WM load may be too difficult for survivors of this age. Furthermore, this decrease in N-back performance as WM load increased raises questions as to whether WM performance is accurately captured by the N-back task in younger survivors. When assessing the entire sample, small but significant positive correlations were found between nonverbal WM and 1 and 2-back accuracy. However, age impacted the association between N-back performance and standard measures of WM; the highest level of N-back load (2-back accuracy) was associated with standard measures of WM (verbal and nonverbal), but only in survivors 15 years of age and older. At the 2-back load, younger survivor (8–10, 11–14 groups) N-back performance correlated with a measure of attention only. It was not until the easier 1-back load, all age groups demonstrated N-back accuracy association with WM.

One potential explanation for the varying N-back associations with standard measures by age is explained by developmental differences in task performance. Studies examining brain activation during N-back performance have found differential cortical activation by age (Ciesielski et al., 2006; Crone, Wendelken, Donohue, van Leijenhorst, & Bunge, 2006). In line with our results, it is suggested that WM ability is influenced by a developmental shift of cortical function through adolescence. This cognitive shift facilitates WM maintenance and manipulation, skills needed for greater WM capacity. Additionally, WM cognitive strategies during N-back performance may be modality specific by age and level of WM load. It has been shown that there are different, as well as overlapping regions of activation when comparing auditory to visual WM tasks (Rodriguez-Jimenez et al., 2009). Although our N-back task stimulus is presented as a string of consonants, we don’t see a verbal WM association until the 2-back load in survivors ≥ 15 years old. Despite the verbal qualities of the task stimulus, our N-back is presented visually. This presentations style may have primed older survivors to use a visual spatial strategy until demands of task load influenced recruitment of verbally mediated WM skills. In contrast, younger survivors appear to recruit a visual/spatial cognitive strategy effective only at 1-back load, possibly due to limitations in developmental ability.

Our findings on standard clinical measures are consistent with other literature specific to neurocognitive outcomes for ALL survivors (Krull et al., 2008). Although ALL survivors’ mean neurocognitive performance appears average, rates of impairment on these measures are higher than what is observed in healthy populations. This suggests that only a subset of patients is at risk for impairment, perhaps due to genetic susceptibility (Krull et al., 2013). We found it necessary to evaluate the ability of the N-back to predict WM impairment, as defined by scores ≤ 1 SD below the mean on standard clinical measures, in ALL survivors. Our results show that as the N-back load increases, task performance becomes more accurate in identifying both verbal and nonverbal WM impairment. The exception is seen in survivors 8–10 on nonverbal WM; where 1-back load demonstrates the highest level of correspondence with WM impairment. As age increases we see better correspondence between N-back performance and both verbal and nonverbal WM impairment. Thus, the N-back is most useful in older survivors at the 2-back load in assessing WM impairment. Although greater sensitivity is demonstrated for verbal WM impairment, AUC and correct classification rate is higher for nonverbal WM, suggesting greater overall utility for this domain.

This study was not without limitations. We did not have access to a healthy age-matched control group, and are unable to comment on how survivor N-back performance may demonstrate aberrant results in accuracy or RT compared to a healthy population. We did not have access to functional activation of cortical networks evoked by N-back performance, and although we were able to identify differences in task performance by our established age groups, these groupings and associated results may not translate to similar findings in cortical activation of working memory networks. Despite having a large sample overall, stratification of age groups produced smaller subgroups that may have limited power in analysis. Further, N-back classification accuracy of WM impairment may have been reduced by the small subgroup age stratification. Lastly, due to the cross sectional nature of this study, we cannot comment on changes in N-back performance as survivors age.

In summary, this study demonstrates the N-back to be an effective tool in assessing WM function, with highest concordance with standardized measures observed in survivors ≥ 15 years of age at the 2-back load. Although 2-back accuracy in survivors ≥ 15 is sensitive to verbal WM impairment, correlations combined with ROC analyses suggests the N-back to be more strongly associated with nonverbal WM. Convergent validity between the N-back and nonverbal WM was demonstrated in younger survivors at the easier 1-back load. Future imaging studies should take in to account N-back task performance. As seen with our results, low accuracy N-back performance may not capture WM association.