Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Jun 23.
Published in final edited form as: N Engl J Med. 2007 May 3;356(18):1842–1852. doi: 10.1056/NEJMoa066397

Long-Term Effect of Diabetes and Its Treatment on Cognitive Function

The Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications (DCCT/EDIC) Study Research Group*

PMCID: PMC2701294  NIHMSID: NIHMS65656  PMID: 17476010

Abstract

BACKGROUND

Long-standing concern about the effects of type 1 diabetes on cognitive ability has increased with the use of therapies designed to bring glucose levels close to the non-diabetic range and the attendant increased risk of severe hypoglycemia.

METHODS

A total of 1144 patients with type 1 diabetes enrolled in the Diabetes Control and Complications Trial (DCCT) and its follow-up Epidemiology of Diabetes Interventions and Complications (EDIC) study were examined on entry to the DCCT (at mean age 27 years) and a mean of 18 years later with the same comprehensive battery of cognitive tests. Glycated hemoglobin levels were measured and the frequency of severe hypoglycemic events leading to coma or seizures was recorded during the follow-up period. We assessed the effects of original DCCT treatment-group assignment, mean glycated hemoglobin values, and frequency of hypoglycemic events on measures of cognitive ability, with adjustment for age at baseline, sex, years of education, length of follow-up, visual acuity, self-reported sensory loss due to peripheral neuropathy, and (to control for the effects of practice) the number of cognitive tests taken in the interval since the start of the DCCT.

RESULTS

Forty percent of the cohort reported having had at least one hypoglycemic coma or seizure. Neither frequency of severe hypoglycemia nor previous treatment-group assignment was associated with decline in any cognitive domain. Higher glycated hemoglobin values were associated with moderate declines in motor speed (P=0.001) and psychomotor efficiency (P<0.001), but no other cognitive domain was affected.

CONCLUSIONS

No evidence of substantial long-term declines in cognitive function was found in a large group of patients with type 1 diabetes who were carefully followed for an average of 18 years, despite relatively high rates of recurrent severe hypoglycemia. (ClinicalTrials.gov number, NCT00360893.)


Over time, improving glycemic control decreases the risk of microvascular, peripheral neuropathic, and macrovascular complications of type 1 diabetes.14 However, it is unclear whether type 1 diabetes and its treatment have substantial effects on the structure and function of the central nervous system.58 The widespread use of intensive therapies designed to achieve glycemic control near the nondiabetic range and the attendant increased risk of severe hypoglycemia9 have elevated concern about the effects of hypoglycemia on the central nervous system.

The Diabetes Control and Complications Trial (DCCT) incorporated a comprehensive battery of cognitive tests to evaluate the effect of diabetes treatment and recurrent hypoglycemic events on cognitive ability. At an average of 6.5 years of follow-up, no untoward effects were associated with either treatment type (conventional or intensive) or with the number of severe hypoglycemic episodes during the DCCT.5,10 However, the DCCT involved a short follow-up of adolescent and young adult patients (mean age, 27 years) who had had relatively few hypoglycemic events. Although these initial findings were promising, longer-term follow-up was needed to determine whether the increase by a factor of three in the frequency of severe hypoglycemia among patients receiving intensive therapy during the DCCT adversely affected cognitive ability over time. Moreover, a longer-term study would allow for an investigation of the potential effects on cognition of recurrent, severe hypoglycemia during the post-DCCT follow-up period; of persistent differences in glycemic control, as indicated by the glycated hemoglobin value; and of the increased age of the cohort members and the increased duration of their diabetes.

To address these questions, we repeated the cognitive evaluation 12 years after the end of the DCCT as part of the follow-up Epidemiology of Diabetes Interventions and Complications (EDIC) study. Thus, the average total follow-up time was 18 years. The longitudinal follow-up of this carefully characterized cohort with type 1 diabetes addressed three specific questions: Was assignment to intensive versus conventional therapy during the DCCT associated with differences in cognitive decline over the 18-year follow-up period? Is a history of severe hypoglycemic events leading to coma and seizures associated with cognitive decline? Is the level of long-term glycemic control, as measured by glycated hemoglobin values, associated with cognitive decline?

METHODS

PATIENTS

Between 1983 and 1989, a total of 1441 patients from 13 through 39 years of age with type 1 diabetes were enrolled in the DCCT. The DCCT study population consisted of two cohorts. Patients in the primary-prevention cohort had received a diagnosis of diabetes 1 to 5 years previously, had no retinopathy, and had a urinary albumin excretion rate of less than 40 mg per 24 hours. The secondary-intervention cohort consisted of patients who had had diabetes for 1 to 15 years, had nonproliferative retinopathy ranging from very mild to moderate, and had a urinary albumin excretion rate of 200 mg per 24 hours or less at baseline.

A total of 711 patients were randomly assigned to intensive therapy. Intensive therapy consisted of three or more insulin injections daily or subcutaneous infusion of insulin with an external pump, guided by frequent self-monitoring of blood glucose levels. The target preprandial blood glucose level was between 3.9 and 6.7 mmol per liter (70 and 120 mg per deciliter), the target glycated hemoglobin value (measured monthly) was within the nondiabetic range (<6.0%), and a goal of therapy was to prevent severe hypoglycemia. The remaining 730 patients were assigned to conventional therapy. These patients received one or two insulin injections daily; there were no target blood glucose levels, and the therapeutic goal was freedom from symptoms of hyperglycemia and from frequent or severe hypoglycemia.

During the 6.5-year DCCT follow-up, the median glycated hemoglobin values of the two treatment groups were maintained at a separation of 1.9 percentage points (7.1% in the intensive-treatment group and 9.0% in the conventional-treatment group). At the end of the DCCT, in 1993, intensive therapy was recommended for all patients, since it had been shown to be highly effective in reducing long-term complications of diabetes.1 Patients in the conventional-treatment group were given training in aspects of intensive therapy and were then returned to their own health care providers for diabetes care. In 1994, a total of 1375 (96%) of the 1428 surviving members volunteered to participate in the EDIC observational follow-up study.11 As previously reported, between-group differences in the median glycated hemoglobin values narrowed during the 12 years of the EDIC follow-up study to 0.2 percentage point (8.0% in the group that had previously received intensive treatment vs. 8.2% in the group previously receiving conventional treatment, P=0.03). In 2004, of the surviving eligible participants, 1144 (85%) were reevaluated with the cognitive-test battery.

COGNITIVE-TEST PROTOCOL

Cognitive testing, as originally described for the DCCT,5 was conducted at each site by personnel who were trained and certified by the DCCT/EDIC Central Neuropsychological Coding Unit. The test protocol, which required 4 to 5 hours to complete, included the following widely used, well-validated tests that were administered initially during the DCCT: five subtests (Similarities, Digit Span, Digit Symbol, Block Design, and Object Assembly) from the Wechsler Adult Intelligence Scale,12 four subtests (Category, Tactual Performance, Trail Making, and Finger Tapping) from the Halstead–Reitan Neuropsychological Test Battery,13 the Logical Memory and Visual Reproduction subtests from the Wechsler Memory Scale,14 the Digit Vigilance Test,15 the Grooved Pegboard Test,16 the Verbal Fluency Test,17 the Four-Word Short-Term Memory Test,18 the Symbol-Digit Learning Test,19 and the Embedded Figures Test.19 The tests were administered in a fixed order. Capillary blood glucose levels were routinely monitored immediately before the testing and at its midpoint to rule out the presence of hypoglycemia during testing. If a patient was found to have a blood glucose level at or below 3.9 mmol per liter (70 mg per deciliter), testing was stopped, the patient was given a snack, and after a wait of at least 15 minutes, testing was resumed when the reading returned to at least 5.0 mmol per liter (90 mg per deciliter).

The tests were scored by technicians at the Central Neuropsychological Coding Unit who were unaware of treatment assignment and other biomedical variables. The results were sent to the Data Coordinating Center, where the data were entered, verified, and edited for out-of-range values and other errors.

The compliance and performance of the patients during the testing session were rated systematically by the personnel administering the cognitive assessment. Information obtained from the patients was rated as “mostly accurate” to “completely accurate” for 97% of the patients. The testing staff also reported that 99% of the patients were “somewhat willing” to “very willing” to try their best throughout the testing session and that 94% of the patients had a “clear understanding” of the test instructions.

COGNITIVE DOMAINS

During the DCCT, 24 test variables had been chosen a priori to be of particular diagnostic value when applied to patients with type 1 diabetes. For each of these 24 test variables, a standardized z score was calculated, with the mean and standard deviation from the baseline assessment of the DCCT cohort used as references.5 These standardized scores provided a unit-free measurement of the relative improvement (positive sign) or deterioration (negative sign) in performance as compared with the total group at baseline. To reduce the number of comparisons, the 24 standardized scores were grouped into eight cognitive domains consistent with standard neuropsychological assessment strategies.20 For each domain, the simple average of the standardized scores was used to represent the change from baseline, with equal weight assigned to each test.

BIOMEDICAL EVALUATIONS

During the EDIC study, each patient underwent an annual examination that involved history taking, physical examination, an electrocardiogram, and laboratory tests, including tests for serum creatinine and glycated hemoglobin levels; the examination was conducted by the same methods used during the DCCT.21 As part of the history, the patients reported the presence of sensory symptoms of peripheral neuropathy. At entry to the DCCT, potential subjects were excluded if they had symptomatic sensory neuropathy. Data from the year in which cognitive testing was performed were used to characterize the level of symptomatic neuropathy at the time of the reevaluation. A criterion for inclusion in the DCCT was a best corrected visual acuity of 20/25 or better for the primary-prevention cohort and 20/32 or better for the secondary-intervention cohort. Best corrected visual acuity was measured at 4-year intervals during the EDIC study. The results of the visual acuity examination closest to the date of cognitive reevaluation were used to characterize the level of visual acuity at the time of cognitive reevaluation. Fasting lipid profiles were determined and 4-hour urine collections for measurement of albumin excretion and creatinine clearance rates were performed in alternate years during the EDIC study.11 Hypertension was defined as a systolic blood pressure of at least 140 mm Hg, a diastolic blood pressure of at least 90 mm Hg, previously documented hypertension, or the use of antihypertensive agents. Hypercholesterolemia was defined as a serum level of low-density lipoprotein (LDL) cholesterol of at least 130 mg per deciliter (3.4 mmol per liter) or the use of lipid-lowering agents. Renal insufficiency was defined as a centrally measured serum creatinine level of at least 2.0 mg per deciliter (177 µmol per liter), treatment with dialysis, or renal transplantation.

Glycated hemoglobin values were measured in a central laboratory by high-performance liquid chromatography21 quarterly during the DCCT and annually during the EDIC study. The time-weighted mean glycated hemoglobin value during the period of the DCCT and the 12 years of the EDIC study was computed, with each DCCT value weighted by one quarter of a year and each EDIC study value weighted by 1 year.

PSYCHIATRIC SYMPTOMS

Psychiatric symptoms were assessed with the Symptom Checklist-90-Revised, which was administered annually during the DCCT and once during the EDIC study in the same year that the cognitive testing was performed.22,23 For this report, the depression scale was used to assess the effects of mood state on cognitive function.

DEFINITION OF SEVERE HYPOGLYCEMIA

During the DCCT, severe hypoglycemia was defined as any event, including seizure or coma, that required the assistance of another person and in which the blood glucose level was less than 2.8 mmol per liter (50 mg per deciliter) or the symptoms were subsequently reversed by oral carbohydrate, injected glucagon, or intravenous glucose.1 At quarterly visits, study coordinators asked about the occurrence of hypoglycemia since the last visit, and all such events were reported to the Data Coordinating Center as soon as possible after their occurrence. Twenty-seven percent of severe hypoglycemic episodes involved coma or seizure.9 During the EDIC study, severe hypoglycemic events that occurred in the 3 months before the annual visit were documented on the annual history form, and further details surrounding these events were recorded. For the purposes of this article, severe hypoglycemic events are limited to those leading to coma, seizure, or both.

STATISTICAL ANALYSIS

Demographic and clinical characteristics were compared with the use of the Wilcoxon rank-sum test to evaluate the differences between the treatment groups in ordinal and numeric variables.24 The contingency chi-square test was used for categorical variables; when the sample size was small, Fisher’s exact test was used.24

Separate analysis-of-covariance models were used to assess the effects of treatment group (intensive vs. conventional), mean glycated hemoglobin values stratified according to thirds (<7.4%, 7.4 to 8.8%, or >8.8%), and frequency of severe hypoglycemia (zero, one to five, or more than five reported events) on the standardized quantitative score for each of the eight cognitive domains. Each model was adjusted for age at baseline, sex, years of education, length of follow-up, visual acuity, self-reported sensory loss due to peripheral neuropathy, and (to control for the effects of practice) the number of cognitive tests taken during the interval since the start of the DCCT. The results are presented as the average increase or decrease in the standardized score from the DCCT baseline within or between groups or the change per unit in a quantitative covariate. Nominally significant results (P<0.01) are reported.

RESULTS

Table 1 presents the characteristics of the cohort at baseline and at the 18-year follow-up. There were no statistically significant differences between the two treatment groups at baseline. At EDIC study year 12, the age of the patients ranged from 29 to 62 years, with a mean (±SD) of 45.7±6.8 years. The only statistically significant difference between the treatment groups was in the percentage of patients with retinopathy (89% in the intensive-treatment group and 97% in the conventional-treatment group, P<0.001), reflecting the effects of previous DCCT interventions. By year 12 of the EDIC study, 3 patients (1 in the intensive-treatment group and 2 in the conventional-treatment group) had a history of stroke, a decrease in vision had occurred in 5 patients (1 in the intensive-treatment group and 4 in the conventional-treatment group) so that they had sufficient vision only to count fingers, and 20 patients (7 in the intensive-treatment group and 13 in the conventional-treatment group) had renal insufficiency.

Table 1.

Characteristics of the Patients.*

Characteristic At Entry into DCCT (1983–1989) At Year 12 of EDIC Study (2005)
Intensive Treatment
(N=588)
Conventional Treatment
(N=556)
Intensive Treatment
(N=588)
Conventional Treatment
(N=556)
Female sex (%) 49 45 49 45
White race (%) 97 96 97 96
Mean age (yr) 27±7 27±7 46±7 45±7
Adult (%) 89 88 100 100
College graduate (%) 30 27 39 37
Marital status (%)
   Never married 43 44 10 10
   Married 50 52 74 74
   Formerly married 7 4 16 16
Occupation (%)
   Professional or technical 32 33 53 51
   Unemployed or retired 1 <1 6 7
Retinopathy (%) 52 50 89 97
Duration of diabetes (yr) 6±4 6±4 24±5 24±5
Glycated hemoglobin (%) 9.0±1.6 9.0±1.6 7.8±1.2 7.6±1.2
Visual acuity ≥20/40 (%) 0 0 4 5
Peripheral neuropathy (%)** 9 14 30 30
Blood pressure (mm Hg)
   Systolic 113±12 115±12 121±14 120±14
   Diastolic 72±9 73±9 74±9 74±9
Treated hypertension (%) ND ND 40 40
Lipids (mg/dl)††
   Total cholesterol 177±33 174±33 182±34 177±34
   LDL cholesterol 110±29 108±29 109±29 106±28
Lipid-lowering medication (%) ND ND 45 43
Current cigarette smoker (%) 23 22 14 12
Symptom Checklist-90-Revised Depression score‡‡ 51±10 51±10 49±12 49±11
Verbal IQ 112±11 112±10 ND ND
Full-scale IQ 114±10 114±10 ND ND
*

Plus-minus values are means ±SD. ND denotes no data, and LDL low-density lipoprotein.

Race was self-assigned. “White” denotes white, non-Hispanic.

Adults are defined as persons 18 years of age or older.

P<0.01 by the Wilcoxon rank-sum test or the chi-square test comparing conventional and intensive treatment.

The DCCT baseline value is the eligibility value.

Visual acuity was measured with a Snellen chart. At baseline in the DCCT, all patients had visual acuity of 20/32 or better. In the EDIC study, visual acuity of 20/40 or worse in at least one eye was recorded in 44 patients.

**

The DCCT baseline definition of peripheral neuropathy is pain or numbness in the hands only, taken from the Neurological History and Examination form. The EDIC study definition is pain or numbness in the hands or feet, taken from the Annual Medical History and Examination form.

††

To convert values for cholesterol to millimoles per liter, multiply by 0.02586.

‡‡

Scores range from 30 to 80, with higher scores indicating worse performance.

The normative mean IQ score is 100±15.

During the entire 18-year follow-up, a total of 1355 episodes of coma or seizure were reported (896 in 262 patients in the intensive-treatment group and 459 in 191 patients in the conventional-treatment group) (Table 2). Of the 53 deaths during the DCCT and the EDIC study, 3 were attributed to hypoglycemia and all 3 occurred during the EDIC study.

Table 2.

Severe Hypoglycemic Events.*

No. of Events DCCT EDIC Study Total 18-Yr Follow-up of the DCCT
and the EDIC Study
Intensive
Treatment
(N=588)
Conventional
Treatment
(N=556)
Intensive
Treatment
(N=588)
Conventional
Treatment
(N=556)
Intensive
Treatment
(N=588)
Conventional
Treatment
(N=556)
0 Events (no. of patients) 364 445 465 424 326 365
≥1 Event (no. of patients) 224 111 123 132 262 191
   1–5 Events 190 104 119 127 212 175
   >5 Events 34 7 4 5 50 16
Total no. of events 653 205 243 254 896 459
*

Severe hypoglycemic events were defined as those leading to coma or seizure. All hypoglycemic events from the DCCT are documented. Hypoglycemic events from the EDIC study are documented for the 3-month period before the annual visit.

The number of events ranged from 6 to 23 in the intensive-treatment group and from 6 to 13 in the conventional-treatment group.

Table 3 summarizes the raw scores for each test, stratified according to treatment group. The mean scores were well within normal limits when compared with those of a large sample of healthy persons without diabetes.25 Figure 1 shows the cognitive test results for each domain according to the original treatment assignment, the cumulative number of severe hypoglycemic events (zero, one to five, and more than five), and the degree of metabolic control (mean glycated hemoglobin values divided into thirds).

Table 3.

Raw Cognitive Test Scores.*

Test Range
of Scores
At Entry into DCCT
(1983–1989)
At Year 12 of EDIC
Study (2005)
Normative Data
Intensive
Treatment
(N=588)
Conventional
Treatment
(N=556)
Intensive
Treatment
(N=588)
Conventional
Treatment
(N=556)
Problem solving
   Similarities (scaled scores) 1–19 12.6±2.5 12.7±2.4 14.0±2.2 13.9±2.3 11.1±2.4
   Category test (no. of errors) 0–208 37.2±22.6 32.7±19.9 26.3±20.7 24.0±19.3 32.6±21.8
Learning (no. correct)
   Symbol-Digit Learning 0–28 24.1±4.5 24.3±4.4 24.1±4.5 24.1±4.5 21.4±6.1
   Tactual Performance Memory 0–10 7.4±1.5 7.4±1.6 7.5±1.5 7.6±1.5 6.8±1.6
Immediate memory (no. correct)
   Visual Reproductions 0–17 14.7±2.1 14.6±2.3 14.3±2.4 14.4±2.2 13.0±2.3
   Four-Word Short-Term Memory 0–60 40.3±9.1 39.6±9.6 39.0±10.5 39.2±10.4 36.6±10.9
   Logical Memory 0–43 20.0±5.8 19.6±5.5 20.0±6.9 19.8±5.5 16.8±5.8
   Digit Symbol 0–9 7.7±1.6 7.8±1.5 7.5±1.8 7.7±1.7 7.6±1.7
Delayed recall (no. correct)
   Visual Reproductions 0–17 15.5±1.8 15.4±1.8 14.7±2.2 15.0±2.0 13.2±2.8
   Logical Memory 0–43 16.4±5.5 16.2±5.4 17.1±6.3 17.2±5.8 13.0±5.3
Spatial information
   Embedded Figures (sec) 0–60 7.4±3.3 7.1±3.2 6.5±3.1 6.4±2.9 6.3±3.2
   Object Assembly (scaled scores) 1–19 12.1±2.9 12.2±2.9 13.9±2.8 14.0±2.7 11.2±3.0
   Block Design (scaled scores) 1–19 13.0±2.6 13.1±2.6 14.3±2.6 14.2±2.6 11.4±2.7
   Tactual Peformance Test (min) 0–30 10.9±3.8 10.8±3.8 11.4±4.8 11.5±4.6 12.1±5.1
Attention
   Digit Vigilance (sec) 0–1000 368.3±72.5 378.3±86.4 388.1±88.5 397.3±87.5 358.8±80.7
   Digit Vigilance (no. of errors) 0–100 4.5±5.1 4.8±5.3 7.7±8.3 7.8±8.3 5.5±8.3
   Digit Span (scaled scores) 1–19 11.8±2.9 11.8±2.8 12.4±2.6 12.4±2.9 10.8±2.6
Psychomotor and mental efficiency
   Verbal Fluency (no. correct) 0–180 42.3±12.4 42.9±12.3 45.7±13.2 45.6±13.1 40.9±12.6
   Digit Symbol, 90-sec total (no. correct) 10–90 65.1±11.1 65.0±11.0 62.3±11.4 61.9±11.4 60.1±12.8
   Trail Making, part B (sec) 0–300 52.3±16.6 52.9±19.9 54.4±20.0 55.0±18.8 57.3±24.7
   Grooved Pegboard, dominant hand (sec) 0–300 65.6±9.8 65.9±9.6 72.3±17.3 73.8±16.1 66.6±12.2
   Grooved Pegboard, nondominant hand (sec) 0–300 70.2±12.3 70.0±11.0 79.0±18.6 80.5±20.0 71.4±14.5
Motor speed
   Finger Tapping, dominant hand (no. of taps in 10 sec) 0–80 48.7±6.5 48.7±7.0 50.6±7.3 50.6±7.7 50.8±5.9
   Finger Tapping, nondominant hand (no. of taps in 10 sec) 0–80 44.6±6.0 44.7±6.1 45.3±6.7 44.6±6.9 45.2±6.0
*

Plus-minus values are means ±SD. Normative data are based on approximately 175 adults without diabetes who were tested with the same test battery in the mid-to-late 1980s at the University of Pittsburgh Medical Center.

Higher scores indicate better performance, except where noted.

Higher scores indicate poorer performance.

P<0.01 by the Wilcoxon rank-sum test comparing conventional and intensive treatment.

Figure 1. Effects of DCCT Treatment Group, Severe Hypoglycemia, and Glycated Hemoglobin on Changes in Cognition, from Entry into DCCT to Year 12 in the EDIC Study.

Figure 1

The bars show the changes within cognitive domains between cognitive testing at baseline in DCCT and follow-up testing (a mean of 18 years after baseline) expressed as changes in z scores for intensive versus conventional treatment (Panel A), frequency of episodes of severe hypoglycemia (coma or seizure) (Panel B), and mean glycated hemoglobin values (Panel C). Across the three groups, higher levels of glycated hemoglobin were associated with moderate declines in psychomotor efficiency (P<0.001) and motor speed (P=0.001), but no other cognitive domain was affected significantly. Cognitive domains are numbered as follows: 1, problem solving; 2, learning; 3, immediate memory; 4, delayed recall; 5, spatial information; 6, attention; 7, psychomotor efficiency; and 8, motor speed.

Neither the original treatment assignment nor the cumulative number of hypoglycemic events influenced performance in any cognitive domain. Higher values of glycated hemoglobin were associated with moderate declines in psychomotor efficiency (P<0.001) and motor speed (P=0.001). The degree of self-reported symptoms of depression, as indexed by median T scores from the Symptom Checklist-90-Revised, was also associated with poorer performance on measures of learning, immediate memory, and psychomotor efficiency. There were no significant interactions between any of the three key predictors (treatment-group assignment, frequency of severe hypoglycemic events, and glycated hemoglobin value) and any covariate.

The analyses were repeated using the broader definition of hypoglycemia, which includes episodes in which the patient is sufficiently incapacitated to require the assistance of another person. The results obtained with the use of the broad definition were similar to those obtained with the narrow definition.

The analyses were repeated again after exclusion of patients with a history of stroke, impaired vision, or severe kidney disease. The results were similar, with one exception: psychomotor efficiency was significantly worse (P=0.002) in patients receiving conventional therapy.

DISCUSSION

The increased risk of severe hypoglycemia, which accompanies intensive diabetes treatment aimed at achieving blood glucose levels in the normal range, remains one of the primary barriers to the implementation of such treatment. The potential cognitive consequences of severe hypoglycemia, including coma and seizure, are a key concern for clinicians, patients, and families. Patients often wonder whether recurrent hypoglycemia will lead to persistent problems in their ability to think or will have negative effects on school performance or future employability. Our study found no evidence of substantial long-term declines in cognitive function in a large group of patients with type 1 diabetes who were carefully followed for an average of 18 years, despite relatively high rates of recurrent severe hypoglycemia.

These results do not mean that severe hypoglycemia is entirely benign. It is well established that an extended episode of profound hypoglycemia, such as one with a blood glucose level below 1.0 mmol per liter (18 mg per deciliter), can induce massive cerebral energy failure with a corresponding development of neuronal necrosis.26 Less severe episodes of severe hypoglycemia (e.g., those with a blood glucose level in the range of 2.8 to 3.6 mmol per liter [50 to 65 mg per deciliter]) are also known to disrupt brain activity transiently, can lead to short-term cognitive impairment, and can increase the risk of motor vehicle accidents.27,28 Nevertheless, with the exception of several small, cross-sectional studies,2931 most researchers either have failed to find effects or have found only relatively weak effects of recurrent hypoglycemia on brain structure and function in children and adults with diabetes.68,3235

Previous examinations of the DCCT cohort found that after an average follow-up of 6.5 years, cognitive function was not adversely affected by recurrent severe hypoglycemia.5,10 In the EDIC study, after 18 years of follow-up of 85% of the available DCCT participants, no deleterious effects of previous intensive therapy or recurrent hypoglycemia were evident, despite the fact that our patients have had substantially longer exposure to diabetes and its glycemic changes, have had more episodes of severe hypoglycemia, and are now entering a later phase of their lives. By using the same comprehensive, extensively validated test battery that has been used in many other studies, we can draw conclusions about the longitudinal course of cognitive functioning in patients with type 1 diabetes. Our findings of minimal or no effects of either previous intensive treatment or severe hypoglycemia should be reassuring for patients with type 1 diabetes for whom intensive therapy is strongly recommended.

Better glycemic control may have subtle beneficial effects on cognitive ability, mirroring the recognized benefit of near-normal glycemic control to the retina, kidney, peripheral nerves, and cardiovascular system.14 For example, those patients with worse metabolic control (glycated hemoglobin values >8.8%) performed approximately 9% more slowly on measures of psychomotor efficiency than those with better control (glycated hemoglobin <7.4%). Our findings are consistent with recent literature demonstrating that poor metabolic control, with subsequent development of micro- and macrovascular complications, is associated with the development of mild cognitive impairments35,36 and subtle abnormalities in brain structure37 and activation.38 These structural and functional changes, which may reflect cerebral microangiopathy,39,40 are moderate in degree. Moreover, there is no evidence that they adversely affect patients’ activities of daily living. Efforts to prevent the occurrence of microvascular complications may reduce the risk of neurocognitive deficits as well. The relationship noted between higher levels of depressive symptoms and poorer performance on measures of learning and memory is consistent with the extensive literature indicating that depression can adversely affect cognitive function, independent of other biomedical and psychosocial characteristics.41

The long-term, comprehensive follow-up study presented here has substantial strengths; however, there are notable gaps that must be kept in mind. Although some patients received a diagnosis of diabetes as very young children, this subgroup is small, and systematic data on glycated hemoglobin values and severe hypoglycemic events before they entered the DCCT are not available. We also do not have information about the effects of intensive therapy on the elderly or those living for more than 30 years with diabetes. Finally, these results from the selected DCCT cohort should be applied carefully to the total population of patients with type 1 diabetes.

The findings of this study provide an important message about the safety of intensive diabetes therapy for those receiving a diagnosis of diabetes as adolescents or young adults. Within the aforementioned limits, we can be confident that although acute hypoglycemic events can be dangerous at the time they occur, recurrent severe episodes associated with intensive diabetes therapy, as administered in the DCCT, do not appear to have long-term adverse effects on the cognitive capacity of patients with type 1 diabetes. This conclusion lends further support to the use of intensive diabetes therapy to reduce the long-term risks of retinopathic, nephropathic, neuropathic, and cardiovascular complications in type 1 diabetes.

Acknowledgments

Supported by a grant (5 R01 DK062218-02) from and contracts with the Division of Diabetes, Endocrinology and Metabolic Diseases of the National Institute of Diabetes and Digestive and Kidney Diseases and by the General Clinical Research Centers Program, National Center for Research Resources.

Dr. Jacobson reports serving on medical advisory boards for Pfizer and Amylin. Dr. Ryan reports receiving consulting fees from GlaxoSmithKline and Amylin and lecture fees from Eli Lilly. No other potential conflict of interest relevant to this article was reported.

APPENDIX

The following persons and institutions participated in the DCCT/EDIC Study Research Group: Study Chairs — S. Genuth, D.M. Nathan, B. Zinman (vice-chair), O. Crofford (past); Albert Einstein College of Medicine — J. Crandall, M. Phillips, M. Reid, J. Brown-Friday, S. Engel, J. Sheindlin, H. Martinez (past), H. Shamoon (past), H. Engel (past); Case Western Reserve University — W. Dahms (deceased), M. Palmert, L. Mayer, S. Pendegras, H. Zegarra, D. Miller, L. Singerman, S. Smith-Brewer, R. Gubitosi-Klug, P. Gaston, S. Genuth (past); Cornell University Medical Center — D. Brillon, M.E. Lackaye, V. Reppucci, T. Lee, M. Heinemann (past); Henry Ford Health System — F. Whitehouse, M. McLellan, D. Kruger, J.D. Carey, E. Angus, M. Croswell, A. Galpirn (past); International Diabetes Center — R. Bergenstal, M. Johnson, M. Spencer, K. Morgan, D. Etzwiler (deceased), D. Kendall (past), D. Noller (past); Joslin Diabetes Center — A. Jacobson, E. Golden, R. Beaser, O. Ganda, O. Hamdy, J. Rosenzweig, H. Wolpert, P.G. Sharuk, P. Arrigg, A. Burwood, L. Rand (past); Massachusetts General Hospital — D.M. Nathan, M. Larkin, J. Godine, D. Moore, E. Cagliero, P. Lou, S. Fritz (past); Mayo Foundation — J. Service, G. Ziegler, J. Pach, R. Colligan; Medical University of South Carolina — M. Lopes-Virella, J. Colwell, K. Hermayer, M. Brabham, J. Soule, A. Belvins, J. Parker, D. Lee, P. Lindsey, M. Bracey, K. Lee, A. Farr (past), S. Elsing (past), T. Thompson (past), J. Selby (past), T. Lyons (past), S. Yacoub-Wasef (past), M. Szpiech (past), D. Wood (past), R. Mayfield (past); Northwestern University — M. Molitch, B. Schaefer, L. Jampol, A. Lyon, M. Gill, J. Mathura, Z. Strugula, L. Kaminski, J. Shankle, P. Astlesford, D. Blackburn, S. Ajroud-Driss, O. Stone, C. West, I. Burnett-Zeigler, D. Weinberg (past); University of California, San Diego — O. Kolterman, G. Lorenzi, and M. Goldbaum; University of Iowa — W. Sivitz, M. Bayless, T. Weingeist, E. Stone, H. Culver Boldt, K. Gehres, S. Russell, J. Bayless, J. Kramer, J. Long, R. Zeither (past); University of Maryland School of Medicine — M. Hebdon, T. Donner, S. Johnsonbaugh, J. Gordon, A. Kowarski (past), D. Ostrowski (past), T. Donner, S. Steidl (past), B. Jones (past), D. Counts (past); University of Michigan — W. Herman, C. Martin, R. Pop-Busui, A.K. Vine, S. Elner, E. Feldman, J. Albers, D. Greene (past), M.J. Stevens (past); University of Minnesota — J. Bantle, B. Rogness, T. Olsen, E. Steuer (past), P. Rath (past), R. Johnston (past); University of Missouri — D. Hainsworth, S. Hitt, J. Giangiacom, D. Goldstein (past); University of New Mexico — D. Schade, J. Canady, M. Burge, J.M. Schluter, A. Das, D. Hornbeck (past); University of Pennsylvania — S. Schwartz, P. A. Bourne, B.J. Maschak-Carey (past), L. Baker (deceased), S. Braunstein, A. Brucker; University of Pittsburgh — T. Orchard, N. Silvers, C. Ryan, T. Songer, B. Doft, S. Olson, R.L. Bergren, L. Lobes, M. Fineman, L. Freisberg, L. Lobes, P. Paczan Rath, D. Becker, A. Drash (past); University of South Florida — A. Morrison, J. Vaccaro-Kish, M.L. Bernal, J. Malone, P.R. Pavan, N. Grove, M.N. Iyer, A.F. Burrows, E.A. Tanaka (past), C. Berger (past), R. Gstalder (past); University of Tennessee — S. Dagogo-Jack, C. Wigley, H. Ricks, A. Kitabchi, M.B. Murphy, S. Moser, D. Meyer, A. Iannacone, E. Chaum, M. Bryer-Ash (past), S. Schussler (past), H. Lambeth (past); University of Texas Southwestern Medical Center — P. Raskin, S. Strowig, Y.-G. He, A. Edwards (past), J. Alappatt (past), C. Wilson (past), S. Park (past); University of Toronto — B. Zinman, A. Barnie, S. MacLean, R. Devenyi, M. Mandelcorn, M. Brent, S. Rogers, A. Gordon; University of Washington — J. Palmer, S. Catton, J. Brunzell, J. Ginsberg, J. Kinyoun, L. Van Ottingham (past); University of Western Ontario — J. Dupre, J. Harth, D. Nicolle, C. Canny (past); Vanderbilt University — M. May, J. Lipps, A. Agarwal, T. Adkins, L. Survant, R. Lorenz (past), S. Feman (past); Washington University, St. Louis — N. White, L. Levandoski, I. Boniuk, G. Grand, M. Thomas, D. Burgess, D. Joseph, K. Blinder, G. Shah, J. Santiago (deceased); Yale University School of Medicine — W. Tamborlane, P. Gatcomb, K. Stoessel, K. Taylor (past); Clinical Coordinating Center (Case Western Reserve University) — W. Dahms (deceased), R. Trail, J. Quin, P. Gaston, M. Palmert; Data Coordinating Center (George Washington University, Biostatistics Center) — J. Lachin, P. Cleary, D. Kenny (past), J. Backlund, W. Sun, B. Rutledge, B. Waberski, K. Klumpp, K. Chan, L. Diminick, D. Rosenberg (past), B. Petty (past), A. Determan (past), C. Williams (past), L. Dews, M. Hawkins; National Institute of Diabetes and Digestive and Kidney Diseases Program Office — C. Cowie, J. Fradkin, C. Siebert (past), R. Eastman (past); Central Fundus Photograph Reading Center (University of Wisconsin) — R. Danis, M. Davis, L. Hubbard, P. Geithman (past), L. Kastorff, M. Neider, D. Badal, B. Esser, K. Miner, H. Wabers, K. Glander, J. Joyce, N. Robinson, C. Hurtenbach, C. Hannon; Central Biochemistry Laboratory (University of Minnesota) — M. Steffes, J. Bucksa, B. Chavers; Central Carotid Ultrasound Unit (New England Medical Center) — D. O’Leary, L. Funk (past), J. Polak, A. Harrington; Central ECG Reading Unit (University of Minnesota) — R. Crow (past), B. Gloeb (past), S. Thomas (past), C. O’Donnell (past); Central ECG Reading Unit (Wake Forest University) — R. Prineas C. Campbell; Central Neuropsychological Coding Unit — C. Ryan, D. Sandstrom, T. Williams, M. Geckle, E. Cupelli, F. Thoma, B. Burzuk, T. Woodfill; Central ANS Reading Unit (Mayo Clinic) — P. Low, C. Sommer, K. Nickander; Computed Tomography Reading Center (Harbor UCLA Research and Education Institute) — R. Detrano, N. Wong, M. Fox, L. Kim, R. Oudiz; External Advisory Committee — G. Weir (Chair), C. Clark, R. D’Agostino, M. Espeland, B. Klein, T. Manolio, L. Rand, D. Singer, M. Stern; Molecular Risk Factors Program Project (Medical University of South Carolina) — M. Lopes-Virella, W.T. Garvey, T.J. Lyons, A. Jenkins, R. Klein, G. Virella, A.A. Jaffa, D. Lackland, M. Brabham (past), D. McGee (past), D. Zheng (past), R.K. Mayfield (past); Genetic Studies Group (Hospital for Sick Children) — A. Paterson, A. Boright, S. Bull, L. Sun, S. Scherer (past), B. Zinman; Lipoprotein Distribution/Obesity Group (University of Washington) — J. Brunzell, J. Hokanson, S. Marcovina, J. Purnell, S. Sibley, S. Deeb, K. Edwards; Neuropsychological Testers — C. Manning, A. Ruocco, J. Tinker, J.L. Thomas, M. Shikhman, L. Krishnamurthy, S. Scott, T. Baines, B. Wong, O. Nakash, M. Baity, S. Kotyk-Sellnow, S. Swenson, D. Devine, T. Meyer, M. Khanna, S. Dreyer, D. Randal, S. Scott, J. Kramer, J. Bayless, C. Kraft, S. Welnel, S. Swisher, M. Pramuka, L. Murphy, C. Hunter, J. Shaw, E. Montague, J. Scully, M. Roger, C. Gouchie, B. Folley, D. Perantie, S. Warren, C. Dorr, D. Cuglietto, O. Stone, C. Boyd, R. Spencer, S. Kernen, K. Wells, T. Yanover, A. Spencer; Editor, EDIC Publications — D.M. Nathan.

Footnotes

*

Participants in the DCCT/EDIC Study Research Group are listed in the Appendix.

REFERENCES

  • 1.The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329:977–986. doi: 10.1056/NEJM199309303291401. [DOI] [PubMed] [Google Scholar]
  • 2.Nathan DM, Lachin J, Cleary P, et al. Intensive diabetes therapy and carotid intima-media thickness in type 1 diabetes mellitus. N Engl J Med. 2003;348:2294–2303. doi: 10.1056/NEJMoa022314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.The Diabetes Control and Complications Trial/Epidemiology of Diabetes Interventions and Complications Research Group. Retinopathy and nephropathy in patients with type 1 diabetes four years after a trial of intensive therapy. N Engl J Med. 2000;342:381–389. doi: 10.1056/NEJM200002103420603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Nathan DM, Cleary PA, Backlund JY, et al. Intensive diabetes treatment and cardiovascular disease in patients with type 1 diabetes. N Engl J Med. 2005;353:2643–2653. doi: 10.1056/NEJMoa052187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.The Diabetes Control and Complications Trial Research Group. Effects of intensive diabetes therapy on neuropsychological function in adults in the Diabetes Control and Complications Trial. Ann Intern Med. 1996;124:379–388. doi: 10.7326/0003-4819-124-4-199602150-00001. [DOI] [PubMed] [Google Scholar]
  • 6.Musen G, Lyoo IK, Sparks CR, et al. Effects of type 1 diabetes on gray matter density as measured by voxel-based morphometry. Diabetes. 2006;55:326–333. doi: 10.2337/diabetes.55.02.06.db05-0520. [DOI] [PubMed] [Google Scholar]
  • 7.Brands AMA, Kessels RPC, Hoogma RP, et al. Cognitive performance, psychological well-being, and brain magnetic resonance imaging in older patients with type 1 diabetes. Diabetes. 2006;55:1800–1806. doi: 10.2337/db05-1226. [DOI] [PubMed] [Google Scholar]
  • 8.Reichard P, Pihl M. Mortality and treatment side-effects during long-term intensified conventional insulin treatment in the Stockholm Diabetes Intervention Study. Diabetes. 1994;43:313–317. doi: 10.2337/diab.43.2.313. [DOI] [PubMed] [Google Scholar]
  • 9.The Diabetes Control and Complications Trial Research Group. Hypoglycemia in the Diabetes Control and Complications Trial. Diabetes. 1997;46:271–286. [PubMed] [Google Scholar]
  • 10.Austin EJ, Deary IJ. Effects of repeated hypoglycemia on cognitive function: a psychometrically validated reanalysis of the Diabetes Control and Complications Trial data. Diabetes Care. 1999;22:1273–1277. doi: 10.2337/diacare.22.8.1273. [DOI] [PubMed] [Google Scholar]
  • 11.Epidemiology of Diabetes Interventions and Complications (EDIC) Study Group. Design, implementation, and preliminary results of a long-term follow-up of the Diabetes Control and Complications Trial cohort. Diabetes Care. 1999;22:99–111. doi: 10.2337/diacare.22.1.99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Wechsler D. Manual for the Wechsler Adult Intelligence Scale. New York: Psychological Corporation; 1955. [Google Scholar]
  • 13.Reitan RM, Wolfson D. The Halstead-Reitan Neuropsychological Test Battery: theory and clinical interpretation. Tucson, AZ: Neuropsychology Press; 1993. [Google Scholar]
  • 14.Wechsler D. A standardized memory scale for clinical use. J Psychol. 1945;19:87–95. [Google Scholar]
  • 15.Lewis RF. Digit Vigilance Test. Lutz, FL: Psychological Assessment Resources; 1990. [Google Scholar]
  • 16.Ruff RM, Parker SB. Gender- and age-specific changes in motor speed and eye-hand coordination in adults: normative values for the Finger Tapping and Grooved Pegboard Tests. Percept Mot Skills. 1993;76:1219–1230. doi: 10.2466/pms.1993.76.3c.1219. [DOI] [PubMed] [Google Scholar]
  • 17.Mitrushina MN, Boone KB, D’Elia LF. Handbook of normative data for neuropsychological assessment. New York: Oxford University Press; 1999. [Google Scholar]
  • 18.Morrow LA, Ryan CM. Normative data for a working memory test: the Four Word Short-Term Memory Test. Clin Neuropsychol. 2002;16:373–380. doi: 10.1076/clin.16.3.373.13850. [DOI] [PubMed] [Google Scholar]
  • 19.Ryan CM, Morrow LA, Bromet EJ, Parkinson DK. Assessment of neuropsychological dysfunction in the workplace: normative data from the Pittsburgh Occupational Exposures Test Battery. J Clin Exp Neuropsychol. 1987;9:665–679. doi: 10.1080/01688638708405209. [DOI] [PubMed] [Google Scholar]
  • 20.Lezak MD. Neuropsychological assessment. 3rd ed. New York: Oxford University Press; 1995. p. 1026. [Google Scholar]
  • 21.Steffes M, Cleary P, Goldstein D, et al. Hemoglobin A1C measurements over nearly two decades: sustaining comparable values throughout the Diabetes Control and Complications Trial and the Epidemiology of Diabetes Interventions and Complications Study. Clin Chem. 2005;51:753–758. doi: 10.1373/clinchem.2004.042143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Diabetes Control and Complications Trial Research Group. Influence of intensive diabetes treatment on quality-of-life outcomes in the Diabetes Control and Complications Trial. Diabetes Care. 1996;19:195–203. doi: 10.2337/diacare.19.3.195. [DOI] [PubMed] [Google Scholar]
  • 23.Derogatis LR. SCL-90-R manual: administration, scoring and procedures. Towson, MD: Clinical Psychometric Research; 1983. [Google Scholar]
  • 24.Snedecor G, Cochran WG. Statistical methods. 8th ed. Ames: Iowa State University Press; 1989. [Google Scholar]
  • 25.Ryan CM. Diabetes, aging, and cognitive decline. Neurobiol Aging. 2005;26(Suppl 1):21–25. doi: 10.1016/j.neurobiolaging.2005.09.006. [DOI] [PubMed] [Google Scholar]
  • 26.Auer RN. Hypoglycemic brain damage. Metab Brain Dis. 2004;19:169–175. doi: 10.1023/b:mebr.0000043967.78763.5b. [DOI] [PubMed] [Google Scholar]
  • 27.Cox DJ, Gonder-Frederick LA, Kovatchev BP, Julian DM, Clarke WL. Progressive hypoglycemia’s impact on driving simulation performance: occurrence, awareness and correction. Diabetes Care. 2000;23:163–170. doi: 10.2337/diacare.23.2.163. [DOI] [PubMed] [Google Scholar]
  • 28.Cox DJ, Penberthy JK, Zrebiec JF, et al. Diabetes and driving mishaps: frequency and correlations from a multinational survey. Diabetes Care. 2003;26:2329–2334. doi: 10.2337/diacare.26.8.2329. [DOI] [PubMed] [Google Scholar]
  • 29.Deary IJ, Crawford JR, Hepburn DA, Langan SJ, Blackmore LM, Frier BM. Severe hypoglycemia and intelligence in adult patients with insulin-treated diabetes. Diabetes. 1993;42:341–344. doi: 10.2337/diab.42.2.341. [DOI] [PubMed] [Google Scholar]
  • 30.Gold AE, Deary IJ, Jones RW, O’Hare JP, Reckless JPD, Frier BM. Severe deterioration in cognitive function and personality in five patients with long-standing diabetes: a complication of diabetes or a consequence of treatment? Diabet Med. 1994;11:499–505. doi: 10.1111/j.1464-5491.1994.tb00314.x. [DOI] [PubMed] [Google Scholar]
  • 31.Perros P, Deary IJ, Sellar RJ, Best JJ, Frier BM. Brain abnormalities demonstrated by magnetic resonance imaging in adult IDDM patients with and without a history of recurrent severe hypoglycemia. Diabetes Care. 1997;20:1013–1018. doi: 10.2337/diacare.20.6.1013. [DOI] [PubMed] [Google Scholar]
  • 32.Brands AMA, Biessels G-J, De Haan EH, Kappelle LJ, Kessels RP. The effects of type 1 diabetes on cognitive performance: a meta-analysis. Diabetes Care. 2005;28:726–735. doi: 10.2337/diacare.28.3.726. [DOI] [PubMed] [Google Scholar]
  • 33.Wysocki T, Harris MA, Mauras N, et al. Absence of adverse effects of severe hypoglycemia on cognitive function in school-aged children with diabetes over 18 months. Diabetes Care. 2003;26:1100–1105. doi: 10.2337/diacare.26.4.1100. [DOI] [PubMed] [Google Scholar]
  • 34.Kramer L, Fasching P, Madl C, et al. Previous episodes of hypoglycemic coma are not associated with permanent cognitive brain dysfunction in IDDM patients on intensive insulin treatment. Diabetes. 1998;47:1909–1914. doi: 10.2337/diabetes.47.12.1909. [DOI] [PubMed] [Google Scholar]
  • 35.Ferguson SC, Blane A, Perros P, et al. Cognitive ability and brain structure in type 1 diabetes: relation to microangiopa-thy and preceding severe hypoglycemia. Diabetes. 2003;52:149–156. doi: 10.2337/diabetes.52.1.149. [DOI] [PubMed] [Google Scholar]
  • 36.Ryan CM, Geckle MO, Orchard TJ. Cognitive efficiency declines over time in adults with Type 1 diabetes: effects of micro- and macrovascular complications. Dia-betologia. 2003;46:940–948. doi: 10.1007/s00125-003-1128-2. [DOI] [PubMed] [Google Scholar]
  • 37.Wessels AM, Simsek S, Remijnse PL, et al. Voxel-based morphometry demonstrates reduced gray matter density on brain MRI in patients with diabetic retinopathy. Diabetologia. 2006;49:2474–2480. doi: 10.1007/s00125-006-0283-7. [DOI] [PubMed] [Google Scholar]
  • 38.Wessels AM, Rombouts SA, Simsek S, et al. Microvascular disease in type 1 diabetes alters brain activation: a functional magnetic resonance imaging study. Diabetes. 2006;55:334–340. doi: 10.2337/diabetes.55.02.06.db05-0680. [DOI] [PubMed] [Google Scholar]
  • 39.Patton N, Aslam T, MacGillivray T, Pattie A, Deary IJ, Dhillon B. Retinal vascular image analysis as a potential screening tool for cerebrovascular disease: a rationale based on homology between cerebral and retinal microvasculatures. J Anat. 2005;206:319–348. doi: 10.1111/j.1469-7580.2005.00395.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Ryan CM. Diabetes and brain damage: more (or less) than meets the eye? Diabetologia. 2006;49:2229–2233. doi: 10.1007/s00125-006-0392-3. [DOI] [PubMed] [Google Scholar]
  • 41.Rabbitt P, Donlan C, Watson P, Mc-Innes L, Bent N. Unique and interactive effects of depression, age, socioeconomic advantage, and gender on cognitive performance of normal healthy older people. Psychol Aging. 1995;10:307–313. doi: 10.1037//0882-7974.10.3.307. [DOI] [PubMed] [Google Scholar]

RESOURCES