Abstract
Objective:
Autonomic symptoms may occur frequently in diabetic and other neuropathies. There is a need to develop a simple instrument to measure autonomic symptoms in subjects with neuropathy and to test the validity of the instrument.
Methods:
The Survey of Autonomic Symptoms (SAS) consists of 11 items in women and 12 in men. Each item is rated by an impact score ranging from 1 (least severe) to 5 (most severe). The SAS was tested in observational studies and compared to a previously validated autonomic scale, the Autonomic Symptom Profile (ASP), and to a series of autonomic tests.
Results:
The SAS was tested in 30 healthy controls and 62 subjects with neuropathy and impaired glucose tolerance or newly diagnosed diabetes. An increased SAS score was associated with the previously validated ASP (rank order correlation = 0.68; p < 0.0001) and with quantitative measures of autonomic function: a reduced quantitative sudomotor axon reflex test sweat volume (0.31; p < 0.05) and an abnormal 30:15 ratio (0.53; p < 0.01). The SAS shows a high sensitivity and specificity (area under the receiver operating characteristic curve 0.828) that compares favorably with the ASP. The SAS scale domains had a good internal consistency and reliability (Cronbach α = 0.76). The SAS symptom score was increased in neuropathy (95% confidence interval [CI] 2.99–4.14) compared to control (95% CI 0.58–1.69; p < 0.0001) subjects.
Conclusions:
The SAS is a new, valid, easily administered instrument to measure autonomic symptoms in early diabetic neuropathy and would be of value in assessing neuropathic autonomic symptoms in clinical trials and epidemiologic studies.
Impaired glucose regulation (IGR) is associated with peripheral neuropathy in at least 40% of cases.1,2 The neuropathy associated with IGR and early diabetes is a small-fiber neuropathy that is often accompanied by mild autonomic symptoms and abnormalities.3–8 A recent consensus statement by the American Diabetes Association recognizes that glycemic burden is a strong predictor of adverse outcomes and that IGR represents a continuum of risk.9 Thus, for the purposes of this study, subjects with neuropathy and prediabetes or newly diagnosed diabetes are described as having IGR or early diabetic neuropathy.
There are few validated scores of autonomic symptoms and even fewer that have been evaluated in subjects with neuropathy. One validated scale is the Autonomic Symptom Profile (ASP). The Composite Autonomic Symptom Scale (COMPASS) was developed to provide an aggregate score of the autonomic symptoms in the ASP with weighting according to clinical relevance. When the ASP was used to assess subjects with neurogenic autonomic failure and control patients, the COMPASS scores correlated well with the Composite Autonomic Scoring Scale (CASS) obtained from autonomic testing.10 However, even though the correlation between autonomic symptoms and the CASS was better in type 1 than it was in type 2 diabetes, the correlation was weak overall.3 This observation was ascribed to mild autonomic symptoms in diabetic patients and thus a reduced strength of the association. Furthermore, this highlights the need for a sensitive, brief, and easy-to-use questionnaire for autonomic neuropathy symptoms that can be used in both research studies and clinical practice.
METHODS
Standard protocol approvals, registrations, and patient consents.
All subjects with neuropathy and normal subjects were consented according to the ethical standards committees on human experimentation (Michigan, Maryland, and Utah) and written informed consent was obtained on all participating subjects. Some of the subjects are participants in ClinicalTrials.gov NCT00780559.
Questionnaire.
The questionnaire was designed to provide a succinct evaluation of autonomic symptoms in subjects with mild neuropathy. Questions were evaluated in subjects with early neuropathy and only symptoms that were found to be reproducibly present in these subjects were included. The Survey of Autonomic Symptoms (SAS) Scale assesses both the presence of symptoms and the degree of severity. The scale is intended to 1) provide assessment of the type of mild autonomic symptoms observed in early diabetic neuropathy and in autonomic neuropathies, 2) ensure that questions are unambiguous and can be easily understood by patients and research subjects, and 3) serve as an instrument that can be used in clinical trials, clinical practice, and large epidemiologic studies. The questions were developed from questions used routinely in practice in patients with neuropathy and then individual elements of the scale were evaluated for their reliability and sensitivity. From these questions, 11 (women) or 12 (men) questions were most frequently positive in subjects with early diabetic neuropathy. These questions assess the following autonomic symptom domains: orthostatic, sudomotor symptoms, vasomotor, gastrointestinal, urinary, and sexual dysfunction (table 1). The questions were designed to minimize ambiguity and required a yes or no response to symptoms occurring in the 6-month period prior to administration. The subject was then asked to indicate the degree of severity of the symptom, with 1 being the least severe and 5 the most severe, to determine the total symptom impact score (TIS).
Table 1.
Number of symptoms reported: —— (sum of column A, 0–12 for men and 0–11 for women); total symptom impact score: —— (sum of column B, 0–60 for men and 0–55 for women).
Study design.
Data in this study were obtained from subjects enrolled in the Impaired Glucose Tolerance Causes Neuropathy Study,2,4 the Improving Neuropathy and Mobility in Early Diabetes study, and the University of Maryland Neuromuscular and Department of Neurology Database. All subjects with polyneuropathy had IGR. IGR includes early type 2 diabetes mellitus (within 2 years of diagnosis), impaired glucose tolerance (IGT), and impaired fasting glucose (IFG) based on standardized ADA criteria.11 Subjects with IGR were evaluated with nerve conduction studies (NCS), other electrophysiologic tests such as quantitative sensory testing (QST) (vibration detection threshold and cold detection threshold), the quantitative sudomotor axon reflex test (QSART), and cardiac autonomic neuropathy (CAN) testing. Subjects also had skin biopsies performed at the calf and thigh and the intraepidermal nerve fiber density (IENFD) was measured. The criteria for inclusion within the study were IGR confirmed on at least 2 separate occasions, signs and symptoms of peripheral neuropathy, and an abnormality in at least one of the following: NCS, QST, or QSART, or IENFD.2,4 Subjects were excluded from the study if other causes of neuropathy existed as previously described.2
Normal subjects were recruited as part of the University of Maryland Neuromuscular or Neurology Database. All normal subjects were examined by one of the authors (J.W.R. or L.Z.) and their medical records were carefully reviewed to exclude subjects with neurologic or autonomic disorders or those taking any medications which may induce autonomic changes. Both normal and IGR neuropathy subjects also completed the ASP.3,10 The ASP consists of 73 questions assessing the following 9 domains of autonomic symptoms: orthostatic (9 items); secretomotor, including sudomotor symptoms (8 items); male sexual dysfunction (8 items); urinary (3 items); gastrointestinal, including gastroparesis, diarrhea, and constipation (14 items); pupillomotor, including visual symptoms (7 items); vasomotor (11 items); reflex syncope (5 items); and sleep function (8 items) as previously described.10 This is scored to provide the COMPASS. The COMPASS is based on key scorable areas of the autonomic nervous system based on presence, severity, distribution, frequency, and progression of symptoms.3,10
Autonomic testing.
When medically permissible, subjects with neuropathy were asked to discontinue any medications that could alter the results of their autonomic tests for 24 to 48 hours before testing and to refrain from consuming caffeine during this time. None of the control subjects were taking medications known to influence the results of autonomic tests. The QSART was purchased from WR Electronics (Stillwater, MN) and performed as previously described.4
CAN testing (WR Electronics) was used to assess the following. 1) Heart rate variability to deep breathing. The ratio of the heart rate response during expiration and inspiration, the expiration:inspiration (E:I) ratio, and heart rate range (HRR) were measured. 2) The Valsalva ratio and the beat-to-beat blood pressure measurements during the Valsalva maneuver. The subject was asked to maintain an expiration pressure of 40 mm Hg for 15 seconds. (3) Beat-to-beat blood pressure change during a 10-minute 70-degree head-up tilt compared to the resting supine blood pressure. (4) The 30:15 ratio. All these tests and their normative values have been previously described.12–15
Construct validity.
The SAS was compared with the ASP/COMPASS and with the CASS to assess the degree of criterion validity. The CASS consists of a comprehensive battery of autonomic tests that has previously been shown to be quantitative, sensitive, specific, reproducible, and standardized.16,17 The SAS was then compared to measures of autonomic function including the QSART sweat response, 30:15 ratio, E:I ratio, HRR, and tilt response.
Statistical design.
Analysis was performed using SPSS version 18. Pearson correlation coefficients were used to examine pairwise correlation between normally distributed variables. Spearman rank order correlations were used for data analysis for the ASP/COMPASS score and subscores and the CASS score and subscores because these scores are not normally distributed10 and a valid transformation of the data were not possible. Receiver operating characteristic (ROC) curves were calculated and compared as previously described.18 Internal consistency for the construct items was determined using Cronbach α. Statistical significance was defined as a 2-tailed p value <0.05, and data are presented as mean ± SEM.
RESULTS
General clinical features of the subjects.
A total of 93 subjects completed the SAS and the COMPASS. There were 38 women (mean age 59.37 ± 1.34 years) and 25 men (mean age 59.20 ± 1.63 years) with neuropathy. Of these, 94% had IGT or IFG, and 6% had early diabetes. In the control group, there were 18 women (mean age 56.94 ± 2.86) and 12 men (mean age 49.25 ± 1.78 years). Mean ages were not different between men and women with neuropathy and control women, but the mean age of control men was less than control women (p < 0.01). Despite the lower mean male control age, there was no difference between gender or age for the SAS symptom score or TIS in either controls or neuropathy subjects (figure 1). However, there was a difference between groups for both the SAS symptom score (p < 0.0001; 95% confidence interval [CI] control: 0.58–1.69; neuropathy: 2.99–4.14) and the TIS (p < 0.0001; 95% CI control: 1.51–5.0207; neuropathy: 8.11–11.92) between control and neuropathy subjects (figure 1). There was also no association between the SAS symptom score or TIS and the weight, height, or body mass index of the subject. There was a strong association between the baseline IENFD and the SAS (p < 0.001; table e-1 on the Neurology® Web site at www.neurology.org).
Validation of SAS domains with other measures of autonomic function.
The SAS showed a strong association with the ASP total score for all domains, and the secretomotor, vasomotor, and orthostatic intolerance ASP domains (table 2). There was a weaker association between the SAS and the ASP bladder dysfunction and diarrhea domains. There was no association between the SAS and the ASP erectile dysfunction, gastric paresis, and other domains. Importantly, an increased SAS symptom score or TIS was associated with a reduced forearm or foot sweat volume on the QSART (table e-2) and also with a reduced 30:15 ratio. However, there was no association between the SAS symptom or TIS and the E:I ratio, HRR, Valsalva ratio (table 2), or an abnormal tilt table response (symptom score: odds ratio [OR] 1.29, 95% CI 0.87–1.91, p = 0.19; TIS: OR 1.08, 95% CI 0.96–1.21, p = 0.19). The CASS total score or any of the CASS subscores (adrenergic, cardiogenic, or sudomotor) were not associated with the SAS using a nonparametric Spearman rank order analysis. Furthermore, when the CASS was expressed as a dichotomous variable and a logistic regression was performed, there was still no association between the CASS sudomotor score and the SAS symptom score (OR 1.19, 95% CI 0.95–1.50, p = 0.13) or the SAS TIS (OR 1.07, 95% CI 0.99–1.16, p = 0.08). There was also no association between the CASS total score and the SAS symptom score (OR 1.12, 95% CI 0.891–1.40, p = 0.33) and SAS TIS (OR 1.05, 95% CI 0.97–1.13, p = 0.20).
Table 2.
Abbreviations: ASP = Autonomic Symptom Profile; CAN = cardiac autonomic neuropathy; CASS = Composite Autonomic Scoring Scale; E:I = expiration:inspiration ratio; HR = heart rate; NS = not significant; QSART = quantitative sudomotor axon reflex test; SAS = Survey of Autonomic Symptoms.
ROC for the SAS and ASP scores.
In assessing autonomic symptoms in subjects with early diabetic neuropathy, the ROC sensitivity/specificity analysis indicated that the SAS symptom score showed a slightly greater sensitivity and specificity throughout its dynamic range than the ASP score (figure 2). The area under the curve (AUC) was as follows: SAS symptom score −0.828 (SEM 0.047, 95% CI 0.737–0.920) compared to the ASP score −0.812 (SEM 0.057, 95% CI 0.700–0.925). Based on the SAS symptom score ROC curve, a cutpoint of greater than zero would provide 95% sensitivity and 50% specificity and a cutpoint of greater than 3 would provide greater than 90% specificity and greater than 65% sensitivity in determining disease. Based on the SAS TIS ROC curve, a cutpoint of greater than 1 would provide greater than 90% sensitivity and greater than 50% specificity and a cutpoint greater than 7 would provide greater than 90% specificity and greater than 60% sensitivity. Using previously described methods,18 there was no difference between the SAS symptom score and ASP ROC curves (p = 0.682).
Internal consistency of internal reliability and frequency of the SAS domains.
Internal consistency reliability testing using Cronbach α provided a value of 0.76, indicating that the scale domains are measuring the same overall construct. All items in the SAS showed a high degree of interitem correlation. However, of all the domains, item 10—“Do you have persistent constipation?”—added least to the overall reliability of the SAS. For subjects with neuropathy, the 3 items showing the greatest reliability were in order: 1) Item 5, “Is sweating in your feet decreased compared to the rest of your body?” 2) Item 4, “Are your feet colder than the rest of your body?” 3) Item 11, “Do you have leaking of urine?” The items addressing gastrointestinal function showed the lowest construct reliability in subjects with neuropathy. The most common symptoms reported by subjects with neuropathy are indicated in table 3. The most common items—“Do you have a dry mouth or dry eyes?” and “Are your feet colder than the rest of your body?”—have similar frequencies in both men and women.
Table 3.
Abbreviations: NA = not available; TIS = total symptom impact score.
DISCUSSION
The SAS was designed to assess autonomic symptoms in subjects with neuropathy and was found to have a slightly greater sensitivity and specificity in subjects with early diabetic neuropathy than the ASP. The questions in the SAS were designed to improve subject understanding and to reduce uncertainty in the responses. Currently available autonomic questionnaires are lengthy, complex, and take training and considerable time to score. In contrast, the SAS can be rapidly completed and scored. Furthermore, interpretation of the SAS is not dependent on age, gender, body mass index, and other factors. This provides for a flexible and more universally acceptable scale to assess autonomic function in subjects with neuropathy. However, further validation of the SAS is needed in blinded and longitudinal clinical studies.
There was a difference in both the SAS symptom score and the TIS between control and neuropathy subjects. This indicates that the SAS has power to distinguish between control subjects and those with peripheral neuropathy. The SAS symptom score and TIS are not affected by age, gender, body mass index, weight, or height. This indicates that the SAS would perform well across subject groups in a clinical trial or epidemiologic study and would be less likely to be affected by common confounding variables. The SAS demonstrated a strong association with the ASP total score and for the ASP domains of secretomotor, vasomotor, and orthostatic intolerance. This is consistent with the observation that symptoms that tested these 3 autonomic domains were most commonly reported by subjects with neuropathy and also showed the greatest internal consistency. Furthermore, both the SAS symptom score and TIS showed no association between the SAS and the ASP domains of gastrointestinal dysfunction and gastrointestinal symptoms showed the lowest internal consistency. These results may be explained by the fact that this study examined subjects with IGR and early diabetic neuropathy. Although patients with type 2 diabetes commonly have autonomic symptoms, the symptoms are usually mild and in one study the syncope and gastrointestinal symptom domains on the ASP did not differ from control subjects.3
The SAS was further validated by an association between an increased SAS symptom score or TIS and a reduced forearm or foot sweat volume on QSART or a reduced 30:15 ratio. However, there was no association seen between the SAS symptom score or TIS and the E:I ratio, HRR, Valsalva ratio, abnormal tilt table response, CASS total score, or CASS subscores. These findings are consistent with a previous study that found only a weak association between autonomic symptom scores on the ASP and autonomic deficits on the CASS in patients with diabetes.3 Only a few subjects were taking a medication with significant anticholinergic properties that may reduce sweating, and these were discontinued for at least 24 hours prior to performing the QSART.19 Thus, these medications would not significantly affect the results in this study. These previous studies highlight the need to examine autonomic symptoms independently of autonomic deficits. The present study raises the possibility that early diabetic neuropathy is associated with a mild autonomic neuropathy that current autonomic tests are not sensitive enough to detect or that autonomic symptom scores overrate for the presence of autonomic neuropathy.
Compared to controls, subjects with IGR have been shown to have greater abnormalities in most cardiovascular reflex tests and greater heart rate variability characterized by the triangle index.8 In contrast, another study examined patients with newly diagnosed IGT with a battery of autonomic tests including heart rate variation variability, heart rate response to deep breathing, heart rate response to Valsalva maneuver, blood pressure response to standing up quickly, and skin sympathetic skin response (SSR) that evaluates postganglionic sympathetic sudomotor function but is less precise than the QSART.7 They found no difference compared to controls in measures of CAN. However, they did find lower amplitudes of the SSR in the IGT group compared to healthy controls that is consistent with the presence of a sudomotor autonomic neuropathy. The importance of abnormal sudomotor responses in subjects with IGR was also confirmed in other studies.2,6 This finding indicates that sudomotor fibers tend to be affected earlier in autonomic neuropathy in patients with IGR and that CAN may develop at a later stage or may require more sensitive tests to detect it than tests commonly used in the clinic.
In future studies the SAS could be adjusted to exclude the questions that individually were less reliable; however, excluding these questions would not affect the overall reliability of the test. A potential weakness of the study is the difference in the mean age of control men vs control women and individuals with neuropathy. This may be because it can be difficult to find age-matched male controls who have a normal neurologic examination, have no evidence of peripheral autonomic neuropathy, and are not on any medications that may even mildly affect autonomic functioning. However, despite the younger age of the control men, this had no affect on the validity of the study because there was no difference between age for the SAS symptom score or TIS in either control or neuropathy subjects. Future studies should address performance of the SAS in larger more diverse populations of subjects and in groups of subjects with other types of neuropathy. Despite these caveats, a validated questionnaire such as the SAS that is sensitive enough to detect mild autonomic neuropathy, is simple to complete, and performs consistently in subjects could potentially aid in the early detection and diagnosis of diabetic autonomic neuropathy.
Supplementary Material
ACKNOWLEDGMENT
The authors thank Dr. Phillip Low, Mayo Clinic, Rochester, MN, for providing the Autonomic Symptom Profile (ASP) and the Composite Autonomic Symptom Scale (COMPASS), Kristina Janevski, and Susan Nalepa for assistance with data collection.
- ASP
- Autonomic Symptom Profile
- AUC
- area under the curve
- CAN
- cardiac autonomic neuropathy
- CASS
- Composite Autonomic Scoring Scale
- CI
- confidence interval
- COMPASS
- Composite Autonomic Symptom Scale
- E:I
- expiration:inspiration ratio
- HRR
- heart rate range
- IENFD
- intraepidermal nerve fiber density
- IFG
- impaired fasting glucose
- IGR
- impaired glucose regulation
- IGT
- impaired glucose tolerance
- NCS
- nerve conduction studies
- OR
- odds ratio
- QSART
- quantitative sudomotor axon reflex test
- QST
- quantitative sensory testing
- ROC
- receiver operating characteristic
- SAS
- Survey of Autonomic Symptoms
- SSR
- sympathetic skin response
- TIS
- total symptom impact score
Supplemental data at www.neurology.org
DISCLOSURE
Dr. Zilliox receives research support from the NIH. Dr. Peltier receives research support from the NIH/NINDS. Dr. Wren receives research support from Genentech, Inc., Oakland University-William Beaumont Hospital Multidisciplinary Research Award, and the Crohn's and Colitis Foundation of America. A. Anderson receives research support from the Office of Research Development (Medical Research Service and Rehabilitation Service), Department of Veterans Affairs. Dr. Smith serves on a scientific advisory board for Baxter International Inc.; serves as an Associate Editor for Education, AAN.com; serves as a consultant for Allergan, Inc., Merz Pharmaceuticals, LLC, and NeurogesX; serves on the speakers' bureau for Allergan, Inc.; and receives research support from the NIH and the American Diabetes Association. Dr. Singleton receives research support from the NIH and the American Diabetes Association. Dr. Feldman serves on a scientific advisory board for Novartis; serves on the editorial boards of Annals of Neurology and the Journal of the Peripheral Nervous System; receives royalties from UpToDate, Inc.; and receives research support from the NIH, the American Diabetes Association, the Juvenile Diabetes Research Foundation, and the Diabetes Research Foundation. Dr. Alexander serves as an Associate Editor for the Journal of Gerontology Med Sci; receives research support from the NIH/NIA, the Office of Research and Development, Medical Service and Rehabilitation Research and Development Service of the Department of Veterans Affairs, and the Dorothy and Herman Miller Fund for Mobility Research in Older Adults. Dr. Russell has a patent pending re: The SAS survey (University of Maryland, Veterans Administration); and receives research support from Baxter International Inc., the NIH, the Juvenile Diabetes Research Foundation, the Office of Research Development (Medical Research Service and Rehabilitation Service), Department of Veterans Affairs, and the American Diabetes Association.
REFERENCES
- 1. Sumner CJ, Sheth S, Griffin JW, Cornblath DR, Polydefkis M. The spectrum of neuropathy in diabetes and impaired glucose tolerance. Neurology 2003;60:108–111 [DOI] [PubMed] [Google Scholar]
- 2. Smith AG, Russell JW, Feldman EL, et al. Lifestyle intervention for prediabetic neuropathy. Diabetes Care 2006;29:1294–1299 [DOI] [PubMed] [Google Scholar]
- 3. Low PA, Benrud-Larson LM, Sletten DM, et al. Autonomic symptoms and diabetic neuropathy: a population-based study. Diabetes Care 2004;27:2942–2947 [DOI] [PubMed] [Google Scholar]
- 4. Peltier A, Smith AG, Russell JW, et al. Reliability of quantitative sudomotor axon reflex testing and quantitative sensory testing in neuropathy of impaired glucose regulation. Muscle Nerve 2009;39:529–535 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Singleton JR, Bixby B, Russell JW, et al. The Utah Early Neuropathy Scale: a sensitive clinical scale for early sensory predominant neuropathy. J Peripher Nerv Syst 2008;13:218–227 [DOI] [PubMed] [Google Scholar]
- 6. Grandinetti A, Chow DC, Sletten DM, et al. Impaired glucose tolerance is associated with postganglionic sudomotor impairment. Clin Auton Res 2007;17:231–233 [DOI] [PubMed] [Google Scholar]
- 7. Isak B, Oflazoglu B, Tanridag T, Yitmen I, Us O. Evaluation of peripheral and autonomic neuropathy among patients with newly diagnosed impaired glucose tolerance. Diabetes Metab Res Rev 2008;24:563–569 [DOI] [PubMed] [Google Scholar]
- 8. Putz Z, Tabak AG, Toth N, et al. Noninvasive evaluation of neural impairment in subjects with impaired glucose tolerance. Diabetes Care 2009;32:181–183 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Executive summary: standards of medical care in diabetes: 2010. Diabetes Care 2010;33(suppl 1):S4–S10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Suarez GA, Opfer-Gehrking TL, Offord KP, Atkinson EJ, O'Brien PC, Low PA. The Autonomic Symptom Profile: a new instrument to assess autonomic symptoms. Neurology 1999;52:523–528 [DOI] [PubMed] [Google Scholar]
- 11. Genuth S, Alberti KG, Bennett P, et al. Follow-up report on the diagnosis of diabetes mellitus. Diabetes Care 2003;26:3160–3167 [DOI] [PubMed] [Google Scholar]
- 12. Mari A, Pacini G, Murphy E, Ludvik B, Nolan JJ. A model-based method for assessing insulin sensitivity from the oral glucose tolerance test. Diabetes Care 2001;24:539–548 [DOI] [PubMed] [Google Scholar]
- 13. Denq JC, O'Brien PC, Low PA. Normative data on phases of the Valsalva maneuver. J Clin Neurophysiol 1998;15:535–540 [DOI] [PubMed] [Google Scholar]
- 14. Clinical Autonomic Disorders, 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2008 [Google Scholar]
- 15. Zilliox L, Russell JW. Acute autonomic neuropathies. In: Gilman S, ed. Neurology Medlink: The Information Resource for Clinical Neurology. San Diego, CA: Arbor Publishing; 2009 [Google Scholar]
- 16. Low PA. Composite Autonomic Scoring Scale for laboratory quantification of generalized autonomic failure. Mayo Clin Proc 1993;68:748–752 [DOI] [PubMed] [Google Scholar]
- 17. Low PA, Benrud-Larson LM, Sletten DM, et al. Autonomic symptoms and diabetic neuropathy: a population-based study. Diabetes Care 2004;27:2942–2947 [DOI] [PubMed] [Google Scholar]
- 18. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–845 [PubMed] [Google Scholar]
- 19. Pop-Busui R, Herman WH, Feldman EL, et al. DCCT and EDIC studies in type 1 diabetes: lessons for diabetic neuropathy regarding metabolic memory and natural history. Curr Diab Rep Epub 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.