Highly cited publications in pediatric neurosurgery

Abstract

Object

The number of citations a publication receives can be used as a surrogate for the impact that article has made on its discipline. This study identifies and characterizes the most cited articles in pediatric neurosurgical journals as of April 2013.

Methods

We examined four clinical pediatric neurosurgery journals. The 100 most cited articles in the overall literature and the top 50 articles from 2002 to 2012 were examined. The following information was recorded for each article: number of authors, country of origin, citation-count adjusted for number of years in print, topic, and level of evidence.

Results

The 100 most cited articles appeared in three of the four journals: Child's Brain, Pediatric Neurosurgery and Child's Nervous System. Publication dates ranged from 1975 to 2006; 21 were prospective studies, 64 were retrospective, and 81 were either class 4 evidence (case series, n=70) or review articles (n=11). Citations ranged from 65 to 193 (mean of 90); average adjusted citation count per year was 4.5. The 50 most cited articles from 2002 to 2012 appeared in Child's Nervous System, Pediatric Neurosurgery, and JNS: Pediatrics. Four were prospective studies, 25 were retrospective, and 38 of the total (76 %) were either class 4 evidence (n=24) or review articles (n=14). Citations ranged from 41 to 125 (mean of 54); average adjusted citation count per year was 6.3.

Conclusion

An original paper in pediatric neurosurgery having a total citation count of 50 or more, and an average citation count of 5 per year or more can be considered a high impact publication.

Introduction

The information age, while providing undeniable benefit to the medical field, has also created a unique set of difficulties for today's researcher. The meteoric rise in biomedical information and resources has often provided too much rather than too little information to answer our questions. PubMed has over 23 million citations, and it is predicted there are well over 50 million scholarly journal articles as of 2009 [1]. The doubling time for the scholarly journal articles is predicted to be 24 years; if this proves true, by the year 2033, there may be over 100 million scholarly journal articles in existence [1]. The study of bibliometrics provides a way to quantitatively analyze the immense volume of biomedical information that is currently available.

Citation analysis is defined as the construction and application of a series of indicators of the impact, influence, or quality of scholarly work, derived from citation data [2]. Citation analysis, first introduced many years ago by Eugene Garfield, can be used as a surrogate for the impact of an individual article or a journal [3, 4]. In principle, the number of citations an individual publication receives can be a quantitative measure of readership and, theoretically, the ability of that article to generate discussion, controversy, change, and further research. Recently, there have been several studies in academic medicine to establish landmark articles within various specialties using citation analysis. This has been done in anesthesiology [5], critical care [6], dermatology [7], emergency medicine [8], forensic science [9–11], ophthalmology [12], orthopedics [13], otolaryngology [14], pediatric orthopedic surgery [15], plastic surgery [16, 17], and urology [18, 19]. In 2010, Ponce et al. published the top 100 most cited works in neurosurgery and a list of articles that had 400 or more citations; so-called “citation classics” [20, 21]. There were, however, few articles in which children composed the majority of the patient population. Out of their top 100 list, only five could conceivably be classified under the topic of pediatric neurosurgery: (1) Gardner's seminal publication on the development of syringomyelia (1965) [22]; (2) the original pathologic description of dysembryoplastic neuroepithelial tumors by Daumas-Duport et al. (1988) [23]; (3) the randomized trial of chemotherapy for children with medulloblastoma by Evans et al. (1990) [24]; (4) the treatment of intracranial germ cell tumors by Jennings et al. (1985) [25]; and (5) the description of shaken-baby syndrome by Duhaime et al. (1987) [26]. The relative lack of representation of pediatric neurosurgery articles is due in part to the influence of adult neurosurgical topics that command much larger audiences, such as oncology, vascular, and trauma. In fact, Ponce et al. did not include a “pediatric” subcategory when classifying their top 100 most cited articles and the “citation classics,” but they did include the topic of “CSF” for the latter with a total of four articles. The purpose of this study is to locate and characterize the most highly cited contributions within pediatric neurosurgery journals.

Methods

Journal identification

A search was performed in April 2013 using Thomson Reuters Journal Citation Reports^® (JCR) (http://wokinfo.com/products_tools/analytical/jcr/) for the year 2011. The terms ‘pediatrics’, ‘neurosurgery’, ‘surgery’, ‘brain’, and ‘neurology’ were queried using the “Specific Journal Search” function within JCR. Journals that were related to pediatric neurosurgery were identified. Journals that were not directly related to clinical pediatric neurosurgery (i.e., related to basic science research) were excluded.

Citation analysis

A total of four journals were identified from the JCR search: (1) Child's Brain; (2) Child's Nervous System; (3) Journal of Neurosurgery: Pediatrics; and (4) Pediatric Neurosurgery. These journals were then compiled into a single search within Thomson Reuters Web of Science [27]. This search returned a list of all articles (n=9,217) present from these journals in Web of Science (WOS) records. These articles were then sorted in descending order based on citation count. The top 100 most cited articles were extracted from this list for analysis in April 2013. An additional search was performed in June 2013 using the same criteria to identify the top 50 most cited articles over the last 10 years (2002–2012). An ‘adjusted’ citation count or index was calculated for each article by dividing the total number of citations by the years since initial publication. This adjusted citation count can be viewed as the average number of citations that paper has received each year since it was published.

Data

The following information was obtained from each article: number of authors; institution; country of origin; type of research (retrospective or prospective); topic (hydrocephalus, oncology, vascular, trauma, functional/epilepsy/spasticity, congenital spine/tethered cord, congenital cranial/craniosynostosis/arachnoid cyst, and other); and level of evidence (Levels 1–5 using the Oxford Centre for Evidence-Based Medicine—i.e., OCEBM—2011 Levels of Evidence, Appendix 1, http://www.cebm.net/index.aspx?o=5653, and the accompanying Table of Evidence Glossary) [28]. The abstract or, when necessary, the entire article was read in detail to determine these characteristics and categories.

Results

The top 100 overall most cited works within pediatric neurosurgery are listed in Table 1. The top 50 most cited works within pediatric neurosurgery over the past decade (2002–2012) are presented in Table 2.

Table 1.

List of top 100 most cited journal articles in pediatric neurosurgical journals

Article	Rank	Citations	Adjusted rank	Adjusted citations	Class
Hoffman HJ, Hendrick EB, Humphreys RP (1976) The tethered spinal cord: its protean manifestations, diagnosis and surgical correction. Childs Brain 2: 145–155	1	193	29	5.22	4
Bowman RM, McLone DG, Grant JA, Tomita T, Ito JA (2001) Spina bifida outcome: a 25–year prospective. Pediatric Neurosurgery 34: 114–120	2	186	1	15.50	4
Taylor A, Butt W, Rosenfeld J, Shann F, Ditchfield M, Lewis E, Klug G, Wallace D, Henning R, Tibballs J (2001) A randomized trial of very early decompressive craniectomy in children with traumatic brain injury and sustained intracranial hypertension. Childs Nervous System 17: 154–162	3	182	2	15.17	2
Moss ML (1975) Functional anatomy of cranial synostosis. Childs Brain 1: 22–33	4	141	59	3.71	Review
Bruce DA, Raphaely RC, Goldberg AI, Zimmerman RA, Bilaniuk LT, Schut L, Kuhl DE (1979) Pathophysiology, treatment and outcome following severe head injury in children. Childs Brain 5: 174–191	5	138	51	4.06	4
Brockmeyer D, Abtin K, Carey L, Walker ML (1998) Endoscopic third ventriculostomy: an outcome analysis. Pediatric Neurosurgery 28: 236–240	6	134	5	8.93	4
Raimondi AJ, Hirschauer J (1984) Head injury in the infant and toddler—Coma scoring and outcome scale. Childs Brain 11: 12–35	7	130	44	4.48	4
Teo C, Jones R (1996) Management of hydrocephalus by endoscopic third ventriculostomy in patients with myelomeningocele. Pediatric Neurosurgery 25: 57–63	7	130	10	7.65	4
Buxton N, Macarthur D, Malucci C, Punt J, Vloeberghs M (1998) Neuroendoscopic third ventriculostomy in patients less than 1 year old. Pediatric Neurosurgery 29: 73–76	9	129	8	8.60	4
Rickert CH, Paulus W (2001) Epidemiology of central nervous system tumors in childhood and adolescence based on the new WHO classification. Childs Nervous System 17: 503–511	10	128	3	10.67	4
Bruce DA, Schut L (1979) Spinal lipomas in infancy and childhood. Childs Brain 5: 192–203	11	122	60	3.59	4
Di Rocco C, Marchese E, Velardi F (1994) A survey of the first complication of newly implanted CSF shunt devices for the treatment of nontumoral hydrocephalus—Cooperative survey of the 1991–1992 Education Committee of the ISPN. Childs Nervous System 10: 321–327	12	121	17	6.37	3
Piatt JH, Carlson CV (1993) A search for determinants of cerebrospinal fluid shunt survival: retrospective analysis of a 14-year institutional experience. Pediatric Neurosurgery 19: 233–242	13	117	21	5.85	4
Kestle J, Drake J, Milner R, Sainte-Rose C, Cinalli G, Boop F, Piatt J, Haines S, Schiff S, Cochrane D, Steinbok P, MacNeil N (2000) Long-term follow-up data from the shunt design trial. Pediatric Neurosurgery 33: 230–236	14	116	6	8.92	2
Pierre-Kahn A, Zerah M, Renier D, Cinalli G, Sainte-Rose C, Lellouch-Tubiana A, Brunelle F, Le Merrer M, Giudicelli Y, Pichon J, Kleinknecht B, Nataf F (1997) Congenital lumbosacral lipomas. Childs Nervous System 13: 298–334	14	116	12	7.25	4
Oberbauer RW, Haase J, Pucher R (1992) Arachnoid cysts in children: a European co-operative study. Childs Nervous System 8: 281–286	16	113	26	5.38	4
Hoffman HJ, Hendrick EB, Dennis M, Armstrong D (1979) Hemispherectomy for Sturge-Weber syndrome. Childs Brain 5: 233–248	17	110	64	3.24	4
Teo C, Rahman S, Boop FA, Cherny B (1996) Complications of endoscopic neurosurgery. Childs Nervous System 12: 248–253	18	109	16	6.41	4
Herrero MT, Barcia C, Navarro JM (2002) Functional anatomy of thalamus and basal ganglia. Childs Nervous System 18: 386–404	19	106	4	9.64	Review
Vinters HV, Fisher RS, Cornford ME, Mah V, Secor DL, Derosa MJ, Comair YG, Peacock WJ, Shields WD (1992) Morphological substrates of infantile spasms: studies based on surgically resected cerebral tissue. Childs Nervous System 8: 8–17	19	106	34	5.05	4
Fasano VA, Broggi G, Barolat-Romana G, Sguazzi A (1978) Surgical treatment of spasticity in cerebral palsy. Childs Brain 4: 289–305	21	105	70	3.00	4
Herman JM, McLone DG, Storrs BB, Dauser RC (1993) Analysis of 153 patients with myelomeningocele or spinal lipoma reoperated upon for a tethered cord—Presentation, management and outcome. Pediatric Neurosurgery 19: 243–249	22	104	30	5.20	4
Pierre-Kahn A, Hirsch JF, Roux FX, Renier D, Sainte-Rose C (1983) Intracranial ependymomas in childhood—Survival and functional results of 47 cases. Childs Brain 10: 145–156	23	102	63	3.40	4
Sanford RA (1994) Craniopharyngioma: results of survey of the American Society of Pediatric Neurosurgery. Pediatric Neurosurgery 21: 39–43	24	100	27	5.26	Other
Adelson PD, Clyde B, Kochanek PM, Wisniewski SR, Marion DW, Yonas H (1997) Cerebrovascular response in infants and young children following severe traumatic brain injury: a preliminary report. Pediatric Neurosurgery 26: 200–207	25	99	18	6.19	4
Aschoff A, Kremer P, Benesch C, Fruh K, Klank A, Kunze S (1995) Overdrainage and shunt technology—A critical comparison of programmable, hydrostatic and variable-resistance valves and flow-reducing devices. Childs Nervous System 11: 193–202	26	98	25	5.44	Review
Raimondi AJ, Robinson JS, Kuwamura K (1977) Complications of ventriculo-peritoneal shunting and a critical comparison of the three-piece and one-piece systems. Childs Brain 3: 321–342	26	98	75	2.72	3
Garrido E, Becker LF, Hoffman HJ, Hendrick EB, Humphreys R (1978) Gangliogliomas in children—A clinicopathological study. Childs Brain 4: 339–346	28	97	73	2.77	4
Renier D, Lajeunie E, Arnaud E, Marchac D (2000) Management of craniosynostoses. Childs Nervous System 16: 645–658	28	97	11	7.46	4
Duhaime AC, Christian C, Moss E, Seidl T (1996) Long-term outcome in infants with the shaking-impact syndrome. Pediatric Neurosurgery 24: 292–298	30	96	22	5.65	4
Packer RJ, Savino PJ, Bilaniuk LT, Zimmerman RA, Schatz NJ, Rosenstock JG, Nelson DS, Jarrett PD, Bruce DA, Schut L (1983) Chiasmatic gliomas of childhood—A reappraisal of natural history and effectiveness of cranial irradiation. Childs Brain 10: 393–403	30	96	66	3.20	4
Reilly PL, Simpson DA, Sprod R, Thomas L (1988) Assessing the conscious level in infants and young children: a paediatric version of the Glasgow coma scale. Childs Nervous System 4: 30–33	30	96	56	3.84	Other
Cohen AR (1993) Endoscopic ventricular surgery. Pediatric Neurosurgery 19: 127–134	33	94	39	4.70	4
Hoppe-Hirsch E, Renier D, Lellouch-Tubiana A, Sainte-Rose C, Pierre-Kahn A, Hirsch JF (1990) Medulloblastoma in childhood: progressive intellectual deterioration. Childs Nervous System 6: 60–65	33	94	50	4.09	3
Peacock WJ, Wehby-Grant MC, Shields WD, Shewmon DA, Chugani HT, Sankar R, Vinters HV (1996) Hemispherectomy for intractable seizures in children: a report of 58 cases. Childs Nervous System 12: 376–384	33	94	24	5.53	4
Bondurant CP, Jimenez DF (1995) Epidemiology of cerebrospinal fluid shunting. Pediatric Neurosurgery 23: 254–258	36	91	32	5.06	Review
Hoppe-Hirsch E, Brunet L, Laroussinie F, Cinalli G, Pierre-Kahn A, Renier D, Sainte-Rose C, Hirsch JF (1995) Intellectual outcome in children with malignant tumors of the posterior fossa: influence of the field of irradiation and quality of surgery. Childs Nervous System 11: 340–345	36	91	32	5.06	4
Matsushima Y, Inaba Y (1984) Moyamoya disease in children and its surgical treatment—Introduction of a new surgical procedure and its follow-up angiograms. Childs Brain 11: 155–170	38	90	69	3.10	Other
Chapman PH (1982) Congenital intraspinal lipomas: anatomic considerations and surgical treatment. Childs Brain 9: 37–47	39	89	71	2.87	4
Kothbauer K, Deletis V, Epstein FJ (1997) Intraoperative spinal cord monitoring for intramedullary surgery: an essential adjunct. Pediatric Neurosurgery 26: 247–254	39	89	23	5.56	Review
Schiffer D, Chio A, Giordana MT, Migheli A, Palma L, Polio B, Soffietti R, Tribolo A (1991) Histologic prognostic factors in ependymoma. Childs Nervous System 7: 177–182	39	89	52	4.05	4
Carrea R, Dowling E, Guevara JA (1975) Surgical treatment of hydatid cysts of central nervous system in pediatric age (Dowlings technique). Childs Brain 1:4–21	42	86	82	2.26	4
Lasjaunias P, Hui F, Zerah M, Garcia-Monaco R, Malherbe V, Rodesch G, Tanaka A, Alvarez H (1995) Cerebral arteriovenous malformations in children—Management of 179 consecutive cases and review of the literature. Childs Nervous System 11: 66–79	43	85	38	4.72	4
Kaplan AM, Albright AL, Zimmerman RA, Rorke LB, Li H, Boyett JM, Finlay JL, Wara WM, Packer RJ (1996) Brainstem gliomas in children—A Children's Cancer Group review of 119 cases. Pediatric Neurosurgery 24: 185–192	44	84	35	4.94	4
Maki Y, Akimoto H, Enomoto T (1980) Injuries of basal ganglia following head trauma in children. Childs Brain 7: 113–123	44	84	76	2.55	4
Stapleton SR, Kiriakopoulos E, Mikulis D, Drake JM, Hoffman HJ, Humphreys R, Hwang P, Otsubo H, Holowka S, Logan W, Rutka JT (1997) Combined utility of functional MRI, cortical mapping, and frameless stereotaxy in the resection of lesions in eloquent areas of brain in children. Pediatric Neurosurgery 26: 68–82	44	84	28	5.25	4
Arens LJ, Peacock WJ, Peter J (1989) Selective posterior rhizotomy: a long-term follow-up study. Childs Nervous System 5: 148–152	47	83	62	3.46	4
Amacher AL, Wellington J (1984) Infantile hydrocephalus: long-term results of surgical therapy. Childs Brain 11: 217–229	48	82	72	2.83	4
Raimondi AJ, Gutierrez FA (1975) Diagnosis and surgical treatment of choroid plexus papillomas. Childs Brain 1: 81–115	48	82	87	2.16	4
Takaku A, Kodama N, Ohara H, Hori S (1978) Brain tumor in newborn babies. Childs Brain 4: 365–375	50	81	78	2.31	4
Johnson DL, Boal D, Baule R (1995) Role of apnea in nonaccidental head injury. Pediatric Neurosurgery 23: 305–310	51	80	46	4.44	4
Hahn YS, Chyung CH, Barthel MJ, Bailes J, Flannery AM, McLone DG (1988) Head injuries in children under 36 months of age—Demography and outcome. Childs Nervous System 4: 34–40	52	79	67	3.16	3
Thompson DN, Malcolm GP, Jones BM, Harkness WJ, Hayward RD (1995) Intracranial pressure in single-suture craniosynostosis. Pediatric Neurosurgery 22: 235–240	52	79	47	4.39	4
Deutsch M, Thomas PR, Krischer J, Boyett JM, Albright L, Aronin P, Langsten J, Allen JC, Packer RJ, Linggood R, Mulhern R, Stanley P, Stehbens JA, Duffner P, Kun L, Rorke L, Cherlow J, Freidman H, Finlay JL, Vietti T (1996) Results of a prospective randomized trial comparing standard dose neuraxis irradiation (3,600 cGy/20) with reduced neuraxis irradiation (2,340 cGy/13) in patients with low-stage medulloblastoma—A Combined Children's Cancer Group-Pediatric Oncology Group Study. Pediatric Neurosurgery 24: 167–176	54	78	42	4.59	2
Goumnerova LC, Frim DM (1997) Treatment of hydrocephalus with third ventriculocisternostomy: outcome and CSF flow patterns. Pediatric Neurosurgery 27: 149–152	54	78	36	4.88	4
Levin HS, Eisenberg HM (1979) Neuropsychological outcome of closed head injury in children and adolescents. Childs Brain 5: 281–292	54	78	80	2.29	4
Rekate HL (1993) Classification of slit-ventricle syndromes using intracranial pressure monitoring. Pediatric Neurosurgery 19: 15–20	54	78	54	3.90	4
Anderson V, Catroppa C, Morse S, Haritou F, Rosenfeld J (2000) Recovery of intellectual ability following traumatic brain injury in childhood: Impact of injury severity and age at injury. Pediatric Neurosurgery 32: 282–290	58	77	20	5.92	4
Sayers MP, Kosnik EJ (1976) Percutaneous third ventriculostomy: experience and technique. Childs Brain 2: 24–30	58	77	91	2.08	4
Tomita T, McLone DG (1993) Radical resections of childhood craniopharyngiomas. Pediatric Neurosurgery 19: 6–14	58	77	55	3.85	4
Hoffman HJ, Humphreys RP, Drake JM, Rutka JT, Becker LE, Jenkin D, Greenberg M (1993) Optic pathway/hypothalamic gliomas: a dilemma in management. Pediatric Neurosurgery 19: 186–195	61	76	58	3.80	4
McGirt MJ, Leveque JC, Wellons JC, Villavicencio AT, Hopkins JS, Fuchs HE, George TM (2002) Cerebrospinal fluid shunt survival and etiology of failures: a seven-year institutional experience. Pediatric Neurosurgery 36: 248–255	61	76	14	6.91	4
Gerosa M, Licata C, Fiore DL, Iraci G (1980) Intracranial aneurysms of childhood. Childs Brain 6: 295–302	63	75	81	2.27	4
Reigel DH, Scarff TB, Woodford JE (1979) Biopsy of pediatric brain stem tumors. Childs Brain 5: 329–340	63	75	85	2.21	4
Almeida GM, Pindaro J, Plese P, Bianco E, Shibata MK (1977) Intracranial arterial aneurysms in infancy and childhood. Childs Brain 3: 193–199	65	74	92	2.06	4
Dirven CM, Mooij JJ, Molenaar WM (1997) Cerebellar pilocytic astrocytoma: a treatment protocol based upon analysis of 73 cases and a review of the literature. Childs Nervous System 13: 17–23	65	74	41	4.63	4
Egnor M, Zheng L, Rosiello A, Gutman F, Davis R (2002) A model of pulsations in communicating hydrocephalus. Pediatric Neurosurgery 36: 281–303	65	74	15	6.73	Other
Sano K, Matsutani M (1981) Pinealoma (germinoma) treated by direct surgery and postoperative irradiation—A long-term follow-up. Childs Brain 8: 81–97	65	74	78	2.31	4
Casey AT, Kimmings EJ, Kleinlugtebeld AD, Taylor WA, Harkness WF, Hayward RD (1997) The long-term outlook for hydrocephalus in childhood—A ten-year cohort study of 155 patients. Pediatric Neurosurgery 27: 63–70	69	73	43	4.56	4
Fischbein NJ, Prados MD, Wara W, Russo C, Edwards MS, Barkovich AJ (1996) Radiologic classification of brain stem tumors: correlation of magnetic resonance imaging appearance with clinical outcome. Pediatric Neurosurgery 24: 9–23	69	73	48	4.29	3
Steinbok P, Irvine B, Cochrane DD, Irwin BJ (1992) Long-term outcome and complications of children born with meningomyelocele. Childs Nervous System 8: 92–96	69	73	61	3.48	4
Ventureyra EC, Higgins MJ (1994) Traumatic intracranial aneurysms in childhood and adolescence—Case reports and review of the literature. Childs Nervous System 10: 361–379	69	73	56	3.84	4
Choi JU, Kim DS, Huh R (1999) Endoscopic approach to arachnoid cyst. Childs Nervous System 15: 285–291	73	72	31	5.14	4
Menezes AH (1995) Primary craniovertebral anomalies and the hindbrain herniation syndrome (Chiari I): data base analysis. Pediatric Neurosurgery 23: 260–269	73	72	53	4.00	4
O'Hayon BB, Drake JM, Ossip MG, Tuli S, Clarke M (1998) Frontal and occipital horn ratio: A linear estimate of ventricular size for multiple imaging modalities in pediatric hydrocephalus. Pediatric Neurosurgery 29: 245–249	73	72	37	4.80	Other
Amacher AL (1980) Craniopharyngioma: the controversy regarding radiotherapy. Childs Brain 6: 57–64	76	71	88	2.15	Review
Glauser TA, Packer RJ (1991) Cognitive deficits in long-term survivors of childhood brain tumors. Childs Nervous System 7: 2–12	76	71	65	3.23	Review
Holness RO, Hoffman HJ, Hendrick EB (1979) Subtemporal decompression for the slit-ventricle syndrome after shunting in hydrocephalic children. Childs Brain 5: 137–144	76	71	89	2.09	4
Sano K, Wakai S, Ochiai C, Takakura K (1981) Characteristics of intracranial meningiomas in childhood. Childs Brain 8: 98–106	76	71	84	2.22	4
Duffner PK, Kischer JP, Sanford RA, Horowitz ME, Burger PC, Cohen ME, Friedman HS, Kun LE, Pediatric Oncology Group (1998) Prognostic factors in infants and very young children with intracranial ependymomas. Pediatric Neurosurgery 28: 215–222	80	70	40	4.67	3
Hoffman HJ, Hendrick EB, Humphreys RP (1975) Manifestations and management of Arnold-Chiari malformation in patients with myelomeningocele. Childs Brain 1: 255–259	80	70	98	1.84	Review
Lasjaunias P, Wuppalapati S, Alvarez H, Rodesch G, Ozanne A (2005) Intracranial aneurysms in children aged under 15 years: review of 59 consecutive children with 75 aneurysms. Childs Nervous System 21: 437–450	80	70	7	8.75	4
Packer RJ, Rood BR, MacDonald TJ (2003) Medulloblastoma: present concepts of stratification into risk groups. Pediatric Neurosurgery 39: 60–67	80	70	13	7.00	Review
Raimondi AJ, Tomita T (1979) Medulloblastoma in childhood: comparative results of partial and total resection. Childs Brain 5: 310–328	80	70	92	2.06	3
Raimondi AJ, Tomita T (1983) Brain tumors during the first year of life. Childs Brain 10: 193–207	80	70	77	2.33	4
Rajakulasingam K, Cerullo LJ, Raimondi AJ (1979) Childhood moyamoya syndrome—Postradiation pathogenesis. Childs Brain 5: 467–475	80	70	92	2.06	4
Amacher AL, Drake CG (1975) Cerebral artery aneurysms in infancy, childhood and adolescence. Childs Brain 1: 72–80	87	69	99	1.82	4
Mori K, Murata T, Hashimoto N, Handa H (1980) Clinical analysis of arteriovenous malformations in children. Childs Brain 6: 13–25	87	69	89	2.09	4
O'Brien M, Parent A, Davis B (1979) Management of ventricular shunt infections. Childs Brain 5: 304–309	87	69	95	2.03	4
Coulon RA, Till K (1977) Intracranial ependymomas in children: a review of 43 cases. Childs Brain 3: 154–168	90	68	97	1.89	4
Storrs BB, Humphreys RP, Hendrick EB, Hoffman HJ (1982) Intracranial aneurysms in the pediatric age-group. Childs Brain 9: 358–361	90	68	86	2.19	4
Beems T, Grotenhuis JA (2002) Is the success rate of endoscopic third ventriculostomy age-dependent? An analysis of the results of endoscopic third ventriculostomy in young children. Childs Nervous System 18: 605–608	92	67	19	6.09	4
Fisher PG, Jenab J, Goldthwaite PT, Tihan T, Wharam MD, Foer DR, Burger PC (1998) Outcomes and failure patterns in childhood craniopharyngiomas. Childs Nervous System 14: 558–563	92	67	45	4.47	3
Takeuchi S, Tsuchida T, Kobayashi K, Fukuda M, Ishii R, Tanaka R, Ito J (1983) Treatment of moyamoya disease by temporal muscle graft ‘encephalo-myo-synangiosis’. Childs Brain 10: 1–15	92	67	83	2.23	4
Abbott R, Forem SL, Johann M (1989) Selective posterior rhizotomy for the treatment of spasticity: a review. Childs Nervous System 5: 337–346	95	66	74	2.75	Review
Dias MS, Walker ML (1992) The embryogenesis of complex dysraphic malformations: a disorder of gastrulation? Pediatric Neurosurgery 18: 229–253	95	66	68	3.14	Review
Osaka K, Handa H, Matsumoto S, Yasuda M (1980) Development of the cerebrospinal fluid pathway in the normal and abnormal human embryos. Childs Brain 6: 26–38	95	66	96	2.00	Other
Suzuki J, Takaku A, Kodama N, Sato S (1975) An attempt to treat cerebrovascular ‘Moyamoya’ disease in children. Childs Brain 1: 193–206	95	66	100	1.74	4
Villani RM, Tomei G, Bello L, Sganzerla E, Ambrosi B, Re T, Barilari MG (1997) Long-term results of treatment for craniopharyngioma in children. Childs Nervous System 13: 397–405	95	66	49	4.13	3
Ronning C, Sundet K, Due-Tonnessen B, Lundar T, Helseth E (2005) Persistent cognitive dysfunction secondary to cerebellar injury in patients treated for posterior fossa tumors in childhood. Pediatric Neurosurgery 41:15–21	100	65	9	8.13	Other

The first rank represents the descending rank order by raw citations. The second rank (‘Adjusted Rank’) is based on the adjusted citation index (number of citations divided by years since initial publication). The column titled “Class” is the class of evidence for the article based on the 2011 “Oxford Centre Level of Evidence” classification scheme

Table 2.

List of top 50 most cited journal articles in pediatric neurosurgical journals from the years 2002–2012

Article	Rank	Citations	Adjusted rank	Adjusted citations	Class
Herrero MT, Barcia C, Navarro JM (2002) Functional anatomy of thalamus and basal ganglia. Childs Nervous System 18: 386–404	1	125	2	11.36	Review
McGirt MJ, Leveque JC, Wellons JC, Villavicencio AT, Hopkins JS, Fuchs HE, George TM (2002) Cerebrospinal fluid shunt survival and etiology of failures: a seven-year institutional experience. Pediatric Neurosurgery 36: 248–255	2	79	9	7.18	4
Egnor M, Zheng L, Rosiello A, Gutman F, Davis R (2002) A model of pulsations in communicating hydrocephalus. Pediatric Neurosurgery 36: 281–303	3	76	14	6.91	Other
Lasjaunias P, Wuppalapati S, Alvarez H, Rodesch G, Ozanne A (2005) Intracranial aneurysms in children aged under 15 years: review of 59 consecutive children with 75 aneurysms. Childs Nervous System 21: 437–450	4	71	4	8.88	4
Packer RJ, Rood BR, MacDonald TJ (2003) Medulloblastoma: present concepts of stratification into risk groups. Pediatric Neurosurgery 39: 60–67	5	70	12	7.00	Review
Solaroglu I, Solaroglu A, Kaptanoglu E, Dede S, Haberal A, Beskonakli E, Kilinc K (2003) Erythropoietin prevents ischemia-reperfusion from inducing oxidative damage in fetal rat brain. Childs Nervous System 19: 19–22	6	69	15	6.90	Other
Beems T, Grotenhuis JA (2002) Is the success rate of endoscopic third ventriculostomy age-dependent? An analysis of the results of endoscopic third ventriculostomy in young children. Childs Nervous System 18: 605–608	7	67	23	6.09	4
Fung LW, Thompson D, Ganesan V (2005) Revascularisation surgery for paediatric moyamoya: a review of the literature. Childs Nervous System 21: 358–364	7	67	5	8.38	Review
Klein O, Pierre-Kahn A, Boddaert N, Parisot D, Brunelle F (2003) Dandy-Walker malformation: prenatal diagnosis and prognosis. Childs Nervous System 19: 484–489	7	67	19	6.70	4
Ronning C, Sundet K, Due-Tonnessen B, Lundar T, Helseth E (2005) Persistent cognitive dysfunction secondary to cerebellar injury in patients treated for posterior fossa tumors in childhood. Pediatric Neurosurgery 41: 15–21	10	66	6	8.25	Other
Cinalli G, Spennato P, Sainte-Rose C, Arnaud E, Aliberti F, Brunelle F, Cianciulli E, Renier D (2005) Chiari malformation in craniosynostosis. Childs Nervous System 21: 889–901	11	64	8	8.00	Review
Koch D, Wagner W (2004) Endoscopic third ventriculostomy in infants of less than 1 year of age: which factors influence the outcome? Childs Nervous System 20: 405–411	11	64	11	7.11	4
Sala F, Krzan MJ, Deletis V (2002) Intraoperative neurophysiological monitoring in pediatric neurosurgery: why, when, how? Childs Nervous System 18: 264–287	11	64	27	5.82	Review
Tulipan N, Sutton LN, Bruner JP, Cohen BM, Johnson M, Adzick NS (2003) The effect of intrauterine myelomeningocele repair on the incidence of shunt-dependent hydrocephalus. Pediatric Neurosurgery 38: 27–33	14	62	22	6.20	4
Chiaretti A, Piastra M, Pulitano S, Pietrini D, De Rosa G, Barbaro R, Di Rocco C (2002) Prognostic factors and outcome of children with severe head injury: an 8-year experience. Childs Nervous System 18: 129–136	15	60	33	5.45	4
Garel C, Chantrel E, Elmaleh M, Brisse H, Sebag G (2003) Fetal MRI: normal gestational landmarks for cerebral biometry, gyration and myelination. Childs Nervous System 19: 422–425	16	57	28	5.70	Other
Tomita T, Bowman RM (2005) Craniopharyngiomas in children: surgical experience at Children's Memorial Hospital. Childs Nervous System 21: 729–746	17	54	18	6.75	4
Schijman E, Steinbok P (2004) International survey on the management of Chiari I malformation and syringomyelia. Childs Nervous System 20: 341–348	18	53	25	5.89	Other
Figaji AA, Fieggen AG, Peter JC (2003) Early decompressive craniotomy in children with severe traumatic brain injury. Childs Nervous System 19: 666–673	19	52	34	5.20	4
Moutard ML, Kieffer V, Feingold J, Kieffer F, Lewin F, Adamsbaum C, Gelot A, Campistol Plana J, van Bogaert P, Andre M, Ponsot G (2003) Agenesis of corpus callosum: prenatal diagnosis and prognosis. Childs Nervous System 19: 471–476	20	51	37	5.10	Other
Aryan HE, Meltzer HS, Park MS, Bennett RL, Jandial R, Levy ML (2005) Initial experience with antibiotic-impregnated silicone catheters for shunting of cerebrospinal fluid in children. Childs Nervous System 21: 56–61	21	50	21	6.25	4
Navarro R, Olavarria G, Seshadri R, Gonzales-Portillo G, McLone DG, Tomita T (2004) Surgical results of posterior fossa decompression for patients with Chiari I malformation. Childs Nervous System 20: 349–356	21	50	30	5.56	4
Lew SM, Kothbauer KF (2007) Tethered cord syndrome: an updated review. Pediatric Neurosurgery 43: 236–248	23	49	7	8.17	Review
Prasad MR, Ewing-Cobbs L, Swank PR, Kramer L (2002) Predictors of outcome following traumatic brain injury in young children. Pediatric Neurosurgery 36: 64–74	23	49	46	4.45	2
Tummala RP, Chu RM, Liu H, Truwit CL, Hall WA (2003) Application of diffusion tensor imaging to magnetic-resonance-guided brain tumor resection. Pediatric Neurosurgery 39: 39–43	23	49	38	4.90	4
Zhao JJ, Yang J, Lin J, Yao N, Zhu Y, Zheng J, Xu J, Cheng JQ, Lin JY, Ma X (2009) Identification of miRNAs associated with tumorigenesis of retinoblastoma by miRNA microarray analysis. Childs Nervous System 25: 13–20	23	49	1	12.25	Other
McLone DG, Dias MS (2003) The Chiari II malformation: cause and impact. Childs Nervous System 19: 540–550	27	48	40	4.80	Review
Oi S, Di Rocco C (2006) Proposal of “evolution theory in cerebrospinal fluid dynamics” and minor pathway hydrocephalus in developing immature brain. Childs Nervous System 22: 662–669	27	48	16	6.86	4
Cuccia V, Zuccaro G, Sosa F, Monges J, Lubienieky F, Taratuto AL (2003) Subependymal giant cell astrocytoma in children with tuberous sclerosis. Childs Nervous System 19: 232–243	29	47	41	4.70	4
Gooch JL, Oberg WA, Grams B, Ward LA, Walker ML (2003) Complications of intrathecal baclofen pumps in children. Pediatric Neurosurgery 39: 1–6	30	46	44	4.60	4
Jallo GI, Biser-Rohrbaugh A, Freed D (2004) Brainstem gliomas. Childs Nervous System 20: 143–153	30	46	35	5.11	Review
Kaufman MH (2004) The embryology of conjoined twins. Childs Nervous System 20: 508–525	30	46	35	5.11	Review
Novegno F, Caldarelli M, Massa A, Chieffo D, Massimi L, Pettorini B, Tamburrini G, Di Rocco C (2008) The natural history of the Chiari Type I anomaly. Journal of Neurosurgery-Pediatrics 2: 179–187	30	46	3	9.20	4
Cochrane DD, Kestle JR (2003) The influence of surgical operative experience on the duration of first ventriculoperitoneal shunt function and infection. Pediatric Neurosurgery 38: 295–301	34	45	45	4.50	3
Kalapurakal JA (2005) Radiation therapy in the management of pediatric craniopharyngiomas—a review. Childs Nervous System 21: 808–816	34	45	29	5.63	Review
Caldarelli M, Massimi L, Tamburrini G, Cappa M, Di Rocco C (2005) Long-term results of the surgical treatment of craniopharyngioma: the experience at the Policlinico Gemelli, Catholic University, Rome. Childs Nervous System 21: 747–757	36	44	31	5.50	4
Jallo GI, Freed D, Epstein F (2003) Intramedullary spinal cord tumors in children. Childs Nervous System 19: 641–649	36	44	47	4.40	Review
Navarro R, Gil-Parra R, Reitman AJ, Olavarria G, Grant JA, Tomita T (2006) Endoscopic third ventriculostomy in children: early and late complications and their avoidance. Childs Nervous System 22: 506–513	36	44	20	6.29	4
Schroeder HW, Oertel J, Gaab MR (2004) Incidence of complications in neuroendoscopic surgery. Childs Nervous System 20: 878–883	36	44	39	4.89	4
Zuccaro G (2005) Radical resection of craniopharyngioma. Childs Nervous System 21: 679–690	36	44	31	5.50	4
Kapp-Simon KA, Speltz ML, Cunningham ML, Patel PK, Tomita T (2007) Neurodevelopment of children with single suture craniosynostosis: a review. Childs Nervous System 23: 269–281	41	43	10	7.17	Review
Raybaud C, Levrier O, Brunel H, Girard N, Farnarier P (2003) MR imaging of fetal brain malformations. Childs Nervous System 19: 455–470	41	43	48	4.30	Other
Beems T, Grotenhuis JA (2004) Long-term complications and definition of failure of neuroendoscopic procedures. Childs Nervous System 20: 868–877	43	42	42	4.67	4
Kan P, Kestle J (2007) Lack of efficacy of antibiotic-impregnated shunt systems in preventing shunt infections in children. Childs Nervous System 23: 773–777	43	42	12	7.00	4
Schijman E (2004) History, anatomic forms, and pathogenesis of Chiari I malformations. Childs Nervous System 20: 323–328	43	42	42	4.67	Review
Sinclair DB, Aronyk K, Snyder T, McKean J, Wheatley M, Bhargava R, Hoskinson M, Hao C, Colmers W (2003) Pediatric temporal lobectomy for epilepsy. Pediatric Neurosurgery 38: 195–205	43	42	49	4.20	4
Vinchon M, Dhellemmes P (2006) Cerebrospinal fluid shunt infection: risk factors and long-term follow-up. Childs Nervous System 22: 692–697	43	42	24	6.00	4
Selcuki M, Vatansever S, Inan S, Erdemli E, Bagdatoglu C, Polat A (2003) Is a filum terminale with a normal appearance really normal? Childs Nervous System 19: 3–10	48	41	50	4.10	Other
Tubbs RS, Lyerly MJ, Loukas M, Shoja MM, Oakes WJ (2007) The pediatric chiari I malformation: a review. Childs Nervous System 23: 1239–1250	48	41	17	6.83	Review
Zhao JJ, Hua YJ, Sun DG, Meng XX, Xiao HS, Ma X (2006) Genome-wide microRNA profiling in human fetal nervous tissues by oligonucleotide microarray. Childs Nervous System 22: 1419–1425	48	41	26	5.86	Basic Science

Top 100 articles

Demographics

Three journals provided all the articles in the top 100 list: Child's Brain (39 articles), Pediatric Neurosurgery (33 articles), and Child's Nervous System (28 articles) (Fig. 1a). Articles were published from 1975 to 2006 with most productive single and 5-year period being 1979 (n=9) and 1994–1999 (n=31), respectively (Fig. 2a). The number of authors ranged from 1 to 20 with a mean, median, and mode of 4, 4, and 2, respectively. The country of the first author was most frequently the United States (n=51), followed by Canada (n=9) and Japan (n=9) (Fig. 3a). The institution with the most articles was Northwestern University in Chicago, Illinois (n=5), followed by Hôpital Necker-Enfants Malades in Paris, France with four articles.

Fig. 1.

Pie chart presenting journals providing articles for a the top 100 overall most cited list and b the top 50 most cited articles over the past 10 years (2002–2012)

Fig. 2.

Articles per year for a the top 100 list and b the top 50 list

Fig. 3.

Article counts by country of origin for a the top 100 list and b the top 50 list

Topic of research, study design and level of evidence

The most popular topic was oncology (n=28) followed by hydrocephalus (n=23) and vascular (n=12) (Fig. 4a). There were 21 prospective studies and 64 retrospective studies. Based on the OCEBM 2011 Levels of Evidence Classification System, three articles were classified as level 2, nine were classified as level 3, and seventy were classified as level 4. Eighteen articles could not be classified, including 11 review articles and 7 ‘Other’ articles: introduction of a new pediatric version of the Glasgow Coma Scale (overall rank no. 32); a survey of pediatric neurosurgeons regarding the management of craniopharyngioma (no. 24); description of a new surgical technique for moyamoya (no. 38); an introduction of the frontal and occipital horn ratio (no. 75); a cross-sectional study of cognitive dysfunction in survivors of posterior fossa tumors (no. 100); and two theoretical models of CSF flow (no. 67 and 97).

Fig. 4.

Article counts by topic for a the top 100 list and b the top 50 list

Citation analysis

The top 100 overall most cited articles were referenced an average of 90±26 times (median 81; range, 65–193). The average adjusted citation count was 4.5±2.5 (median 4.1; range, 1.7–15.5). When comparing the adjusted ranking to the overall, the change in position ranged from −55 to +91 with a mean absolute rank change of 21±19. Of the 24 papers that had 100 or more citations, 17 (71 %) had an adjusted citation value of ≥5.0. Of those papers with less than 100 citations, 17 (22 %) had an adjusted citation value of ≥5.0. The paper with the highest combined citations overall (186, rank no. 2) and adjusted citation count (15.5, rank no. 1) was by Bowman et al.

Top 50 articles in the last decade (2002–2012)

Demographics

Three journals provided all the articles in the top 50 list: Child's Nervous System (38 articles), Pediatric Neurosurgery (11 articles), and Journal of Neurosurgery: Pediatrics (1 article) (Fig. 1b). The largest number of articles (n=16) were published in 2003 (Fig. 2b). The number of authors ranged from 1 to 11, with a mean, median, and mode of 4, 5, and 5, respectively. The country of origin of the first author was most frequently the United States (n=18), followed by France (n=6) and Italy (n=5) (Fig. 3b). The institution with the most articles included in the top 50 was again Northwestern University in Chicago, Illinois (n=5).

Topic of research, study design and level of evidence

Hydrocephalus-related research was the most common topic (n=13), then oncology (n=10) and congenital spine (8) (Fig. 4b). There were four prospective studies and 25 retrospective studies. There was one level 2 article, one level 3, and 24 level 4. The 24 articles that could not be classified were 14 review and ten ‘Other’ articles: three basic science (overall rank no. 6, 23 and 49); two radiological studies on MRI landmarks for gestation and myelination (nos. 16 and 42); a model of CSF flow (no. 3); an international survey on Chiari malformations (no. 18); a pathologic analysis of the filum terminale (no. 50); and two neurocognitive studies (nos. 10 and 20).

Citation analysis

The top 50 articles over the last decade were cited an average of 54±15 times (median 49; range, 41–125). The average adjusted citation count was 6.3±1.7 (median 5.9; range, 4.1–12.3). When comparing the adjusted ranking to the overall, the change in position ranged from −23 to +31 with a mean absolute rank change of 11±9. Of the 22 papers that had 50 or more citations, all had an adjusted citation value of ≥5.0. Of those papers with less than 50 citations, 15 (54 %) had an adjusted citation value of ≥5.0. The paper with the highest combined overall (125, rank no. 1) and adjusted citation count (11.36, rank no. 2) was by Herrero et al.

Discussion

In this paper, we present the top 100 most cited articles found in the literature and the top 50 cited articles from the last decade (2002–2012) in pediatric neurosurgery. Ponce et al. recently published the top 100 cited in neurosurgery, yet there was a distinct paucity of pediatric neurosurgery articles in this list [20]. This is due to the fact that pediatric neurosurgical organizations are young and the readership is relatively very small compared with adult neurosurgery. The International Society for Pediatric Neurosurgery (ISPN) and the Pediatric Section of the American Association of Neurological Surgeons (AANS) were both founded in 1972 and the American Society of Pediatric Neurosurgery (ASPN) in 1978. Child's Brain was the first official journal of the ISPN, but split into two journals—Pediatric Neurosurgery and Child's Nervous System—on January 1, 1985 when ISPN changed their publishing company. The official journal of the ASPN is the Journal of Neurosurgery: Pediatrics, which began publication initially as a quarterly supplement in 2004 but is now monthly.

Our results

Based on citation analysis, an original (i.e., non-review article) pediatric neurosurgery publication will achieve elite status amongst all publications from dedicated pediatric neurosurgery journals (what we term a ‘historical classic’) if it acquires a total citation count of 100 or more and an adjusted citation count of five or greater. Likewise, an article within the last 10 years will attain similar status if it accrues 50 or more citations and, again, an adjusted citation count of five or greater (what we term a ‘contemporary classic’). Citations in pediatric neurosurgery are dwarfed by those in the adult literature. Based on our search, the highest number of citations for a pediatric paper was 193 compared with 1,515 as reported by Ponce et al. in 2010, an almost eightfold difference.

The adjusted citation count produced significant shifts in rank for the top 100 (mean absolute rank change 21±19, range, −55 to +91) and top 50 (mean absolute rank change 11±9; range, −23 to +31) lists. The time-adjusted citation count provides another metric by which to judge the citation history of a paper. It can be viewed as a measure of the paper's ‘relevance’. A publication reaching high citation counts more recently will have a higher adjusted citation count than an older publication with the same total number of citations. Therefore, papers with both high total citation and adjusted citation values will likely still retain relevance to a pediatric neurosurgeon's practice. Publications with a high total citation count but low adjusted citation index are more likely of historical interest and may not reflect the current knowledge, controversy, and clinical applications of the topic. Further work could include the creation of a metric that creates a sort of ‘heat-index’ of an article that would track the number and rate of change of citations over a particular time period, such as a 2- or 5-year window. Such an index could provide the reader a better means of identifying articles that are receiving high attention (i.e., relevance), while removing the historical bias from citation count accumulation over time. Another possible measure of relevance would be the number of times an article is downloaded from a journal's website. This would likely provide the most up-to-date information on what topic(s) are currently generating the greatest interest.

Many of the articles included within this study were clinical, retrospective with regards to data collection, and uncontrolled (meaning there was a lack of identifiable comparison groups). As such, they represent the lowest level of clinical evidence (level 4). If we are to use bibliometrics—of which citation analysis is a part—as a means to analyze an author's publication history and impact, then review articles deserve special mention. Review articles, in general, are typically well-cited, particularly in the first few years of their publication. There were 11 review articles in the top 100 list and 14 in the top 50. Although the creation of a review article is time-consuming and can be of great interest to the readership by synthesizing and summarizing data on a particular topic, it does not represent original research (although an argument can be made with meta-analyses). As such, the citations obtained by a review article should not be given the same weight as citations given to a publication with an original contribution.

Limitations

Our work has a number of limitations. The first is equating citation count with importance and impact of an article [2]. The citation count may not accurately reflect an article's actual impact on a field in the case of a review article, as previously mentioned [29]. There is time bias with citations. Newer articles that are recognized as having important—even critical—new information that are immediately relevant to one's practice simply may not have had sufficient time to accumulate citations. We have attempted to adjust for this bias by introducing the time-adjusted citation count.

Our method likely missed a number of key articles by searching only journals that are dedicated to pediatric neurosurgery. There are undoubtedly important pediatric neurosurgery articles that were published in journals such as Neurosurgery or Journal of Neurosurgery (before the introduction of the Pediatrics supplement) that we did not include. Also, the accuracy—and thus quality—of citation analysis is dependent on the accuracy and quality of the search engine that is used. Currently, there are three citation databases in existence: Google Scholar, Scopus, and WOS. Google Scholar's database is more inclusive but has multiple inaccuracies and less up-to-date citation information. Scopus database has a wider journal scope but is limited to publications after 1995 compared with WOS, which includes publications since the early 1900s [30].

Conclusion

Within the limits of citation analysis, we have compiled the top 100 most cited and top 50 most recently cited publications in pediatric neurosurgery. These lists provide a practical guide to users wishing to familiarize themselves with some of the most important studies within pediatric neurosurgery. We have introduced the time-adjusted citation count, which can be viewed as the average number of citations an article receives per year since publication as an attempt to quantify an article's ‘relevance’. Based on our analysis, a ‘historical classic’ is defined as an original paper with 100 or more total citations and an average of five or more citations per year. A ‘contemporary classic’ is one that has 50 or more citations and an average of five or more citations per year. Such lists should be made available on web sites for ISPN, ASPN, or the Pediatric Section of the AANS/CNS and periodically updated as a means to evaluate the history, evolution, and future direction of research in pediatric neurosurgery.

Table 3.

The 2011 Oxford centre for evidence-based medicine levels of evidence [28]

Question	Level 1a	Level 2a	Level 3a	Level 4a	Level 5
How common is the problem?	Local and current random sample surveys (or censuses)	Systematic review of surveys that allow matching to local circumstancesb	Local non-random sampleb	Case-seriesb	n/a
Is this diagnostic or monitoring test accurate? (Diagnosis)	Systematic review of cross sectional studies with consistently applied reference standard and blinding	Individual cross sectional studies with consistently applied reference standard and blinding	Non-consecutive studies, or studies without consistently applied reference standardsb	Case–control studies, or “poor or non-independent reference standard”b	Mechanism-based reasoning
What will happen if we do not add a therapy? (Prognosis)	Systematic review of inception cohort studies	Inception cohort studies	Cohort study or control arm of randomized triala	Case-series or case–control studies, or poor quality prognostic cohort studyb	n/a
Does this intervention help? (Treatment Benefits)	Systematic review of randomized trials or n-of-1 trials	Randomized trial or observational study with dramatic effect	Non-randomized controlled cohort/follow-up studyb	Case-series, case–control studies, or historically controlled studiesb	Mechanism-based reasoning
What are the COMMON harms? (Treatment Harms)	Systematic review of randomized trials, systematic review of nested case–control studies, n-of-1 trial with the patient you are raising the question about, or observational study with dramatic effect	Individual randomized trial or (exceptionally) observational study with dramatic effect	Non-randomized controlled cohort/follow-up study (post-marketing surveillance) provided there are sufficient numbers to rule out a common harm. (For long-term harms the duration of follow-up must be sufficient.)b	Case-series, case–control, or historically controlled studiesb	Mechanism-based reasoning
What are the RARE harms? (Treatment Harms)	Systematic review of randomized trials or n-of-1 trial	Randomized trial or (exceptionally) observational study with dramatic effect
Is this (early detection) test worthwhile? (Screening)	Systematic review of randomized trials	Randomized trial	Non-randomized controlled cohort/follow-up studyb	Case-series, case–control, or historically controlled studiesb	Mechanism-based reasoning

^aLevel may be graded down on the basis of study quality, imprecision, indirectness (study PICO does not match questions PICO), because of inconsistency between studies, or because the absolute effect size is very small; level may be graded up if there is a large or very large effect size

^bAs always, a systematic review is generally better than an individual study