Abstract
Background:
Clinicians appear to obtain emergent neuroimaging for children with headaches based on the presence of red flag findings. However, little data exists regarding the prevalence of these findings in emergency department populations, and whether the identification of red flag findings is associated with potentially unnecessary emergency department neuroimaging.
Objectives:
We aimed to determine the prevalence of red flag findings and their association with neuroimaging in otherwise healthy children presenting with headaches to the emergency department. Our secondary aim was to determine the prevalence of emergent intracranial abnormalities in this population.
Methods:
A prospective cohort study of otherwise healthy children 2–17 years of age presenting to an urban pediatric emergency department with non-traumatic headaches was undertaken. Emergency department physicians completed a standardized form to document headache descriptors and characteristics, associated symptoms, and physical and neurological exam findings. Children who did not receive emergency department neuroimaging received 4-month telephone follow-up. Outcomes included emergency department neuroimaging and the presence of emergent intracranial abnormalities.
Results:
We enrolled 224 patients; 197 (87.9%) had at least one red flag finding on history. Several red flag findings were reported by more than a third of children, including: Headache waking from sleep (34.8%); headache present with or soon after waking (39.7%); or headaches increasing in frequency, duration and severity (40%, 33.1%, and 46.3%). Thirty-three percent of children received emergency department neuroimaging. The prevalence of emergent intracranial abnormalities was 1% (95% CI 0.1, 3.6). Abnormal neurological exam, extreme pain intensity of presenting headache, vomiting, and positional symptoms were independently associated with emergency department neuroimaging.
Conclusions:
Red flag findings are common in children presenting with headaches to the emergency department. The presence of red flag findings is associated with emergency department neuroimaging, although the risk of emergent intracranial abnormalities is low. Many children with headaches may be receiving unnecessary neuroimaging due to the high prevalence of non-specific red flag findings.
Keywords: Pediatric, headache, emergency department, red flag findings, emergent intracranial abnormality, brain tumor, computed tomography, magnetic resonance imaging
Introduction
Background
More than 400,000 children present annually to emergency departments (EDs) in the US with a chief complaint of headache, comprising approximately 1% of ED visits (1–3). Most of these children have headaches that are primary (e.g. migraines), or secondary to conditions such as respiratory infections (2–4). A small but meaningful proportion (0.5–1.0%) are associated with intracranial abnormalities requiring emergent identification, such as brain tumors, intracranial hemorrhages, or strokes (2,5–7).
Although emergent intracranial abnormalities are uncommon, clinicians in the ED obtain computed tomography (CT) or magnetic resonance imaging (MRI) scans for as many as 36% of these children (1). The overuse of CT exposes children to unnecessary radiation, with an estimated rate of inducing lethal malignancies between 1 in 1000 and 1 in 5000 CT scans (8,9). MRI, which has become increasingly utilized in US EDs, may require procedural sedation with its associated risks, including growing concerns that anesthetics and sedatives may be harmful to the developing brain (10–17). Finally, unnecessary neuroimaging may also identify inconsequential findings that lead to unnecessary testing and interventions, and unwarranted patient and parental concerns (18).
Importance
ED clinicians likely obtain emergent neuroimaging for children with headaches based on the presence of red flag findings, such as early-morning headaches or vomiting, that have been associated with the presence of emergent intracranial abnormalities (19–23). However, these red flag findings were identified primarily from retrospective studies of children already diagnosed with intracranial abnormalities (4,6,24–36). Little prospective data exists on the prevalence of red flag findings in well-defined ED populations of otherwise healthy children with headaches. If the prevalence of red flag findings being used to prompt ED neuroimaging is significantly greater than that of emergent intracranial abnormalities, and these red flag findings are associated with ED neuroimaging, there may be the need to reconsider how or which red flag findings are used for decision-making in order to reduce unnecessary neuroimaging in children.
Goals of this investigation
We aimed to determine the prevalence of red flag findings and the association between clinical variables and ED neuroimaging in otherwise healthy children presenting with headaches to the ED. Our secondary aim was to determine the prevalence of emergent intracranial abnormalities in a well-described population of children. Finally, we explored the reasons endorsed by physicians for obtaining ED neuroimaging for children with headaches.
Methods
Study design and setting
We conducted a prospective cohort study in an urban pediatric ED with an annual census of approximately 55,000 visits. The institutional review board approved this study with waiver of informed consent for collection of ED data. We obtained verbal consent for completion of telephone follow-up.
Selection of participants
From October 2015 to April 2017, we enrolled children 2 to 17 years of age. Children were eligible if they presented with a headache as their chief complaint, or if headache was identified on review of systems and was of concern to the clinician, such that a consultation by a neurologist was obtained or recommended, neuroimaging (cranial CT or MRI) was completed in the ED or recommended as an outpatient, or outpatient follow-up with a neurologist or primary care physician for headache evaluation was recommended.
We excluded children if they had a documented temperature of ≥38°C in the ED or at home; neuroimaging (cranial CT or MRI) prior to their index ED visit; an abnormal baseline neurological exam; or any history of intracranial surgery, structural abnormalities, or other chronic condition that was a clear risk factor for an intracranial abnormality including, but not limited to: sickle cell disease; immunocompromised state; history of, or current, neoplasm or any intracranial lesion or shunt; collagen vascular disease; coagulopathy or currently taking an anticoagulant; history of pseudotumor cerebri; known pregnancy. We also excluded patients with a head injury in the past 7 days or if previously enrolled in the study.
Procedures
A pediatric emergency medicine attending or fellow or a general pediatrician attending completed a standard case report form for each eligible patient (Supplemental Figure). Forms were completed prior to ED neuroimaging, if obtained. The decision to obtain ED neuroimaging was at the discretion of the attending physician caring for the patient. Patients for whom ED neuroimaging was not completed received a follow-up telephone call 4 months after their index ED visit to assess for emergent intracranial abnormalities. Parents were asked for details regarding any interim outpatient neuroimaging, neurosurgical intervention, interventional radiologic procedure, or medical intervention (e.g. chemotherapy, fibrinolytic therapy).
For patients who had neither ED neuroimaging nor telephone follow-up completed, we reviewed medical and radiological records 6 months after their index ED visit to evaluate for completion of any neuroimaging, neurosurgical interventions, interventional radiologic procedures, or medical interventions. Patients with emergent intracranial abnormalities identified during the 6-month review were included in determination of emergent intracranial abnormality prevalence. Patients were considered lost to follow-up if they did not undergo ED neuroimaging, the 4-month follow-up telephone call was incomplete, and no emergent intracranial abnormality was identified by the 6-month medical and radiological review. Enrolled patients lost to follow-up (i.e. no outcome determined) were not included in the determination of emergent intracranial abnormality prevalence.
We identified eligible but not enrolled (i.e. missed eligible) patients by reviewing the electronic health record, using ICD-9 and ICD-10 codes with a headache or headache-related diagnosis, and for diagnoses that were considered relevant intracranial findings (Supplemental Table 1). If eligible, data was collected to compare the missed eligible patients to those enrolled, including whether ED neuroimaging was obtained, neuroimaging results, and ED disposition.
ED data collected
Clinical findings from the history and physical were documented for each patient. These findings were chosen to include common headache-related variables and red flag findings described in prior literature and guidelines, and in consultation with two pediatric neurologists with specific expertise in childhood headaches (19–23). For patients who received neuroimaging, physicians were asked to document in free-text their reason(s) for ordering the test.
Outcomes
The primary outcome was completion of ED neuroimaging (cranial CT or MRI). The secondary outcome was the presence or absence of an emergent intracranial abnormality. A priori, we defined intracranial abnormalities based on an iterative consensus process involving a group of seven pediatric emergency medicine physicians, five pediatric neurosurgeons, and four pediatric neurologists from five geographically distinct tertiary pediatric hospitals across the US. Supplemental Table 1 describes the four clinically-oriented categories resulting from this consensus process: Emergent intracranial abnormalities; serious intracranial abnormalities; incidental intracranial abnormalities; and pseudotumor cerebri. Patients with documented sinus changes or disease were not categorized as having an intracranial finding.
An emergent intracranial abnormality was defined as an intracranial finding for which, if identified in the ED, one of the following interventions would be indicated at that time: a) A surgical intervention; b) a medical intervention; or c) hospital admission to monitor for potential clinical deterioration, to obtain additional diagnostic evaluation (e.g. imaging, biopsy), or to perform a planned intervention specifically related to the intracranial abnormality. In order for a diagnosis to be considered an emergent intracranial abnormality, >70% of the consensus working group had to classify the finding as emergent. For those patients who had no CT or MRI performed in the ED or within the 4-month follow-up period, we defined the absence of an emergent intracranial abnormality as not having a neurosurgical intervention, interventional radiologic procedure, or medical intervention (e.g. chemotherapy, fibrinolytic therapy) performed for an intracranial abnormality when assessed at 4 months after the index ED visit.
A serious intracranial abnormality was defined as an intracranial finding that is typically asymptomatic but has the potential to be the cause of a headache (depending on characteristics such as size or location of the finding) and, therefore, may potentially require emergent intervention. An incidental intracranial abnormality was defined as an intracranial finding that did not require immediate intervention, may or may not require outpatient follow up, and would be unlikely to be the cause of a headache. Although termed “incidental” in accordance with prior literature, these findings may still be concerning to patients and families (18). Supplemental Table 1 lists the specific findings on CT or MRI assigned to each clinically-oriented category.
Patients diagnosed with pseudotumor cerebri were classified as their own category because, even though its identification in the ED would prompt an intervention at that time, the diagnosis cannot typically be made by neuroimaging alone. Rather, opening pressures obtained by lumbar puncture are required, and since our patients would not be routinely receiving lumbar punctures and follow-up would not rule out the presence of pseudotumor cerebri, this diagnosis could not be reliably classified as an emergent intracranial abnormality.
Analyses
We used proportions to describe the prevalence of headache red flag findings. We calculated relative risks with 95% CIs and conducted standard bivariable analyses, appropriate for variable type and distribution, to identify clinical findings associated with patients receiving ED neuroimaging. In addition to assessing all findings individually, a priori we grouped specific findings from patient history and physical examination into clinically-relevant composite categories that reflected a specific disease (e.g. subarachnoid hemorrhage), a common underlying pathophysiology (e.g. positional symptoms), or that were conceptually similar (e.g. extreme pain intensity of presenting headache, change in headache characteristic). Individual red flag findings that were not related to other findings in these ways were analyzed on their own (e.g. occipital location of headache). Supplemental Table 2 shows the specific findings grouped into each category. No single finding was included in more than one category. We also created a category of “possible migraine headache” by using the higher-sensitivity version of the Irma Criteria (i.e. requiring three out of six migraine criteria) (37).
We conducted logistic regression analyses using the clinically-relevant categories of patient history and physical examination to identify factors independently associated with ordering ED neuroimaging. The categories were used instead of individual variables in order to accommodate a reasonable number of factors in the regression analyses. All variables analyzed were categorical. We conducted backward stepwise regression with confirmation using forward stepwise regression. We included variables that had plausible association with the outcome and had potential association (p<0.1) on bivariable analysis. We removed from the final model those variables with p>0.05. We assessed for multicollinearity of variables using variance inflation factors and model goodness-of-fit based on the Hosmer-Lemeshow test. We aimed to include in the models a maximum of one variable for every 10 patients who had ED neuroimaging. Finally, we summarized clinicians’ free-text reasons for obtaining CTs and used descriptive statistics to describe the results. We conducted all statistical analyses using SPSS (version 24; IBM Corporation, Armonk, NY).
Results
Characteristics of study subjects
Between October 2015 and April 2017, we enrolled 224 (65.9%) of 340 eligible children, whose characteristics are shown in Table 1. Figure 1 shows the number of patients screened, patients excluded, and missed eligible patients. The characteristics of the missed eligible population were similar to those of the study population (Supplemental Table 3). Twenty-five (11.2%) enrolled patients did not complete the 4-month follow-up telephone call. None of these 25 patients were identified with documented neurosurgical, interventional radiologic, or medical interventions on 6-month medical and radiological record review. Therefore, 25 (11.2%) enrolled patients were lost to follow-up, with no outcome determination. These patients were similar to the enrolled patients who had outcomes determined with regard to median age (11 years, IQR 8.5, 14.0) and sex (60% females).
Table 1.
Patient characteristics and clinical management (n=224).
| n (%) | |
|---|---|
| Age in years, median (IQR) | 12 (8, 15) |
| 2—4 years old, n (%)1 | 6 (2.7) |
| 2–5 years old, n (%)1 | 12 (5.4) |
| 2–6 years old, n (%)1 | 26 (11.6) |
| Female, n (%) | 118 (52.7) |
| Race/ethnicity, n (%) | |
| Hispanic or Latino | 170 (75.9) |
| Black or African-American | 21 (9.4) |
| White | 12 (5.4) |
| Asian | 2 (0.8) |
| More than one | 1 (0.4) |
| Don’t know | 3 (1.3) |
| Missing | 15 (6.7) |
| Primary language, n (%) | |
| English | 147 (65.6) |
| Spanish | 68 (30.4) |
| Missing | 9 (4) |
| Patient took an analgesic(s) for headache within 12 hours of presenting to ED, n (%)2 | 142 (63.4) |
| Non-steroidal anti-inflammatory drug | 91 (40.6) |
| Acetaminophen | 60 (26.8) |
| Triptan (e.g. sumatriptan) | 1 (0.4) |
| Metaclopramide or compazine | 0 |
| Other | 14 (6.3) |
| Received neuroimaging in emergency department, n (%) | |
| Any neuroimaging | 74 (33) |
| CT | 23 (10.3) |
| MRI | 51 (22.7) |
| Disposition, n (%) | |
| Home | 216 (96.4) |
| Admit, general inpatient service | 7(3.1) |
| Admit, PICU | 1 (0.4) |
| Received outpatient neuroimaging, n (%)3 | |
| No outpatient neuroimaging | 102 (81.6) |
| Any neuroimaging | 23 (18.4) |
| CT | 4 (3.2) |
| MRI | 17 (13.6) |
| Both CT and MRI | 1 (0.8) |
| Unsure of neuroimaging type | 1 (0.8) |
These age groupings were chosen to show the number of children that would be considered at higher risk of emergent intracranial abnormality due to young age, based on different age cutoffs that have been described in the literature.
Patients may have taken more than one type of analgesic.
Only includes patients who did not receive neuroimaging in the ED and completed follow up (n=125).
IQR: interquartile range; CT: computed tomography; MRI: magnetic resonance imaging; PICU: pediatric intensive care unit.
Figure 1.
Patient enrollment.
Main results
Tables 2–5 display the prevalence of individual red flag findings on history and physical examination as well as the prevalence of findings within the clinically-relevant composite categories. Supplemental Table 4 displays the prevalence of non-red flag headache-related findings. There were 197 (87.9%) patients with at least one red flag finding on history. Several red flag findings were reported by more than a third of children, including presenting and/or prior headaches waking from sleep (34.8%), presenting and/or prior headaches present or soon after waking in the morning (39.7%), and headaches increasing in frequency, duration or severity (40%, 33.1%, and 46.3%, respectively). The most common headache-related symptom was vomiting (31.7%), with early-morning vomiting and vomiting awakening the patient from sleep being much less common (5.8% and 4.9%, respectively). Table 3 notes that only 20 (8.9%) children had abnormal neurological exams. The most common abnormal neurological finding was papilledema, present in five (2.2%) children. However, the clinician was unsure of the presence of papilledema in 44 (19.6%) children.
Table 2.
Headache characteristics: Bivariable association with receiving ED neuroimaging.
| Characteristic | n (%) | Received ED neuroimaging if characteristic was present, n/N (%) | Received ED neuroimaging if characteristic was absent, n/N (%) | RR (95% CI) | Clinician “unsure” whether characteristic present, n (%) | Missing |
|---|---|---|---|---|---|---|
| Headache(s) wakes from sleep1 | 78 (34.8) | 44/78 (56.4) | 29/144 (20.1) | 2.8 (1.9, 4.1) | 2 (0.9) | 0 |
| Headache(s) on or soon after waking in the morning1 | 89 (39.7) | 44/89 (49.4) | 28/130 (21.5) | 2.3 (1.6, 3.4) | 4 (1.8) | 1 (0.4) |
| Improves over course of the day | 15 (6.7) | 8/15 (53.3) | 25/48 (52.1) | 1 (0.6, 1.8) | 0 | 28 (12.5) |
| Headache(s) precipitated/caused or worsened by lying down1 | 45 (20.1) | 21/45 (46.7) | 50/171 (29.2) | 1.6 (1.1, 2.4) | 7 (3.1) | 1 (0.4) |
| Headache(s) precipitated/caused or worsened by coughing, sneezing, straining, or any Valsalva maneuver1 | 39 (17.4) | 19/39 (48.7) | 47/169 (27.8) | 1.8 (1.2, 2.6) | 14 (6.7) | 1 (0.4) |
| Headache(s) precipitated/caused by exertion or physical activity1 | 50 (22.3) | 29/50 (58) | 38/156 (24.4) | 2.4 (1.7, 3.4) | 16 (7.1) | 2 (0.9) |
| Presenting headache reached maximal intensity in less than one minute | 46 (20.5) | 21/46 (45.7) | 43/146 (29.5) | 1.6 (1, 2.3) | 0 | 32 (14.3) |
| – Less than one minute, longer than one second | 18 (8) | 8/18 (44.4) | 57/186 (30.6) | 1.5 (0.8, 2.5) | 0 | 0 |
| – Less than one second | 11 (4.9) | 7/11 (63.6) | 58/193 (30.1) | 2.1 (1.3, 3.5) | 0 | 0 |
| Worst headache of patient’s life2 | 69 (43.1) | 32/69 (46.4) | 41/149 (27.5) | 1.7 (1.2, 2.4) | 5 (3.1) | 1 (0.6) |
| Severe pain intensity of presenting headache | 93 (41.5) | 40/93 (43) | 32/129 (24.8) | 1.7 (1.2, 2.5) | 0 | 0 |
| Occipital location of headache1 | 43 (19.2) | 13/43 (30.2) | 61/181 (33.7) | 0.9 (0.5, 1.5) | 0 | 0 |
| Chronic progressive temporal pattern2,3 | 22 (9.8) | 16/22 (72.7) | 51/163 (31.3) | 2.3 (1.7, 3.3) | 0 | 0 |
| Increasing headache:2 | ||||||
| –frequency | 64 (40) | 27/64 (42.2) | 46/152 (30.3) | 1.4 (1, 2) | 6 (3.8) | 2 (1.3) |
| –duration | 53 (33.1) | 29/53 (54.7) | 43/159 (27) | 2 (1.4, 2.9) | 7 (4.4) | 5 (3.1) |
| –severity | 74 (46.3) | 39/74 (52.7) | 34/141 (24.1) | 2.2 (1.5, 3.1) | 5 (3.1) | 4 (2.5) |
| Change in headache:2 | ||||||
| –location | 20 (12.5) | 10/20 (50) | 60/189 (31.7) | 1.6 (1, 2.6) | 13 (8.1) | 2 (1.3) |
| –quality/character | 21 (13.1) | 13/21 (61.9) | 56/183 (30.6) | 2 (1.4, 3) | 18 (11.3) | 2 (1.3) |
Presenting and/or prior headaches.
Only includes patients who have had more than one headache in their life (n=160).
Chronic progressive temporal pattern is defined as headaches that gradually increase in frequency and severity over time (21).
Table 5.
Clinically-relevant composite categories of variables from patient history and examination: Bivariable association with receiving ED neuroimaging.
| Category | n (%) | Received ED neuroimaging if finding was present, n/N (%) | Received ED neuroimaging if finding was absent, n/N (%) | RR (95% CI) | Unsure or missing2 |
|---|---|---|---|---|---|
| Positional symptoms1* | 136 (60.7) | 65/136 (47.8) | 9/88 (10.2) | 4.7 (2.5, 8.9) | 0 |
| Abnormal neurological exam finding (other than altered mental status) * | 20 (8.9) | 16/20 (80) | 58/202 (28.7) | 2.8 (2.0, 3.8) | 2 (0.9) |
| Extreme pain intensity of presenting headache* | 126 (56.3) | 54/126 (42.9) | 17/92 (18.5) | 2.3 (1.4, 3.7) | 6 (2.7) |
| Vomiting (any) * | 71 (31.7) | 37/71 (52.1) | 37/153 (24.2) | 2.2 (1.5, 3.1) | 0 |
| Valsalva- or exertion-related symptoms* | 64 (28.6) | 31/64 (48.4) | 33/136 (24.3) | 2.0 (1.4, 2.9) | 24 (10.7) |
| Neurological or neurocognitive dysfunction* | 68 (30.4) | 33/68 (48.5) | 41/156 (26.3) | 1.8 (1.3, 2.6) | 0 |
| Change in headache characteristics* | 100 (44.6) | 42/100 (42) | 31/115 (27) | 1.6 (1.1, 2.3) | 9 (4.0) |
| Symptoms concerning for subarachnoid hemorrhage* | 72 (32.1) | 31/72 (43.1) | 43/152 (28.3) | 1.5 (1.1, 2.2) | 0 |
| Young age (i.e. 2–6 years old) | 26 (11.6) | 9/26 (34.6) | 65/133 (32.8) | 1.1 (0.6, 1.9) | 0 |
| Occipital location of headache | 43 (19.2) | 13/43 (30.2) | 61/181 (33.7) | 0.9 (0.5, 1.5) | 0 |
| Abnormal physical exam finding | 7 (3.1) | 1/7 (14.3) | 68/205 (33.2) | 0.4 (0.1, 2.7) | 12 (5.4) |
A symptom such as headache or vomiting that changes or is associated with a change in position, or prolonged supine position.
Unsure and missing variables from individual findings were combined when categories created, and all analyzed as missing.
p<0.05.
Table 3.
Headache-related symptoms and characteristics: Bivariable association with receiving ED neuroimaging.
| Symptom or characteristic | n (%) | Received ED neuroimaging if symptom or characteristic was present, n/N (%) | Received ED neuroimaging if symptom or characteristic was absent, n/N (%) | RR (95% CI) | Clinician “unsure” whether symptom or characteristic present, n (%) | Missing |
|---|---|---|---|---|---|---|
| Age 2–6 years old | 26 (11.6) | 9/26 (34.6) | 65/133 (32.8) | 1.1 (0.6, 1.9) | 0 | 0 |
| Vomiting1 | 71 (31.7) | 37/71 (52.1) | 37/153 (24.2) | 2.2 (1.5, 3.1) | 0 | 0 |
| Vomiting with, or soon after, waking in the morning1 | 13 (5.8) | 12/13 (92.3) | 59/207 (28.5) | 3.2 (2.4, 4.2) | 4 (1.8) | 0 |
| Vomiting awakens patient from sleep1 | 11 (4.9) | 6/11 (54.5) | 66/210 (31.4) | 1.7 (1, 3.1) | 3(1.3) | 0 |
| Neck pain | 29 (12.9) | 12/29 (41.4) | 62/195(31.8) | 1.3 (0.8, 2.1) | 0 | 0 |
| Neck stiffness | 7 (3.1) | 3/7 (42.9) | 71/217 (32.7) | 1.3 (0.5, 3.1) | 0 | 0 |
| Focal motor weakness2 | 9 (4) | 7/9 (77.8) | 67/148 (31.2) | 2.5 (1.7, 3.7) | 0 | 0 |
| Sensory changes2 | 21 (9.4) | 12/21 (57.1) | 62/203 (30.5) | 1.9 (1.2, 2.9) | 0 | 0 |
| Unsteadiness2 | 40 (17.9) | 22/40 (55) | 50/178 (28.1) | 2 (1.4, 2.8) | 3 (1.3) | 3 (1.3) |
| Abnormal movements, non-seizure2 | 12 (5.4) | 8/12 (66.7) | 66/212 (31.1) | 2.1 (1.4, 3.3) | 0 | 0 |
| Seizures2 | 0 | 0 | 0 | |||
| Episodes of confusion2 | 8 (3.6) | 4/8 (50) | 70/216 (32.4) | 1.5 (0.8, 3.2) | 0 | 0 |
| Double vision2 | 18 (8) | 9/18 (50) | 43/81 (34.7) | 1.4 (0.9, 2.4) | 6 (2.7) | 76 (33.9)3 |
| Brief or transient (“a few seconds”) episodes of vision loss2 | 16(7.1) | 7/16 (43.8) | 42/124 (33.9) | 1.3 (0.7, 2.4) | 7(3.1) | 77 (34.4)3 |
| -Occurs with coughing, sneezing, straining, any Valsalva maneuver, or bending over | 5 (2.2) | 1/5 (20) | 2/7 (28.6) | 0.7 (0.1, 5.8) | 0 | 4 (1.8) |
| Hearing problems2 | 4 (1.8) | 1/4 (25) | 73/220 (33.2) | 0.7 (0.1, 4.2) | 0 | 0 |
| Declining school performance2 | 5 (2.2) | 4/5 (80) | 70/149 (32) | 2.5 (1.6, 4) | 0 | 0 |
| Behavior, mood, or personality change2 | 10 (4.5) | 6/10 (60) | 68/214 (31.8) | 1.9 (1.1, 3.2) | 0 | 0 |
Associated with presenting and/or prior headaches.
Within the past 6 months of index ED visit.
This variable was not collected until starting with patient 74.
There were 74 (33%) children with headaches who received ED neuroimaging, with 68.9% of studies obtained being MRIs. Of the 197 patients with at least one red flag finding on history, 73 (37.6%) received ED neuroimaging. Of the 27 patients with no red flag findings on history, only one received ED neuroimaging. Twenty patients had an abnormal neurological exam and 16 (80%) of them received ED neuroimaging. There were 58 (25.9%) patients with no abnormal neurological findings who received ED neuroimaging; all of these patients had at least one red flag finding on history.
On logistic regression analysis, several categories of red flag findings were independently associated with the conduct of ED neuroimaging, including: abnormal neurological exam, extreme pain intensity of presenting headache, vomiting (any), and positional symptoms (Table 6). Of patients with any red flag findings within these four categories, 74 (40.9%) received ED neuroimaging. A presenting headache duration of 1 day or less decreased the likelihood of receiving ED neuroimaging. The results were similar for both backward and forward stepwise regression. Variance inflation factors for all variables were <2. The Hosmer-Lemeshow test showed no evidence of poor fit (p=0.42)
Table 6.
Clinically-relevant composite categories of variables from patient history and examination independently associated with ED neuroimaging (p<0.05)1.
| Category | n (%) | Adjusted OR (95% CI) |
|---|---|---|
| Abnormal neurological exam | 20 (8.9) | 7.4 (1.7, 33.2) |
| Extreme pain intensity of presenting headache | 126 (56.3) | 5.5 (2.1, 14.4) |
| Vomiting (any) | 71 (31.7) | 4.6 (1.9, 10.7) |
| Positional symptoms2 | 136 (60.7) | 3.4 (1.4, 8.7) |
| Duration of current headache ≤1 day | 76 (33.9) | 0.3 (0.1, 0.7) |
Categories analyzed in the model were: Abnormal neurological exam finding (other than altered mental status); neurological or neurocognitive dysfunction; change in headache characteristics; extreme pain intensity of presenting headache; positional symptoms; symptoms concerning for subarachnoid hemorrhage; Valsalva- or exertion-related symptoms; vomiting (any); headache duration ≤1 day; and headache duration >1 week (see Supplemental Table 2 for specific red flag findings in each category).
A symptom such as headache or vomiting that changes or is associated with a change in position, or prolonged supine position.
Two (1%, 95% CI 0.1, 3.6) patients were diagnosed with emergent intracranial abnormalities; their findings are detailed in Supplemental Table 5. One of the patients (Patient #2) was diagnosed with a complex migraine headache at her index ED visit and did not receive emergent neuroimaging. She was discharged with a plan to attend outpatient neurology follow-up and neuroimaging. She returned to the ED prior to her appointments with a focal seizure, with neuroimaging at that time identifying the lesion. There were three patients (1.5%) with serious intracranial abnormalities, 14 (7%) with incidental intracranial abnormalities, and two (1%) with pseudotumor cerebri (Supplemental Table 6).
Of the 74 children who underwent ED neuroimaging, physicians documented their reasons for ordering emergent neuroimaging for 55 (Supplemental Table 7). Fifty-two of the 55 (94.5%) underwent ED neuroimaging because of a concerning history, with the most common reasons being headaches that awoke the patient from sleep (46.1%), headaches present in the morning or with waking (26.9%) or headaches of increasing frequency (19.2%).
Discussion
In this prospective study, previously identified red flag findings were present in the majority of children assessed in the ED for headaches and were associated with obtaining ED neuroimaging. The frequency of red flag findings was in contrast to the rarity of emergent intracranial abnormalities in this well-described population. Our findings highlight the need to more clearly understand how or which red flag findings should be used to decide whether emergent neuroimaging is indicated.
Red flag findings are the foundation of existing guidelines and recommendations regarding the use of neuroimaging. However, there is no clear consensus on which findings should be used for decision-making, resulting in a large number of findings proposed to be concerning enough to warrant emergent neuroimaging. For instance, authoritative groups such as the American Academy of Neurology, the Child Neurology Society, and the American College of Radiology have recommended neuroimaging for a host of findings, including: The presence of an abnormal neurological exam; concurrent seizures; recent or sudden onset of severe headache; change in type of headache, including increasing frequency, duration and intensity; headaches with associated neurological dysfunction; headaches that awaken a child from sleep or occur on arising; intense, prolonged, and incapacitating headaches with an absent family history for migraine; and vomiting (19,20). Other recommendations include obtaining neuroimaging for children younger than 6 years old, progression of symptoms for less than 2 months, an inability to describe the quality of pain, headaches of occipital location, recent or sudden onset (i.e. thunderclap) or that are of a concerning pattern (i.e. precipitated by Valsalva maneuver, positional, progressive), or if there is no family history of a primary headache disorder (21–23). Given the high prevalence in our study of children having at least one of these findings, the substantial proportion that had ED neuroimaging might be expected. Specific red flag findings (such as headache that wakes from sleep) were also commonly cited as reasons for ordering ED neuroimaging.
The presence of an abnormal neurological examination is also considered a red flag finding in a child with headache, and an indication for emergent neuroimaging (19–23). Although a normal neurological exam was common in our population, it did not appear reassuring enough for clinicians to forego ED neuroimaging when other red flag findings were present on history. One potential explanation could be clinician concern that abnormal neurological findings in a child with an emergent intracranial abnormality might be a later development, following the initial endorsement of headache and red flag findings on history (33,38). Additionally, clinicians were frequently “unsure” of the presence or absence of important exam findings, such as papilledema. The high rates of unsure status may also reflect the challenge faced in an ED setting, when children with headaches may be too symptomatic or too young to comply with a complete neurological exam.
Interestingly, the prevalence of emergent intracranial abnormalities we observed (1%) appears on initial review to be lower than the 3% to 8.1% reported in prior studies of children presenting to EDs with non-traumatic headaches (3,4,6,7,39,40). However, prior studies often failed to exclude patients with fever, known intracranial abnormalities (e.g. VP shunts), or risk factors for intracranial abnormalities. These studies also did not differentiate their findings into categories such as emergent intracranial abnormality, serious intracranial abnormalities, incidental intracranial abnormalities, and pseudotumor cerebri, which are important distinctions to front-line clinicians. Using our definition of emergent intracranial abnormalities, the prevalence of emergent intracranial abnormalities in all but one of these studies decreased to 0.3% to 1% (3,6,7,39,40). The one differing study did not state whether patients with underlying risk factors or prior intracranial abnormalities were excluded (4).
Our results strongly suggest the need to conduct large prospective studies with sufficient numbers of patients to enable accurate stratification of the risk of emergent intracranial abnormalities based on specific key findings from patient history and physical examination. The association between clinical findings and emergent intracranial abnormalities in children with headaches has mainly been derived from retrospective studies of children with known intracranial abnormalities (26,27,32,33,41–43). The relative lack of prospective studies of sufficient rigor and sample size was recognized more than a decade ago by the American Academy of Neurology and the Child Neurology Society, and more recently by the Society for Academic Emergency Medicine (19,44).
Finally, the use of neuroimaging in our study was at the higher end of the 6.3% to 41% range noted in prior studies of children evaluated in both general and pediatric EDs with headaches (1–3,5–7,39,45,46). One possible reason for the broad range of estimates is the different inclusion and exclusion criteria among the studies. This variability in ED neuroimaging could also reflect specific institutional protocols, local variations in practice, or differences in availability of resources for outpatient follow-up or neuroimaging.
Limitations
Our study was a convenience sample and had a substantial proportion of missed eligible patients. However, the missed population had similar characteristics to those enrolled. Although the use of 4-month telephone follow-up likely captured the patients with emergent intracranial abnormalities, it would not necessarily identify the presence of serious intracranial abnormalities and pseudotumor cerebri if the patient did not receive any neuroimaging or further investigations (e.g. lumbar puncture). Additionally, we did not collect certain variables that may be considered red flag findings, such as witnessed loss of consciousness or diz-ziness. We also did not have enough patients who received ED neuroimaging to be able to analyze each red flag finding individually, although we were able to identify categories of findings associated with imaging practices. We did not ask clinicians whether or not they felt that completing the case report forms influenced their decision to obtain neuroimaging, although the prevalence of ED neuroimaging observed in our study was similar to that reported in prior ED settings (1,3,5). We also did not evaluate the interrater reliability of the variables collected. However, many similar variables obtained from history and some from physical exam in children with seizures and closed head injury have been shown to have at least moderate reliability (47,48). Finally, we did not collect any information about the clinicians evaluating the children. Clinician characteristics, such as years in practice, could potentially be related to decisions to obtain neuroimaging. Of note, the majority of clinicians who evaluate children in our emergency department are board-certified in pediatric emergency medicine.
Conclusions
Red flag findings are common in otherwise healthy children presenting with headaches to the ED. The presence of red flag findings is associated with ED neuroimaging, although the risk of emergent intracranial abnormalities is low. Our data suggest that many children with headaches receive unnecessary neuroimaging due to the high prevalence of non-specific red flag findings. In order to optimize the use of ED neuroimaging, further study is necessary to determine which clinical findings most accurately identify children with headaches who are at high and low risk of emergent intracranial abnormalities.
Supplementary Material
Table 4.
General physical and neurological examination findings: Bivariable association with receiving ED neuroimaging.
| Finding | n (%) | Received ED neuroimaging if finding was present, n/N (%) | Received ED neuroimaging if finding was absent, n/N (%) | RR (95% CI) | Clinician “unsure” whether finding present, n (%) | Missing |
|---|---|---|---|---|---|---|
| General physical examination findings | ||||||
| Ill-appearing general appearance1 | 0 | 0 | 73 (32.6) | |||
| Altered mental status1 | 0 | 0 | 74 (33) | |||
| Neck stiffness | 2 (0.9) | 0/2 (0) | 72/218 (33) | 0.5 (0, 6.4) | 0 | 4 (1.8) |
| Neurocutaneous findings | 5 (2.2) | 1/5 (20) | 69/208 (33.2) | 0.6 (0.1, 3.5) | 5 (2.2) | 6 (2.7) |
| Neurological examination findings | ||||||
| Head tilt | 2 (0.9) | 0/2 (0) | 71/215 (33) | 0.5 (0, 6.4) | 2 (0.9) | 5 (2.2) |
| Abnormal speech | 0 | 2 (0.9) | 5 (2.2) | |||
| Cranial nerve abnormality | 2 (0.9) | 2/2 (100) | 72/216 (33.3) | 3 (2.5, 3.6) | 5 (2.2) | 1 (0.4) |
| Extraocular movements abnormal | 1 (0.4) | 0/1 (0) | 74/222 (33.3) | 0.7 (0.1, 8.3) | 0 | 1 (0.4) |
| Pupillary reactivity abnormal | 0 | 2 (0.9) | 1 (0.4) | |||
| Nystagmus present | 0 | 2 (0.9) | 2 (0.9) | |||
| Papilledema present | 5 (2.2) | 5/5 (100) | 52/167 (31.1) | 3.2 (2.6, 4) | 44 (19.6) | 8 (3.6) |
| Visual fields abnormal | 1 (0.4) | 1/1 (100) | 43/143 (30.1) | 3.3 (2.6, 4.3) | 67 (29.9) | 13 (5.8) |
| Focal motor function abnormality | 2 (0.9) | 2/2 (100) | 72/221 (32.6) | 3.1 (2.5, 3.7) | 0 | 1 (0.4) |
| Focal sensory function abnormality | 2 (0.9) | 2/2 (100) | 69/207 (33.3) | 3 (2.5, 3.6) | 13 (5.8) | 2 (0.9) |
| Deep tendon reflexes abnormal | 1 (0.4) | 1/1 (100) | 66/196 (33.7) | 3 (2.4, 3.6) | 22 (9.8) | 5 (2.2) |
| Babinski reflex present | 1 (0.4) | 1/1 (100) | 62/165 (37.6) | 2.7 (2.2, 3.2) | 50 (22.3) | 8 (3.6) |
| Gait abnormal | 2 (0.9) | 2/2 (100) | 65/213 (30.5) | 3.3 (2.7, 4) | 7 (3.1) | 2 (0.9) |
| Tandem gait abnormal | 4(1.8) | 4/4 (100) | 56/187 (29.9) | 3.3 (2.7, 4.2) | 26 (11.6) | 7 (3.1) |
| Pronator drift present | 0 | 22 (9.8) | 6 (2.7) | |||
| Romberg test positive | 4 (1.8) | 3/4 (75) | 57/191 (29.8) | 2.5 (1.4, 4.6) | 23 (10.3) | 6 (2.7) |
| Dysmetria present | 1 (0.4) | 1/1 (100) | 64/202 (31.7) | 3.2 (2.6, 3.9) | 18 (8) | 3 (1.3) |
This variable was not collected until starting with patient 74.
Clinical implications.
Red flag findings are common in children presenting with headaches to the ED.
The presence of red flag findings is associated with ED neuroimaging, although the risk of emergent intracranial abnormalities is low.
Many children with headaches may be receiving unnecessary neuroimaging due to the high prevalence of non-specific red flag findings.
Acknowledgements
The authors would like to thank all of their colleagues who completed the case report forms; Dr. Nathan Kuppermann for his mentorship and oversight in the development of this project; Drs. Robert Fryer and Shannon Babineau for their assistance in developing the case report form; Drs. Nathan Kuppermann, Robert Hickey, Elizabeth Powell, Lynn Babcock and Steven Chan for their assistance in defining and classifying intracranial abnormalities; and Leonor Suarez and Christine Sison for their assistance with the study.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: National Center for Advancing Translational Sciences, (Grant/Award Number: ‘UL1 TR000040’)
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- 1.Sheridan DC, Meckler GD, Spiro DM, et al. Diagnostic testing and treatment of pediatric headache in the emergency department. J Pediatr 2013; 163: 1634–1637. [DOI] [PubMed] [Google Scholar]
- 2.Burton LJ, Quinn B, Pratt-Cheney JL, et al. Headache etiology in a pediatric emergency department. Pediatr Emerg Care 1997; 13: 1–4. [DOI] [PubMed] [Google Scholar]
- 3.Kan L, Nagelberg J and Maytal J. Headaches in a pediatric emergency department: Etiology, imaging, and treatment. Headache 2000; 40: 25–29. [DOI] [PubMed] [Google Scholar]
- 4.Lewis DW and Qureshi F. Acute headache in children and adolescents presenting to the emergency department. Headache 2000; 40: 200–203. [DOI] [PubMed] [Google Scholar]
- 5.Richer L, Graham L, Klassen T, et al. Emergency department management of acute migraine in children in Canada: A practice variation study. Headache 2007; 47: 703–710. [DOI] [PubMed] [Google Scholar]
- 6.Conicella E, Raucci U, Vanacore N, et al. The child with headache in a pediatric emergency department. Headache 2008; 48: 1005–1011. [DOI] [PubMed] [Google Scholar]
- 7.Scagni P and Pagliero R. Headache in an Italian pediatric emergency department. J Headache Pain 2008; 9: 83–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Brenner D, Elliston C, Hall E, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. Am J Roentgenol 2001; 176: 289–296. [DOI] [PubMed] [Google Scholar]
- 9.Brenner DJ. Estimating cancer risks from pediatric CT: going from the qualitative to the quantitative. Pediatr Radiol 2002; 32: 228–221, discussion 242–244. [DOI] [PubMed] [Google Scholar]
- 10.Coté CJ, Notterman DA, Karl HW, et al. Adverse sedation events in pediatrics: A critical incident analysis of contributing factors. Pediatrics 2000; 105: 805–814. [DOI] [PubMed] [Google Scholar]
- 11.Coté CJ, Karl HW, Notterman DA, et al. Adverse sedation events in pediatrics: Analysis of medications used for sedation. Pediatrics 2000; 106: 633–644. [DOI] [PubMed] [Google Scholar]
- 12.Ing C, DiMaggio C, Whitehouse A, et al. Long-term differences in language and cognitive function after childhood exposure to anesthesia. Pediatrics 2012; 130: e476–e485. [DOI] [PubMed] [Google Scholar]
- 13.Flick RP, Katusic SK, Colligan RC, et al. Cognitive and behavioral outcomes after early exposure to anesthesia and surgery. Pediatrics 2011; 128: e1053–e1061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wilder RT, Flick RP, Sprung J, et al. Early exposure to anesthesia and learning disabilities in a population-based birth cohort. Anesthesiology 2009; 110: 796–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ikonomidou C, Bosch F, Miksa M, et al. Blockade of NMDA receptors and apoptotic neurodegeneration in the developing brain. Science 1999; 283: 70–74. [DOI] [PubMed] [Google Scholar]
- 16.Ramsay JG and Rappaport BA. SmartTots: A multidisciplinary effort to determine anesthetic safety in young children. Anesth Analg 2011; 113: 963–964. [DOI] [PubMed] [Google Scholar]
- 17.Scheinfeld MH, Moon J-Y, Fagan MJ, et al. MRI usage in a pediatric emergency department: An analysis of usage and usage trends over 5 years. Pediatr Radiol 2017; 47: 327–332. [DOI] [PubMed] [Google Scholar]
- 18.Rogers AJ, Maher CO, Schunk JE, et al. Incidental findings in children with blunt head trauma evaluated with cranial CT scans. Pediatrics 2013; 132: e356–e363. [DOI] [PubMed] [Google Scholar]
- 19.Lewis DW, Ashwal S, Dahl G, et al. Practice parameter: Evaluation of children and adolescents with recurrent headaches: Report of the Quality Standards Subcommittee of the American Academy of Neurology and the Practice Committee of the Child Neurology Society. Neurology 2002; 59: 490–498. [DOI] [PubMed] [Google Scholar]
- 20.Strain JD. ACR Appropriateness Criteria on headache-child. J Am Coll Radiol 2007; 4: 18–23. [DOI] [PubMed] [Google Scholar]
- 21.Lewis DW and Koch T. Headache evaluation in children and adolescents: When to worry? When to scan? Pediatr Ann 2010; 39: 399–406. [DOI] [PubMed] [Google Scholar]
- 22.Gofshteyn JS and Stephenson DJ. Diagnosis and management of childhood headache. Curr Probl Pediatr Adolesc Health Care 2016; 46: 36–51. [DOI] [PubMed] [Google Scholar]
- 23.Kabbouche MA and Cleves C. Evaluation and management of children and adolescents presenting with an acute setting. Semin Pediatr Neurol 2010; 17: 105–108. [DOI] [PubMed] [Google Scholar]
- 24.Ahmed MAS, Martinez A, Cahill D, et al. When to image neurologically normal children with headaches: Development of a decision rule. Acta Paediatr 2010; 99: 940–943. [DOI] [PubMed] [Google Scholar]
- 25.Rho Y-I, Chung H-J, Suh E-S, et al. The role of neuroimaging in children and adolescents with recurrent headaches – multicenter study. Headache 2011; 51: 403–408. [DOI] [PubMed] [Google Scholar]
- 26.Lanphear J and Sarnaik S. Presenting symptoms of pediatric brain tumors diagnosed in the emergency department. Pediatr Emerg Care 2014; 30: 77–80. [DOI] [PubMed] [Google Scholar]
- 27.Medina LS, Pinter JD, Zurakowski D, et al. Children with headache: Clinical predictors of surgical space-occupying lesions and the role of neuroimaging. Radiology 1997; 202: 819–824. [DOI] [PubMed] [Google Scholar]
- 28.Hsiao H-J, Huang J-L, Hsia S-H, et al. Headache in the pediatric emergency service: A medical center experience. Pediatr Neonatol 2014; 55: 208–212. [DOI] [PubMed] [Google Scholar]
- 29.Ansell P, Johnston T, Simpson J, et al. Brain tumor signs and symptoms: Analysis of primary health care records from the UKCCS. Pediatrics 2010; 125: 112–119. [DOI] [PubMed] [Google Scholar]
- 30.Kundra M, Stankovic C, Gupta N, et al. Epidemiologic findings of cancer detected in a pediatric emergency department. Clin Pediatr (Phila) 2009; 48: 404–409. [DOI] [PubMed] [Google Scholar]
- 31.Reulecke BC, Erker CG, Fiedler BJ, et al. Brain tumors in children: Initial symptoms and their influence on the time span between symptom onset and diagnosis. J Child Neurol 2008; 23: 178–183. [DOI] [PubMed] [Google Scholar]
- 32.Maytal J, Bienkowski RS, Patel M, et al. The value of brain imaging in children with headaches. Pediatrics 1995; 96: 413–416. [PubMed] [Google Scholar]
- 33.Honig PJ and Charney EB. Children with brain tumor headaches. Distinguishing features. Am J Dis Child 1982; 136: 121–124. [DOI] [PubMed] [Google Scholar]
- 34.Wilne SH, Ferris RC, Nathwani A, et al. The presenting features of brain tumours: A review of 200 cases. Arch Dis Child 2006; 91: 502–506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Bear JJ, Gelfand AA, Goadsby PJ, et al. Occipital headaches and neuroimaging in children. Neurology 2017; 89: 469–474. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Genizi J, Khourieh-Matar A, Assaf N, et al. Occipital headaches in children: Are they a red flag? J Child Neurol 2017; 32: 942–946. [DOI] [PubMed] [Google Scholar]
- 37.Trottier ED, Bailey B, Lucas N, et al. Diagnosis of migraine in the pediatric emergency department. Pediatr Neurol 2013; 49: 40–45. [DOI] [PubMed] [Google Scholar]
- 38.Wilne S, Collier J, Kennedy C, et al. Progression from first symptom to diagnosis in childhood brain tumours. Eur J Pediatr 2012; 171: 87–93. [DOI] [PubMed] [Google Scholar]
- 39.Lateef TM, Grewal M, McClintock W, et al. Headache in young children in the emergency department: Use of computed tomography. Pediatrics 2009; 124: e12–e17. [DOI] [PubMed] [Google Scholar]
- 40.Massano D, Julliand S, Kanagarajah L, et al. Headache with focal neurologic signs in children at the emergency department. J Pediatr 2014; 165: 376–382. [DOI] [PubMed] [Google Scholar]
- 41.The epidemiology of headache among children with brain tumor. Headache in children with brain tumors. The Childhood Brain Tumor Consortium. J Neurooncol 1991; 10: 31–46. [DOI] [PubMed] [Google Scholar]
- 42.Chu ML and Shinnar S. Headaches in children younger than 7 years of age. Arch Neurol 1992; 49: 79–82. [DOI] [PubMed] [Google Scholar]
- 43.Lewis DW and Dorbad D. The utility of neuroimaging in the evaluation of children with migraine or chronic daily headache who have normal neurological examinations. Headache 2000; 40: 629–632. [DOI] [PubMed] [Google Scholar]
- 44.Finnerty NM, Rodriguez RM, Carpenter CR, et al. Clinical decision rules for diagnostic imaging in the emergency department: A research agenda. Acad Emerg Med 2015; 22: 1406–1416. [DOI] [PubMed] [Google Scholar]
- 45.Eapen A, Sivaswamy L, Agarwal R, et al. Management of pediatric migraine in a tertiary care versus community based emergency department: An observational pilot study. Pediatr Neurol 2014; 50: 164–170. [DOI] [PubMed] [Google Scholar]
- 46.Rossi R, Versace A, Lauria B, et al. Headache in the pediatric emergency department: A 5-year retrospective study. Cephalalgia 2018; 38: 1765–1772. [DOI] [PubMed] [Google Scholar]
- 47.Dayan PS, Lillis K, Bennett J, et al. Interobserver agreement in the assessment of clinical findings in children with first unprovoked seizures. Pediatrics 2011; 127: e1266–e1271. [DOI] [PubMed] [Google Scholar]
- 48.Gorelick MH, Atabaki SM, Hoyle J, et al. Interobserver agreement in assessment of clinical variables in children with blunt head trauma. Acad Emerg Med 2008; 15: 812–818. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.