Abstract
Context:
Self-reported symptoms (SRS) scales comprise one aspect of a multifaceted assessment of sport-related concussion. Obtaining SRS assessments before a concussion occurs assists in determining when the injury is resolved. However, athletes may present with concussion-related symptoms at baseline. Thus, it is important to evaluate such reports to determine if the variables that are common to many athletic environments are influencing them.
Objective:
To evaluate the influence of a history of concussion, sex, acute fatigue, physical illness, and orthopaedic injury on baseline responses to 2 summative symptom scales; to investigate the psychometric properties of all responses; and to assess the factorial validity of responses to both scales in the absence of influential variables.
Design:
Cross-sectional study.
Setting:
Athletic training facilities of 6 National Collegiate Athletic Association institutions.
Patients or Other Participants:
The sample of 1065 was predominately male (n = 805) collegiate athletes with a mean age of 19.81 ± 1.53 years.
Main Outcome Measure(s):
Participants completed baseline measures for duration and severity of concussion-related SRS and a brief health questionnaire.
Results:
At baseline, respondents reporting a previous concussion had higher composite scores on both scales (P ≤ .01), but no sex differences were found for concussion-related symptoms. Acute fatigue, physical illness, and orthopaedic injury increased composite SRS scores on both duration and severity measures (P ≤ .01). Responses to both scales were stable and internally consistent. Confirmatory factor analysis provided strong evidence for the factorial validity of the responses of participants reporting no fatigue, physical illness, or orthopaedic injury on each instrument.
Conclusions:
A history of concussion, acute fatigue, physical illness, and orthopaedic injury increased baseline SRS scores. These conditions need to be thoroughly investigated and controlled by clinicians before baseline SRS measures are collected.
Keywords: baseline evaluation, factorial validity, Postconcussion Symptom Scale
Key Points.
Within the multifaceted concussion assessment protocol, clinicians should use a standardized self-reported symptoms summative scale that provides reliable, valid responses.
Variables such as a previous concussion, fatigue, physical illness, and orthopaedic injury can alter normal baseline responses.
If the athlete is currently experiencing fatigue, physical illness, or orthopaedic injury, baseline testing should be postponed until the athlete improves.
An athlete with a history of concussion or an elevated self-reported symptoms score should be thoroughly evaluated.
Self-reported symptoms (SRS) are a necessary component of any clinical examination. Symptoms are the basis for extending the diagnostic process, but using SRS as the only diagnostic measure after sport-related concussion is problematic. Thus, sports medicine clinicians have been encouraged to incorporate a multifaceted approach toward injury assessment and to use caution when interpreting responses to SRS scales.1–5 The need for caution stems from the bias that may be present among athletes who are typically motivated to return to play and may feign the absence of SRS.6 Additionally, postconcussion symptoms are by no means unique to concussive injuries. In other words, some postconcussive symptoms occur in persons who have not sustained concussions, rendering the specificity of alleged postconcussive symptoms suspect.7–11 Therefore, clinicians must be able to differentiate normal baseline rates of concussion-like symptoms from those directly attributable to an acute concussive incident.
Summative SRS scales typically use Likert-type scaling and, as a result, offer clinicians more information than is typically obtained with simple dichotomous measures (yes, symptom is experienced versus no, symptom is not experienced).1,9 Capturing information beyond symptom presence allows changes in SRS, including symptom duration, severity, and intensity, to be better categorized and understood. Measuring symptom duration, severity, and intensity at baseline and after injury is the most optimal method for dealing with bias related to using SRS.1,9 Even when many of these aforementioned measurement concerns are left uncontrolled or are inconsistent, a recently published meta-analysis2 supports the continued use of summative SRS scales as part of the multifaceted approach to concussion assessment and management. Hence, given the beneficial information provided by the responses to summative SRS scales, it becomes crucial to discuss and investigate those variables that may cause unwanted variability in SRS scores.
Signs and symptoms related to changes in cognition, memory, reaction time, attention, information processing, balance, and SRS all serve as potential markers of the concussive injury.12–24 Measuring each of these markers can provide sports medicine clinicians with information needed to make safe return-to-play decisions. With regard to SRS, summative scales measuring 24-hour symptom duration or severity are the most commonly applied instruments, but knowledge about the potentially confounding effects of acute medical conditions on scores derived from these measures is lacking.17,22,23,25–27
Therefore, the primary purpose of our study was to evaluate the influence of a history of concussion, sex, fatigue, physical illness, and orthopaedic injury on baseline responses to 2 summative, concussion-related symptom scales that have been demonstrated to provide reliable and valid responses.22,23 Our secondary purpose was to investigate the psychometric properties of both scales to extend data on scale consistency and validity. The final purpose was to remove any variables related to the inflation of composite scores and assess model-data fit to evaluate the factorial validity and structural integrity of responses to both scales.
METHODS
Participants
Participants in the cross-sectional analysis (N = 1065) were sports participants in National Collegiate Athletic Association institutions (Table 1). Volunteers ranged from 18 to 27 years of age (mean = 19.81 ± 1.53 years) and were predominantly male (n = 805). All participants signed an informed consent document, and the study was approved by the institutional review board.
Table 1.
Instrumentation
The 9-item Head Injury Scale (HIS) is a summative 7-point Likert-type scale instrument designed to measure the overall duration (length of symptom experienced over a 24-hour period) of concussion-related symptoms.21,22 Two-day test-retest reliability was established with a subsample of randomly selected participants (n = 80). The 9-item HIS stability was R = 0.85. Both administrations were demonstrated to be internally consistent, with the Cronbach α estimates greater than 0.94. Instructions to the respondent read, “Here is a list of symptoms that people often feel when they have a concussion. After reading each symptom please circle the number that best describes how long you have experienced the symptom during the previous 24-hour period (today).” Items were rated on a 7-point Likert-type scale with the response options ranging from 0 to 6: 0, I have never experienced this symptom; 1, I have experienced this symptom very briefly today; 3, I have experienced this symptom sometimes during today (about half the day long); 6, I have always experienced this symptom today (all day long). Item responses were then summed to provide an overall composite score.
The 9-item severity scale was designed to represent the 2 scales commonly used in the literature (Postconcussion Symptom Scale and Graded Symptom Checklist) to capture the overall severity of symptoms experienced.17,26,27 Two-day test-retest reliability was established with a subsample of randomly selected participants (n = 80). The 9-item severity stability was R = 0.86. Both administrations were demonstrated to be internally consistent, with the Cronbach α estimates greater than 0.97. The summative scale of concussion-related symptoms has also been demonstrated to produce reliable and valid baseline responses.23 Instructions to the respondent read, “The previous scale asked you to report how long you have experienced each of the listed symptoms during the previous 24-hour period (today). Now we would like for you to please circle the number that best describes how severe the symptom has felt during the previous 24-hour period (today).” As with the HIS, items were rated on a 7-point Likert-type scale (0–6), but response options were written to reflect item severity: 0, I have not experienced this symptom; 1, I have felt mild problems with this symptom today; 3, I have felt moderate problems with this symptom today; 6, I have felt severe problems with this symptom today. Item responses were then summed to provide an overall composite score.
The Brief History Questionnaire was a 2-page, 27-item questionnaire designed for this study to collect participant descriptive and demographic information. It included information related to a history of sport-related concussion, presence of fatigue from daily activities before baseline testing, acute physical illness, acute orthopaedic injury, type of sport played, years of exposure to the played sport, and type of position played. Participants were asked to answer these questions to the best of their abilities.
Procedures
The baseline measures were collected by certified athletic trainers trained in the administration of this self-report battery. The HIS was administered in conjunction with the severity scale and Brief Health Questionnaire. Informed consent forms were completed before participants entered the study.
Data Analysis
After assessing the tenability of assumptions for a parametric t test, we selected a nonparametric technique because of the violations of normality demonstrated by a graphical analysis using the skewness and kurtosis values presented in Table 2. Multiple Mann-Whitney U tests were conducted as an analogous statistical approach to evaluate the influences of preexisting conditions (independent variables: history of concussion, sex, fatigue, orthopaedic injury, and physical illness) on SRS responses (dependent variables: mean scores for 9-item HIS and 9-item severity scale) to each scale. Bonferroni methods were used to set the a priori α level at P ≤ .01 (.05/5 = .01) to decrease the likelihood of committing a multiple-comparisons error. Effect sizes for these nominal by interval relationships were calculated as η and were squared to provide a measure of the variance accounted for in the dependent variable by the independent variable.
Table 2.
RESULTS
Descriptive Statistics
The means, SDs, and skewness and kurtosis values for responses to the 9-item HIS and severity scale are reported in Table 2. The frequencies of each symptom (HIS duration and severity) are described along with previously reported frequencies from related studies in Table 3.
Table 3.
Baseline Responses
Differences with moderate effect sizes were demonstrated between the baseline responses of athletes reporting a history of concussion and those reporting no history on both the HIS and severity scales (U = 85 886.5, P ≤ .01, η2 = 0.06, and U = 81 631.5, P ≤ .01, η2 = 0.05, respectively). No differences due to sex were observed (U = 101 782.5, P = .498, η2 = 0.05, and U = 98 024, P = .114, η2 = 0.05, respectively). However, those athletes reporting fatigue (U = 37 438.5, P ≤ .01, η2 = 0.21, and U = 37 090, P ≤ .01, η2 = 0.20, respectively), physical illness (U = 45 743, P ≤ .01, η2 = 0.07, and U = 44 598, P ≤ .01, η2 = 0.05, respectively), acute orthopaedic injury (U = 57 858.5, P ≤ .01, η2 = 0.07, and U = 57 446.5, P ≤ .01, η2 = 0.07, respectively) had higher composite scores on both the 9-item HIS and 9-item severity scale than those without these conditions.
Based upon these findings, we removed all the significant variables deemed to be clinically variable (fatigue, physical illness, orthopaedic injury) from the analysis and reevaluated the effects of the 2 stable variables (sex and history of concussion, n = 681). Removing the stable variables did not change the statistical relationships between sex (U = 40 485.5, P = .04, η2 = 0.06 for the HIS and U = 39 757, P = .02, η2 = 0.07 for the severity scale) and history of concussion (U = 36 002.5, P ≤ .01, η2 = 0.07 for the HIS and U = 34 047, P ≤ .01, η2 = 0.06 for the severity scale). We also conducted a post hoc examination of the factorial validity of both scales after respondents reporting influencing variables were removed from the data set. These variables elevated overall composite scores, thus increasing the variance among items and increasing the likelihood of good model-data fit, but their removal would not alter the 3-factor, higher-order model shown to fit in previously published data.22,23 We tested our assumptions by following protocols described in prior publications.22,23
The 3-factor, higher-order measurement model represented a good fit to the 9-item HIS (χ224 = 118.37, root mean square error of approximation [RMSEA] = 0.07 [90% confidence interval = 0.06, 0.09], nonnormed fit index [NNFI] = 0.93, comparative fit index [CFI] = 0.95), and the 3-factor, higher-order measurement model represented a good fit to the 9-item severity scale (χ224 = 140.85, RMSEA = 0.08 [90% confidence interval = 0.07, 0.09], NNFI = 0.91, CFI = 0.94). For both scales, the χ2 statistic was significant (P < .05), but this statistic assumes the correct model and is affected by sample size.29–34 The RMSEA values were 0.07 and 0.08, which exceed and meet the threshold, respectively.30–32,34,35 Also, the lower bound of the 90% confidence interval was below the threshold (0.08) and approached or met 0.06. The NNFI and CFI provided good support for the 9-item, 3-factor model for both scales. The NNFI and the CFI values exceeded the 0.90 standard, and the CFI met the 0.95 threshold for the 9-item HIS, representing a strong model to data fit.30,32,34 These findings confirm the factorial validity of both self-report scales in the absence of potentially confounding variables.
DISCUSSION
We found that nonconcussed athletes reported a constellation of SRS related to concussion (Table 3). These findings are consistent with previously reported results in nonconcussed athletes and studies involving healthy nonathlete groups.9–11,22,23 Furthermore, baseline composite scores from measures designed to characterize SRS were (1) inflated by a history of concussion, fatigue, physical illness, and orthopaedic injury; (2) consistent internally and across time; and (3) factorially valid when confounding clinical variables were removed.
Baseline scores of concussion-related symptom measures serve as assistive criteria for sports medicine clinicians making return-to-play decisions. This role is important because such guidelines recommend that athletes be asymptomatic before returning to full activity.1,3,5,36–38 Thus, baseline responses to measures of SRS require characterization and understanding before they are used as benchmarks for assessing the athlete and determining if he or she is asymptomatic. The most common symptoms reported during a nonconcussed baseline evaluation (fatigue, headache, difficulty concentrating, drowsiness, and trouble falling asleep) are also among the most commonly reported during a postconcussion evaluation.22,39–41 Although much emphasis has been placed upon symptom reports after a concussive injury, our findings support the need to investigate baseline rates for concussion-related symptoms. Examining how symptom reports are influenced by preexisting variables that are common to sports is also tremendously important. Clinicians commonly obtain baseline measures at the most convenient time, such as after a workout or when an athlete is in the athletic training facility receiving treatment. Our results demonstrate that athletes who have participated in workouts or are physically ill or experiencing acute orthopaedic injury tend to report a higher level of baseline symptoms than those who had not been working out or were not ill or injured.
An important caveat to this investigation relates to the particular instruments we chose to obtain our data. Historically, SRS research (in both sport- and non–sport-related populations) has been limited by the lack of psychometric and measurement evidence and instrument standardization. Because of this, we not only used instruments that possess reported evidence of such properties22,23 but also extended this support by evaluating and confirming the stability and internal consistency of responses to both instruments. In addition, we realized that prior investigators22,23 evaluating the factorial validity of responses to the HIS and Graded Symptom Checklist instruments did not control for variables that we found inflated baseline composite scores. A tenet of the techniques used to provide validity evidence (structural equation modeling) is that higher scores result in greater item variability, which potentially increases the probability of good model-data fit. Therefore, we evaluated the factorial validity (via confirmatory factor analysis) of the responses to the HIS and severity scales with respondents reporting the confounding conditions (those that could theoretically be controlled by the clinician: fatigue, physical illness, orthopaedic injury) via confirmatory factor analysis while controlling for these confounding factors. We hypothesized that even in the absence of these conditions, the constellation of documented symptoms and the theoretically supported latent relationships (cognitive, somatic, neurobehavioral) would maintain their integrity. Our findings via confirmatory factor analysis supported our hypothesis, with strong continued evidence for the factorial validity of responses to the 9-item HIS and the 9-item severity scale. Our data suggest that responses to the 9-item HIS and the 9-item severity scale (headache, nausea, balance difficulty, fatigue, drowsiness, trouble falling asleep, difficulty concentrating, feeling “in a fog,” feeling “slowed down”) remain cohesive as a constellation of symptoms in the absence of variables that can artificially increase baseline SRS scores, so controlling for these conditions before a baseline assessment does not affect the validity of the inferences that can be drawn from the obtained scores. The importance of this finding is that clinicians can expect to obtain accurate and meaningful baseline responses to validated measures of SRS, thus allowing them to ascertain when an individual athlete has become asymptomatic and returned to preinjury (ie, baseline) status.
We found no difference in the composite scores of the 9-item HIS and the 9-item severity scale between males and females, which is consistent with previous baseline research.10,42 A novel aspect to this study was the comparison of athletes reporting a history of concussion versus those without such a history. A history of concussion is related to decreased neurocognitive performance, a predisposition to incur another concussion, and a resulting slower recovery from that subsequent episode.26,40 Those with a history of concussion reported higher composite scores for symptom duration and symptom severity than those without such a history. However, we did not investigate concussion history beyond the single question answered in a dichotomous fashion. Therefore, without a more detailed understanding of our respondents' concussion histories, we were unable to describe how they specifically affected baseline composite SRS scores. Certainly our findings support the need for additional inquiry with a more thorough approach to recording concussion histories to help identify the specific effects of concussions (ie, number, severity and timing) upon baseline SRS. Regardless, baseline SRS score appears to be sensitive to a report of concussion history. Typically, the logistics and large numbers associated with preseason physical examinations or preconcussion baseline assessments (or both) make it difficult to obtain individual medical histories. Thus, if clinicians are aware that higher composite baseline scores on SRS scales are associated with a history of concussion, those athletes who require more attention and further assessment within a group screening setting can be identified.
RECOMMENDATIONS AND CONCLUSIONS
We recommend that clinicians employ a standardized SRS summative scale that provides reliable and valid responses within their multifaceted concussion assessment protocols. Also, we encourage clinicians to control for variables that are common to sport and may increase normal baseline responses. Clinicians should provide athletes with a mechanism to report their history of concussion as well as the presence of fatigue, physical illness, and orthopaedic injury before they complete a preseason baseline SRS scale. Baseline ratings from athletes reporting any of the 3 “controllable” variables should be obtained at a later time, when the condition is no longer being experienced. Lastly, athletes reporting a history of concussion or possessing an elevated composite SRS score should be identified and given a thorough individual clinical evaluation to determine the history of concussion and any potential reasons for the elevated symptom scale. Implementing these recommendations will further offset the inherent weaknesses of the self-report mechanism and provide sports medicine clinicians with appropriate information on which to base return-to-play decisions.
Acknowledgments
This project was funded by a National Athletic Trainers' Association Research & Education Foundation Doctoral Dissertation Grant (303DGP0007) and the Kindig Research Award from the Department of Kinesiology, The University of Georgia.
REFERENCES
- 1.Guskiewicz K. M., Bruce S. L., Cantu R. C., et al. National Athletic Trainers' Association position statement: management of sport-related concussion. J Athl Train. 2004;39(3):280–297. [PMC free article] [PubMed] [Google Scholar]
- 2.Broglio S. P., Puetz T. W. The effect of sport concussion on neurocognitive function, self-report symptoms and postural control. Sports Med. 2008;38(1):53–67. doi: 10.2165/00007256-200838010-00005. [DOI] [PubMed] [Google Scholar]
- 3.Aubry M., Cantu R., Dvorak J., et al. Summary and agreement statement of the 1st International Symposium on Concussion in Sport, Vienna 2001. Clin J Sport Med. 2002;12(1):6–11. doi: 10.1097/00042752-200201000-00005. [DOI] [PubMed] [Google Scholar]
- 4.McCrory P., Johnston K., Meeuwisse W., et al. Summary and agreement statement of the 2nd International Conference on Concussion in Sport, Prague 2004. Clin J Sport Med. 2005;15(2):48–55. doi: 10.1097/01.jsm.0000159931.77191.29. [DOI] [PubMed] [Google Scholar]
- 5.Cantu R. C., Aubry M., Dvorak J., et al. An overview of concussion consensus statements since 2000. Neurosurg Focus. 2006;21(4):E3. doi: 10.3171/foc.2006.21.4.4. [DOI] [PubMed] [Google Scholar]
- 6.McCrea M., Hammeke T., Olsen G., Leo P., Guskiewicz K. Unreported concussion in high school football players: implications for prevention. Clin J Sport Med. 2004;14(1):13–17. doi: 10.1097/00042752-200401000-00003. [DOI] [PubMed] [Google Scholar]
- 7.Bigler E. D. Neuropsychology and clinical neuroscience of persistent post-concussive syndrome. J Int Neuropsychol Soc. 2008;14(1):1–22. doi: 10.1017/S135561770808017X. [DOI] [PubMed] [Google Scholar]
- 8.Binder L. M. Persisting symptoms after mild head injury: a review of the postconcussive syndrome. J Clin Exp Neuropsychol. 1986;8(4):323–346. doi: 10.1080/01688638608401325. [DOI] [PubMed] [Google Scholar]
- 9.Gouvier W. D., Cubic B., Jones G., Brantley P., Cutlip Q. Postconcussion symptoms and daily stress in normal and head-injured college populations. Arch Clin Neuropsychol. 1992;7(3):193–211. [PubMed] [Google Scholar]
- 10.Iverson G. L., Lange R. T. Examination of “postconcussion-like” symptoms in a healthy sample. Appl Neuropsychol. 2003;10(3):137–144. doi: 10.1207/S15324826AN1003_02. [DOI] [PubMed] [Google Scholar]
- 11.Wang Y., Chan R. C., Deng Y. Examination of postconcussion-like symptoms in healthy university students: relationships to subjective and objective neuropsychological function performance. Arch Clin Neuropsychol. 2006;21(4):339–347. doi: 10.1016/j.acn.2006.03.006. doi:10.1016/j.acn.2006.03.006. [DOI] [PubMed] [Google Scholar]
- 12.Macciocchi S. N., Barth J. T., Littlefield L. M. Outcome after mild head injury. Clin Sports Med. 1998;17(1):27–36. doi: 10.1016/s0278-5919(05)70058-2. [DOI] [PubMed] [Google Scholar]
- 13.Guskiewicz K. M. Postural stability assessment following concussion: one piece of the puzzle. Clin J Sport Med. 2001;11(3):182–189. doi: 10.1097/00042752-200107000-00009. [DOI] [PubMed] [Google Scholar]
- 14.Guskiewicz K. M., Riemann B. L., Perrin D. H., Nashner L. M. Alternative approaches to the assessment of mild head injury in athletes. Med Sci Sports Exerc. 1997;29(7):S213–S221. doi: 10.1097/00005768-199707001-00003. [DOI] [PubMed] [Google Scholar]
- 15.Collins M. W., Grindel S. H., Lovell M. R., et al. Relationship between concussion and neuropsychological performance in college football players. JAMA. 1999;282(10):964–970. doi: 10.1001/jama.282.10.964. [DOI] [PubMed] [Google Scholar]
- 16.Lovell M., Collins M., Bradley J. Return to play following sports-related concussion. Clin Sports Med. 2004;23(3):421–441. doi: 10.1016/j.csm.2004.04.001. [DOI] [PubMed] [Google Scholar]
- 17.Lovell M. R., Collins M. W. Neuropsychological assessment of the college football player. J Head Trauma Rehabil. 1998;13(2):9–26. doi: 10.1097/00001199-199804000-00004. [DOI] [PubMed] [Google Scholar]
- 18.Maroon J. C., Lovell M. R., Norwig J., Podell K., Powell J. W., Hartl A. Cerebral concussion in athletes: evaluation and neuropsychological testing. Neurosurgery. 2000;47(3):659–669. doi: 10.1097/00006123-200009000-00027. [DOI] [PubMed] [Google Scholar]
- 19.Notebaert A. J., Guskiewicz K. M. Current trends in athletic training practice for concussion assessment and management. J Athl Train. 2005;40(4):320–325. [PMC free article] [PubMed] [Google Scholar]
- 20.Oliaro S., Anderson S., Hooker D. Management of cerebral concussion in sports: the athletic trainer's perspective. J Athl Train. 2001;36(3):257–262. [PMC free article] [PubMed] [Google Scholar]
- 21.Peterson C. L., Ferrara M. S., Mrazik M., Piland T., Elliott T. Evaluation of neuropsychological stability following cerebral domain scores and postural concussion in sports. Clin J Sport Med. 2003;13(4):230–237. doi: 10.1097/00042752-200307000-00006. [DOI] [PubMed] [Google Scholar]
- 22.Piland S. G., Motl R. W., Ferrara M. S., Peterson C. L. Evidence for the factorial and construct validity of a self-report concussion symptoms scale. J Athl Train. 2003;38(2):104–112. [PMC free article] [PubMed] [Google Scholar]
- 23.Piland S. G., Motl R. W., Guskiewicz K. M., McCrea M., Ferrara M. S. Structural validity of a self-report concussion-related symptom scale. Med Sci Sports Exerc. 2006;38(1):27–32. doi: 10.1249/01.mss.0000183186.98212.d5. [DOI] [PubMed] [Google Scholar]
- 24.McCrory P. R., Ariens M., Berkovic S. F. The nature and duration of acute concussive symptoms in Australian football. Clin J Sport Med. 2000;10(4):235–238. doi: 10.1097/00042752-200010000-00002. [DOI] [PubMed] [Google Scholar]
- 25.Lovell M. R., Iverson G. L., Collins M. W., et al. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. 2006;13(3):166–174. doi: 10.1207/s15324826an1303_4. [DOI] [PubMed] [Google Scholar]
- 26.Guskiewicz K. M., McCrea M., Marshall S. W., et al. Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA concussion study. JAMA. 2003;290(19):2549–2555. doi: 10.1001/jama.290.19.2549. [DOI] [PubMed] [Google Scholar]
- 27.McCrea M., Guskiewicz K. M., Marshall S. W., et al. Acute effects and recovery time following concussion in collegiate football players: the NCAA concussion study. JAMA. 2003;290(19):2556–2563. doi: 10.1001/jama.290.19.2556. [DOI] [PubMed] [Google Scholar]
- 28.Chan R. C. Base rate of post-concussion symptoms among normal people and its neuropsychological correlates. Clin Rehabil. 2001;15(3):266–273. doi: 10.1191/026921501675253420. [DOI] [PubMed] [Google Scholar]
- 29.Bentler P. M. Comparative fit indexes in structural modeling. Psychol Bull. 1990;107(2):238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
- 30.Bentler P. M., Bonett D. G. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull. 1980;88(3):588–606. [Google Scholar]
- 31.Bollen K. A. Structural Equations With Latent Variables. New York, NY: John Wiley; 1989. [Google Scholar]
- 32.Jöreskog K. Structural equation models. In: Bollen K. A., Long J. S., editors. Testing Structural Equation Models. Newbury Park, CA: Sage; 1993. pp. 294–316. [Google Scholar]
- 33.Hu L., Bentler P. M. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model. 1999;6:1–55. [Google Scholar]
- 34.Steiger J. H. Structural model evaluation and modification: an interval estimation approach. Multivar Behav Res. 1990;25(2):173–180. doi: 10.1207/s15327906mbr2502_4. [DOI] [PubMed] [Google Scholar]
- 35.Kelly J. Sports-related recurrent brain injuries: United States. JAMA. 1997;277(15):1190–1191. Reprinted from MMWR Morb Mortal Wkly Rep. 1997:46(10);224–227. [PubMed] [Google Scholar]
- 36.Cantu R. C. Return to play guidelines after a head injury. Clin Sports Med. 1998;17(1):45–60. doi: 10.1016/s0278-5919(05)70060-0. [DOI] [PubMed] [Google Scholar]
- 37.Cantu R. C. Cerebral concussion in athletes: evaluation and neuropsychological testing: comment. Neurosurgery. 2000;47(3):669–670. doi: 10.1097/00006123-200009000-00027. [DOI] [PubMed] [Google Scholar]
- 38.Collins M. W., Lovell M. R., Iverson G. L., Cantu R. C., Maroon J. C., Field M. Cumulative effects of concussion in high school athletes. Neurosurgery. 2002;51(5):1175–1179. doi: 10.1097/00006123-200211000-00011. [DOI] [PubMed] [Google Scholar]
- 39.Collins M. W., Iverson G. L., Lovell M. R., McKeag D. B., Norwig J., Maroon J. On-field predictors of neuropsychological and symptom deficit following sports-related concussion. Clin J Sport Med. 2003;13(4):222–229. doi: 10.1097/00042752-200307000-00005. [DOI] [PubMed] [Google Scholar]
- 40.Wong J. L., Regennitter R. P., Barrios F. Base rate and simulated symptoms of mild head injury among normals. Arch Clin Neuropsychol. 1994;9(5):411–425. [PubMed] [Google Scholar]