Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Apr 1.
Published in final edited form as: J Sci Med Sport. 2017 Aug 24;21(4):363–367. doi: 10.1016/j.jsams.2017.08.008

Validation of HeadCount-2w for Estimation of Two-Week Heading: Comparison to Daily Reporting in Adult Amateur Player

Michael L Lipton 1,2,5,6, Chloe Ifrah 1,2, Walter F Stewart 7, Roman Fleysher 1,2, Martin J Sliwinski 8, Mimi Kim 4, Richard B Lipton 3,4
PMCID: PMC5826783  NIHMSID: NIHMS901532  PMID: 28882435

Abstract

Objective

To validate the HeadCount-2w questionnaire for estimation of 2-week soccer heading by comparison to daily electronic diary reporting over the same two-week period.

Design

Prospective observational study.

Methods

Adult amateur soccer players completed HeadCount-daily, comprising 14 daily at-home assessments of soccer play and heading via a tablet PC. Following the 14 day period, players completed HeadCount-2w, a web-based two-week-recall questionnaire on soccer and heading. intraclass correlation coefficient (ICC) was estimated between HeadCount-daily, the reference standard, and HeadCount-2w estimates of heading during the same 2-week period.

Results

53 participants (38 men) reported a mean of 24.36 (median = 11.76) headers during 2 weeks via HeadCount-daily and a mean of 38.34 (median = 15.0) headers for the same 2 weeks via HeadCount-2w. The ICC comparing 2-week heading from HeadCount-daily and HeadCount-2w was 0.85. Linear regression of the log-transformed Headcount-daily on Headcount-2w data yielded a slope of 0.71 (p<0.001; 95% CI 0.54–0.82), suggesting that heading tends to be over-estimated by HeadCount-2w relative to HeadCount-daily. Slope estimates for men (0.65) and women (0.71) were similar.

Conclusion

HeadCount, a self-administered web-based survey, is valid for self-reporting 2-week heading in adult amateur players, supporting its use in future research and as a simple and low-cost technique for exposure monitoring.

Keywords: Soccer, Exposure Quantification, Head Injury

INTRODUCTION

In soccer, purposeful heading is an integral component of play that entails repeated head impacts, often of low intensity1. Although heading has been associated with adverse consequences in players at various levels (see references2–6), the causative role of subconcussive heading in either short- or long-term alteration of brain function and structure remains controversial7,8. Practical and reliable means for quantifying exposure to heading are essential to effectively evaluating potential risks from heading.

Heading can be assessed by direct observation of play or video recordings of play, wearable miniature accelerometry and self-report. While direct observation is assumed to be the gold-standard, it is prone to observer bias, requires careful training and monitoring of observers, and involves substantial labor cost for sideline personnel. Direct observation is not sutiable for large scale epidemiologic studies. Wearable miniature accelerometry is a potential solution for characterizing exposure, but available methods have not yet proven to be reliable9,10. Player self-report is a low-tech, low-cost means to capture exposure, but is also prone to potential limitations, such as recall bias.

We developed HeadCount-2w, a self-administered web-based questionnaire to capture heading exposure during the prior two weeks, and previously reported its validation in a cohort of NCAA Division 1 collegiate players11. In the present study, we sought to validate HeadCount-2w in an adult amateur population, which represents a large segment of soccer players and is dispersed among numerous leagues and teams over a wide geographic area. To evaluate potential effects of heading in this population, it will be important to have proven scalable low cost means available to quantify exposure. To this end, we evaluated the validity of the HeadCount-2w-based estimate compared to daily reports captured using an electronic diary (HeadCount-daily), over the same 2 week period.

METHODS

The Einstein Soccer Study is a multi-faceted longitudinal study of heading and its consequences in adult amateur soccer players approved by the Albert Einstein College of Medicine Institutional Review Board. Details of the overall study are reported elsewhere12. Inclusion criteria were: age 18–55; at least 5 years of active amateur soccer play; current active amateur soccer play; 6 months of amateur soccer play per year; and English language fluency. Exclusion criteria were: schizophrenia, bipolar disorder; known neurological disorder; pregnancy; and medical contraindication to MRI (relevant to a separate arm of the study). Herein we describe validation and calibration of HeadCount-2w, for recall of 2-week heading, in a sub-sample of participants.

Participants were eligible if they play soccer more than twice a week, commited to visit the lab every 3 months over a 12-month period and agreed to complete the at-home electronic daily (HeadCount-daily) and two-week HeadCount (HeadCount-2w) questionnaires. Participants completed one or more epochs, which included HeadCount-daily every day for 14 days, followed by HeadCount-2w the day after the final daily questionnaire. Participants were aware at the outset that they would be asked to complete HeadCount-2w following the series of HeadCount-daily assessment. Two participants were excluded from the analysis due to not playing soccer during the daily assessment period. We assessed the distribution of 2-week heading agreement (HeadCount-2w – HeadCount-daily) and consequently excluded 3 participants who were extreme outliers on report of heading (exceeding 2 standard deviation).

HeadCount-daily Heading Questionnaire

HeadCount-daily was administered on a custom-programmed tablet PC (Android Tab 2-Version 4.1.2; Hardware version i705.03 or Tab 4-Version 4.4.4; Hardware Version T337V.03) issued to each participant, which alarms to prompt them to complete the survey each evening for 14 consecutive days. Each daily assessment begins asking whether the participant played soccer that day and then queries the number of headers performed. The entire assessment, which also incorporates questions on symptoms, injuries and environmental exposures (e.g., smoking and alcohol) as well as brief cognitive tests, requires less than 10 minutes to complete.

HeadCount-2w Heading Questionnaire

HeadCount-2w was administered remotely as a web-based questionnaire immediately following completion of HeadCount-daily (above). Details on the development of HeadCount have been reported previously including details on the format and structure of the questionnaire11. In brief, HeadCount asks a series of identical questions regarding practice or competition in outdoor and indoor settings, including 1) the number of competitive soccer games and practice sessions during the two-week period, 2) the average number of headers during a typical game and 3) the average number of headers during a typical practice. Total heading during the two-week recall period was estimated as (headings per outdoor game × number of outdoor games) + (headings per outdoor practice × number of outdoor practices) + (headings per indoor game × number of indoor games) + (headings per indoor practice × number of indoor practices).

Data Analysis

To validate the 2-week estimate of heading obtained with HeadCount-2w, we assessed its level of agreement with daily reporting obtained with HeadCount-daily over the same 14-day period. To derive a standardized estimate of 2-week heading from HeadCount-daily devoid of missing data from players who did not complete all 14 daily reports, we took the average of the number of headers reported across all daily reports, multiplied by 14 as the HeadCount-daily estimate of total 2-week heading. Because distributions of heading across participants were highly left-skewed for both HeadCount-2w and HeadCount-daily, we log-transformed both measures prior to analysis. We computed the intraclass correlation coefficient (ICC) between the two log-transformed heading estimates for the same two-week period. We also plotted the estimates of HeadCount-2w against Headcount-daily as the reference and fit the following linear regression model:

log(Total-HeadingHeadCount-daily)=α+βlog(Total-HeadingHeadCount-2w)

to characterize the relationship between the two approaches to estimation of heading. Analyses were completed using IBM SPSS Version 24.0.0.0 32-bit for Windows.

RESULTS

53 participants (38 men) age 18–52 (mean 27; median 24) completed one or more epochs of HeadCount-daily immediately followed by HeadCount-2w. A total of 138 epochs were completed; 14 participants completed 1 epoch, 11 completed 2 epochs, 10 completed 3 epochs and 18 completed 4 epochs. An average of 12.4 days of daily reporting (median 14, with 27/53 participants completing all 14 assessments) was completed for each HeadCount-daily epoch. Average 2-week heading reported was 24.36 (median = 11.76) reported via HeadCount-daily and 38.34 (median=15) via HeadCount-2w. The Bland-Altman plot in Figure 1 shows the distribution of the individual differences between HeadCount-daily and HeadCount-2w (Figure 1) shows the distribution of agreement between the two questionnaires.

Figure 1.

Figure 1

Open in a new tab

Bland-Altman plot of the difference between 2-week heading measured by HeadCount-daily and HeadCount-2w vs. mean heading from HeadCount-2w for 138 2-week reporting epochs completed by 53 participants. Overall good agreement is demonstrated, with slight underestimation by HeadCount-2w at lower reported heading and overestimation at the highest heading levels.

Agreement of HeadCount-2w with HeadCount-daily

The ICC was 0.85 (95% CI 0.76–0.90) between the estimate of 2-week heading from HeadCount-2w and that from HeadCount-daily. The ICC for women was 0.87 (95% CI: 0.74–0.94) similar to the ICC in men (0.80; 95% CI: 0.68–0.87).

The relationship of HeadCount-2w to HeadCount-daily and the corresponding regression line with slope of 0.71 (p<0.001; 95% CI 0.54–0.82) are shown in Figure 2. The slopes for men and women were similar, 0.65 (p<0.001; 95% CI 0.42–0.83) and 0.71 (p<0.001; 95% CI 0.62–0.89), respectively. The α (intercept term) for the regression line was estimated to be 0.54 (95% CI 0.07–1.01) for the entire cohort, 0.76 (95% CI 0.07–1.46) for men and 0.24 (95% CI −0.20–0.69) for women. These results suggest that HeadCount-2w systematically overestimates 2-week heading in comparison to HeadCount-daily when headcounts are higher but slightly understimates 2-week heading at lower levels.

Figure 2.

Figure 2

Open in a new tab

Plots of 2-week heading measured by HeadCount-2w vs. HeadCount-daily in the total sample (top; 138 epochs completed by 53 participants), men only (middle; 99 epochs completed by 38 participants) and women only (bottom; 39 epochs completed by 15 participants). Solid Line: Linear Regression Line; Dashed Line: Reference with slope=1 and intercept=0,0; Dotted Lines: 95% CI of slope.

DISCUSSION

Two-week recall obtained with a web-based questionnaire (HeadCount-2w) showed excellent agreement (ICC of 0.85) in comparison to daily electronic reporting (HeadCount-daily). We found that HeadCount-2w and HeadCount-daily performed similarly across men and women, tending to overestimate exposure in both groups, particularly for epochs with higher reported heading.

Notwithstanding its role in symptomatic events including concussion, most heading does not produce clinically notable symptoms12. Thus, in order to determine whether “subconcussive” repetitive heading leads to short or long-term changes in brain structure and/or function, reliable and valid means for quantifying exposure to heading are needed. Several factors must be considered in designing approaches to characterize exposure to heading. To assess relative exposure across large numbers of players over a wide geographic area, as in our study, the assessment methodology must be readily distributed across the population of interest. This requirement is distinct from detailed biomechanical characterization of individual events, which requires sophisticated and carefully calibrated measurement devices, such as accelerometers and high speed videography, with subsequent quantification of individual events10. Although wearable and relatively low-cost miniaturized accelerometry is becoming available, these approaches have generally been employed in limited controlled settings, such as school sports teams, and have yet to be validated for widespread unsupervised and unobserved use9. Player self-report thus provides an attractive, highly practical, low-cost means to assess exposure for research and intervention. Reporting of absolute number of headers on the same day that they occur is intuitively more reliable than using other recall intervals. The longer the recall interval, the more likely it is that error variance is introduced. We selected a two-week recall interval to balance the need for more information on head impact exposures that could be recalled with reasonable reliability and accuracy. To some degree, 2-week recall of heading may rely on an individual’s typical exposure for each venue (game/practice; indoor/outdoor) weighted by variation in that experience. We thus aimed to validate a 2-week recall-based estimate of exposure to heading against daily recall of heading.

Several factors may account for over-estimation in our recall-based measures. One factor is telescoping, the tendency to remember past events closer to the present than they actually occurred13. In this context, participants may report heading events prior to the two-week recall interval as if they occurred in that recall interval. In addition, days with more symptoms, such as headache, may be more memorable; if greater exposure is associated with more symptoms, players may preferentially recall days with greater exposure leading to overestimation. A carryover effect could also occur, where responding to the electronic diary (HeadCount-daily) each day inflates estimation of heading on HeadCount-2w at the end of the two week interval. In order to compare estimates of heading during the same 2-week period, our validation compared daily reporting (HeadCount-daily) to 2-week recall (HeadCount-2w), where the latter was administered immediately following the 14 days of daily self-report. We therefore considered the possibility that completion of HeadCount-daily immediately prior to HeadCount-2w might bias the HeadCount-2w estimate.

Our finding that HeadCount-2w overestimates 2-week heading compared to HeadCount-daily might result from a carryover effect. That is, it may be that daily reporting during the 2 weeks immediately preceding a recall of the same 2-week period causes players to either more accurately estimate or perhaps overestimate their prior 2-week exposure. We performed a secondary confirmatory analysis to address the possibility of a clinically meaningful degree of carryover bias between the two heading measures. We have previously shown that verbal memory, measured using the International Shopping List (ISL) task from Cogstate®14 is associated with heading15. For the secondary analysis, we tested the null hypothesis that the association of heading with cognitive function would not differ for heading measured through HeadCount-daily compared to HeadCount-2w, expecting that if prior HeadCount-daily would bias the HeadCount-2w estimate in a clinically meaningful way, it would alter the association of heading with cognitive function. We then tested the linear association of heading as estimated by HeadCount-2w either following HeadCount-daily or in the absence of prior HeadCount-daily as separate predictors of verbal memory, measured in the same participants as part of a separate arm of the study. For 198 observations, HeadCount-2w was preceded by HeadCount-daily in 59, but not in the remainder. Greater 2-week heading, estimated from HeadCount-2w, was significantly associated with both worse immediate (ISL β=−3.5, p<0.001) and delayed (ISL-Recall β =−1.18, p<0.01) recall. For the subset of 59 observations comprising only HeadCount-2w preceded by daily reporting, we found similar associations on one measure (ISL β=−3.48, p=0.04). On the ISL-Recall, we found a larger effect magnitude (β=−3.04). This finding trended toward, but did not reach significance (p=0.10), which we attribute to a much smaller sample size (59 vs. 198). An additional factor mitigating the likelihood of bias due to a carryover effect is that the structure of HeadCount-daily is inherently different from HeadCount-2w. HeadCount-daily queries on explicit number of headers performed during each 1-day recall period, whereas HeadCount-2w queries on average heading in game or practice contexts and computes an estimate of heading therefrom.

We previously reported that heading in the highest quartile (range 46–750) on HeadCount-2w conferred significantly elevated odds for the occurrence of heading-related moderate to very severe CNS symptoms during the 2-week reporting period (OR=4.79; 95% CI=2.37–9.67)12. Reporting error in the present study (Figure 1), was largest in those reporting the most heading. Thus, most players identified in the group at high risk for symptomatic heading event would still be so classified, even when accounting the reporting error. Overall, therefore, we find that the utility of HeadCount-2w would not be diminished in a meaningful way by the overreporting we have identified.

Our findings indicate that HeadCount-2w provides a robust means to capture relatively short-term exposure to heading. Additionally, HeadCount-2w has been used to characterize symptomatic and cognitive functional outcomes related to recent heading12,16. Because such short-term effects may be transient in nature, it remains important to determine the type of functional effects of longer periods of exposure to heading, a question explicitly beyond the scope of our present analysis. We note, however, that HeadCount-12m, a version of the same questionnaire designed to estimate heading over 1 year, has shown excellent test-retest reliability and has been used to robustly characterize both imaging and neurocognitive changes associated with long-term heading15,16.

Our validation study must be considered in context of several limitations. Most importantly, the findings cannot be explicitly applied to other populations, geographic locations or contexts of play without further study. Youth players, for example, may interact differently with this type of survey. While exposure assessment has particular potential importance in youth soccer, HeadCount should be specifically studied in younger age groups before it is applied in practice. Our validation lacks an explicit gold standard. We previously reported validation of the same instrument against trained sideline observers11. This prior validation, however, was constrained to NCAA Division I players at a single university. We noted its inherent logistical difficulties and that the “gold standard” it employed is itself subject to potential observer bias. Direct observation would not also be feasible for a large longitudinal epidemiologic study. While the reference used in the present study, daily reporting on an electronic diary, has not been explicitly validated, we believe daily reporting of headers is likely to be more accurate than 2-week recall (HeadCount-2w) and that it is highly unlikely that daily reporting is worse than 2-week recall. Under this assumption, our current report likely represents a lower limit on agreement. In other words, daily heading would most likely show better agreement with direct observation than reported previously11. It is possible that concussion-related symptoms, concussive events during the recall period, or prior concussion, might bias reporting. However, based on specific symptom assessment captured by a separate section of HeadCount-2w, only 3/53 participants reported probable concussive events during the recall period and none reported actually experiencing concussion. Further, in order for concussion to impact our findings it would need to bias both daily and 2-week recall estimates to significantly different degrees. We nonetheless conducted a secondary analysis to test agreement of HeadCount-2w with HeadCount-daily separately for those reporting moderate, severe or very severe symptoms (n=19 reports) vs. those who did not (n=119 reports). The ICC was 0.83 and 0.85, repectively. Although agreement is nearly identical across the symptom groups, we nonetheless acknowledge that it is possible we were not powered to detect such an effect due to the low rate of concussive symptoms. Finally, the need for a 2-week recall assessment might be questioned. While it is attractive to consider exclusive use of daily electronic assessment, we believe that a simple questionnaire, which provides a reasonable estimate of short-term exposure is plausibly a more generalizable and useful approach, especially across diverse areas and player populations where access to technology-intensive approaches may be limited. Moreover, availability of multiple validated approaches to assessment has value. Further study can determine the best approach for specific applications, research questions or clinical monitoring.

CONCLUSION

HeadCount-2w is valid for estimation of exposure to heading over two weeks in adult amateur players, supporting its use in future research. In addition, as guidelines for safe and unsafe levels of heading exposure are developed and implemented, HeadCount-2w can provide a simple and low-cost tool for exposure monitoring. Future study should explore similar approaches in different soccer populations and contexts as well as over different periods of exposure.

PRACTICAL IMPLICATIONS.

  • Because heading may be associated with adverse consequences, means to quantify exposure across large dispersed groups are needed to assess potential effects.

  • HeadCount-2w is a simple, inexpensive and valid means for ascertaining heading over a 2-week period.

  • HeadCount-2w provides means to capture and monitor player exposure.

Acknowledgments

Study Funded by NIH (R01 NS082432) and The Dana Foundation David Mahoney Neuroimaging Program but there has been no financial assistance with the project.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Spiotta AM, Bartsch AJ, Benzel EC. Heading in soccer: dangerous play? Neurosurgery. 2012;70(1):1–11. doi: 10.1227/NEU.0b013e31823021b2. discussion 11. [DOI] [PubMed] [Google Scholar]
  • 2.Witol AD, Webbe FM. Soccer heading frequency predicts neuropsychological deficits. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists. 2003;18(4):397–417. [PubMed] [Google Scholar]
  • 3.McCrory PR. Brain injury and heading in soccer. BMJ. 2003;327(7411):351–352. doi: 10.1136/bmj.327.7411.351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Stephens R, Rutherford A, Potter D, Fernie G. Neuropsychological impairment as a consequence of football (soccer) play and football heading: a preliminary analysis and report on school students (13–16 years) Child Neuropsychol. 2005;11(6):513–526. doi: 10.1080/092970490959629. [DOI] [PubMed] [Google Scholar]
  • 5.Rutherford A, Stephens R, Potter D. The neuropsychology of heading and head trauma in Association Football (soccer): a review. Neuropsychology review. 2003;13(3):153–179. doi: 10.1023/a:1025525613477. [DOI] [PubMed] [Google Scholar]
  • 6.Kontos AP, Dolese A, Elbin RJ, Covassin T, Warren BL. Relationship of soccer heading to computerized neurocognitive performance and symptoms among female and male youth soccer players. Brain Inj. 2011;25(12):1234–1241. doi: 10.3109/02699052.2011.608209. [DOI] [PubMed] [Google Scholar]
  • 7.Rodrigues AC, Lasmar RP, Caramelli P. Effects of Soccer Heading on Brain Structure and Function. Front Neurol. 2016;7:38. doi: 10.3389/fneur.2016.00038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Maher ME, Hutchison M, Cusimano M, Comper P, Schweizer TA. Concussions and heading in soccer: a review of the evidence of incidence, mechanisms, biomarkers and neurocognitive outcomes. Brain Inj. 2014;28(3):271–285. doi: 10.3109/02699052.2013.865269. [DOI] [PubMed] [Google Scholar]
  • 9.Brennan JH, Mitra B, Synnot A, et al. Accelerometers for the Assessment of Concussion in Male Athletes: A Systematic Review and Meta-Analysis. Sports Med. 2017;47(3):469–478. doi: 10.1007/s40279-016-0582-1. [DOI] [PubMed] [Google Scholar]
  • 10.Caccese JB, Kaminski TW. Minimizing Head Acceleration in Soccer: A Review of the Literature. Sports Med. 2016;46(11):1591–1604. doi: 10.1007/s40279-016-0544-7. [DOI] [PubMed] [Google Scholar]
  • 11.Catenaccio E, Caccese J, Wakschlag N, et al. Validation and calibration of HeadCount, a self-report measure for quantifying heading exposure in soccer players. Res Sports Med. 2016;24(4):416–425. doi: 10.1080/15438627.2016.1234472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Stewart WF, Kim N, Ifrah CS, et al. Symptoms from repeated intentional and unintentional head impact in soccer players. Neurology. 2017;88(9):901–908. doi: 10.1212/WNL.0000000000003657. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Öztaş Ayhan H, Işiksal S. Memory recall errors in retrospective surveys: A reverse record check study. Quality and Quantity. 2005;38(5):475–493. [Google Scholar]
  • 14.Maruff P, Thomas E, Cysique L, et al. Validity of the CogState brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Arch Clin Neuropsychol. 2009;24(2):165–178. doi: 10.1093/arclin/acp010. [DOI] [PubMed] [Google Scholar]
  • 15.Lipton ML, Kim N, Zimmerman ME, et al. Soccer heading is associated with white matter microstructural and cognitive abnormalities. Radiology. 2013;268(3):850–857. doi: 10.1148/radiol.13130545. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Levitch CF, Zimmerman ME, Lubin N, et al. Recent and Long-Term Soccer Heading Exposure is Differentially Associated with Neuropsychological Function in Amateur Players. Journal of the International Neuropsychological Society. 2017 doi: 10.1017/S1355617717000790. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES