Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2021 Sep 17;16(9):e0257487. doi: 10.1371/journal.pone.0257487

Retrospective survey of youth sports participation: Development and assessment of reliability using school records

Steven Jin 1, Amanda R Rabinowitz 2,*, Jordan Weiss 3, Sameer Deshpande 4, Nitika Gupta 5, Reuben A Buford May 6, Dylan S Small 7
Editor: Javier Brazo-Sayavera8
PMCID: PMC8448309  PMID: 34534246

Abstract

Many youths participate in sports, and it is of interest to understand the impact of youth sports participation on later-life outcomes. However, prospective studies take a long time to complete and retrospective studies may be more practical and time-efficient to address some questions. We pilot a retrospective survey of youth sports participation and examine agreement between respondent’s self-reported participation with high school records in a sample of 84 adults who graduated from high school between 1948 and 2018. The percent agreement between our survey and the school resources for individual sports ranged between 91.5% and 100%. These findings provide preliminary evidence for the reliability of retrospective self-report of youth sports participation. This survey may serve as an efficient approach for evaluating relationships between involvement in youth sports and health outcomes later in adulthood.

Introduction

Playing high school sports has historically been and continues to be a prominent part of adolescence and young adulthood. In the United States, nearly eight million students played a high school sport during the 2018–2019 school year [1]. However, participation in high school sports has recently declined (from 7,980,886 in 2017–2018 to 7,937,491 in 2018–2019), led by declines in football participation, marking the first decline in participation for more than 30 years [1]. Reasons for this decline may be multifaceted and partially attributed to growing concern over the short- and long-term implications on mental and cognitive health of playing collision sports [2]. State-level appropriations for public school athletic programs were also reduced in response to the Great Recession [3], which, some have argued, is justified in response to potential deleterious effects on health and detracting from academic achievement [4]. To date, however, numerous studies that examined associations between participation in school sports and health or academic achievement found no association [5,6] while others reported that playing high school sports may have positive effects on educational attainment and future labor outcomes [7,8]. Additionally, many papers have reported positive associations between sports participation and physical fitness, brain function [9,10], and social and emotional health [11]. Meanwhile, a growing but mixed body of work links sports-related head trauma to mental and cognitive health deficits, prompting concern about the potential risks of adolescent participation in contact sports [12].

An ideal study of the long-term health implications of playing high-school sports would involve the collection of detailed, prospective information on sports participation with longitudinal follow-up that includes measures of cognitive and psychosocial functioning.

However, longitudinal studies of this magnitude are costly and the scientific community would have to wait decades for study participants to age to a point when the long-term risks of contact sports may manifest [13]. An alternative approach is to capitalize on existing longitudinal studies that collect both retrospective and prospective data, such as the National Longitudinal Study of Adolescent to Adult Health (Add Health). These large-scale studies are nationally representative and contain data spanning economic, health, and psychosocial outcomes.

The prospective identification of brain injury by observation or clinical diagnosis, is the gold standard method for injury ascertainment. This method is often too costly or unfeasible in many research designs. Fortunately, retrospective self-report measures have been developed and validated to collect incidence of prior head injury, such as the OSU-TBI ID [14] and the BISQ [15]. Data support that these instruments are valid and reliable indicators of prior brain trauma, including mild traumatic brain injury (mTBI) with associated loss of consciousness. Such measures have already been integrated into survey studies of long-term health. However, due to the absence of a known reliable survey to assess early-life sports participation, studies have been limited in their collection of information related to sports participation in early-life. This hampers the ability to contextualize possible deleterious effects of head injury against the backdrop of physical, emotional, and social benefits of youth sports participation.

To fill this gap, this study introduces a retrospective sports participation questionnaire administered electronically to adults who graduated from high school between 1948 and 2018. We evaluate the reliability of the questionnaire with respect to high school sports participation and find it to be reliable. Retrospective assessment of youth sports participation has the advantage of allowing investigators to capitalize on existing longitudinal datasets that have already tracked cognitive and health outcomes into later-life when some of the deleterious effects of brain injury are thought to manifest. Our survey represents a feasible and cost-effective approach for obtaining retrospective sports participation data.

Materials and methods

Survey development

Survey items were developed by the authors. Items were designed to capture information about the duration and intensity of exposure (i.e., sports participation). Intensity was operationalized as both weekly hours of participation and the competitive level of the team. A draft version of the survey was implemented using the Qualtrics platform for web-based survey administration. An electronic link to the survey was shared with two athletic trainers for comment after which the survey was piloted among a group of undergraduate research assistants (RAs) who were not involved in survey development. Feedback from that process was incorporated, and the survey was updated for content, clarity, and ease of use. The finalized version was sent back to RAs for quality assurance before deployment in the study sample.

Survey

In our survey, subjects were asked to select the sports in which they participated from a list of radio buttons. The survey used a series of blocks to assess sports participation at various stages of one’s education. This included a block for grades K-6, grades 7–8, grades 9–12, and college. For each block corresponding to a specific period in the life course, respondents were asked to select the sports in which they participated from a list of 15 radio buttons. The specific 15 sports (listed in Table 1) included in our survey were selected based on sports participation surveys conducted by the National Federation of State High School Associations as well as knowledge of other sports that were popular at the schools where the pilot study described in the next section was conducted. Based on each respondent’s selections, binary variables were constructed for participation in each sport during high school. Respondents also indicated whether, at each specific period, they had any physical limitations that prevented them from participating in sports. For each selected sport at each time period, respondents were asked to report (using radio buttons) the specific grades in which they participated, whether their team (a) traveled for games or tournaments, (b) participated in playoffs, (c) won a championship, or (d) none of the above. Respondents also indicated how many hours per week (0 hours, 1–4 hours, 5–9 hours, 10–14 hours, 15–19 hours, 20+ hours) they spent practicing and playing games for each specific sport, in addition to the position(s) they played. Our survey is provided in the supplementary materials.

Table 1. Percent agreement and internal consistency for survey responses and school resources.

Sport Percent Agreement Self-reported Participation
Total %
Football 98.8 10 12.2
Basketball 92.7 22 26.8
Baseball 91.5 10 12.2
Soccer 96.3 24 29.3
Lacrosse 91.5 15 18.3
Wrestling 98.8 9 11
Cross Country 93.9 10 12.2
Track and Field 92.7 2 2.4
Swimming 95.1 15 18.3
Softball 97.6 9 11
Volleyball 98.8 6 7.3
Tennis 96.3 9 11
Golf 97.6 3 3.7
Cheerleading 96.3 1 1.2
Bowling 100 2 2.4
Open in a new tab

Sample collection

A pilot study was conducted to assess the reliability of the survey by constructing a convenience sample that distributed the survey through alumni e-mail lists and social media groups from high schools where authors (D.S. and A.R) are alumni. The volunteer sample consisted of all alumni from these two high schools who responded to a request to fill out the survey. These high schools offered an advantage in that they maintain yearbook and/or online database records of student sports participation. The availability of such data from these high schools provided a straightforward method for evaluating the accuracy of self-reported high school sports participation data. Yearbook data were reviewed by two independent RAs on site at the individual schools. RAs abstracted data on sports participation by examining physical yearbooks or reviewing information from digital archives. The inspection of yearbooks involved examining all possible sources of sports participation information, including team photographs, rosters, and descriptions of extracurriculars for individual students. When multiple archival sources were available, all information was considered for the purposes of coding sport participation.

Disagreements in coding were rare, but when they occurred, they were resolved by consensus with input from all authors. The Institutional Review Board of the University of Pennsylvania reviewed the procedures of this study and determined that the study met eligibility criteria for IRB review exemption. Because the IRB judged the risk to be minimal to participants and the risk/benefit ratio to be reasonable, the IRB approved our process of providing the information about how the survey would be used to participants and not requiring explicit consent.

Statistical analysis

We examined the agreement between respondent’s self-reported sports participation and the information obtained from yearbooks/online databases by examining the percentage of survey responses that were in agreement with these external sources. For each individual sport, there were four possible outcomes: 1) the survey respondent reported participation that was confirmed by the school record, 2) the survey respondent reported participation but it was not confirmed by the school record, 3) the survey respondent did not report sports participation but there was a record of sports participation in the school yearbook/database, or 4) the survey respondent and the school record did not report participation. We removed cases from analysis if they could not be validated due to irregularities with the school resources available to us, e.g. if a sport was not included in the school record for a given year.

Results

Eighty-two respondents completed the survey. The average time it took to complete the survey was 15.8 minutes (SD = 23.7 minutes). Modal completion time for the full sample was 6 minutes. The mean age of the sample was 42.2 years (SD = 14.4y); 61% (n = 50) of the sample was male and 83% (n = 68) of respondents were white. Respondents reported playing an average of 2.02 (SD = 1.09) sports.

The percent agreement between our survey and the school resources for individual sports ranged between 91.5% and 100% (Table 1). Table 1 also displays the frequency of self-reported participation for each sport.

To illustrate the duration and intensity information gathered through the survey, we also report data for several measures of intensity, such as years of play, weekly hours spent participating in practice and games, and team achievements (i.e. playoff appearances and championships won) for the most frequently participated in sports—basketball and soccer (Table 2).

Table 2. Sports participation duration and intensity.

Basketball (N = 17) Soccer (N = 20)
Years spent playing sport
    Median 3 6
    Minimum 1 1
    1st Quartile 1 2
    3rd Quartile 5 11
    Max 12 13
Hours per week spent in practice
    1–4 0 1
    5–9 7 11
    10–14 9 7
    15–19 0 0
    20+ 1 1
Hours per week spent in games
    1–4 6 9
    5–9 8 9
    10–14 2 1
    15–19 0 0
    20+ 1 1
High school team achievements
    Made playoffs % 47.1 60
    Won championships % 11.8 35
Open in a new tab

Discussion

In this paper, we introduce a new survey for gathering retrospective data on youth-sports participation. Comparisons of participants’ responses with high school yearbook data suggested that most respondents were able to reliably report their high school athletic experiences. Out of 15 sports evaluated, agreement between survey data and school records was over 90% for all 15 sports. This was notable, considering that the average age of survey respondents was 42 years old, which would indicate that many participants in our sample were at least 25 years post-high school. Our study provides preliminary evidence that individuals are reliable reporters of their high school sports-participation, and the survey presented here is one instrument that could be used for this purpose. However, this is a pilot study with small sample sizes and further research is warranted.

Of note, our results support the validity of self-reported retrospective assessment of the highest frequency and highest concussion risk sports evaluated in the present sample. The self-reported retrospective assessment of football, basketball, soccer, and wrestling participation all demonstrated excellent agreement with participation recorded in school yearbooks.

Limitations

There are limitations of the present study that bear noting. The survey queries sports participation throughout childhood and adolescence. However, we were only able to evaluate the reliability of participants’ reports of high school sports. Hence, the present data do not allow us to evaluate the reliability of retrospective reports of sports in earlier childhood. Although we collected information about the intensity of sports participation, we did not have corroborative yearbook data to evaluate the reliability of these responses. Some of the sports we evaluated have very small sample sizes. The sample was a convenience sample comprised of the authors’ alumni networks, and not a representative sample of different types of schools across geographic regions. Hence it is possible that characteristics of the sample may have influenced our results in unanticipated ways.

Inconsistencies between survey responses and school records may have been due to memory errors on the part of survey respondents. There are two types of memory errors that would result in inaccurate self-report: retrieval errors (or failure to report prior participation that did take place) and confabulations (or erroneous report of sports participation that did not occur). Inconsistencies due to memory failure would most likely show the pattern of higher sports participation rates indicated by school records as compared to survey responses, as confabulatory errors are relatively uncommon for the type of autobiographical information assessed in the present study—memory for sustained regular participation in a highly engaging experience, rather than a brief non-distinctive repetitive event (i.e. a single instance of taking a daily medication [16]). However, aside from cheerleading, cross country, skiing, and frisbee, the counts for self-reported participation in all other sports were greater than or equal to the corresponding participation counts obtained from school resources. This finding suggests that inconsistency between self-reported participation and school records may be due to errors in record keeping, rather than memory failures on the part of the survey respondents. These discrepancies could be due to inaccurate school records related to student absence on the day of team yearbook photos or errors in listing all athletes’ names in photo captions.

Future directions

Limitations notwithstanding, the present study supports the reliability of retrospective self- reported sports participation, particularly in the highest frequency, highest concussion risk sports. This survey may be an efficient way to incorporate youth sports participation into existing datasets for research purposes, providing an opportunity to better characterize the risks and benefits of youth participation in contact sports on cognitive, physical, and emotional health across the lifespan.

Further assessment of the reliability of this survey could be a next step to integrating it in a large, representative sample. In doing so, we can better understand the long-term implications of adolescent sports participation on trajectories of health as well as financial and psychosocial well-being. Future studies may seek to replicate the present findings in a larger sample, which would enable the opportunity to compare reliability across different sports. Our survey contains measures of sports-intensity (time spent in participation and competitive levels) but we were not able to evaluate their reliability. Sports-intensity is potentially important for investigators to consider, because it may be relevant to both psychological and physical effects of athletic participation that may bear on health risks and benefits [17,18]. Future work could evaluate the reliability of the sports-intensity using collateral interviews (e.g. coaches, parents), local newspaper reports, and school records.

There are a number of longitudinal data sets that have been following individuals from childhood/adolescence into later life. The addition of reliable instruments for retrospective self-report of youth sports participation to these data sets would provide an efficient and cost- effective method for examining the effects of childhood/adolescent sports participation on later life outcomes.

Supporting information

S1 Survey. Survey used for study.

(PDF)

Click here for additional data file. (971.2KB, pdf)
S1 Dataset. Sports participation data.

(XLSX)

Click here for additional data file. (50KB, xlsx)
S2 Dataset. Cleaned participation data used for analysis.

(XLSX)

Click here for additional data file. (14.3KB, xlsx)
S1 Text. Dataset and survey question explanations.

(RTF)

Click here for additional data file. (4.2KB, rtf)

Acknowledgments

We thank all the survey participants and special thanks to Susan Paterson, Tim Seminerio, Genevieve Strycharz, Stephen Loy, Timothy Hetrich and Corey Jones for their help with accessing school resources.

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

SJ was supported by the Penn Undergraduate Research Mentoring program (https://www.curf.upenn.edu/content/penn-undergraduate-research-mentoring-program-purm) and the Wharton Dean’s Research Fund. NG was supported by the Leonard Davis Institute of Health Economics SUMR Scholars program (https://ldi.upenn.edu/education/penn-ldi-training-programs/sumr/). JW received funding from the Population Research Training Grant (NIH T32 HD007242) awarded to the Population Studies Center at the University of Pennsylvania by the NIH’s Eunice Kennedy Shriver National Institute of Child Health and Human Development (https://www.nichd.nih.gov/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.National Federation of State High School Associations. Participation in High School Sports Registers First Decline in 30 Years. 2019:3–6. [Google Scholar]
  • 2.Hart J, Kraut MA, Womack KB, et al. Neuroimaging of cognitive dysfunction and depression in aging retired national football league players. JAMA Neurol. 2013;70(3):326–335. doi: 10.1001/2013.jamaneurol.340 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Leachman M, Masterson K, Figueroa E. A Punishing Decade for School Funding. Cent Budg Policy Priorities. 2017:1–17. https://www.cbpp.org/sites/default/files/atoms/files/11- 29-17sfp.pdf. [Google Scholar]
  • 4.Ripley A. The Case Against High-School Sports. Atl Mon. 2013:72–78. [Google Scholar]
  • 5.Guo X, Meyerhoefer CD, Peng L. Participation in school-sponsored sports and academic spillovers: new evidence from the early childhood longitudinal survey. Appl Econ. 2019. doi: 10.1080/00036846.2018.1527466 [DOI] [Google Scholar]
  • 6.Ransom MR, Ransom T. Do High School Sports Build or Reveal Character? IZA Discuss Pap. 2017;11110(11110). [Google Scholar]
  • 7.Kniffin KM, Wansink B, Shimizu M. Sports at Work: Anticipated and Persistent Correlates of Participation in High School Athletics. J Leadersh Organ Stud. 2015;22(2):217–230. doi: 10.1177/1548051814538099 [DOI] [Google Scholar]
  • 8.Lunn PD, Kelly E. Participation in School Sport and Post-School Pathways: Evidence from Ireland. Natl Inst Econ Rev. 2015;232(1):51–66. doi: 10.1177/002795011523200106 [DOI] [Google Scholar]
  • 9.Alosco ML, Stanek KM, Galioto R, et al. Body mass index and brain structure in healthy children and adolescents. Int J Neurosci. 2014;124(1):49–55. doi: 10.3109/00207454.2013.817408 [DOI] [PubMed] [Google Scholar]
  • 10.Ou X, Andres A, Pivik RT, Cleves MA, Badger TM. Brain gray and white matter differences in healthy normal weight and obese children. J Magn Reson Imaging. 2015;42(5):1205–1213. doi: 10.1002/jmri.24912 [DOI] [PubMed] [Google Scholar]
  • 11.Eime RM, Young JA, Harvey JT, Charity MJ, Payne WR. A systematic review of the psychological and social benefits of participation in sport for adults: Informing development of a conceptual model of health through sport. Int J Behav Nutr Phys Act. 2013;10. doi: 10.1186/1479-5868-10-135 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Tarnutzer AA, Straumann D, Brugger P, Feddermann-Demont N. Persistent effects of playing football and associated (subconcussive) head trauma on brain structure and function: A systematic review of the literature. Br J Sports Med. 2017;51(22):1592–1604. doi: 10.1136/bjsports-2016-096593 [DOI] [PubMed] [Google Scholar]
  • 13.Caruana EJ, Roman M, Hernández-Sánchez J, Solli P. Longitudinal studies. J Thorac Dis. 2015;7(11):E537–E540. doi: 10.3978/j.issn.2072-1439.2015.10.63 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Corrigan JD, Bogner J. Initial reliability and validity of the Ohio State University TBI identification method. J Head Trauma Rehabil. 2007;22(6):318–29. doi: 10.1097/01.HTR.0000300227.67748.77 [DOI] [PubMed] [Google Scholar]
  • 15.Hibbard MR, Ashman TA, Spielman LA, Chun D, Charatz HJ, Melvin S. Relationship between depression and psychosocial functioning after traumatic brain injury. Arch Phys Med Rehabil. 2004;85:43–53. doi: 10.1016/j.apmr.2003.08.116 [DOI] [PubMed] [Google Scholar]
  • 16.Tourangeau R. Remembering what happened: Memory Errors and Survey Reports. Sci self-report Implic Res Pract. 2000;(January 2000):29–47. [Google Scholar]
  • 17.O’Connor KL, Baker MM, Dalton SL, Dompier TP, Broglio SP, Kerr ZY. Epidemiology of sport-related concussions in high school athletes: National athletic treatment, injury and outcomes network (NATION), 2011–2012 through 2013–2014. J Athl Train. 2017;52(3):175–185. doi: 10.4085/1062-6050-52.1.15 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Downs A., & Ashton J. Vigorous physical activity, sports participation, and athletic identity: Implications for mental and physical health in college students. J Sport Behav. 2011;34(3):228. [Google Scholar]

Decision Letter 0

Javier Brazo-Sayavera

13 Jan 2021

PONE-D-20-27226

Retrospective survey of youth sports participation: development and validation using school records

PLOS ONE

Dear Dr. Jin,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Due to the current world situation is being difficult to find reviewers with time to help with the manuscripts. It has been a difficult way but we have a couple of reviewers that have made a great review. After their consideration, a major revision is needed. 

In addition, the journal policy implies to do the data fully available. Then, please, address this issue as well. 

Please submit your revised manuscript by Feb 20 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Javier Brazo-Sayavera, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy of the full version, in both the original language and English, as Supporting Information.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper attempts to introduce and validate a retrospective survey of sports participation by utilising respondents self-reported sports participation data and validating responses using yearbook and online data bases. The introduction to the paper raised some interesting points, specifically around the investigation of physical and psychological risks of long-term sports participation, however no attempt seemed to be made to answer these questions There is also a lack of clarity relating to the methods section, particularly around the items used in the questionnaire – especially what was to be validated and what was not. The data set relating to intensity did not seem to be available. More broadly, it is felt that the aim of the paper and the associated research question is underdeveloped and unclear – the purpose and the need for the survey is questioned. As such, it is believed the paper is not currently suitable for publication. Below is a sectional review of the paper – it is hoped the authors find the more focused comments useful.

Introduction

The introduction begins with an opening line, and much of the opening paragraph, highlighting the detrimental effects of repeated head injury, however the authors do not follow this sporting and social dilemma within the survey investigation. The need to undertake work to survey longitudinal data relating to head trauma and sports participation versus the dilemma of the positive effects of sports participation were established on page 1 and 2 – however this did not seem to continue through the remaining sections of the paper.

For example, the authors need to clarify why participants were not asked if they suffered concussion during their time at high school. Why was this question not included in the survey – especially when they included questions relating to intensity, which in the end could not be validated?

Page 3 paragraph 2, line 1-5: A decline in sports participation was noted, however these figures were not substantiated. For example, if a drop in 2018-19 was recorded what were the previous years figures?

Page 4, 1st paragraph, line 5-7: mixed findings from previous research (and there seems to be numerous studies available) warrants further research investigating the impact of high school sports on concurrent and later-life outcomes. This process of investigation does not seem to be associated with the aims and survey development outlined? It was felt the introduction takes the reader on an important journey and raises important issues, but the path does not lead anywhere.

It may be useful for the authors to utilise the introduction in a more focused way – i.e. what the gap in the current literature is and where this paper fits within that. Within this, it would also be useful to offer more detail on how existing survey tools and questionnaires require this improvement.

Methods

Survey development:

Line 1: What survey items were developed by the authors and what informed their inclusion? How did this compare with other surveys? What are the differences – this will assist in understanding why we need this survey?

Line 1 and 2: Sports were included for the potential of concussion/risk to hit on head; however, this was not a question in the survey (see above point).

How many items were included in the survey?

Results:

In the discussion section it is noted this is indeed a pilot study. It would have been useful to highlight this in the introduction or methods. This can account for the low numbers associated with the paper.

Also, as the number of items were not included in the survey development section it is difficult to comment on the sample size. However, it seems, for some sports, a high agreement scores were achieved for low participant numbers.

Further clarity is required regarding what participants were agreeing to (the items) in order to achieve full recall after at least 20+ years out of high school. For example, in the ‘weakness’ section of the discussion it states that the intensity data could not be validated – therefore was only a single binary question relating to type of sport and confirmation of participation requested? This is unclear.

Did the information contained in High School year books assist in item formation? If it did not, the study assumes all high school yearbooks will contain comparative data. This may be the case if the items used for validation was a simple yes/no to the sports the respondents participated in – but this is not clear.

Discussion:

Page 10 line 1: This feels like quite an optimistic statement, mostly due to sample sizes when breaking the findings down to specific sports.

In the limitations part (there is no specific limitations section) it is noted that the study provides no way to evaluate the validity of the sports intensity measures – this is a crucial omission, as it does not provide the opportunity to delve deeper into the psychological and physical effects of athletic participation. In addition, the first paragraphs of the study seem to outline what data could or would have been useful to collect, rather than focusing on what was collected. In order to rectify this the authors may wish to consider a restructure the paper.

Also, in the limitations (weakness) section the use of a convenience sample was described – this should be noted and acknowledged in the methods section.

In the final paragraph there is first mention of ‘pilot study’ – this should have been detailed in the methods section.

It may be useful for the authors to restructure the discussion section by having more specific sections. For example, the authors offer some discussion on possible applications, however this could be more focused. In addition, a separate limitations section would also be useful.

Reviewer #2: Big picture

The topic the authors address is a relevant issue in the field and the authors should be commended on their attempt to validate a measure of previous sport participation. The study over all was well written and presented, however there are some concerns that need to be addressed.

Specifically the use of Cronbach’s alpha and Cohen’s kappa as appropriate for this study i needed. Previously internal consistency is shown as correlations between different items or sub scales on the same measure not between measures, and inter-rater reliability is between separate observers, RA's etc. using the same measure, again not between measures. Justification for using these methods for a between measures analysis is needed.

"For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available." Any data on hours, years, measures of intensity, as the authors labeled it, is missing from the submission.

Details

Introduction

I found it difficult to see the connection of the introduction (particularly the first three paragraphs) to the purpose and results of the paper. While the introduction was clear and well written, the link between the focus on contact sports and concussions as the supporting literature and the validation of the tool itself was missing. I would suggest cutting the first section on contact sports and CTE unless authors can explicitly link it to the validating of the measure on all sport participation. There is literature that highlights a need for a valid measure of previous sport participation, and this would be better suited for the introduction. Further the authors highlight the optimal ways of collecting data (prospective, longitudinal) but then settle on a retrospective method, while they provide some information as to why the other methods aren’t often used, there is no explanation why their retrospective method is used or why it is beneficial.

Sample Methods

The authors need to explain why photo and yearbook rosters were used over other methods. The authors themselves point out that people could often be away for picture day and year books could have incorrect or missing names. Additionally, depending when the photos were taken athletes could have quit the team soon after and therefore the information would not match. It is an interesting and novel approach, but further justification is needed.

Statistical analysis

“self-report used as one source of data and the school record used as another”

The authors should explain why these atypical approaches are suitable for their purposes, especially on top of the approach of percent agreement. Are these approaches appropriate to use for inter-rater reliability or internal consistency? If so, the authors need to provide examples of when this has been done in the past or why this should be considered an appropriate use of Cronbach’s alpha and Cohen’s kappa.

Results

The authors should provide justification for removing participants who took a long time to complete. Was completing the survey in one time session required? Why was interruption a problem ?

The purpose of the paper was to validate a measure of sport participation, yet there is not indication of an effort to validate measure of intensity. The authors collected information on hours, year and achievements, but this section was not validated? I suggest the authors explain why they did not validate this section, show that is was validated or remove it from the paper. The authors did mention that the year-book does not provide measures on intensity but there are other methods for validating this information, or if it does not fit with the year-book method then it should be removed.

Table 1.

Title is misleading, “Interrater Reliability for Survey Responses and School Resources” as this table also shows, internal consistency and precent agreement?

Discussion

Again the authors highlight concerns around using the yearbook information to validate measures “inconsistency between self-reported participation and school records may be due to errors in record keeping, rather than memory failures on the part of the survey respondents”

and therefore the authors need to make a stronger argument for use of yearbooks.

The discussion highlights the need for information on intensity, but the study fails to validate this information. Such a focus on intensity and consequences of intensity in the discussion seems out of place when measures of intensity were not validated.

Supplemental information

The authors provide the data for the coding and analysis of sports but there is not data with intensity information ?

Overall, the study provides a novel approach to validating a measure of high school sport participation and is an important endeavour, however the paper requires further explanations and some reorganizing to be suitable for publication.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Alexandra Mosher

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Sep 17;16(9):e0257487. doi: 10.1371/journal.pone.0257487.r002

Author response to Decision Letter 0

30 Mar 2021

Reviewer 1 Comment 1: This paper attempts to introduce and validate a retrospective survey of sports participation by utilising respondents self-reported sports participation data and validating responses using yearbook and online databases. The introduction to the paper raised some interesting points, specifically around the investigation of physical and psychological risks of long-term sports participation, however no attempt seemed to be made to answer these questions There is also a lack of clarity relating to the methods section, particularly around the items used in the questionnaire – especially what was to be validated and what was not. The data set relating to intensity did not seem to be available. More broadly, it is felt that the aim of the paper and the associated research question is underdeveloped and unclear – the purpose and the need for the survey is questioned. As such, it is believed the paper is not currently suitable for publication. Below is a sectional review of the paper – it is hoped the authors find the more focused comments useful.

Thanks for the useful comments. We have edited the paper to clarify that the aim of the paper is to introduce a survey of retrospective sports participation and to validate aspects of the survey. We have edited the abstract and introduction to clarify these aims. For example, we have rewritten the first two lines of the abstract to be, “Many youth participate in sports and it is of interest to understand the impact of youth sports participation on later-life outcomes. However, prospective studies take a long time to complete and retrospective studies may be more practical and time-efficient to address some questions.” We have edited the introduction and discussion to clarify that the survey is intended to help with investigating physical and psychological risks and benefits of sports participation, but we do not answer questions about the risks and benefits in this paper. We have rewritten the methods section to clarify which responses in the survey were validated and which were not. We have made the data set related to intensity available.

Reviewer 1 Comment 2: The introduction begins with an opening line, and much of the opening paragraph, highlighting the detrimental effects of repeated head injury, however the authors do not follow this sporting and social dilemma within the survey investigation. The need to undertake work to survey longitudinal data relating to head trauma and sports participation versus the dilemma of the positive effects of sports participation were established on page 1 and 2 – however this did not seem to continue through the remaining sections of the paper. For example, the authors need to clarify why participants were not asked if they suffered concussion during their time at high school. Why was this question not included in the survey – especially when they included questions relating to intensity, which in the end could not be validated?

We agree with the reviewer that the introduction could be better structured to set up the aims of the study. In response to this comment, we have restructured the introduction to lead with this discussion of youth sports participation, which is the subject of the survey. We did not include survey items on history of brain injury/concussion because validated measures for assessing history of brain injury history are already available. We now include this information in the introduction.

Reviewer 1 Comment 3: Page 3 paragraph 2, line 1-5: A decline in sports participation was noted, however these figures were not substantiated. For example, if a drop in 2018-19 was recorded what were the previous years figures?

The number of participants in high school sports declined from 7,980,886 in 2017-2018 to 7,937,491 in 2018-2019. These figures come from the National Federation of State High School Associations. Our revised introduction now includes these figures.

Reviewer 1 Comment 4. Page 4, 1st paragraph, line 5-7: mixed findings from previous research (and there seems to be numerous studies available) warrants further research investigating the impact of high school sports on concurrent and later-life outcomes. This process of investigation does not seem to be associated with the aims and survey development outlined? It was felt the introduction takes the reader on an important journey and raises important issues, but the path does not lead anywhere.

It may be useful for the authors to utilise the introduction in a more focused way – i.e. what the gap in the current literature is and where this paper fits within that. Within this, it would also be useful to offer more detail on how existing survey tools and questionnaires require this improvement.

We have now restructured the introduction per the reviewers’ suggestion, putting discussion of youth sports participation up front, and also adding detail on the current measures available for assessing prior brain injury. In the penultimate paragraph of the introduction, we directly state the gap in the current literature, and how the survey presented in the paper fills that gap.

Reviewer 1 Comment 5: What survey items were developed by the authors and what informed their inclusion? How did this compare with other surveys? What are the differences – this will assist in understanding why we need this survey? Sports were included for the potential of concussion/risk to hit on head; however, this was not a question in the survey (see above point). How many items were included in the survey?

We developed our survey using a series of blocks to assess sports participation at various stages of one's education. This included a block for grades K-6, grades 7-8, grades 9-12, and college. We structured the survey this way to assess the timing and duration of sports participation while inherently collecting information about competitiveness (eg, participation in college sports is likely to be more competitive than grades 7-8 or high school [grades 9-12]). The four blocks (K-6, grades 7-8, grades 9-12, and college) were identical in the sense that the same series of questions and response options were provided. In addition, we included an introductory block to collect demographic information on the respondent (e.g., age, gender, race, ethnicity, educational attainment, and where they attended high school).

The sports included in our survey were derived, in part, from the high school athletics participation survey conducted by the National Federation of State High School Associations. We also took cues from the Life History Survey 2017 of the Health and Retirement Study which assessed whether respondents participated in sports during junior/middle and high school. However, one potential limitation of the aforementioned survey is that it has not been validated. Thus, we developed the survey items for our survey by drawing from these existing surveys in addition to collecting detailed information about high school attendance history which allowed us to validate survey responses using historical yearbook data.

In total, there were 545 survey items but these were based on branching logic; if an individual participated in every listed support at each time point, they would have responded to 545 queries.

Reviewer 1 Comment 6: In the discussion section it is noted this is indeed a pilot study. It would have been useful to highlight this in the introduction or methods. This can account for the low numbers associated with the paper. Also, as the number of items were not included in the survey development section it is difficult to comment on the sample size. However, it seems, for some sports, a high agreement scores were achieved for low participant numbers. Further clarity is required regarding what participants were agreeing to (the items) in order to achieve full recall after at least 20+ years out of high school. For example, in the ‘weakness’ section of the discussion it states that the intensity data could not be validated – therefore was only a single binary question relating to type of sport and confirmation of participation requested? This is unclear. Did the information contained in High School year books assist in item formation? If it did not, the study assumes all high school yearbooks will contain comparative data. This may be the case if the items used for validation was a simple yes/no to the sports the respondents participated in – but this is not clear.

We have now highlighted the fact that our study is indeed a pilot study earlier in the abstract and introduction of our paper. We have also added an acknowledgement of the small sample sizes -- see last sentence of first paragraph of the Discussion. The high school yearbooks were not used to design our survey items. Rather, we used the yearbook to validate the survey results.

We have provided further clarity about the survey and what respondents were agreeing to by adding a subsection “Survey” to the Methods Section of our revised manuscript in which we write “In our survey, subjects were asked to select the sports in which they participated from a list of radio buttons. The survey used a series of blocks to assess sports participation at various stages of one's education. This included a block for grades K-6, grades 7-8, grades 9-12, and college. For each block corresponding to a specific period in the life course, respondents were asked to select the sports in which they participated from a list of radio buttons. Using this, binary variables were constructed for participation in each sport during high school. Respondents also indicated whether, at each specific period, they had any physical limitations that prevented them from participating in sports. For each selected sport at each time period, respondents were asked to report (using radio buttons) the specific grades in which they participated, whether their team (a) traveled for games or tournaments, (b) participated in playoffs, (c) won a championship, or (d) none of the above. Respondents also indicated how many hours per week (0 hours, 1-4 hours, 5-9 hours, 10-14 hours, 15-19 hours, 20+ hours) they spent practicing and playing games for each specific sport, in addition to the position(s) they played.”

Reviewer 1 Comment 7: Page 10 line 1: This feels like quite an optimistic statement, mostly due to sample sizes when breaking the findings down to specific sports. In the limitations part (there is no specific limitations section) it is noted that the study provides no way to evaluate the validity of the sports intensity measures – this is a crucial omission, as it does not provide the opportunity to delve deeper into the psychological and physical effects of athletic participation. In addition, the first paragraphs of the study seem to outline what data could or would have been useful to collect, rather than focusing on what was collected. In order to rectify this the authors may wish to consider a restructure the paper. Also, in the limitations (weakness) section the use of a convenience sample was described – this should be noted and acknowledged in the methods section. In the final paragraph there is first mention of ‘pilot study’ – this should have been detailed in the methods section. It may be useful for the authors to restructure the discussion section by having more specific sections. For example, the authors offer some discussion on possible applications, however this could be more focused. In addition, a separate limitations section would also be useful.

Thanks for these helpful comments. We have qualified the statement originally on Page 10, line 1: “Our study provides preliminary evidence that individuals are reliable reporters of their high school sports-participation, and the survey presented here is one instrument that could be used for this purpose. However, this is a pilot study with small sample sizes and further research is warranted.”

We have edited the first sentence of the “Sample Collection” section of the Methods section to acknowledge that this is a pilot study with a convenience sample: “A pilot study was conducted to assess the reliability of the survey by constructing a convenience sample that distributed the survey through alumni e-mail lists and social media groups from high schools where authors (D.S. and A.R) are alumni.”

We have restructured the discussion to have more specific sections and provided a more focused direction of possible applications in our “Future Directions” section.

Reviewer 2 Comment 1: The topic the authors address is a relevant issue in the field and the authors should be commended on their attempt to validate a measure of previous sport participation. The study overall was well written and presented, however there are some concerns that need to be addressed. Specifically the use of Cronbach’s alpha and Cohen’s kappa as appropriate for this study is needed. Previously internal consistency is shown as correlations between different items or sub scales on the same measure not between measures, and inter-rater reliability is between separate observers, RA's etc. using the same measure, again not between measures. Justification for using these methods for a between measures analysis is needed. Any data on hours, years, measures of intensity, as the authors labeled it, is missing from the submission.

We appreciate the reviewer’s overall assessment of our work. We concede that our use of Cronbach’s alpha and Cohen’s kappa is somewhat nonstandard. In our revised manuscript, we have removed the analysis based on Cohen’s kappa. We have, however, retained the analysis based on Cronbach’s alpha for the following reason. Essentially, we have two ostensible measurements of the single construct, sports participation. Computing Cronbach’s alpha allows us to assess to the extent to which these two measures really are measuring the same thing. We have included this justification in our revised manuscript. We will make a de-identified data file available.

Reviewer 2 Comment 2: I found it difficult to see the connection of the introduction (particularly the first three paragraphs) to the purpose and results of the paper. While the introduction was clear and well written, the link between the focus on contact sports and concussions as the supporting literature and the validation of the tool itself was missing. I would suggest cutting the first section on contact sports and CTE unless authors can explicitly link it to the validating of the measure on all sport participation. There is literature that highlights a need for a valid measure of previous sport participation, and this would be better suited for the introduction. Further the authors highlight the optimal ways of collecting data (prospective, longitudinal) but then settle on a retrospective method, while they provide some information as to why the other methods aren’t often used, there is no explanation why their retrospective method is used or why it is beneficial.

We have now restructured the introduction (see R1.2), cutting some of the background on CTE, and adding a more focused discussion of how the present study fills gaps in the literature. We have also added the following to the final paragraph of the introduction,

“Retrospective assessment of youth sports participation has the advantage of allowing investigators to capitalize on existing longitudinal datasets that have already tracked cognitive and health outcomes into later-life, when some of the deleterious effects of brain injury are thought to manifest.”

Reviewer 2 Comment 3. The authors need to explain why photo and yearbook rosters were used over other methods. The authors themselves point out that people could often be away for picture day and year books could have incorrect or missing names. Additionally, depending when the photos were taken athletes could have quit the team soon after and therefore the information would not match. It is an interesting and novel approach, but further justification is needed.

We considered several other potential sources against which to validate the self-reported sports participation data. Unfortunately many of them, such as retrospective reports from parents, siblings, coaches, and friends, are subject to the same potential recall biases as the self-reports themselves. In the absence of prospectively collected participation data, yearbooks and school records offer a nearly-contemporaneous source of sports participation that is not subject to recall bias.

That being said, the referee is correct that yearbooks are not 100% reliable. Nevertheless, our preliminary results suggest that there may be considerable agreement between retrospective self-reports and yearbook records.

Reviewer 2 Comment 4: The authors should explain why these atypical approaches are suitable for their purposes, especially on top of the approach of percent agreement. Are these approaches appropriate to use for inter-rater reliability or internal consistency? If so, the authors need to provide examples of when this has been done in the past or why this should be considered an appropriate use of Cronbach’s alpha and Cohen’s kappa.

In a certain sense, we have two items (yearbook record and self-report) that purport to measure the same thing (sports participation). As a result, we argue that it is useful to consider the internal consistency of these two items and to report the value of Cronbach’s alpha.

Reviewer 2 Comment 5. The authors should provide justification for removing participants who took a long time to complete. Was completing the survey in one time session required? Why was interruption a problem?

We thank the reviewer for their comments. We wish to make a point of clarification regarding the removal of participants. These respondents were not removed from the analytic sample; we only excluded them to calculate the adjusted mean survey completion time after accounting for outliers. However, we agree that this may be a point of confusion in the manuscript. As such, we removed this comment from the main text.

Reviewer 2 Comment 6: The purpose of the paper was to validate a measure of sport participation, yet there is no indication of an effort to validate measure of intensity. The authors collected information on hours, year and achievements, but this section was not validated? I suggest the authors explain why they did not validate this section, show that it was validated, or remove it from the paper. The authors did mention that the year-book does not provide measures on intensity but there are other methods for validating this information, or if it does not fit with the year-book method then it should be removed.

We considered other methods for validating intensity items. Detailed school reports of participation intensity were not available for this purpose. We considered collateral report from either parents or other school sources, however, these reports would have been burdensome to acquire, and we were not at all confident that they would be appropriate measures for validating this information-- that is, the retrospective reports from collaterals would also be subject to memory biases. Hence, we determined that there was no appropriate comparison standard for evaluating the validity of these items in the present study.

Despite this limitation, we have decided to include the intensity data in the manuscript. These data highlight an aspect of the survey that captures more granular information on sports participation, which we believe is a potential strength of the survey. The predictive validity of these data could be established in future work examining relationships between sports participation intensity and outcomes of interest. Such an examination is outside of the scope of the current work, and an important direction for future research.

Reviewer 2 Comment 7: Table 1: Title is misleading, “Interrater Reliability for Survey Responses and School Resources” as this table also shows, internal consistency and percent agreement?

Thank you for pointing this out. We will rename the table “Percent Agreement and Internal Consistency for Survey Responses and School Resources”

Reviewer 2 Comment 8: Again the authors highlight concerns around using the yearbook information to validate measures “inconsistency between self-reported participation and school records may be due to errors in record keeping, rather than memory failures on the part of the survey respondents” and therefore the authors need to make a stronger argument for use of yearbooks. The discussion highlights the need for information on intensity, but the study fails to validate this information. Such a focus on intensity and consequences of intensity in the discussion seems out of place when measures of intensity were not validated.

We appreciate the reviewer’s pointing this out. As we argue in response to Reviewer 2 Comment 3, yearbooks are not subject to the same biases as self-report or collateral interviews with parents, friends, and school officials. The relatively high agreement observed in our pilot study suggests that there may not be substantial errors in record keeping. We have reduced the emphasis on intensity in the discussion of the revised manuscript.

Reviewer 2 Comment 9: The authors provide the data for the coding and analysis of sports but there is not data with intensity information ?

We have now included a data file that contains intensity information such as years played, hours spent per week, and team/individual achievements.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (29.6KB, docx)

Decision Letter 1

Javier Brazo-Sayavera

27 May 2021

PONE-D-20-27226R1

Retrospective survey of youth sports participation: development and validation using school records

PLOS ONE

Dear Dr. Jin,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Pay attention to reviewer's comments in order to address all the specific issues. We are sorry for the delay but it is too difficult to find reviewers with enough expertise for this review. 

Please submit your revised manuscript by Jul 11 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Javier Brazo-Sayavera, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Partly

Reviewer #3: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: No

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: I appreciated that the authors took the time to respond to all comments and to revise some aspects of the paper, however there are still some concerns that need to be addressed.

BIG PICTURE

The main concern is still around the use of Cronbachs alpha. If you are only looking at number of sports from two different sources, then technically you only have two items. There are concerns around using cronbach’s alpha with only two items See, Eisinga, R., Te Grotenhuis, M., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown?. International journal of public health, 58(4), 637-642. The authors might consider only using percent agreement in this study.

There is a concern around the use of the word validity and validation in the study. Finding reliability does not necessarily mean that a measure is valid. It can be misleading to use the word validity when there was no measure of validity in the study. This may be incorrect, so I leave it up to the editors discretion.

Survey development / Survey

I am unsure how the sports included in the study were chosen at the authors state two different reasons. “The sports included in the survey were selected for their popularity at the high school/collegiate level and the potential risk for a hit on the head or concussion” and

“The specific 15 sports (listed in Table 1) included in our survey were selected based on sports participation surveys conducted by the National Federation of State High School Associations.”

Please clarify which was the correct criteria for inclusion.

Statistical analysis

Please edit this sentence it says context twice and reads awkwardly - “ In our context, high values of Cronbach’s alpha in our context indicate that the retrospective self-reports and yearbook records are roughly measuring the same construct, sports participation”

Discussion

The last sentence mentions of ‘all three metrics’ which three are you referring to? Or do you mean both metrics?

Limitations

I do not know what you mean by “validity sample” in the sentence “ The validity sample was a convenience sample comprised of the authors’ alumni networks”

Future directions

Again the use of validity seems misleading ‘Limitations not withstanding, the present study supports the validity of retrospective self-reported sports participation, particularly in the highest frequency, highest concussion risk sports.”

I believe you supported the reliability of the measure?

Reviewer #3: Thank you for responding to reviewers' recommendations thoroughly. I do think that in the absence of actual longitudinal studies, this retrospective method with some improvements could fill the current gap. I would remove all associations with contact sport reference, and keep it broad and just concentrate on validation of the survey as a method, confirming that long term sport participation need to be investigated for its benefits too, precisely because of the large drop in participation observed in recent years. Just a small issue of typos - participants' not participant's. Check manuscript throughout.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Alexandra Mosher

Reviewer #3: Yes: Dr Erika Borkoles

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Sep 17;16(9):e0257487. doi: 10.1371/journal.pone.0257487.r004

Author response to Decision Letter 1

17 Jul 2021

Reviewer 2 Comment 1: BIG PICTURE

The main concern is still around the use of Cronbachs alpha. If you are only looking at number of sports from two different sources, then technically you only have two items. There are concerns around using cronbach’s alpha with only two items See, Eisinga, R., Te Grotenhuis, M., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown?. International journal of public health, 58(4), 637-642. The authors might consider only using percent agreement in this study.

Response: We are grateful for the helpful references. We agree with the referee that Cronbach’s alpha may be inappropriate in this setting and we have removed this analysis from our revised manuscript. We now report only percent agreement between the yearbook record and survey self-report.

Reviewer 2 Comment 2: There is a concern around the use of the word validity and validation in the study. Finding reliability does not necessarily mean that a measure is valid. It can be misleading to use the word validity when there was no measure of validity in the study. This may be incorrect, so I leave it up to the editors discretion.

Response: Thank you for bringing up the distinction between reliability and validity. We have replaced the word “validity” with “reliability” throughout the manuscript and retitled the manuscript “The sample was a convenience sample comprised of the authors’ alumni networks, and not a representative sample of different types of schools across geographic regions.”

Review 2 Comment 3: Survey development / Survey

I am unsure how the sports included in the study were chosen at the authors state two different reasons. “The sports included in the survey were selected for their popularity at the high school/collegiate level and the potential risk for a hit on the head or concussion” and

“The specific 15 sports (listed in Table 1) included in our survey were selected based on sports participation surveys conducted by the National Federation of State High School Associations.”

Please clarify which was the correct criteria for inclusion.

Response: Thanks for pointing out this discrepancy. We have clarified the correct criteria for inclusion as “The specific 15 sports (listed in Table 1) included in our survey were selected based on sports participation surveys conducted by the National Federation of State High School Associations as well as knowledge of other sports that were popular at the schools where the pilot study described in the next section was conducted.”

Reviewer 2 Comment 4: Statistical analysis

Please edit this sentence it says context twice and reads awkwardly - “ In our context, high values of Cronbach’s alpha in our context indicate that the retrospective self-reports and yearbook records are roughly measuring the same construct, sports participation”

Response: Thanks for mentioning this. We have removed this sentence because we have removed the use of Cronbach’s alpha.

Reviewer 2 Comment 5: Discussion

The last sentence mentions of ‘all three metrics’ which three are you referring to? Or do you mean both metrics?

Response: Thanks for pointing this out. We edited this sentence to “Of note, our results support the validity of self-reported retrospective assessment of the highest frequency and highest concussion risk sports evaluated in the present sample. The self-reported retrospective assessment of football, basketball, soccer and wrestling participation all demonstrated excellent agreement with participation recorded in school yearbooks”

Reviewer 2 Comment 6: Limitations

I do not know what you mean by “validity sample” in the sentence “ The validity sample was a convenience sample comprised of the authors’ alumni networks”

Response: Thanks for bringing this up. We removed the word “validity” so that the sentence is now “The validity sample was a convenience sample comprised of the authors’ alumni networks, and not a representative sample of different types of schools across geographic regions.”

Reviewer 2 Comment 7: Future directions

Again the use of validity seems misleading ‘Limitations not withstanding, the present study supports the validity of retrospective self-reported sports participation, particularly in the highest frequency, highest concussion risk sports.”

I believe you supported the reliability of the measure?

Response: Thanks for bring this up. We have replaced “validity” with “reliability” in the sentence so it now reads: “Limitations not withstanding, the present study supports the reliability of retrospective self-reported sports participation, particularly in the highest frequency, highest concussion risk sports.”

---------------------------------------------------------------------------------------------------------------------

Reviewer 3 Comment 1: Thank you for responding to reviewers' recommendations thoroughly. I do think that in the absence of actual longitudinal studies, this retrospective method with some improvements could fill the current gap. I would remove all associations with contact sport reference, and keep it broad and just concentrate on validation of the survey as a method, confirming that long term sport participation need to be investigated for its benefits too, precisely because of the large drop in participation observed in recent years.

Response: We have removed the paragraph reviewing the literature on contact sports and risk of traumatic brain injury. Now the introduction is more broadly focused on the long-term implications of sports participation.

Reviewer 3 Comment 2: Just a small issue of typos - participants' not participant's. Check manuscript throughout.

Response: Thanks for pointing this out. We have replaced participant’s with participants’ throughout the manuscript.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (17.1KB, docx)

Decision Letter 2

Javier Brazo-Sayavera

3 Sep 2021

Retrospective survey of youth sports participation: development and validation using school records

PONE-D-20-27226R2

Dear Dr. Jin,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Javier Brazo-Sayavera, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Please, review the whole text to correct the typos.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #3: No

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: (No Response)

Reviewer #3: The authors satisfactorily addressed the reviewers' comments. There are a lot of typos in the text. Please do address these.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #3: Yes: Erika Borkoles

Acceptance letter

Javier Brazo-Sayavera

9 Sep 2021

PONE-D-20-27226R2

Retrospective survey of youth sports participation: development and  assessment of reliability using school records

Dear Dr. Jin:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Javier Brazo-Sayavera

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Survey. Survey used for study.

    (PDF)

    Click here for additional data file. (971.2KB, pdf)
    S1 Dataset. Sports participation data.

    (XLSX)

    Click here for additional data file. (50KB, xlsx)
    S2 Dataset. Cleaned participation data used for analysis.

    (XLSX)

    Click here for additional data file. (14.3KB, xlsx)
    S1 Text. Dataset and survey question explanations.

    (RTF)

    Click here for additional data file. (4.2KB, rtf)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (29.6KB, docx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (17.1KB, docx)

    Data Availability Statement

    All relevant data are within the paper and its Supporting Information files.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES