Best practices for reporting survey‐based research

In our previous papers, we reviewed the best practices for the development and distribution of surveys and summarized the process for collecting validity evidence to support the use of one's survey results. ¹ , ² The final challenge for authors of survey‐based research is to succinctly, yet completely, document their approach to survey design and administration and to report their results. In Figure 1, we seek to highlight the best practices for reporting survey‐based research.

FIGURE 1.

Best practices for reporting survey‐based research.

Several published guidelines offer authors a roadmap to inform their approach to reporting. It is important to consider that these guidelines pertain to research using surveys to assess outcome measures and not solely the reporting of the de novo development of a survey instrument. Sharma and colleagues ³ recently published a consensus‐based checklist for reporting of survey studies (CROSS). The CROSS checklist contains 40 items, derived from a modified Delphi process, that offers specific, granular, and comprehensive recommendations; it is a good starting point for authors reporting survey‐based research. Artino and colleagues ⁴ also published a reporting checklist for survey‐based research articles submitted to Academic Medicine. Their checklist differs from the CROSS guidelines in that they emphasize that authors should integrate a framework‐based validity argument throughout their manuscript. The Standards for Educational and Psychological Testing (i.e., the Standards) have adopted and adapted Messick's unified validity framework, defining five sources of validity evidence: content, response process, internal structure, relations to other variables, and consequences of testing. ⁵ , ⁶ Content validity refers to the appropriateness of survey content in light of the construct being measured. Response process refers to the cognitive and psychological processes survey takers are engaged in while completing the survey. The internal structure of the survey is evaluated through statistical analysis (e.g., reliability and factor analyses) of the relationships between the survey items and the underlying constructs. Evaluating expected relationships between survey variables and other externally measured variables provides additional evidence for the validity of one's survey results and inferences. Consequences of testing, as outlined by Messick and the Standards, refer to the positive or negative, intended or unintended effects of survey use.

Readers should also note two caveats for the recommendations outlined in Figure 1. First, it is important to tailor your reporting to the stakes of the study being reported, to the requirements of the journal, and to the rigor required to publish that work. In other words, not all of the recommendations listed will be applicable to every survey study. For example, a survey assessing the implementation of a new, local educational curricula on a small group of learners would not necessarily employ focus groups or cognitive interviews but would still benefit from a clear description of survey constructs and adherence to best practices for survey item writing. Second, readers should note that some of the reporting recommendations may be reasonably placed in different sections of the paper than those outlined here. It is more important that reporting be complete than that any one reporting recommendation be placed in a particular location within the manuscript.

In the introduction, it is recommended that authors delineate why a survey was used and to define the constructs that the survey aims to measure. The methods section should fully describe the development of the survey including any adaptation from previously published instruments. Authors should also summarize the process for writing and revising survey items including any expert reviews, pretesting (to include cognitive interviewing), and/or pilot testing. The methods section should also describe the means of survey administration, target population, and follow‐up practices; it should also outline any planned statistical analysis. In addition, a copy of the full survey instrument should be provided in the final manuscript or an online supplement (if space is limited). Furthermore, authors should consider using the results section as an opportunity to summarize the collected validity evidence by reporting measures of internal structure (e.g., Cronbach's alpha, factor analysis) and the relationship of survey variables to conceptually related constructs. Authors may additionally refer back to their derivation processes outlined in the introduction and methods sections to support the survey's content and respondents’ response processes. Finally, in their discussion section, authors should provide an overall interpretation of the study, describe any limitations, and hypothesize on positive or negative consequences of survey administration.

Hill J, Chuko J, Ogle K, Gottlieb M, Santen SA, Artino AR Jr. Best practices for reporting survey‐based research. AEM Educ Train. 2024;8:e10929. doi: 10.1002/aet2.10929