Open Science Practices in Communication Sciences and Disorders: A Survey

Abstract

Purpose:

Open science is a collection of practices that seek to improve the accessibility, transparency, and replicability of science. Although these practices have garnered interest in related fields, it remains unclear whether open science practices have been adopted in the field of communication sciences and disorders (CSD). This study aimed to survey the knowledge, implementation, and perceived benefits and barriers of open science practices in CSD.

Method:

An online survey was disseminated to researchers in the United States actively engaged in CSD research. Four-core open science practices were examined: preregistration, self-archiving, gold open access, and open data. Data were analyzed using descriptive statistics and regression models.

Results:

Two hundred twenty-two participants met the inclusion criteria. Most participants were doctoral students (38%) or assistant professors (24%) at R1 institutions (58%). Participants reported low knowledge of preregistration and gold open access. There was, however, a high level of desire to learn more for all practices. Implementation of open science practices was also low, most notably for preregistration, gold open access, and open data (< 25%). Predictors of knowledge and participation, as well as perceived barriers to implementation, are discussed.

Conclusion:

Although participation in open science appears low in the field of CSD, participants expressed a strong desire to learn more in order to engage in these practices in the future.

Supplemental Material and Open Science Form:

https://doi.org/10.23641/asha.21569040

Accurate and reliable research is a cornerstone of evidence-based practice and a key driver of progress in the field of communication sciences and disorders (CSD). In the past decade, a number of concerns have been raised surrounding the reproducibility of research findings across many disciplines (e.g., psychology, education, biology, and ecology; Baggerly & Coombes, 2009; Fraser et al., 2018; Makel & Plucker, 2014; Open Science Collaboration, 2015). These concerns were sparked by the “replication crisis” in psychology; the Open Science Collaboration attempted to replicate 100 experimental and correlational studies and found that only 36% of the replicated studies had statistically significant findings compared with 97% of the original studies (Open Science Collaboration, 2015). In addition, the effect sizes in the replicated studies were approximately half the original effects, suggesting that selective reporting and/or publication bias may have inflated effects in the original studies.

At the core of the replication crisis is an overall lack of transparency in scientific studies as well as the use of questionable research practices, which has been argued to have originated from an ongoing culture of academia that incentivizes publication quantity over quality (Munafò et al., 2017). Lack of transparency can mean incomplete reporting of methodology or failure to provide access to materials, protocols, data sets, or publications (Samsa & Samsa, 2019). Questionable research practices are practices used by researchers with the intention of enhancing the likelihood of finding evidence to support their hypotheses. Examples of these practices include the selective reporting of findings, hypothesizing after results are known (HARK-ing), and deciding to collect more data after finding nonsignificant results with a given sample size, among others (John et al., 2012).

Selective reporting of findings refers to the practice of deliberately not fully or accurately reporting research findings to serve the researcher's agenda and hide undesirable findings. It can also be used to report only significant findings. HARK-ing refers to researchers making hypotheses after they have seen the study results. HARK-ing can be detrimental as it can propagate a finding that was due to statistical error (Type I error) and translate it into theory. Both selective reporting of finding and HARK-ing lead to the loss of an opportunity to communicate what did not work in research, which is equally important to what did work (Kerr, 1998). Collecting more data after finding nonsignificant results can lead to p-hacking, where a researcher is knowingly making choices after seeing the results in order to get to a significant finding. The consequences of p-hacking include a waste of time and resources, an increase in the number of false positives, and a biased literature base that does not replicate. Some of these practices, such as collecting additional data, may be necessary in the context of exploratory work. The practices are problematic, however, when they are undisclosed and used selectively to get interesting findings that would otherwise not exist. Therefore, the use of questionable research practices can lead to the publication of misleading findings that cannot be replicated. Journal publication biases encouraging selective reporting of findings indeed contribute to the ongoing replication crisis (Ioannidis et al., 2014). To mitigate these issues, the field of psychology has increasingly adopted open science practices (Nelson et al., 2018). The call to reconsider scientific methods and processes has also been seen across other fields such as cancer research (Begley & Ellis, 2012) and strategic management in business (Hubbard et al., 1998).

To date, no empirical evidence has examined the extent to which open science practices have been adopted in the field of CSD. The closest proxy comes from a relevant medical field that includes research from CSD scientists—otolaryngology, where less than 6% of 300 randomly selected articles published between 2014 and 2018 reported reproducible and transparent research practices (Johnson et al., 2020). Consequently, there is a need to better understand the knowledge, attitudes, and implementation of open science in CSD.

Open Science and Potential Benefits to CSD

Open science refers to a collection of research practices that aim to increase the accessibility and transparency of science. These practices include preregistration, self-archiving, gold open access, and open data defined in Table 1. The principles of open science can be incorporated into all stages of the scientific process, from preregistering a study plan to sharing study materials or increasing access to research publications. Implementing open science, however, is not an all-or-nothing endeavor; researchers can incrementally add open science practices to their workflow with the long-term goal to open their science.

Table 1.

Open science practices definitions.

Practice	Definition
Preregistration	The practice of documenting the research plan, study design, hypotheses, and/or analyses prior to data collection and submitting it to a registry. Preregistration separates hypothesis-generating (exploratory) versus hypothesis-testing (confirmatory) research.
Self-archiving	The act of making a version of a manuscript legally and freely available online on a lab/personal website or in a repository. The version may be the submitted, accepted, or published version of the manuscript, depending on publisher policy.
Gold open access	Unrestricted public availability of a research paper on the Internet through formal publication systems (e.g., Open Access Publishers). Gold open access indicates that researchers paid money to the publishers for them to make their work available online through open access.
Open data	Unrestricted public availability of research data and/or any resource necessary for the collection of these data (methodology, protocol, software packages, etc.), generally through online repositories.

Open science has a number of potential benefits for the field of CSD—both in terms of scientific discovery and evidence-based practice for clinicians. First, open science is associated with increased transparency and reproducibility, potentially facilitating a higher quality of research output (Hardwicke et al., 2020; OECD, 2015; Rubin, 2020). Higher quality, reproducible research can make science more cost effective, as scientific discoveries are more robust. An estimate of the cost of irreproducible preclinical research in the United States alone, for example, is $28 billion annually (Freedman et al., 2015). Second, open science reduces duplication and costs associated with creating, transferring, and reusing data since the efficiency of science is improved (OECD, 2015). This improved efficiency may help to reduce the time associated with clinical uptake of research findings, currently thought to take approximately 17–20 years for health care research (Balas & Boren, 2000). Third, open science can increase the global impact of research, as it promotes collaboration and faster knowledge transfer (OECD, 2015). Optimizing assessment, diagnosis, and treatment practices in CSD is a global issue and one that serves to benefit from worldwide collaboration. Ultimately, more widespread adoption of open science practices could result in a more robust, transparent, and replicable body of literature, as well as an increased rate of clinical translation and implementation.

Factors Affecting Adoption of Open Science Practices

Several factors may affect the uptake of open science practices. These factors can pertain to the level of an individual scientist, their institution, and more broadly to the field (Zečević et al., 2021). At the individual level, these factors can include age, seniority, position, as well as knowledge of and attitudes toward open science (Houtkoop et al., 2018; Toribio-Flórez et al., 2021; Zhu, 2017). Researchers may face financial limitations in paying the fees associated with publishing gold open access or may fear being scooped if they make their data open (Bahlai et al., 2019). In health care research, in particular, scientists may be unsure how to share data while maintaining patient privacy and confidentiality (Kostkova et al., 2016). Power hierarchies can also affect early career researchers as more senior collaborators may not want to adopt open science practices (Bahlai et al., 2019).

At the institutional level, uptake may be affected by the availability of funding and infrastructure to support scientists in adopting open science practices (Zečević et al., 2021). Larger institutions may be better equipped to provide financial support to offset costs associated with opening up research and to provide dedicated support through education and training on open science (Bahlai et al., 2019; Zečević et al., 2021).

At the field level, the movement toward open science can be influenced by structural support and incentives for open science. For instance, tenure committees and funding agencies do not uniformly recognize and/or incentivize the contribution of nontraditional research outputs, such as open research materials and data, to progress in the field. Additionally, not all journals facilitate open science by accepting preprints for publication as well as publishing registered reports. A recent study found that CSD journals currently have a low level of encouraging researchers to participate in open science practices as measured by the TOP (Transparency and Openness Promotion) Factor metric (Schroeder et al., 2022). A small number of journals in CSD, however, have begun to facilitate and promote open science practices. For example, the Journal of Speech, Language, and Hearing Research and Language Learning now accept registered reports (a publication format that involves full peer review of the methods protocol and an in-principle acceptance for publication before data collection begins—regardless of the outcome of the study; Chambers, 2019; Marsden et al., 2018; Storkel & Gallun, 2022) and Ear and Hearing introduced a badge system to reward authors who implement open science practices (Svirsky, 2020). The badge reward system is also available to authors who publish in any of the American Speech-Language-Hearing Association (ASHA) journals. Additionally, the ASHA journals adopted the TOP guidelines, which encourage the use of open science practices. These include, for example, requiring an explicit Data Availability Statement from authors. These journal initiatives suggest that open science practices are feasible in the field of CSD; however, multifaceted support across all levels may be necessary for widespread adoption.

Study Aims

The goal of this study was to survey researchers in CSD to better understand their knowledge, implementation, and perceived benefits of open science practices, as well as to identify barriers to implementation. For the purposes of this work, we defined four-core practices of interest (hereby referred to as open science practices): preregistration, self-archiving, gold open access, and open data. Specifically, we aimed to

1. Describe CSD researchers' knowledge and perceived benefit of open science practices;

2. Describe the frequency of CSD researchers' participation in open science practices;

3. Report perceived barriers to implementation of open science practices;

4. Examine the relationship between demographics and knowledge and participation in these open science practices; and

5. Examine whether perceived knowledge or benefit differs across practices.

Hypotheses

1. We hypothesized that participants would report “low” knowledge of open science practices, which we defined as a median score of 3 or lower on a 6-point Likert scale;

2. We hypothesized that participants would report “low” participation in open science practices, which we defined as ≤ 50%;

3. We explored known barriers in the implementation of open science practices included in this study (preregistration, self-archiving, gold open access, and open data) and their associations with knowledge and participation in open science practices;

4. We hypothesized that participants with less research experience and in more junior roles would report higher knowledge of open science practices but would not report higher participation;

5. We hypothesized that the perceived knowledge and benefit of preregistration and gold open access would be higher than other open science practices (open data and self-archiving).

Findings on the field's current state of open science practices have the potential to elucidate directions for growth within the field.

Method

This study was approved by the University of Georgia Institutional Review Board. The study preregistration, data, and analysis code can be found on the Open Science Framework: https://osf.io/2f7xp/. No deviations from the preregistration were encountered for statistical analyses. Thematic analysis of participants' open-text responses to survey questions was added to the analysis plan following preregistration, which is reflected in an addendum.

Survey Development

A 57-item online survey was created to examine research scientists' knowledge, participation, and barriers to implementing open science practices (available in Supplemental Material S1). To develop the survey, we first performed a literature search to identify studies examining similar constructs in related fields. Our final survey was adapted from Toribio-Flórez et al. (2021) who sought to explore attitudes toward open science among early career researchers in the Max Planck Society. Our survey followed a similar format as the original survey by aiming to assess the knowledge, attitudes, perceived benefit, and implementation of each open science practice. We modified the Toribio-Flórez et al. (2021) survey to include a 6-point Likert scale for questions exploring degree responses and adapted the demographic questions for our sample of CSD scholars. Our survey was created and distributed through the Qualtrics platform. It was composed of seven sections: (a) informed consent, (b) eligibility screening, (c) demographic information, (d) preregistration, (e) self-archiving, (f) gold open access, and (g) open data.

The demographics section included eight questions. Participants were asked to indicate (a) their research position/job title, (b) what year their PhD was awarded (if applicable), (c) years of experience conducting research in CSD, (d) the Carnegie classification of their current institution (Indiana University Center for Postsecondary Research, 2021), (e) their research area, (f) an approximate number of peer-reviewed manuscripts submitted in the past 3 years, (g) type(s) of regular research engagement, and (h) background in authoring scientific research.

The remaining sections of the survey asked participants about their knowledge, participation, and perceived benefits and barriers of implementing the four-core open science practices of interest: preregistration, self-archiving, gold open access, and open data, defined in Table 1 earlier. Participants were presented with nine to 12 questions in each section. Response methods to questions included Likert rating scales (1–6; 1 = not at all; 6 = extremely; 2–5 not labeled), slider scales (0%–100%), yes/no, multiple-choice, and forced-choice options. All multiple choice and forced-choice options included a free-text “Other” option to write in alternative responses.

During development, the survey was shared with six external, unaffiliated researchers for pilot assessment. Pilot participants were required to be actively engaged in publishing scientific research but not necessarily in the field of CSD. The researchers completed the survey and provided feedback regarding the clarity of instructions and questions, adequacy of response options, potential information omission, and time requirement to complete the survey. The survey questions were revised based on the feedback from the expert judges prior to its formal dissemination.

Participation Criteria

Inclusion criteria for survey participation included (a) active engagement in research in the field of CSD and (b) residence in the United States. We defined engagement as participation in any aspect of the research process. This included doctoral students, postdoctorates, research scientists, and faculty members, but not undergraduate students. The survey was restricted to participants based in the United States as open science practices are expected to differ by country. For example, 20% of funders in the United Kingdom mandate that resulting publications be made open access, compared with less than 5% in the United States (Open Science Monitor, 2019). Examining country-specific differences in policy was beyond the scope of this study.

Procedure

Convenience sampling was used to examine a representative sample of researchers in the field of CSD. Recruitment occurred using two primary methods. First, we identified all universities in the United States with CSD programs through the ASHA EdFind website (https://find.asha.org/ed; February to March 2021). For each CSD department on EdFind, we manually searched for contact information for the department chair. We sent a recruitment e-mail to all of the identified department chairs (n = 311), including a description of the survey and inclusion/exclusion criteria, and asked them to forward the survey to eligible students, faculty, and staff. A reminder follow-up e-mail was sent 2 weeks after the initial contact. Our second recruitment method was promoting the survey across social media platforms, specifically, Instagram, Facebook, and Twitter, and to an ASHA Special Interest Group (SIG 13, Swallowing and Swallowing Disorders [Dysphagia]) of which one author was a member. The survey was open for 6 weeks (July to August 2021).

Statistical Analysis

R (v. 4.0.1) was used for descriptive and inferential statistical analyses (R Core Team, 2018) with the following packages: ordinal (v12.10) for cumulative link regression models, car (v3.0) for likelihood ratio tests, and lsmeans (v2.30) for post hoc comparisons (Christensen, 2019; Fox, 2019; Lenth, 2016). For descriptive analyses within each open science practice, frequencies were used for categorical variables, medians and interquartile ranges for ordinal variables, and means and standard deviations for continuous variables. For inferential analyses, we used an alpha level of .05. Multiple post hoc pairwise comparisons were conducted with Tukey's honestly significant difference test.

To examine the relationship between demographic variables and knowledge of each open science practice (measured on a Likert scale from 1 to 6), cumulative link ordinal regression models were performed with the following independent variables: years of research experience, Carnegie classification of the participants' institution, year doctorate degree was awarded, and current research position. Binary logistic regression models were performed with a dependent variable of prior participation with an open science practice and the aforementioned demographic predictors.

To examine differences in knowledge, behaviors, and perceived benefit between open science practices, we performed separate ordinal regression models. Open science practice was included as a dummy-coded categorical predictor in the full model. A likelihood ratio test then compared models with and without this predictor. If this test was statistically significant, follow-up pairwise comparisons were performed to examine differences between each type of open science practice.

For inferential statistical models with multiple predictors, we used a model-fitting procedure that began with the null model, iteratively added a predictor, and then examined the Akaike information criterion (AIC) to determine model fit. AIC values of at least 2 were required for the inclusion of a predictor variable (Burnham & Anderson, 2004). In order to account for multicollinearity, independent variables with a variance inflation factor > 3 or correlation > 0.80 were excluded from the model. Residuals from the final full model were examined to ensure that assumptions were satisfied. If statistical models did not converge, we scaled continuous predictors or collapsed categorical predictors, as necessary. Due to unequal distributions, two variables were collapsed: the Carnegie classification (R1 versus all other categories) and research position (PhD student vs. all other categories). Nagelkerke's R2 and Tjur's R2 served as measures of variance explained for cumulative link ordinal and binary logistic models, respectively. Likelihood ratios (LRs) were used as a measure of effect size for model comparisons. LRs between 2 and 5 were considered “small,” “moderate” between 5 and 10, and “large” if greater than 10 (McGee, 2002).

Thematic Analysis

Respondents were given the opportunity to elaborate on their responses concerning barriers to implementation of open science practices through open-ended text responses on the survey. These responses were analyzed using coding reliability thematic analysis (Braun et al., 2019). The first author (M.E.A.) read through the responses and identified potential themes. The third author (H.L.L.) reviewed the themes and collapsed them into larger overarching themes. After the themes were identified, the first author coded the responses of participants for each open science practice. To establish the reliability of the coding procedure, the fourth author (M.A.K.) coded 25% of the statements for each open science practice (percent agreement = 64%). Consensus coding was conducted between M.E.A. and M.A.K. for statements where discrepancies were found until agreements were reached. An example of a discrepancy included the following statement that was coded by M.E.A. as “lack of buy-in” but by M.A.K. as “worry about confidentiality:” “…I spend a lot of time and money developing experiments and running subjects. So, to just hand that data over to someone else doesn't seem quite fair.” After meeting for consensus coding, we decided that “Interest in retaining data for own analyses” better reflected the statement. Additional points of discrepancy occurred where M.A.K. coded N/A for statements she was unsure about (14% of statements), for example, “I try to publish open access.” After the consensus meeting, this statement was coded as “preferred gold.” Therefore, both authors discussed all points of discrepancy until both authors were in full agreement on the codes for those statements.

Results

Demographics

Two-hundred sixty-four participants responded to the survey. Two participants reported living outside the United States, and 17 did not indicate whether they lived in the United States. Eleven participants reported that they did not engage in research, and five did not respond to this question. Therefore, 245 participants met the eligibility criteria. Of these 245 participants, 23 did not complete any survey questions. Thus, our final sample for analysis was 222 participants.

The majority of participants were doctoral students (38.29%) or assistant professors (23.87%). Most respondents were employed at an R1 (i.e., very high research activity) institution (58.56%; see Figure 1). On average, participants reported 10.33 years of research experience (SD = 8.57) and 8.78 publications submitted in the past 3 years (SD = 36.02). Of 136 participants with a PhD, the average length of time since PhD was awarded was 8.95 years (SD = 9.59). The most frequently reported research areas were language learning and/or language disorders, neurogenic communication disorders, cognitive aspects of communication, and swallowing and/or swallowing disorders (see Figure 2).

Figure 1.

Distribution of (A) participants' research positions and (B) Carnegie classifications.

Figure 2.

Distribution of participant research areas in communication sciences and disorders. CSD = communication sciences and disorders.

Preregistration

Knowledge, Participation, and Barriers

Response methods to questions included Likert rating scales (1–6; 1 = not at all; 6 = extremely; 2–5 not labeled). Participants reported median scores of 3 (IQR = 2) for knowledge, 4 (IQR = 2) for desire to learn more, 3 (IQR = 2) for extent of barriers, 4 (IQR = 1) for benefits of preregistration to daily, 4 (IQR = 2) for benefits to research, and 4 (IQR = 2) for benefits to public society (see Figure 3). The distribution of knowledge, participation, and barriers to preregistration by research position are shown in Supplemental Material S2. Twenty-five percent of respondents reported previously preregistering at least one study. These projects were primarily preregistered at either the researcher's personal website (32%) or the Open Science Framework (30%). PROSPERO (10.70%), clinicaltrials.gov (16.1%), AsPredicted (5.4%), and internal institutional review board (IRB) submission (5.4%) were also used. Among respondents who reported previously preregistering a study, the mean percentage of their studies that were preregistered was 44% (SD = 27%). Thirty-six percent of respondents stated that they planned to preregister a study in the next year. The most common barrier to preregistration was a lack of knowledge on how to preregister a study (see Table 2). This barrier was also frequently reported in the analysis of free-text responses (see Table 3), in addition to a lack of buy-in from others and a perception that preregistration occurs through existing processes (e.g., when submitting an ethics application to the institutional review board).

Figure 3.

Preregistration: knowledge, barriers, and perceived benefit.

Table 2.

Perceived barriers to preregistration.

Perceived barrier	Frequency
“I don't know how to preregister my work”	100
“Lack of time is why I don't preregister my studies”	78
“I feel that it limits my ability to change the study moving forward”	70
“I have never heard of preregistration”	65
“Lack of buy-in from colleagues/the field to preregistration”	65
“I fear that other authors might steal my work”	45
“No barriers”	21
“I don't feel like my research needs to be fully open”	21
“Institutional/university policies are a barrier”	15
Other (free-text response)	14

Note. Participants were allowed to select more than one answer.

Table 3.

Thematic analysis of barriers to preregistration.

Theme	Example response	Frequency
Lack of knowledge and experience	“No orientation and complicated process”	5
Preregistration considered to occur elsewhere	“clinicaltrials.gov was required in IRB submission, uncertain how to preregister in ‘full’ format”	4
Lack of buy-in from others	“I conduct basic science research where preregistration is not the norm”	4
Negative perceptions	“pre-registration appears to completely unnecessary and dangerous to a research project and program”	3
The impact of the study design and related policies	“I think the type of study is important here - RTCs absolutely should be pre-registered (as required by clinical trials.gov) but for chart reviews or student projects that are limited in scope, pre-registration may just be yet another administrative hurdle with little scientific/societal benefit.”	2
Lack of time	“…I don't have the time to look into it”	2

Note. A single response can include several themes.

Demographic Predictors of Knowledge and Participation in Preregistration

Due to a high correlation (r = .92) between years since doctoral degree was awarded and years of research experience, we excluded the former variable from all inferential model fitting procedures. Research years of experience (AIC Δ = 1.41), Carnegie classification (AIC Δ = 0.45), and research position (AIC Δ = 0.40) did not improve model fit compared to a null model.

For participation in preregistration, research years of experience did not uniquely contribute to the model (AIC Δ = 1.79). However, Carnegie classification (AIC Δ = 3.72) and research position (AIC Δ = 3.12) improved model fit. Results showed that Carnegie classification, χ²(3) = 5.49, p = .139, and research position, χ²(7) = 10.95, p = .141, did not show statistically significant associations with participation in preregistration (R ² = .065).

Self-Archiving

Knowledge, participation, and barriers. Participants reported median scores of 4 (IQR = 2) for knowledge of self-archiving, 5 (IQR = 3) for desire to learn more, 3 (IQR = 2) for extent of barriers, 5 (IQR = 3) for benefits of self-archiving to daily, 5 (IQR = 2) for benefits to research, and 5 (IQR = 2) for benefits to public society (see Figure 4). The distribution of knowledge, participation, and barriers to self-archiving by research position are shown in Supplemental Material S3. Thirty-eight percent of participants reported previously self-archiving, including on a personal website (42.86%), lab or university website (34.52%), institutional repository (16.67%), external server (19.05%), and social networking site (39.29%). Fifty-nine percent of participants reported planning to self-archive in the next year. The most common barriers included difficulty interpreting copyright rules (40.99%) and journal policies (45.94%; see Table 4). Analysis of the free-text responses also highlighted lack of knowledge and time and a preference for publishing gold open access as barriers to self-archiving (see Table 5).

Figure 4.

Self-archiving: knowledge, barriers, and perceived benefit.

Table 4.

Perceived barriers to self-archiving.

Perceived barrier	Frequency
“Journal policies are a barrier to self-archiving”	102
“Copyright rules are too difficult to figure out”	91
“I don't know how to self-archive”	71
“Publishing in open access journals costs too much”	71
“Lack of time is why I don't self-archive”	52
“I have never heard of self-archiving”	39
“No barriers”	31
“Institutional/university policies are a barrier”	24
“Lack of buy-in from colleagues/the field to self-archiving”	20
“I don't feel like my research needs to be fully open”	3
Other (free-text response)	7

Note. Participants were allowed to select more than one answer.

Table 5.

Thematic analysis of barriers to self-archiving.

Theme	Example response	Frequency
Lack of knowledge	“I fully support complete open access to research. Particularly in our field of CSD, I think this is a moral imperative, but I have little to no knowledge about how this works policy/rights/finance-wise.”	2
Preferred gold	“I routinely pay open access fees to make work accessible after peer review.”	2
Lack of time and resources	“I do self-archive now, but it takes time”	3

Note. A single response can include several themes.

Demographic predictors of knowledge and participation in self-archiving. For predictors of knowledge of self-archiving, research years of experience (AIC Δ = 5.55) uniquely contributed to the model, whereas Carnegie classification (AIC Δ = 1.54) and research position (AIC Δ = 0.93) did not improve model fit. Results showed that participants with more research experience reported greater knowledge of self-archiving, LR χ²(1) = 24.14; p < .001, R ² = .036.

For participation in self-archiving, Carnegie classification (AIC Δ = 3.77) and research position (AIC Δ = 5.47) uniquely contributed to the model, whereas research years of experience did not improve model fit (AIC Δ = 0.77). Results showed that Carnegie classification, χ²(3) = 1.68, p = .642, and research position, χ²(7) = 8.53, p = .288, did not show statistically significant associations with participation in self-archiving (R ² = .007).

Gold Open Access

Knowledge, participation, and barriers. Participants reported median scores of 3 (IQR = 3) for knowledge of gold open access, 4 (IQR = 2) for desire to learn more, 4 (IQR = 2) for extent of barriers, 4 (IQR = 2) for benefits of gold open access to daily, 4 (IQR = 3) for benefits to research, and 5 (IQR = 3) for benefits to public society (see Figure 5). The distribution of knowledge, participation, and barriers to gold open access by research position are shown in Supplemental Material S4. Twenty-two percent of participants reported previously using gold open access. Among these participants, they reported approximately 45% (SD = 26%) of papers published gold open access. Eighteen percent of participants reported planning to use it in the next year. The most common barriers included journal cost (59%) and lack of buy-in from colleagues/the field to pay for publishing (22.97%; see Table 6). The analysis of free-text responses also revealed a lack of interest in publishing gold open access (see Table 7).

Figure 5.

Gold open access: knowledge, barriers, and perceived benefit.

Table 6.

Perceived barriers to gold open access.

Perceived barrier	Frequency
“Publishing in open access journals costs too much”	131
“I don't feel like I need to pay to publish my work”	72
“I find that there is a lack of buy-in from colleagues/the field to pay for publishing”	51
“I don't know how to publish in gold open-access journals”	49
“I have never heard of gold open access”	44
“Institutional/university policies are a barrier”	27
“No barriers”	19
“I don't feel like my research needs to be fully open”	4
Other (free-text response)	16

Note. Participants were allowed to select more than one answer.

Table 7.

Thematic analysis of barriers to gold open access.

Theme	Example response	Frequency
Lack of interest/negative perceptions	“I don't WANT to pay to publish my work because I shouldn't have to. My work is funded by taxpayer dollars and should be publicly available.” “I find it unethical to have to pay to publish”	8
Financial barriers	“Cost”	7
Other	“Lack of institutional support for research in general” “Open access journals generally have requirements for preregistration/open science that I have not followed to the letter from the time that I conceived of the study”	2

Note. A single response can include several themes.

Demographic predictors of knowledge and participation in gold open access. For predictors of knowledge of gold open access, research years of experience (AIC Δ = 4.22), research position (AIC Δ = 2.21), and Carnegie classification (AIC Δ = 2.88) contributed to the model. Results showed main effects of Carnegie classification, LR χ²(3) = 125.20, p < .001, and research position, LR χ²(7) = 65, p < .001, R ² = .098, whereas research experience was nonsignificant, LR χ²(1) = 0.78, p = .377. Specifically, the probability of higher self-reported knowledge of gold open access increased by 4% for each additional year of research experience. Post hoc pairwise comparisons between Carnegie classifications were nonsignificant (p > .05; Supplemental Material S5).

For participation in gold open access, Carnegie classification (AIC Δ = 3.96) and research position uniquely contributed to the model (AIC Δ = 6.38), whereas research years of experience did not (AIC Δ = 0.20). Results showed that Carnegie classification, χ²(3) = 2.03, p = .567, and research position, χ²(7) = 7.62, p = .368, did not show statistically significant associations with participation in gold open access (R ² = .001).

Open Data

Knowledge, perceived benefit, participation, and barriers. Participants reported median scores of 4 (IQR = 1) for knowledge of sharing open data, 4 (IQR = 2) for desire to learn more, 3 (IQR = 2) for extent of barriers, 4 (IQR = 2) for benefits of sharing open data to daily, 5 (IQR = 3) for benefits to research, and 4 (IQR = 3) for benefits to public society (see Figure 6). The distribution of knowledge, participation, and barriers to sharing open data by research position are shown in Supplemental Material S6. Twenty-six percent of participants reported previously sharing open data, and 37% reported planning to share open data in the next year. The most common barriers included lack of knowledge on how to share open data (34.68%) and concern for the confidentiality of participants (31.53%; see Table 8). Analysis of the free-text responses also revealed confidentiality concern as a key barrier (see Table 9).

Figure 6.

Open data: knowledge, barriers, and perceived benefit.

Table 8.

Perceived barriers to sharing open data.

Perceived barrier	Frequency
“I don't know how to share open data”	77
“I fear for the confidentiality of my participants (can be identified)”	70
“Lack of time is why I don't share open data”	61
“I fear for my copyright over the data I'm sharing”	61
“Lack of buy-in from colleagues/the field to sharing open data”	44
“Institutional/university policies are a barrier”	36
“No barriers”	31
“I have never heard of open data”	26
“I don't feel like my research needs to be fully open”	19
Other (free-text response)	13

Note. Participants were allowed to select more than one answer.

Table 9.

Thematic analysis of barriers to open data.

Theme	Example response	Frequency
Concerns about confidentiality	“I work with vulnerable populations, Even though I can fully de-identify my data, I want my families to feel protected.”	3
Lack of time/resources	“I do try to share code now, but preparing it to be publicly available takes time…” “Lack of a central source for depositing data”	3
Worry about perceptions/judgment	“I also worry about what others will think of my code.” “Not sure anyone would be interested in or know how to read my data”	2
IRB/institutional policies	“It is very difficult to get IRB approval to make data (especially audio recordings of speech) publicly available. This greatly lengthens the amount of time it takes to get IRB approval, and there is a ton of pushback.”	3
Interest in retaining data for own analyses	“I spend a lot of time and money developing experiments and running subjects. So, to just hand that data over to someone else doesn't seem quite fair.” “We are not yet done working on the dataset”	2
Other	“I need to learn more about how to do it properly” “Getting scooped.”	2

Note. A single response can include several themes.

Demographic predictors of knowledge and participation in sharing open data. For predictors of knowledge of open data, research years of experience (AIC Δ = 3.32), Carnegie classification (AIC Δ = 2.12), and research position (AIC Δ = 7.27) contributed to the model. Results showed that years of Carnegie classification, LR χ²(3) = 164.63, p < .001, and research position, LR χ²(7) = 80.02, p < .001, were significantly associated with knowledge of sharing open data; however, years of research experience, LR χ²(1) = 0.29, p = .588, was nonsignificant (R ² = .097). Post hoc pairwise comparisons examining differences between Carnegie classifications showed that participants from R1 (very high research activity) institutes reported higher knowledge compared to doctoral/professional institutions (p = .013). All other comparisons of Carnegie classifications were nonsignificant (p > .05; Supplemental Material S7). All pairwise comparisons between research positions were nonsignificant (p > .05; Supplemental Material S8).

For participation in sharing open data, Carnegie classification (AIC Δ = 2.84) and research position uniquely contributed to the model (AIC Δ = 3.84), whereas research years of experience did not improve model fit (AIC Δ = 1.63). Results showed that Carnegie classification, χ²(3) = 2.18, p = .536, and research position, χ²(7) = 10.16, p = .180, did not show statistically significant associations with participation in sharing open data (R ² = .007).

Differences Between Open Science Practices

There was a significant main effect of open science practices on perceived knowledge, χ²(3) = 22.40, p < .001, R ² = .027; see Figure 7. Specifically, the knowledge of self-archiving (p = .013), open data (p = .001), and gold open access (p = .005) was higher than preregistration. All other pairwise comparisons between open science practices on knowledge were nonsignificant (p > .05; Supplemental Material S9).

Figure 7.

Comparison of knowledge and perceived benefit between open science practices.

There was a significant main effect of open science practices on perceived benefit to the daily life of a researcher, χ²(3) = 34.30, p < .001; R ² = .039. Specifically, self-archiving was viewed as more beneficial to the daily life of a researcher compared to preregistration (p < .001) and gold open access (p < .001). Open data were also rated as more beneficial compared with preregistration (p = .01). All other pairwise comparisons were nonsignificant (p > .05; Supplemental Material S10).

There was a significant main effect of open science practices on perceived benefit to one's research field, χ²(3) = 31.48, p < .001; R ² = .036. Specifically, self-archiving was viewed as more beneficial to research fields compared to preregistration (p < .001) and gold open access (p < .001). Open data were also rated as more beneficial compared to preregistration (p = .004). All other pairwise comparisons were nonsignificant (p > .05; Supplemental Material S11).

There was a significant main effect of open science practices on perceived benefit to public society, χ²(3) = 31.86, p < .001; R ² = .036. Specifically, self-archiving was viewed as more beneficial to public society compared with preregistration (p < .001), gold open access (p = .014), and open data (p < .001). All other pairwise comparisons were nonsignificant (p > .05; Supplemental Material S12).

Discussion

There has been a growing movement to promote open science practices to improve the transparency, openness, and replicability of research. In the social sciences, the adoption of these practices has rapidly increased over the past decade, potentially signaling a shift in cultural and normative scientific values (Christensen et al., 2020). Despite this high rate of implementation in adjacent disciplines, it remains unclear whether researchers in the field of CSD are familiar with these practices and implement them in their own research. This study had five aims. Specifically, we sought to (a) describe CSD researchers' knowledge and perceived benefit of open science practices, (b) describe the frequency of CSD researchers' participation in open science practices, (c) report perceived barriers to implementation of open science practices, (d) examine the relationship between demographics and knowledge and participation in these open science practices, and (e) examine whether perceived knowledge or benefit differs across practices. Across all open science practice areas, we hypothesized overall low knowledge and low participation, higher knowledge in more junior scientists, and the highest perceived knowledge and benefit from preregistration and gold open access. A discussion of these questions with respect to each open science practice is provided below.

Overall, our findings demonstrate that CSD researchers report low knowledge related to preregistration and gold open access, as well as low participation across all core open science practices. However, many reported a strong desire to learn more and engage in these practices in the future. The key barriers that may impede the adoption of open science practices in the field of CSD include lack of knowledge, time, and costs associated with implementation. Collectively, these findings suggest that initiatives to increase knowledge and reduce barriers in the implementation of open science practices are desired by the scientific community.

Knowledge, Participation, and Barriers

Preregistration

Preregistration requires researchers to specify hypotheses, methods, and analyses prior to data collection and/or analysis. This time-stamped document is then made available to readers, so they can identify discrepancies between the original plan and the published study. In regard to our research questions, we predicted low knowledge and low participation in preregistration and a higher perceived knowledge and benefit than self-archiving or open data. Although preregistration has become increasingly popular in the social sciences (Christensen et al., 2020), CSD researchers reported low knowledge of this open science practice. Furthermore, participation was rare with only a quarter of respondents preregistering at least one of their studies. These findings suggest that preregistration is not commonly used in the field of CSD, which may be due, in part, to a gap in knowledge and familiarity with preregistering a research study. Indeed, nearly half of participants reported that they did not know how to preregister, whereas those with a history of preregistering reported that this was done for nearly half of their studies. Therefore, initiatives to promote preregistration will require training on the logistics of preregistration, which may, in turn, promote participation.

Our qualitative findings highlight several common misconceptions of preregistration. Many participants reported concerns that they would be unable to change aspects of their study once it was preregistered. Preregistration facilitates transparency by allowing readers to understand when decisions were made or changed and how that compares to the authors' published work. If deviations are required, then the preregistration is updated and revisions are disclosed in the published manuscript (Claesen et al., 2019). In these situations, some may view deviating from a preregistered protocol as indicative of poor quality. However, deviations are common and may be expected if data collection or analysis issues arise (Claesen et al., 2019). A preregistration merely ensures that these deviations are recorded and made publicly available. Finally, there was confusion related to the definition of a preregistration, with many participants reporting that an ethics application to an institutional review board or a protocol uploaded to a personal website satisfied the requirements. Instead, preregistrations are most commonly submitted to a third-party registry, such as the Open Science Framework, which provides a public and shareable time-stamped record of the protocol and subsequent revisions.

Advocates of preregistration contend that this practice affords many unique benefits that would otherwise be considered questionable without a time-stamped document, such as distinguishing confirmatory versus exploratory aims, specifying optional stopping rules, and using one-tailed statistical tests to optimize power (Rubin, 2020). Findings from this study suggest that CSD researchers acknowledge its potential benefits to not only their individual work and research field but also public society as a whole. However, it should be noted that preregistration is not a panacea for issues related to reproducibility and questionable research practices and is arguably the most controversial open science practice. Opponents have posited that preregistration alone is ill-equipped to resolve issues related to statistical inference (i.e., p-hacking; Navarro, 2020) and that selective reporting may impede its ability adequately to promote transparency (Claesen et al., 2019). Ultimately, preregistration should be viewed as one tool that, when used in combination with other practices, may enhance the transparency of one's research process. Our findings suggest that while this tool is currently underutilized in the field of CSD, researchers acknowledge its potential benefits and report a desire to learn more.

Self-Archiving

Self-archiving, also known as green open access, is an open access route in which the author shares an accepted version of the peer-reviewed manuscript online or deposits it to a repository such as PubMed or the Open Science Framework. For this variable, we predicted low knowledge and low participation. Contrary to our hypothesis, participants in our sample reported adequate knowledge of self-archiving, with greater knowledge in participants with more research experience. Although only 38% of our sample actively self-archive their research, more than half of participants indicated plans to self-archive in the future, suggesting an increasing awareness and interest in use of this practice. It should be noted that a volunteer-based group aimed at increasing awareness of self-archiving advertised participation in this study through social media. Researchers already following this group on social media (and potentially already curious about open science practices) likely comprised a substantial portion of our participants, which may explain these findings.

Despite participants reporting a high desire to self-archive in the future, several barriers to implementation were noted. Difficulty interpreting copyright rules and journal policies was the most frequently cited barrier. Guidelines on self-archiving can be found on publisher or journal websites, as well as online resources, such as Sherpa Romeo (https://v2.sherpa.ac.uk/romeo/) and ShareYourPaper (https://shareyourpaper.org/). Additional barriers included a lack of knowledge and time. This highlights the potential benefits of conducting training for researchers on self-archiving, particularly for early career researchers. Lack of time also underscores the need for institutional support and dedicated staff to assist in this process. Finally, CSD researchers reported a preference for publishing gold open access over self-archiving. This may reflect the relative ease of paying journals to make work freely available; however, this convenience has a substantial monetary cost that prohibits most researchers from affording gold open access.

The possible benefits of self-archiving include increased access to research findings by clinicians, policymakers, and science communicators. A significant barrier clinicians face in staying up to date with research findings and implementing evidence-based practice is the lack of access to research articles (Thome et al., 2020). It can take 17 years for research findings to be translated to clinical practice (Balas & Boren, 2000); thus, increasing access to research can help decrease the research to practice gap. Other important stakeholders such as policymakers and science communicators need access to research articles in order to make informed policy decisions and share findings with the general public. Scientists can also benefit from self-archiving through increased visibility of their work and cost savings in publishing fees. Studies that are self-archived have been linked to a citation benefit, receiving 30% more citations than research that is made open access by paying a publisher fee (Piwowar et al., 2018). Moreover, self-archiving is free, which opens up funding that may otherwise have been used to pay the article processing fees associated with publishing gold open access.

It is also important to note that many scientists may confuse gold open access availability with the PubMed Central (PMC) Open Access Subset. Per licensing and copyright terms, many manuscripts are available through the PubMed database if research was supported by National Institutes of Health (NIH) funding. Many researchers also use NIH funds to pay for gold open access through the publisher, which means the published and accepted versions (through PubMed) are openly available, further complicating the differentiation between different open versions.

Gold Open Access

Gold open access is the route for open access publications that requires a payment to the publisher to make the manuscript available for anyone without a subscription to read. We predicted low knowledge of and low participation in gold open access publishing, but higher overall perceived knowledge and participation than self-archiving and data sharing. Participants in our sample reported a low level of knowledge of gold open access publishing with greater knowledge in participants with more research experience. Our sample also indicated low levels of participation in gold open access publishing with only 22% of our sample indicating that they have done so in the past. These findings suggest that publishing using the gold open access route is not commonly used in CSD. This finding is contrary to what Toribio-Flórez et al. found in their 2021 article surveying early career researchers from the Max Planck Institute. Participants in their sample showed higher knowledge of gold open access publishing. Thus, researchers in CSD possibly might have lower knowledge of open access publishing; however, the trend of low implementation was also observed in the Toribio-Flórez study where only 31% of their sample indicated previously publishing open access papers.

The most commonly cited barriers for gold open access in our study were the cost and lack of buy-in to pay for publishing. This is likely attributed to the lack of extra funds to cover these costs as doctoral students or if they were not built into research grants, given that more than half of our sample constituted early career researchers.

The thematic analysis of open text responses revealed negative perceptions associated with “paying for publishing” and gold open access. Several researchers indicated that they believed that taxpayer money should not be spent on open access publishing, and others mentioned that it is unethical to have to pay to publish open access. Journals such as Nature contest that article processing charges are necessary if they will provide these articles without a paywall (Else, 2020). A possible solution to researchers who are either impeded by the publication costs or who do not want to support the paying to publish open access model is to self-archive their publications. However, as previously mentioned, lack of time and knowledge can be a barrier to self-archiving. Therefore, institutions can support researchers by connecting them with librarians who are knowledgeable about journal policies and can support CSD scientists in their endeavors to make research accessible for everyone.

Open Data

Open data refer to the public sharing of de-identified research data and/or other resources created for the collection and analysis of these data, typically through online repositories. In this study, we predicted low knowledge and low participation in open data sharing among CSD scientists. Our participants indicated a higher degree of knowledge than was predicted on sharing open data with greater knowledge in participants from R1 (very high research activity) institutes compared with participants from doctoral/professional institutions. However, only a quarter of participants indicated active data sharing. Participants reported a high desire to learn more about this practice, and a third of our sample reported plans to share data over the course of the next year. Similar trends have been reported in related fields on this topic (Christensen et al., 2020; Johnson et al., 2020; Toribio-Flórez et al., 2021). These findings indicate a generally positive outlook on open data sharing; however, barriers continue to prevent implementation.

The most commonly reported barriers to open data sharing reported by our sample of CSD scientists included a lack of knowledge and a concern for participant confidentiality. There are several online guides on how to share data openly, including in the ASHA Journals Academy (ASHA, 2021) and the Inter-university Consortium for Political and Social Research (ICPSR, 2021), as well as many tutorials on this topic (Klein et al., 2018; Martone et al., 2018). Other commonly reported barriers for open data, such as lack of time and lack of buy-in from colleagues, may be associated with a lack of data sharing knowledge at the field level. Studies suggest that the long-term effects of data sharing can lead to improved time and cost efficiency in science and knowledge transfer (Balas & Boren, 2000; Freedman et al., 2015), as well as benefits for individual researchers, such as increased citation rates (Piwowar & Vision, 2013).

Reidentification of open data is a legitimate confidentiality concern for CSD and other human subjects' researchers. Incorporating clear data retention and sharing clauses into IRB submissions and consent forms can reduce confidentiality concerns (Meyer, 2018). Example IRB and consent templates for data retention and sharing are available online (ICPSR, 2021). Following the safe-harbor provision of the Health Insurance Portability and Accountability Act Privacy Rules (Department of Health and Human Services, 2002), researchers can anonymize identifiers through “masking” (i.e., replacing original identifiable variables with random variables), or reducing the resolution of variables (i.e., presenting age ranges instead of true ages) to decrease the chance of reidentification (Barth-Jones, 2012; Meyer, 2018). Synthetically generated data that maintain the statistical distribution of an original data set is also an increasingly popular alternative in health care and biobehavioral research (Chen et al., 2021; Quintana, 2020). Ultimately, scientists hold a professional, ethical responsibility to follow FAIR Data Principles to keep open data Findable, Accessible, Interoperable, and Reusable (Wilkinson et al., 2016).

Demographic Predictors of Knowledge and Participation

Across all four practices studied, Carnegie classification and years of experience were the greatest predictors of knowledge and participation. For both preregistration and self-archiving, higher Carnegie classification and research position predicted a higher level of participation but not level of knowledge. For gold open access publishing, we found that participants with more research experience and who work in institutions with a higher Carnegie classification had a higher level of knowledge of gold open access publishing. Carnegie classification was also significantly associated with knowledge of open data sharing. This finding potentially indicates greater support for open science practices in more research-intensive settings, highlighting a critical need for university- and department-level support (alongside federal- and community-level initiatives) to continue building knowledge in data sharing and other open science practices.

Differences Between Practices

Since open science is an umbrella term for a variety of practices, researchers can decide to participate in one or more practices when deemed appropriate. However, it remains unclear whether there are differences in the knowledge and perceived benefit of these practices by CSD scholars. Our results showed that preregistration has the lowest degree of knowledge compared to other practices. This finding highlights the need for education on preregistration and its potential utility in the field of CSD. Additionally, we examined differences in the perceived benefit of different open science practices to researchers' daily life, research field, and public society. Self-archiving showed a higher perceived benefit in these domains compared with other practices, suggesting that it may be viewed as the most beneficial practice. This may be due, in part, to its relative ease of implementation and immediate deliverables (i.e., readers can access a free and legal version of the published manuscript). However, these results should be interpreted with caution as they may be biased by our sampling methods, as well as recent initiatives to promote self-archiving in CSD. Additionally, our results suggest that open data were perceived as more beneficial compared to preregistration, highlighting the perceived benefits of making data, code, and materials accessible.

Limitations and Future Directions

There are several limitations that warrant discussion. Although we attempted a comprehensive sampling approach to capture various research positions and experiences, our sample was relatively small and skewed toward younger researchers and more research-intensive institutions. The small sample may have been due to the fact that one of the primary methods of recruitment relied on department chairs forwarding the study information to the wider population. This method made it difficult to quantify how much of the target population was reached, and as a result, it was not possible to identify an accurate response rate. Low response rates, however, have also been noted in prior surveys of open science practices (Houtkoop et al., 2018; Paret et al., 2022; Schmidt et al., 2016; Tenopir et al., 2011). In future work, this sampling method may be supplemented through other means, for example, through extracting contact information of authors publishing in CSD journals.

The sampling bias toward younger researchers and more research-intensive institutions may have occurred due to (a) researchers interested in open science being more likely to participate in this study and (b) promoting the survey through social media platforms that promote open science. It is likely that our results overestimate the knowledge and use of open science practices given this sampling bias. Considering this, it is also important to note that many researchers may not be in favor of open science practices (and thus, may not be likely to participate); additional research is necessary to directly study these scientists' perspectives on disadvantages of individual open science practices to obtain a more comprehensive view of this topic. Ultimately, future work should use other more comprehensive means to obtain a representative of CSD researchers.

A second limitation is that we limited our survey to four-core open science practices due to time and resource constraints. Our survey did not incorporate other common practices such as registered reports and replication studies. Future studies surveying CSD scientists' knowledge, participation, and perspectives on additional open science practices are warranted.

Finally, results from the thematic analysis may be interpreted with caution as the number of participants who answered the open text responses were minimal. More comprehensive work on this topic is underway using a qualitative research methodology to interview scientists in CSD to better understand their perceptions of open science practices and barriers to their implementation. Future studies will also investigate doctoral students' explicit training in CSD on open science practices. Ultimately, methodological reforms should be held to the same rigor and standards as empirical research (Devezer et al., 2021). Therefore, the relative benefits of these open science practices on transparency, openness, and replicability in the field of CSD will require future study, as well as ongoing discussion on optimal and nuanced evaluations of their use.

Conclusions

The results of this study show that researchers see a greater benefit for open science practices in their daily research, across their research field, and in public society despite their current level of knowledge and participation in these practices. These findings highlight a critical need for university, departmental, and community support for improving scientists' access and knowledge in implementing open science practices. Although outside the scope of this study, approaches to increase adoption have been discussed in adjacent fields (Gagliardi et al., 2014) and include engagement within and across scholarly communities (Armeni et al., 2021), increased departmental support, and incentives for implementing open science practices, badges to recognize open science in academic journals (Kidwell et al., 2016), and registered reports (Munafò et al., 2017). Adoption of open science practices can be cumulative, as incrementally adding each tool affords additional transparency and openness. Ultimately, increased implementation of these practices may improve the rigor and reproducibility of science in the field of CSD.

Data Availability Statement

The preregistration, data, and analysis script that support the findings of this study are publicly available in the Open Science Framework at https://osf.io/2f7xp/.

Funding Statement

Research reported in this publication was supported by the National Institute of Child Health & Human Development (T32HD007489, appointee: Long) and the University of Wisconsin–Madison. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Mary Alice Keller discloses that this research was supported in part by HCA Healthcare Graduate Medical Education. The views expressed in these publications represent those of the authors and do not necessarily represent the official views of HCA Healthcare or any of its affiliated entities.