Relationships Between the Survey of Organizational Research Climate (SORC) and Self-Reported Research Practices

Abstract

Background

The Survey of Organizational Research Climate (SORC) is a validated tool to facilitate promotion of research integrity and research best practices. This work uses the SORC to assess shared and individual perceptions of the research climate in universities and academic departments and relate these perceptions to desirable and undesirable research practices.

Methods

An anonymous web- and mail-based survey was administered to randomly selected biomedical and social science faculty and postdoctoral fellows in the United States. Respondents reported their perceptions of the research climates at their universities and primary departments, and the frequency with which they engaged in desirable and undesirable research practices.

Results

More positive individual perceptions of the research climate in one’s university or department were associated with higher likelihoods of desirable, and lower likelihoods of undesirable, research practices. Shared perceptions of the research climate tended to be similarly predictive of both desirable and undesirable research practices as individuals’ deviations from these shared perceptions.

Conclusions

Study results supported the central prediction that more positive SORC-measured perceptions of the research climate were associated with more positive reports of research practices. There were differences with respect to whether shared or individual climate perceptions were related to desirable or undesirable practices but the general pattern of results provide empirical evidence that the SORC is predictive of self-reported research behavior.

INTRODUCTION

Unethical or otherwise undesirable research-related behavior that compromises the integrity of science is problematic for a number of reasons. It threatens direct harm to research participants, indirect harm to others who may be subsequently affected by unreliable or invalid research results and compromises the societal value of funds spent on research. Such behavior poses a threat to institutional reputations and ultimately, may undermine confidence and public trust in the scientific enterprise.

Whereas the scientific community has long acknowledged that egregious research misbehaviors such as falsification, fabrication and plagiarism impact the integrity of science, scientists and the community at large have more recently widened the scope of concern to consider that more common everyday misbehaviors(Martinson et al., 2005) may also damage the scientific enterprise(Committee on Assessing Integrity in Research Environments (U.S.) et al., 2002; Steneck, 2004, 2006; Titus et al., 2008; Council of Canadian Academies and The Expert Panel on Research Integrity, 2010; Irish Council for Bioethics, Rapporteur Group, 2010; Steneck and Mayer, 2010; Council of Graduate Schools, 2011). Knowing the true prevalence of all such wrongdoing is highly problematic, if not impossible, due to its hidden nature(Hackett, 1994; Sovacool, 2008), but some recent attempts to probe this question have occurred, primarily in studies of biomedical and social science researchers. Based on self-reports of recent past behavior, the best estimates of formally defined misconduct hover around 1%, while up to a third of researchers have admitted to lesser, but still serious types of undesirable research-related behavior(Martinson et al., 2005; Fanelli, 2009). Causal explanations for misbehavior in science have similarly broadened from a laying of blame at the feet of mendacious, “bad apple” individuals to a recognition that behavior, for better or worse, can be fostered by climates that are more or less conducive to high quality research(Mumford and Helton, 2001; Committee on Assessing Integrity in Research Environments (U.S.) et al., 2002; Sovacool, 2008; Teitelbaum, 2008; DuBois et al., 2012).

In fact, in their report, “Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct”,(Committee on Assessing Integrity in Research Environments (U.S.) et al., 2002) the Institute of Medicine identified specific features of organizational climate believed to be conducive to research integrity, including visible support of organizational leadership for ethical practices, openness to discussion of ethical questions, practices that conform to stated policies, and organizational member perception that ethical conduct is a key expectation. At the time of the report, however, no gold-standard measures of organizational climate existed that research institutions could use to identify dimensions on which they are solid, areas of weakness, or organizational sub-units that may benefit from change initiatives. Such a tool would greatly facilitate the pursuit of a proactive, self-regulatory approach to promoting research integrity and research best practices.

As we have described in a companion article in this journal issue, our team has recently developed and validated the Survey of Organizational Research Climate (SORC). The SORC provides a measure of how respondents perceive the quality of the research environments in which they are immersed and the extent to which their organizational units and institutions support responsible research practices and research integrity. Notably, the SORC measures key institutional-level factors, such as visible ethical leadership, socialization and communication processes, and the presence of policies, procedures and structures to deal with threat to research integrity. These are factors that should be both mutable and subject to influence by local institutional leaders.

While having an ethical organizational climate is valuable in its own right, the “holy grail” of such organizational initiatives is to improve the quality of research produced by organizational members working in environments targeted for improvement(Heitman et al., 2005). We have previously tested the hypothesis that scientists’ perceptions of their working environments play an important role in whether they engage in misbehavior or questionable research practices(Martinson et al., 2006; Anderson et al., 2007). There we focused specifically on perceptions of organizational justice(Colquitt, 2001; Colquitt et al., 2002) to separately assess subjective perceptions of justice with respect to one’s immediate department, university, IRB/IACUC, and to the broader manuscript and peer review processes. By contrast, the items that make up the SORC ask respondents to report on more objective features of the research climate in their organizations, including processes, policies, structures, and outcomes.

Thus, the present work logically extends our prior work by using the SORC to assess perceptions of factors specific to universities and to departments and then relate those perceptions to research-related behaviors. It pursues the researcher-focused question of whether individual researchers with more positive perceptions of the organizational climate for research would be more likely to engage in desirable or ideal research practices and less likely to engage in undesirable behaviors. In keeping with the shift away from an isolated focus on individual perceptions and behaviors, we expand our inquiry of the relationships between organizational climate and behavior to consider the shared perceptions of researchers who work in the same local environment. We made use of the organizational structure of researchers working in departments and universities across the Unites States to assess two climate-focused questions. First, among scientists who share a work environment, are shared perceptions of organizational climate related to the quality of work performed by these scientists? Second, is the likelihood of an individual engaging in desirable or undesirable research practices associated with their holding exceptionally positive or negative perceptions of the research climate relative to the shared perceptions of others in their department. We expected that more positive shared perceptions of the organizational climate would be associated with more desirable and less undesirable research practices, and were curious as to whether this pattern would extend to individual departures from shared perceptions.

METHODS

Data collection

We obtained prior approval for this research from the Regions Hospital Institutional Review Board, the oversight body with responsibility for all research conducted at HealthPartners Institute for Education and Research, and from the University of Arkansas for Medical Sciences Institutional Review Board. The hierarchical sampling frame for this anonymous web-based survey with mailed follow-up included N=2,836 randomly selected biomedical and social science faculty and postdoctoral fellows from 251 departments that receive substantial funding from the National Institutes of Health within 40 academic health centers in top-tier research universities in the United States. The survey asked respondents to report on their perceptions of the research climates at their university and in the department in which they had their primary affiliation, about the frequency with which they engaged in a series of desirable and undesirable research practices, and about their professional and demographic characteristics. Additional detail about the sample frame construction and survey fielding procedures is available separately in a companion article in this issue. See Martinson, Thrush and Crain, in this issue.

Measures

Outcomes: Desirable and undesirable research practices

The key outcomes were respondents’ self-reports of having engaged in desirable and undesirable research practices. Four desirable research practice composites were constructed from items that asked respondents to report how often they engage in each of 10 behaviors reflecting ideals of ethical behavior in science(Steneck, 2004). The composite names and example behaviors were as follows: Maintaining confidentiality and data integrity (e.g., I manage data in ways that maintain data integrity and confidentiality), Playing by the rules (e.g., I comply with regulations and laws that govern research on human subjects), Avoiding favoritism (e.g., On my publications, all co-authors can explain the contributions that justify their authorship) and Following the Golden Rule (e.g., When working with trainees, I set clear rules for things such as performance expectations and intellectual credit)(Martinson et al., 2009). Responses to each item were categorized as to whether the respondent reported “always” engaging in the behavior or not. The composites were coded to represent whether the respondent always engaged in all of the behaviors included in the composite or did not always engage in at least one of them.

Six undesirable research practice composites were constructed from respondents’ self-reports of behaviors they had engaged in that are potentially detrimental to research integrity. We asked survey respondents to indicate whether they had engaged in any of 30 behaviors during the previous three years(Martinson et al., 2005, 2006, 2009). We considered respondents to have engaged in an undesirable research practice if they reported having engaged in any single behavior in a composite at least once in the previous three years.

The Neglect composite consisted of 4 neglectful or careless behaviors (e.g., inadequate record keeping related to research projects). The Top 10 composite consisted of behaviors that unquestionably damage the integrity of science (e.g., overlooking others’ use of flawed data or methods, inappropriately altering or suppressing research results in response to pressure from a commercial funding source). Three items pertained to the Circumvention of federal regulations regarding treatment of humans, animals, or materials related to research (e.g., circumventing or ignoring aspects of human subjects research requirements such as informed consent, confidentiality, etc.). Two asked about Misappropriation of resources or proprietary information (e.g., unauthorized use of proprietary information), and a single item assessed Careless peer review (inappropriate or careless peer review of papers or proposals). The FFP composite encompassed three items to assess fabrication or falsification of data and plagiarism (e.g., using another’s words or ideas without giving proper credit)(Office of Science and Technology Policy, 2000).

Predictors: Survey of Organizational Research Climate (SORC)

We constructed 7 key predictor measures from 32 SORC items that asked respondents about their perceptions of the research climate at their universities and primary departments(Martinson et al., 2012). Two of the SORC subscales assessed aspects of the research climate that operate at the level of the university (Regulatory Quality, RCR Resources) and the remaining 5 subscales asked about aspects of the research climate that operate within academic departments (Integrity Norms, Integrity Socialization, Integrity Inhibitors, Advisor-Advisee Relations, Expectations). Respondents rated each item (1 = Not at All to 5 = Completely) according to their perceptions of the quantity of a specific property existing in their university or department. Higher values on the SORC subscales reflected more positive perceptions of the research climate with the exception of the Integrity Inhibitors subscale, for which higher values reflected more barriers to research integrity.

Professional and demographic classifications

Measures of professional status included years since doctoral degree, whether doctoral degree was received in the US, type of doctoral degrees earned, academic rank, tenure status and department of primary affiliation. Demographic measures included gender, race and ethnicity.

Statistical analysis

Variance components models were estimated to calculate the intraclass correlations (ICC) of SORC subscales, and desirable and undesirable research practice composites among respondents from each academic department at each university. The variance components for the SORC subscales were estimated using general linear mixed regression models (GLMM) that specified a random intercept for each department and nested respondents within their local self-reported departments. Generalized linear mixed models (GzLMM; binomial error, logit link) estimated the variance components for the desirable and undesirable research practices.

The first objective of the inferential analysis was to assess the predictive utility of each raw SORC subscale with respect to desirable and undesirable research practices. These analyses expanded on the variance components models by estimating a separate GzLMM model to quantify how well respondents’ scores on each of the SORC subscales predicted each research practice composite. The SORC parameters were treated as fixed effects and their standard errors were estimated using empirical sandwich estimation (residual pseudo-likelihood estimation, subject-specific linearization). A separate series of GzLMM models was estimated using a model-based approach to calculating standard errors and produced virtually identical results.

The second set of inferential analyses took advantage of the hierarchically structured sample by predicting the likelihood of desirable and undesirable research practices from two empirically independent components for each of the seven SORC dimensions, one that reflected shared perceptions of the research climate in the respondent’s department relative to other departments, and one that reflected respondents’ individual climate perceptions relative to others in their department. The first component was a department mean, which was the average SORC subscale score among all respondents who reported working in the same department at the same university. A higher value on this score denoted departments in which members tended to perceive a better research climate. The second component, the individual component, was a respondent deviation score, calculated by centering each respondent’s raw subscale score around their department mean score. Higher deviation scores denoted respondents who held more positive perceptions of the research climate relative to others who worked in their department, and lower values denoted respondents whose perceptions were more negative relative to others in their immediate environment.

We continued to use GzLMMs to assess the strength of the relationships between climate and practice using the same specifications as the models that predicted research practices from raw SORC scores. In these models, however, practices were predicted by a department mean SORC score, which varied across departments, and a respondent deviation SORC score, which varied across individuals. Positive relationships between department scores and research practices can be interpreted to mean that a practice is more likely to be observed in departments with higher average climate scores. Conversely, a negative relationship can be interpreted to mean that a practice is less likely in departments with higher average climate scores. Positive relationships between respondent deviation scores and practices can be interpreted to mean that respondents who have more positive perceptions of the research climate relative to others in their immediate work environment are more likely to engage in a given practice. A negative relationship can be interpreted to mean that respondents with more positive perceptions than others are less likely to engage in a given practice. Because the department mean scores and respondent deviation scores are statistically independent, it is possible for one, both or neither of these values to predict research practice in the same regression model.

RESULTS

Descriptive Information

There were N=1267 survey respondents (n=952 web, n=316 mailed), representing roughly 50% of the eligible sample (N=2543), with M=5.3 (SD=3.8; inter-quartile range = 3–7) respondents per academic department. Briefly, respondents tended to be non-Hispanic White men who earned their PhD about 20 years ago from an institution in the United States and were currently in tenure track positions. About half worked in academic departments focused on the basic sciences while one in five each worked in medicine and in applied health and sciences. Detailed information about the personal and professional characteristics of survey respondents is available separately(Martinson et al., 2012).

As might be expected, respondents were more likely to report that they always engaged in desirable research practices than to acknowledge undesirable ones (Table 1). The proportion of respondents endorsing desirable research practices (range: Golden Rule 23% to Maintaining Confidentiality 69%) is consistent with previous research conducted among academic researchers in a broader range of disciplines(Martinson et al., 2006). Also consistent with this previous work was the range of respondents reporting undesirable research practices. Neglect was a relatively common practice being reported by nearly half of the respondents. More serious undesirable practices were reported by about 1 in every 6 (Top 10, Circumventing Federal Regulations) to 1 in every 10 (Misappropriation) respondents. Only about 5% of respondents acknowledged reviewing a paper or proposal carelessly or engaging in FFP.

Table 1.

Percent of respondents who self-reported desirable and undesirable research practices and department-level intraclass correlations (ICC).

	n	%	ICC
Desirable research practices
Maintaining Confidentiality	971	69.2	.03
Playing by the Rules	872	62.6	.17
Avoiding Favoritism	951	46.2	.02
Following the Golden Rule	949	22.8	.05
Undesirable research practices
Neglect	935	46.7	.02
Top Ten	936	17.5	.00
Circumventing Federal Regulations	938	14.3	.05
Misappropriation	937	10.5	.11
Careless Peer Review	930	5.4	.14
FFP	936	4.9	.06

The distributions of the raw SORC subscale scores suggested that respondents gave generally positive ratings of the organizational climates at their universities and departments, Ms=3.5–4.1 where 5 is the most positive value, although Integrity Inhibitors (M=2.1) is worded so that a value of 1 represents the most positive climate rating (Table 2). Implicit to the research questions being posed is the notion that researchers who share a work environment will have similar perceptions of the organizational climate. As expected, researchers who reported working in the same academic department tended to rate the climates in their universities and departments more similarly than would researchers at different departments in the same university or at different universities (departmental intraclass correlations, ICCs = .09–.20). Such large proportions of variance at the department level made it feasible to partition raw SORC scores into independent components that represented shared intradepartmental perceptions of the research climate and individual respondents’ deviations from those shared perceptions. Mean department SORC scores were very similar to the respondent level means while their standard deviations were consistently lower than the raw scores. Because each respondent’s deviation score is centered around a department mean, deviation score means and ICCs will both, by definition, be zero.

Table 2.

Descriptive statistics for the raw SORC subscale scores and for department mean and deviation scores.

	raw		department	deviation
SORC subscale	M (SD)	ICC	M (SD)	M (SD)
University
Regulatory Quality	3.78 (.78)	.09	3.76 (.65)	0 (.59)
RCR Resources	3.89 (.77)	.12	3.90 (.57)	0 (.60)
Department / Program
Integrity Norms	4.08 (.69)	.20	4.07 (.56)	0 (.50)
Socialization	3.50 (.90)	.12	3.49 (.74)	0 (.67)
Integrity Inhibitors	2.08 (.83)	.09	2.10 (.64)	0 (.65)
Advisor-Advisee Relations	3.78 (.73)	.11	3.77 (.58)	0 (.55)
Expectations	3.64 (.87)	.11	3.61 (.70)	0 (.67)

SORC scores and research practices

The consistent pattern of relationships between the SORC subscales and research practices was that more positive perceptions of the research climate in one’s university or department were associated with higher likelihoods of desirable, and lower likelihoods of undesirable, research practices.

Respondent ratings on the SORC university focused Regulatory Quality and RCR Resources subscales were positively related to each of the four desirable research behaviors. Respondents who had more positive perceptions of these aspects of the research climate were more likely than those with less positive perceptions to report that they always engaged in all of the behaviors included in the four desirable research practice composites (Table 3). The same pattern of results held for the SORC department focused measures of Integrity Norms, Socialization, and Advisor-Advisee Relations, and to a lesser extent Expectations and Integrity Inhibitors.

Table 3.

Statistically significant regression coefficients from models that predicted always engaging in desirable research practices from raw SORC scores.

	Maintaining Confidentiality	Playing by the Rules	Avoiding Favoritism	Following the Golden Rule
University
Regulatory Quality	.445***	.359***	.205*	.569***
RCR Resources	.659***	.515***	.464***	.878***
Department / Program
Integrity Norms	.810***	.496***	.576***	.926***
Socialization	.507***	.378***	.444***	1.000***
Integrity Inhibitors	−.276**		−.273***
Advisor-Advisee Relations	.570***	.280**	.472***	.743***
Expectations	.251**		.374***	.502***

^***p<.001,

^**p<.01,

^*p<.05

As a few examples of the range of relationships between raw SORC scores and desirable research practices, the predicted likelihood that respondents reported always Playing by the Rules ranges from about 58% among respondents who had a score of 3 on the SORC Regulatory Quality subscale to a predicted likelihood of 74% among those who had a 5. A stronger relationship was observed between SORC Integrity Norms and the Golden Rule composite, where a score of 3 had a predicted likelihood of 9% while 5 was related to a 39% likelihood. Negative relationships obtained between SORC Integrity Inhibitors and two desirable research practices, where the predicted likelihood of always Maintaining Confidentiality dropped from about 76% among respondents with a score of 1 to about 64% for those with a score of 3.

A conceptually similar but more specific pattern of results obtained in the analyses that predicted undesirable practices from respondent SORC scores. Respondents with higher scores on the university focused subscales were less likely to report that they had engaged in the more frequently reported undesirable research practices – Neglect, Top 10, Circumvention of Federal Regulations and Misappropriation (Table 4). Higher Integrity Norms scores on the SORC were related to a lower likelihood of reporting all of the undesirable practices. The SORC Expectations subscale was not related to any of the undesirable practices, while Integrity Inhibitors was related to the more serious undesirable practice composites, FFP and the more inclusive Top 10.

Table 4.

Statistically significant regression coefficients from models that predicted engaging in any undesirable research practices from raw SORC scores.

	Neglect	Top 10	Federal Regs	Misapp	Careless Review	FFP
University
Regulatory Quality	−.535***		−.445***	−.338**
RCR Resources	−.564***	−.353**	−.579***	−.313*
Department / Program
Integrity Norms	−.469***	−.589***	−.604***	−.310*	−.419*	−.460**
Integrity Socialization	−.480***	−.386***	−.559***	−.323**
Integrity Inhibitors		.367***				.461**
Advisor-Advisee Relations	−.444***	−.332**	−.352**		−.308*
Expectations

^***p<.001,

^**p<.01,

^*p<.05

Among respondents who provided a score of 3 on SORC Integrity Socialization, 53% were predicted to have engaged in Neglect, which dropped to 30% for those with a score of 5. Similarly, 22% of respondents providing a score of 3 on SORC Integrity Norms were predicted to violate Federal Regulations, which dropped to 8% for those with a score of 5. The predicted likelihood of engaging in one of the Top 10 most serious practices was 13% among respondents who gave a rating of 1 on the SORC Integrity Inhibitors subscale and increased to 23% among those with a rating of 3.

Shared versus individual perceptions and research practices

The next set of analyses took advantage of the hierarchically structured sample by predicting the likelihood of desirable and undesirable research practices from two empirically independent components of the SORC scores, the average of department members’ perceptions of the research climate, and each respondent’s perception relative to his or her department’s mean. By doing this, the relative strength of the associations between shared and individual perceptions of organizational climate and research practices could be simultaneously assessed.

In general, desirable behaviors were more likely to be reported by researchers in departments with more positive shared perceptions of the research climate, and by researchers who had more positive perceptions than others in their immediate environment. If the department mean was related to desirable research practices (Table 5, top row of each cell) then the researchers’ deviations also tended to be related (Table 5, bottom row of each cell). The notable exception to this pattern is the SORC Integrity Inhibitors subscale. Maintaining Confidentiality and Avoiding Favoritism behaviors were less likely to be reported in departments with stronger shared perceptions that there were barriers to research integrity but variation in individual respondents’ perceptions relative to their colleagues were not related to these practices.

Table 5.

Statistically significant regression coefficients from models that predicted always engaging in desirable research practices from department mean SORC scores (M) and researcher deviations from department means (dev).

	Maintaining Confidentiality	Playing by the Rules	Avoiding Favoritism	Following the Golden Rule
University
Regulatory Quality	M=.604***	.532***	.286*	.896***
	dev=.339**	.264*	ns	.373*
RCR Resources	.772***	.748***	.594***	.869***
	.593***	.403**	.384***	.884***
Department / Program
Integrity Norms	.726***	.489*	.507**	.636**
	.884***	.501**	.635***	1.159***
Integrity Socialization	.605***	.465**	.379**	.850***
	.435***	.327**	.492***	1.100***
Integrity Inhibitors	−.547***		−.427**
	ns		ns
Advisor-Advisee Relations	.645***	ns	.624***	.638**
	.518***	.388**	.368**	.812***
Expectations	ns		.373*	.321*
	.318**		.375***	.614***

^***p<.001,

^**p<.01,

^*p<.05

Figure 1 presents a prototype of how department mean scores and respondent deviation scores simultaneously predict research practices. The separate lines show that desirable practices tend to be more common in departments with more positive shared climate perceptions on the SORC. The slope of each line shows that within departments, researchers with above-average perceptions on the SORC are more likely to engage in desirable behavior than those with below-average perceptions. The regression coefficients presented in Table 5 show that shared perceptions of the research climate are at least as strongly related to desirable research practices as are individual’s perceptions of these climates.

Figure 1.

Prototype of simultaneous relationships between shared and indiviual perceptions of the research climate and research practices.

The relationships between shared and individual climate perceptions and undesirable research practices were less consistent (Table 6). More positive shared and individual perceptions on the SORC were related to less frequent reporting of the relatively common (Neglect) and broadly defined (Top 10, Circumventing Federal Regulations) undesirable research practices, consistent with the pattern displayed in Figure 1.

Table 6.

Statistically significant regression coefficients from models that predicted engaging in any undesirable research practices from department mean SORC scores (M) and researcher deviations from department means (dev).

	Neglect	Top 10	Federal Regs	Misapp	Careless Review	FFP
University
Regulatory Quality	M=−.528***		−.483**	ns
	dev=−.539***		−.418**	−.426*
RCR Resources	−.713***	−.347*	−.696***	ns		−.696*
	−.476***	−.357*	−.519***	−.334*		ns
Department / Program
Integrity Norms	−.311*	−.582**	−.516*			−.733***
	−.609***	−.560**	−.680***			ns
Integrity Socialization	−.430***	−.337*	−.590***	ns
	−.521***	−.425**	−.535***	−.398*
Integrity Inhibitors		.512***			.560*	.709**
		.271*			ns	ns
Advisor-Advisee Relations	−.398**	−.462**	ns			−.753***
	−.477***	ns	−.331*			ns
Expectations

^***p<.001,

^**p<.01,

^*p<.05

Differentiated patterns emerged for the more narrowly defined Misappropriation, Careless Peer Review and FFP composites. Reports of Misappropriation were less likely among respondents who had relatively positive perceptions of the university climate and were related to only one of the department mean scores (Integrity Socialization). Careless peer review tended to be unrelated to climate perceptions. The most serious but least frequently reported of the undesirable practice composites, FFP, was not related to respondents’ perceptions relative to others in their departments on any subscales but FFP was more likely to occur in departments whose members had less positive shared perceptions of the organizational climate for research.

Statistical limitations

These results are subject to some limitations that warrant mention. The 50% response rate raises some concern about non-response bias but does not guarantee it(Groves, 2006). The present response rate is similar to that obtained in a similar survey by the Council of Graduate School’s Project on Scholarly Integrity (http://www.scholarlyintegrity.org/ShowContent.aspx?id=402), similar to our own prior work on research integrity conducted in two samples of academic scientists(Martinson et al., 2006), and higher than a previous study we conducted in a similar sample of biomedical and social science researchers in which a thorough investigation did not find consistent evidence of response bias(Martinson et al., 2010). Another consideration when interpreting these analyses is that the SORC subscales are not statistically independent. As a result, the relationships between correlated SORC subscales and a given behavior are in part attributable to the unique construct measured by each subscale but also attributable in part to overarching perceptions of research climate that are common across subscales. While correlation among SORC subscales complicates their interpretation, we assert that the aspects of the research climate measured by the SORC are also not conceptually independent and therefore the sub-scales measuring them should not be forced to be empirically independent. A final consideration is the relatively low number of respondents per department, which may inflate the variance in department mean scores and reduce the variance in respondent deviation scores. Especially in departments with only a few or non-representative respondents, department mean estimates may be unstable so that variability in respondent scores relative to the true department mean may be misattributed to the department level. As a result, power to detect relationships between respondent deviations and research practices may be decreased. Because the number of respondents per department was relatively constant across all of the estimated models, and respondent level relationships were observed in many of these models, this limitation does not discount the overall pattern of results.

CONCLUSIONS

Our central prediction that more positive perceptions of the research climate, as measured by the SORC, would be associated with more positive research practices was supported by the study results. There were some differences with respect to whether shared or individual perceptions of the research climate were related to reports of desirable or undesirable practices but the general pattern of results demonstrate that the SORC is predictive of self-reported research behavior.

These results provide evidence that this easily administered tool can reliably and intuitively predict desirable and undesirable research-related behavior. The utility of such a tool is that it can enable the assessment of research climate across sub-units of an organization so that improvement efforts, and consequently the quality of research produced at an organization, may be targeted to areas in which they are most needed rather than a one-size-fits-all approach.

Considering respondents’ raw SORC scores, more positive perceptions of the research climate were consistently related to more frequent endorsement of all four of the desirable research practice composites and less frequent reporting of the four “every day” undesirable research practices.

Decomposing SORC scores into shared and individual components, and quantifying the strength of the relationships of each with research practices, revealed a similar but somewhat more discriminating pattern of results. The most noteworthy finding in these analyses was that the shared perceptions of the research climate tended to be more predictive of both desirable and undesirable research practices than individuals’ deviations from these shared perceptions.

More positive shared and individual climate perceptions were both consistently related to more frequent reports of desirable research practices. The notable exception to this pattern was that only shared perceptions of stronger Integrity Inhibitors were associated with less frequent reporting of two desirable research practices.

Both shared and individual climate perceptions also tended to be intuitively related to undesirable yet “every day” research practices. Misappropriation, however, was more likely to be reported only among respondents who had especially negative climate perceptions relative to their colleagues. In contrast, FFP was more likely to be reported in departments where there were more negative shared climate perceptions on four of the seven SORC subscales but was unrelated to respondents’ deviations from the shared departmental perceptions.

The most serious of the undesirable research practices, FFP, was not at all related to individual climate perceptions once they were empirically separated from shared climate perceptions. This fact highlights shortcomings of adopting a “bad apple” approach to monitoring research practice since researchers with especially poor climate perceptions were no more likely to engage in FFP than those with the most positive perceptions. It also underscores the potential benefit to fostering a shared research climate in which, for example, there are relatively positive perceptions of RCR resources, integrity norms and advisor-advisee relationships, and few inhibitors to conducting quality research. The relationships between the Integrity Inhibitors scale and research practices is also noteworthy. To the extent that the Integrity Inhibitors score was related to research practices, stronger shared perceptions of barriers to integrity was related to less desirable practices while individual perceptions of inhibitors were unrelated to practice. The means and standard deviations of the Integrity Inhibitor subscales were similar to those of other subscales, discounting the possibility that a restriction in range at the respondent level prevented these relationships from being statistically significant. It is possible that the presence of negative environmental factors has a qualitatively different impact on behavior than the absence of positive factors, although having only one negatively coded subscale prevents us from observing such a pattern.

Two of the undesirable research practices, careless peer review and FFP, were reported by only 5% or fewer of respondents. The dearth of relationships between individual SORC scores and reports of these practices could therefore be due to either the true absence of relationships between these constructs or to a lack of statistical power. The fact that there were department level relationships between four SORC subscales and FFP, and only one with carelessness, suggests that low power is probably not the only factor responsible for the lack of individual level relationships but it may nonetheless play a role.

In conclusion, our observation of significant associations between departmental level perceptions of organizational climate and a range of both desirable and undesirable research-related behaviors lends credence to the argument that the SORC can serve as a useful tool for facilitating the kind of “reporting and feedback” that elsewhere has been argued to be an effective means for supporting positive culture change, and improving operational outcomes(Leape, 2010). Further bolstering this case is the fact that the SORC measures specific aspects of local organizational climates, such as visible ethical leadership, socialization and communication processes, and the presence of policies, procedures, structures and processes to deal with risks to integrity, that should be subject to influence by interested local institutional leaders and officials. We believe that the generation and feedback of this information may itself engender both a greater awareness among organizational leaders of specifically where and how their organizations are weak, as well as motivate some of them to engage in efforts to address those weaknesses.