Abstract
In this study, we assess the association between academics’ research agendas and their preferences for basic research, applied research, or experimental development. Using a sample of Mexican academics working in some of the country’s most research-oriented universities, we identify three clusters. The largest is composed of applied research-oriented academics, the second largest is composed of basic research-oriented academics, and the smallest is composed of academics who engage in both basic and applied research, and experimental development. The strategic research agendas of the three clusters are distinguished from each other along four main dimensions: Divergence, Discovery, Mentor Influence, and Social Orientation. These findings show that strategic research agendas are associated with preferences for basic research, applied research, or experimental development, but only to some extent. We also extend the Multi-Dimensional Research Agendas Inventory – Revised, a widely used instrument for measuring strategic research agendas, by adding a new dimension, “Government,” and validating the instrument in a new context. We also make the scale available in Spanish for use by academics, practitioners, managers, and administrators in Spanish-speaking countries.
Keywords: Strategic research agendas, Basic research, Applied research, Experimental development, Academic preferences
Introduction
The categorization of research activities into basic research, applied research, and experimental development was formalized by the influential Frascati Manual, published by the Organization for Economic Cooperation and Development (OECD) in 1963 (Godin, 2006). The Frascati Manual was a response to the need to differentiate types of research, given the changes in the development and funding of research in the second half of the twentieth century (Schauz, 2014). According to the Frascati Manual, the main purpose of basic research is the advancement of knowledge regardless of any particular use or application. Applied research has a problem-solving rationale and is oriented towards practical objectives, whereas experimental development involves the use of research knowledge to generate new products and services or improve current ones (OECD, 2015).
Studies on the evolution and purposes of basic and applied research, and experimental development, have been carried out. These often examined sectors and organizations that privilege one research focus over the other, public policies that promote research specialization and/or broadness, and how changes in science and technology influence these research categories (e.g., Fan et al., 2021; Larivière et al., 2018; Salter & Martin, 2001). Some studies are intellectual discussions of the benefits, challenges, and meaning of categorizing research into basic, applied, and experimental development, and of the feasibility of developing other categorization systems (e.g., Godin, 2002). However, despite critiques, the original OECD categorization is still generally accepted and frequently used in policymaking, managerial, and academic circles (Roll-Hansen, 2017; Schauz, 2014). Some studies have linked this research categorization to career-related incentives and the evaluation of academic research. The orientation towards applicability, problem solving, and impact has become explicit in national research projects funded by governments and in the evaluation criteria used by universities (Marques et al., 2017). This has led to concern that the focus on applicability in funding and evaluation has created extrinsic incentives and constraints on academic research and has encouraged a shift from basic to applied research in many North American and European countries (e.g., Zapp & Powell, 2017).1 However, studies have shown that instead of shifting from basic to applied research, academics have adapted by carrying out both types (Bentley et al., 2015), using a complex mix of the two (Gulbrandsen & Kyvik, 2010).
Despite all the studies mentioned above, the preference of individual academics for basic research, applied research, or experimental development is still understudied in the literature. It is known that academics with work experience outside academia tend to prefer applied research and experimental development; it is unsurprising that they bring problems and ideas from their employment experiences to their academic work practices (Gulbrandsen & Thune, 2017). Full professors, although continuing to engage in basic research, also tend to be more engaged in knowledge exchange and commercialization activities, and therefore conduct more applied research and experimental development than assistant and associate professors (Gulbrandsen & Smeby, 2005). Recent research shows that choosing to focus on either basic research or applied research requires academics to make a trade-off between publications and innovations; only a few, known as ambidextrous scholars, manage to balance basic and applied research by relying on network dynamics and collaboration (Werker & Hopp, 2020). Academic women tend to be overrepresented in applied research fields and underrepresented in basic research (Abreu & Grinevich, 2016). In other words, male academics are more engaged in basic research that contributes to scientific progress and female academics are more engaged in applied research that addresses social issues and development (Zhang et al., 2021). Academics in the humanities and natural sciences are more likely to engage in basic research, those in the social and health sciences are more likely to have mixed research focuses, and those in the fields of engineering tend to engage in applied research and experimental development (Gulbrandsen & Kyvik, 2010). Similarly, the analysis of research orientation in 15 countries by Bentley et al. (2015) notes that although there are inter-country differences, academics who specialize in basic research consistently work in settings where applied research is not emphasized, obtain less funding, and are less engaged with social problems. These findings highlight the role of national and institutional policies in shaping academics’ research. It also highlights the role of academics as agents; they use the interactions between their educational and professional backgrounds and structural factors to shape their research strategies and define their identities as researchers (see Sapir, 2017).
The studies discussed above provide valuable insights into the question of who adopts specific research focuses in what settings, but an important gap can be identified: the extent to which academics’ strategic research agendas (SRAs)2 relate to individual preferences for basic research, applied research, and experimental development. Understanding the relationship between SRAs and research preferences is important because studies have shown that SRAs are associated with personal attributes, such as gender (Santos et al., 2021), concepts of research (Santos & Horta, 2020), thinking styles (Santos et al., 2020), and choice of collaborators (Horta et al., 2021). This means that an academic’s SRA is related to research processes, but it is also imbued with the cognitive, judgmental, and decision-making traits of the researcher. The latter have been understudied in relation to research focuses, and therefore, in this paper, we assess the association between academics’ research agendas and their preferences for basic research, applied research, or experimental development. This leads to this study’s research questions:
Are there archetypes in terms of academics’ preference for basic research, applied research, and experimental development?
Are academics’ SRAs associated with their preference for basic research, applied research, and experimental development?
This second question in particular allows us to examine how individual academics’ beliefs, wants, and planning with regards to research are shaped by their predispositions and values (see Mallon et al., 2005) and how these are related to the pursuit of research activities in a spectrum that has basic research on one end and applied research/experimental development on the other (Bentley et al., 2015). The analysis uses a dataset of Mexican academics working in some of the most research-oriented universities in the country. This sample is relevant because most research on academics engaging in basic research, applied research, and experimental development has been conducted using samples from countries with developed scientific systems. One exception is the country comparison of Bentley et al. (2015), but they conduct a broad inter-country comparison and do not focus on developing scientific systems. Moreover, our dataset of academic researchers in Mexican universities allows us to achieve a second aim: validating the Multi-Dimensional Research Agendas Inventory – Revised (MDRAI-R) in a new context (see the following section and Horta & Santos, 2020). The replicability of the MDRAI-R, its translation into Spanish, and the addition of a new dimension, “Government,” represent further developments of this instrument, which can inform future research and managerial practices.
Data and method
SRAs and the MDRAI-R
We use the MDRAI-R (Horta & Santos, 2020), which is an instrument designed to characterize SRAs. It has been widely used in conceptual and empirical studies. It measures eight main dimensions, some of which have sub-dimensions, of the strategic choices and preferences in researchers’ SRAs.
The first dimension, Scientific Ambition, is related to striving for prestige and peer recognition, and the related drive to publish scientific articles (i.e., the need to produce and disseminate knowledge), which are pivotal criteria in contemporary academic careers. The second dimension, Divergence, is a preference for expansion into multiple fields of knowledge, and engagement in multidisciplinary research, a key feature of current science, especially in pioneering topics where single-disciplinary perspectives are insufficient. The third dimension, Discovery, is a preference for topics that have the potential to lead to new scientific discoveries and breakthroughs, a type of research that is typically high risk, high reward. The fourth dimension, Tolerance to Low Funding (TTLF), is the willingness of an academic to engage in research projects with little to no sources of funding. The fifth dimension, Collaboration, represents both the opportunity and the willingness to participate in collaborative ventures. The sixth dimension, Mentor Influence, represents the degree to which an academic’s research is influenced by his or her mentor, typically the Ph.D. supervisor. The Academia Driven dimension measures the extent to which a person’s research agenda is influenced by institutional missions and goals, which may be either scientific communities that the academic identifies with or the university where he or she works. The eighth dimension, Society Driven, represents the degree to which the academic’s research agenda is oriented toward tackling societal problems, and the degree to which consultation with non-academics shapes the research agenda.
In this study, we add a new dimension to the MDRAI-R: the Government dimension, which measures the degree of perceived support (by academics and researchers) that the government provides to different knowledge activities. The introduction of this dimension is relevant because of the increasing influence of public policies and the associated research funding on the academics’ research orientation (Gläser & Laudel, 2016). Government policies that favor the development of higher education and science and technology tend to foster greater levels of research productivity, but also a greater intensity of knowledge and technology transfer behaviors by academics (e.g., Kowalczewska & Behagel, 2019). The introduction of the new Government dimension allows the removal of several redundant items to accommodate the new questions without increasing the survey’s length. Specifically, one item is removed from each of the Divergence, Collaboration, Discovery, and Society Driven scales. Table 1 summarizes the dimensions of the MDRAI-R/ MDRAI-R-S.
Table 1.
Dimensions of the MDRAI-R/MDRAI-R-S
| Dimension | Definition |
|---|---|
| Scientific Ambition |
Prestige The desire to acquire recognition and academic prestige in a given field Drive to Publish Motivated to publish scientific articles |
| Divergence |
Branching out Desire to expand into other fields of study or topics Multidisciplinarity Preference for working in multidisciplinary research ventures |
| Discovery | Preference for working in fields or topics with the potential to lead to scientific discoveries |
| Tolerance to Low Funding (TTLF) | Willingness to work in fields or topics for which research funding is scarce |
| Collaboration |
Willing to Collaborate Desire to engage in collaborative scientific ventures Invited to Collaborate Have the opportunity and the invitations to participate in collaborative scientific ventures |
| Mentor Influence | The researcher’s mentor (Ph.D. or otherwise) holds a degree of influence over his or her work |
| Academia Driven |
Field oriented The extent to which the research agenda is influenced by scientific priorities that the field community determines by consensus Institution oriented The propensity of the researcher to align their research agenda with the strategic research targets of their institution |
| Society Driven |
Society oriented The prevalence of society related challenges in the research agenda Non-academic oriented The influence and participation of laymen and non-experts in the design of the research agenda |
| Government | Perceived level of governmental policies and financial support to science, research, and academia |
Partly adapted from Horta et al. (2021)
SRAs and expected preferences for basic and applied research and experimental development
The relationships between some SRA dimensions and preferences for basic research, applied research, or experimental development are unclear in the literature. For example, Ranga et al. (2003) show that academics’ publication profiles are often a mix of basic and applied research, which suggests that the influences of Scientific Ambition, Discovery, and Collaboration on research preferences are difficult to assess. These dimensions are highlighted in relation to academics’ publication profiles, as a recent study showed that they are associated with academics’ research productivity throughout their careers (Santos et al., 2022). Academics’ willingness to do research even with low or no funding (i.e., TTLF) is also difficult to assess when related to preferences for basic and applied research because funding may be a consideration or may be allocated by funders for some basic and applied research fields or contexts but not for others (e.g., Overland & Sovacool, 2020). However, a negative association between experimental development and a high TTLF score in that dimension may be expected because of the high costs that experimental development projects usually entail (see Hirzel et al., 2018).
The evidence of the role of Mentor Influence in an academic’s specific research focus is also expectedly mixed. Full professors, who are the most common mentors for Ph.D. students and the most influential, tend to be more engaged in applied research and experimental development than associate and assistant professors (e.g., Gulbrandsen & Smeby, 2005). However, a mentor’s influence is strongest in the early stages of an academic’s career. As academics need to publish, at the beginning of their career, they may be required to focus on basic research with some applied research, rather than full-on applied research and experimental development (Santos & Horta, 2018). For the divergence dimension, there is an expectation that academics focused on interdisciplinary and translational research may prefer applied research (Valentin et al., 2016). Considering the emphasis that many scientific communities, governments, and universities are placing on the production of knowledge that can be used by non-academic stakeholders, academics scoring high on the Academia Driven and Government dimensions (Jongbloed et al., 2008) are expected to favor applied research. Similarly, academics who are more socially oriented are likely to engage in applied research and experimental development, as their research will focus on problem-solving, targeted research, and development of products, services, or solutions to a problem (Raynor, 2019).
Data collection
The first step in data collection was identifying all of the academics working in some of the most research-oriented universities in Mexico (UNAM, ITESM, UAM, UANL, and BUAP). A total of 15,093 individuals were identified on university websites. They were contacted via e-mail in three waves between April and July 2021 with an invitation to complete a survey. A total of 1160 valid responses were collected, representing a response rate of 7.68%. The survey began with an informed consent form that the participants were required to sign before proceeding to the translated and updated version of the MDRAI-R (henceforth, MDRAI-R-S) and other questions relevant to the analysis. Table 2 contains details on the sampling, notably the population size per institution, the sample size per institution, and the relative difference in percentage. Overall, across the nine institutions, there is an average distribution difference of 2.98%. A paired samples t-test used to compare the population percentage with the sample percentage for each institution showed no significant differences (t(8) = 1.413, p = 0.195), confirming the similarity of the population’s and the sample’s distribution in terms of institutions.
Table 2.
Population and sample distribution of institutions
| Institution | Population N | Population % | Sample N | Sample % | Difference % |
|---|---|---|---|---|---|
| BUAP | 972 | 6.44 | 117 | 10.10 | 3.66 |
| IBERO | 302 | 2.00 | 35 | 3.00 | 1.00 |
| IPN | 1263 | 8.37 | 140 | 12.10 | 3.73 |
| ITAM | 85 | 0.56 | 11 | 0.90 | 0.34 |
| ITESM | 669 | 4.43 | 87 | 7.50 | 3.07 |
| UAG | 708 | 4.69 | 73 | 6.30 | 1.61 |
| UAM | 3024 | 20.04 | 177 | 15.30 | 4.74 |
| UANL | 936 | 6.20 | 72 | 6.20 | 0.00 |
| UNAM | 7134 | 47.27 | 448 | 38.60 | 8.67 |
Procedure
We conducted several analyses. The first was the validation of the MDRAI-R-S, which used structural equation modeling, specifically confirmatory factor analysis (CFA; Kline, 2016; Marôco, 2010). As this is the most technical section of the paper, the implementation is described in some depth. In the second analysis, we conducted a cluster analysis with three input variables: share of time dedicated to basic research; share of time dedicated to applied research; and share of time dedicated to experimental development. The cluster analysis was an exploratory procedure used to identify patterns in the sample (Hair et al., 2014; Marôco, 2003) and has been used in other studies to categorize individuals based on science indicators (Almeida et al., 2009; Santos & Horta, 2015). The goal of this analysis was to identify research agenda profiles based on the allocation of time to different types of research. For this purpose, a two-step clustering algorithm in SPSS 26 was used, which is generally considered a superior alternative to classical hierarchical clustering (Norusis, 2012; Zhang et al., 1996). Following this clustering procedure, a multinominal regression analysis was performed with the clustering variables as dependent variables, and the MDRAI-R-S dimensions—as well as controls—as predictors. The aim was to identify whether there were differences across research profiles.
Variables
This section defines the variables used in the multinominal regression analysis. The primary independent variables were the SRA dimensions, described above. These were complemented with control variables drawn from previous studies characterizing academics’ preference for basic research, applied research, or experimental development (i.e., Bentley et al., 2015; Gulbrandsen & Kyvik, 2010; Gulbrandsen & Smeby, 2005; Ranga et al., 2003; Werker & Hopp, 2020): gender (reference category: female); field of science3 (FOS; reference category: agricultural sciences); non-academic experience, which indicates whether the academic has work experience outside academia; full professor, which is a dummy variable indicating whether the participant is a full professor; and external funding, which is a dummy variable indicating whether the participant has received external funding in the past 3 years. These control variables allowed us to assess whether our findings matched those of other studies of research preferences, which have generally been undertaken in advanced scientific systems, whereas ours focused on a developing scientific system.
We also used a number of control variables not included in previous empirical research on this topic. For example, academic career duration is a self-explanatory variable that assesses the possibility of academics shifting their focus from research and publications to administration, knowledge exchange, and other activities more related to financial rewards as their careers progress, leading them to focus more on experimental development over time (Mittermeir & Knorr, 1979). Academic mobility may also have a role. In a study of academic inbreeding in Mexico, Horta et al. (2010) find that non-mobile academics are more likely to be engaged in knowledge transfer activities than their peers, suggesting that they are more oriented toward applied research and experimental development than their more mobile peers. Accordingly, we used the non-mobile academics category (i.e., academics hired by the university where they obtained their Ph.D. who remain there for their professional career) as the baseline for our measure of academic mobility. The other categories of mobility were as follows: silver-corded (those currently working in the university where they earned a Ph.D., but who have worked in other universities), adherents (those who were hired by a different university than the one where they completed their Ph.D. and stayed at that university), mobile national (those who have held academic jobs at several Mexican universities), and mobile international (those who have held several academic jobs including some at non-Mexican universities). The final control variables categorized academics by the percentage of time they dedicate to each of the following activities: teaching, research, knowledge exchange, administrative tasks, and supervision of students. It is known that these activities sometimes complement each other, and at other times constrain each other, but how they relate to academics’ focus on basic research, applied research, and experimental development is unknown. Table 3 summarizes the descriptive statistics for the variables employed in this study.
Table 3.
Descriptive statistics for the control variables
| Variable | N | % |
|---|---|---|
| Gender | ||
| Female | 438 | 37.80 |
| Male | 721 | 62.20 |
| FOS | ||
| Agricultural Sciences | 21 | 1.80 |
| Engineering & Technology | 399 | 34.50 |
| Humanities | 43 | 3.70 |
| Medical and Health Sciences | 129 | 11.10 |
| Natural Sciences | 263 | 22.70 |
| Social Sciences | 303 | 26.20 |
| Full professor | ||
| Not full professor | 222 | 20.90 |
| Full professor | 842 | 79.10 |
| Non-academic experience | ||
| No | 368 | 34.50 |
| Yes | 699 | 65.50 |
| External funding | ||
| No | 493 | 47.70 |
| Yes | 540 | 52.30 |
| Mobility | ||
| Inbreeding pure | 219 | 20.70 |
| Silver-corded | 123 | 11.60 |
| Adherents | 248 | 23.40 |
| Mobile national | 256 | 24.20 |
| Mobile international | 212 | 20.00 |
| Variable | Mean | SD |
| Academic career duration | 24.307 | 13.084 |
| Percentage teaching | 32.184 | 24.978 |
| Percentage research | 41.790 | 25.159 |
| Percentage knowledge transfer | 6.498 | 12.777 |
| Percentage admin | 17.957 | 22.072 |
| Percentage supervision | 22.013 | 22.400 |
Results
Analysis 1—MDRAI-R-S validation
Imputation
To specify the model, the missing data were imputed using a linear regression method (Zhang, 2016). This was required, as the built-in function in AMOS for handling missing data does not permit the computation of modification indices (MI). Following the model specification stage, and once MI estimation was complete, a full-information maximum likelihood (FIML) estimation was applied, as this is considered a superior method for managing missing data (Enders & Bandalos, 2001). This analysis therefore incorporates data for the full working sample (N = 1160).
Model specification
As this instrument has been validated using a global sample, and the factorial structure of the items—with the exception of the new scale—is well documented (Horta & Santos, 2020), the specification strategy was merely to replicate the structure identified in previous studies. The new items were specified as new, independent factors. As expected, since the previous validation exercise already solved all detected issues with the scale, the initial solution was immediately admissible with no required re-specification steps.
The second step in the model specification was locating items with poor loadings, as these are a threat to factorial validity. As expected, no items exhibited factorial loadings under the 0.50 threshold (Kline, 2016; Marôco, 2010), so there were no candidates for removal.
The third step in the model specification was evaluating the MIs. Although the initial model already exhibited good fit (as described below), it was decided that MI evaluation should still be done for the sake of completeness. The MIs were scanned at the 11 threshold, which corresponds to a Type I error probability of 0.001 (Marôco, 2010). Although some proposed covariances met the required threshold, none of them were eligible, as they represented inter-factor covariances or non-valid latent factor covariances (Hair et al., 2014; Kline, 2016). Accordingly, the initial model was also the final one.
Fit evaluation
Following best practices, a range of fit indices were used to assess model fit (Barrett, 2007; Kline, 2016): the X2/df index (Arbuckle, 2007), the comparative-fit index (CFI; Bentler, 1990), its parsimony-adjusted variant, the PCFI (Marôco, 2010), and the root-mean-square error of approximation (RMSEA; Steiger et al., 1985).
After model specification, the model was estimated and the fit was qualitatively assessed as good (X2/df = 2.512; CFI = 0.956; PCFI = 0.799; RMSEA = 0.036). Table 4 compares the fit of the MDRIA-R-S with that of the original instrument; they were very similar, confirming the robustness of the instrument even when applied to a completely independent sample.
Table 4.
Model fit evaluation
| Instrument | X2/df | CFI | PCFI | RMSEA |
|---|---|---|---|---|
| MDRAI-R-S | 2.512 | 0.956 | 0.799 | 0.036 |
| MDRAI-R | N/A | 0.953 | 0.850 | 0.037 |
The original study for the MDRAI-R did not estimate X2/df as the very large sample size precluded its use
CFA
The next step was a CFA of the specified model. Figure 1 illustrates the model, and Table 5 presents the factorial loadings of the various items in our analysis and in the original scale. The loadings were very similar, another indication of the scale’s robustness.
Fig. 1.
Measurement model for the MDRAI-R-S, with standardized regression weights (loadings). Note Ellipses indicate latent variables, and squares indicate manifest variables. Disturbance terms are indicated by the latent variables labeled “e.”
Table 5.
Factorial loadings for the MDRAI-R and the MDRAI-R-S
| Code | Item | Loadings | |
|---|---|---|---|
| R | R-S | ||
| A1 | I aim to one day be one of the most respected experts in my field | 0.802 | 0.886 |
| A2 | Being a highly regarded expert is one of my career goals | 0.802 | 0.885 |
| A3 | I aim to be recognized by my peers | 0.704 | 0.690 |
| A5 | I feel the need to constantly publish new and interesting papers | 0.782 | 0.816 |
| A6 | I am constantly striving to publish new papers | 0.873 | 0.766 |
| DV1 | I look forward to diversifying into other fields | 0.720 | 0.764 |
| DV2 | I would be interested in pursuing research in other fields | 0.781 | 0.861 |
| DV4 | I would like to publish in different fields | 0.737 | 0.819 |
| DV5 | I enjoy multi-disciplinary research more than single-disciplinary research | 0.851 | 0.876 |
| DV6 | Multi-disciplinary research is more interesting than single-disciplinary research | 0.877 | 0.860 |
| COL2 | My publications are enhanced by collaboration with other authors | 0.604 | 0.668 |
| COL5 | I enjoy conducting collaborative research with my peers | 0.734 | 0.835 |
| COL7 | My peers often seek to collaborate with me in their publications | 0.741 | 0.850 |
| COL8 | I am often invited to collaborate with my peers | 0.908 | 0.936 |
| COL12 | I am frequently invited to participate in research collaborations due to my reputation | 0.827 | 0.773 |
| M2 | Part of my work is largely due to my Ph.D. mentor | 0.787 | 0.837 |
| M3 | My research choices are highly influenced by my Ph.D. mentor’s opinion | 0.852 | 0.854 |
| M4 | My Ph.D. mentor is responsible for a large part of my work | 0.892 | 0.823 |
| M6 | My Ph.D. mentor largely determines my research topics | 0.931 | 0.853 |
| TTLF1 | Limited funding does not constrain my choice of topic | 0.822 | 0.624 |
| TTLF3 | The availability of research funding for a certain topic does not influence my decision to conduct research on that topic | 0.696 | 0.693 |
| TTLF4 | I am not discouraged by the lack of funding on a certain topic | 0.616 | 0.716 |
| D3 | I prefer “innovative” research to “safe” research, even when the odds of success are much lower | 0.687 | 0.896 |
| D4 | I would rather engage in new research endeavors, even when success is unlikely, than safe research that contributes little to the field | 0.701 | 0.821 |
| D9 | I am driven by innovative research | 0.678 | 0.667 |
| O1 | My choice of topics is determined by my field community | 0.600 | 0.786 |
| O9 | I often decide my research agenda in collaboration with my field community | 0.803 | 0.755 |
| O6 | I adjust my research agenda based on my institution’s demands | 0.759 | 0.811 |
| O7 | My research agenda is aligned with my institution's research strategies | 0.733 | 0.767 |
| S1 | I decide my research topic based on societal challenges | 0.807 | 0.788 |
| S4 | Societal challenges drive my research choices | 0.904 | 0.742 |
| S2 | I choose my research topics based on my interactions with my non-academic peers | 0.769 | 0.655 |
| S3 | I consider my research topics myself, but this consideration often occurs after I hear what my non-academic peers have to say about these topics | 0.732 | 0.789 |
| S6 | I consider the opinions of my non-academic peers when I choose my research topics | 0.868 | 0.734 |
| G1 | The government supports my research field | - | 0.818 |
| G2 | The government supports academic development in general | - | 0.888 |
| G3 | The government uses incentives to support the development of science and technology | - | 0.806 |
Validity, reliability, and sensitivity
We evaluated MDRAI-R-S’ psychometric properties. All of the calculations were conducted using the Validity Master macro in James Gaskin’s Stats Tool Package (2016). The calculations, referred to throughout this discussion, are shown in Table 6.
Table 6.
Validity and reliability evaluation
| Correlations | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CR | AVE | MSV | ASV | Gov | Acad | Soci | Disc | TTLF | Ment | Coll | Div | Ambition | |
| Government | 0.876 | 0.702 | 0.136 | 0.032 | 0.838 | ||||||||
| Academia Driven | 0.740 | 0.588 | 0.837 | 0.169 | 0.155 | 0.767 | |||||||
| Society Driven | 0.704 | 0.544 | 0.837 | 0.192 | 0.211 | 0.915 | 0.738 | ||||||
| Discovery | 0.841 | 0.641 | 0.209 | 0.071 | 0.050 | 0.120 | 0.346 | 0.800 | |||||
| TTLF | 0.719 | 0.461 | 0.136 | 0.040 | 0.369 | 0.077 | 0.219 | 0.311 | 0.679 | ||||
| Mentor | 0.907 | 0.709 | 0.081 | 0.025 | 0.101 | 0.284 | 0.282 | 0.022 | 0.051 | 0.842 | |||
| Collaboration | 0.786 | 0.657 | 0.496 | 0.145 | 0.064 | 0.433 | 0.346 | 0.289 | 0.144 | 0.038 | 0.810 | ||
| Divergence | 0.800 | 0.667 | 0.496 | 0.152 | 0.018 | 0.365 | 0.509 | 0.457 | 0.090 | 0.145 | 0.704 | 0.817 | |
| Ambition | 0.791 | 0.656 | 0.248 | 0.064 | 0.178 | 0.256 | 0.170 | 0.206 | 0.066 | 0.009 | 0.498 | 0.297 | 0.810 |
The diagonal of the correlation’s matrix indicates the square root of the AVE
We first evaluated the factorial, convergent, and discriminant validity of the dimensions (Anastasi & Urbina, 1997). Factorial validity requires all of the items to have loadings of at least 0.50 (Marôco, 2010). This was verified by the CFA, discussed above, which confirmed the factorial validity.
Second, we evaluated the convergent validity, which occurs when the manifest variables exhibit very high loadings into the respective latent variables. A strict measure of this can be attained through the average variance extracted indicator (AVE; Fornell & Larcker, 1981), which is given by
where j indicates a given factor, i a given item, λ a factorial loading, and ε an error term. As per the Fornell–Larcker criterion, an AVE of more than 0.50 indicates convergent validity. This threshold was fully met for all of the sub-scales, with the exception of TTLF, which had an AVE slightly below the cutoff point (0.461). This may have been caused by the exclusion of one of the items belonging to this sub-scale. This suggests that future revisions should reintroduce the item. Nevertheless, as the AVE for TTLF was only a few decimal points under the threshold, it is likely that this will have little practical impact. Interestingly, there was a similar result for the Discovery sub-scale in the original instrument (Horta & Santos, 2020), which seems to have been resolved in this version. Again, this might be related to the removal of one of the items in the Discovery scale. Accordingly, the permanent removal of that item might be warranted.
The third aspect of validity is discriminant validity, which requires that the various sub-scales do not conceptually overlap—in other words, constructs should have a low degree of inter-factor correlations or cross-loadings. We tested this using the maximum shared variance (MSV), which is the square of the highest of the inter-factorial correlations, and the average shared variance (ASV), which is the average of the sum of squared inter-factorial correlations. To demonstrate discriminant validity, the square root of the AVE must exceed the value of all of the inter-factorial correlations; cumulatively, the AVE for a factor must be greater than that factor’s MSV and ASV. These criteria were met for all factors, with the exception of Academia Driven and Society Driven; these two factors exhibited a correlation of 0.915, substantially higher than that observed in the original scale (0.760). There are two possible explanations for this. First, Academia Driven and Society Driven goals might be strongly aligned in Mexico, causing the scores of these sub-scales to naturally converge. Alternately, this alignment of academic and social goals might not be specific to Mexico, but part of a worldwide trend that has developed since the scale was first validated. Although this is speculative—and we currently have no data to test this—the COVID-19 pandemic, which began after the original validation exercise, might have pushed academic and societal goals closer together, with the result that these two sub-scales are no longer fully differentiated. If such a global trend is confirmed, then these two sub-scales might merge at some point in the future. For this study, the implication was that the scores across these two sub-scales were expected to be very highly correlated.
The next psychometric property to be evaluated was reliability. For this purpose, we computed the composite reliability (CR; Fornell & Larcker, 1981). CR is given by the following formula:
with the same notations as the calculation for AVE. The generally accepted threshold for CR is 0.70 (Hair et al., 2014). All of the dimensions exceeded this threshold, demonstrating the reliability of the MDRAI-R-S.
Finally, we calculated the scale’s sensitivity, which is its ability to differentiate between individuals. This property is demonstrated when each item is sufficiently close to a normal distribution (Marôco, 2010), which is commonly achieved when an item’s skewness and kurtosis are under the absolute value of 3 (Kline, 2016). As can be seen in Table 7, this was the case for all of the items.
Table 7.
Sensitivity analysis
| Item | Min | Max | M | SD | Sk | Ku |
|---|---|---|---|---|---|---|
| A1 | 1 | 7 | 5.510 | 1.360 | − 0.767 | 0.502 |
| A2 | 1 | 7 | 5.590 | 1.334 | − 0.942 | 1.015 |
| A3 | 1 | 7 | 5.000 | 1.353 | − 0.603 | 0.682 |
| A5 | 1 | 7 | 5.570 | 1.313 | − 0.899 | 0.814 |
| A6 | 1 | 7 | 5.840 | 1.189 | − 1.257 | 2.314 |
| A7 | 1 | 7 | 5.830 | 1.252 | − 1.331 | 2.378 |
| DV1 | 1 | 7 | 5.420 | 1.211 | − 0.695 | 0.734 |
| DV2 | 1 | 7 | 5.320 | 1.264 | − 0.586 | 0.307 |
| DV4 | 1 | 7 | 5.110 | 1.313 | − 0.397 | 0.004 |
| DV5 | 1 | 7 | 5.570 | 1.311 | − 0.779 | 0.319 |
| DV6 | 1 | 7 | 5.680 | 1.292 | − 0.787 | 0.214 |
| COL2 | 1 | 7 | 5.640 | 1.185 | − 0.881 | 1.098 |
| COL5 | 1 | 7 | 5.970 | 1.009 | − 0.952 | 1.550 |
| COL7 | 1 | 7 | 4.980 | 1.263 | − 0.593 | 0.571 |
| COL8 | 1 | 7 | 5.150 | 1.260 | − 0.640 | 0.656 |
| COL12 | 1 | 7 | 5.070 | 1.208 | − 0.529 | 0.667 |
| M2 | 1 | 7 | 2.990 | 1.641 | 0.367 | − 0.706 |
| M3 | 1 | 7 | 2.710 | 1.570 | 0.538 | − 0.496 |
| M4 | 1 | 7 | 2.790 | 1.634 | 0.539 | − 0.529 |
| M6 | 1 | 7 | 2.630 | 1.559 | 0.576 | − 0.511 |
| TTLF1 | 1 | 7 | 3.810 | 1.802 | 0.082 | − 0.974 |
| TTLF3 | 1 | 7 | 4.650 | 1.693 | − 0.422 | − 0.606 |
| TTLF4 | 1 | 7 | 4.510 | 1.698 | − 0.341 | − 0.648 |
| D4 | 1 | 7 | 5.050 | 1.392 | − 0.567 | 0.205 |
| D3 | 1 | 7 | 5.170 | 1.329 | − 0.558 | 0.216 |
| D9 | 1 | 7 | 5.460 | 1.164 | − 0.565 | 0.520 |
| O1 | 1 | 7 | 4.020 | 1.492 | − 0.238 | − 0.408 |
| O9 | 1 | 7 | 4.260 | 1.459 | − 0.360 | − 0.204 |
| O6 | 1 | 7 | 4.440 | 1.444 | − 0.410 | − 0.140 |
| O7 | 1 | 7 | 4.860 | 1.366 | − 0.536 | 0.255 |
| S1 | 1 | 7 | 4.540 | 1.557 | − 0.387 | − 0.349 |
| S2 | 1 | 7 | 3.960 | 1.596 | − 0.079 | − 0.547 |
| S3 | 1 | 7 | 3.840 | 1.600 | − 0.064 | − 0.586 |
| S4 | 1 | 7 | 4.630 | 1.576 | − 0.401 | − 0.346 |
| S6 | 1 | 7 | 4.040 | 1.535 | − 0.205 | − 0.372 |
| G1 | 1 | 7 | 3.800 | 1.590 | − 0.139 | − 0.681 |
| G2 | 1 | 7 | 3.770 | 1.581 | − 0.080 | − 0.692 |
| G3 | 1 | 7 | 4.240 | 1.622 | − 0.363 | − 0.581 |
Analysis 2—Cluster analysis
Three variables were used as predictors for clustering—the share of time dedicated to basic research, to applied research, and to experimental development. One hundred and twenty seven participants skipped this section of the survey and as such were not eligible for data imputation. The working sample for this analysis was therefore lower (N = 1033). This analysis yielded a three− cluster solution with a good fit: 0.5 on the silhouette measure of cohesion and separation (Kaufman & Rousseeuw, 2009; Rousseeuw, 1987). Table 8 describes the characteristics of these clusters based on the predictor variables.
Table 8.
Mean share of time per activity for each cluster
| Variable | Applied Researchers (N = 433; 41.9%) |
Basic Researchers (N = 371; 35.9%) |
Balanced Researchers (N = 229; 22.2%) |
|---|---|---|---|
| % Basic research | 18.95% | 84.84% | 36.83% |
| % Applied research | 70.94% | 11.77% | 41.55% |
| % Development | 6.38% | 4.29% | 46.29% |
The first cluster, “Applied Researchers,” consisted of academics who allocated most of their time to applied research. They also allocated a reasonable amount of time to basic research, but very little time to experimental development. The second cluster, “Basic Researchers,” showed the opposite pattern, with a large share of time dedicated to basic research, a fraction dedicated to applied research, and a very small amount to experimental development. It is noteworthy that the proportion of basic research in the Basic Researchers’ cluster was substantially higher than the proportion of time allocated to applied research in the Applied Researchers’ cluster, suggesting that the applied researchers were more open to research focus complementarity and less specialized than the basic researchers. Finally, the last cluster, “Balanced Researchers,” distributed their time somewhat equitably across all three research focuses; these academics match the definition of ambidextrous scholars in Werker and Hopp’s (2020) paper. Having classified the academics into these three clusters, the second step of the cluster analysis was to determine whether the SRAs varied between clusters. For this purpose, we computed the average scores of the items for each dimension in each cluster (DiStefano et al., 2009). Additionally, an analysis of variance (ANOVA) was conducted to identify which SRA dimensions differed significantly across clusters (Table 9), with the goal of understanding how SRA are associated with their preferences for the different types of research. Tukey’s HSD post− hoc tests (Tukey, 1953) were used to triangulate specific pairs with differences.
Table 9.
SRA dimensions for each cluster
| Dimension | Basic | Applied | Balanced |
F (2, 1030) |
|||
|---|---|---|---|---|---|---|---|
| M | SD | M | SD | M | SD | ||
| Scientific Ambition | 5.580 | 0.992 | 5.586 | 1.001 | 5.550 | 0.924 | 0.111 |
| Divergence | 5.275 | 1.035 | 5.458 | 1.031 | 5.704 | 0.903 | 12.920*** |
| Collaboration | 5.291 | 0.979 | 5.440 | 0.898 | 5.435 | 0.865 | 3.053* |
| Mentor Influence | 2.569 | 1.374 | 2.849 | 1.445 | 2.962 | 1.405 | 6.569** |
| TTLF | 4.364 | 1.451 | 4.399 | 1.296 | 4.177 | 1.389 | 2.065 |
| Discovery | 5.109 | 1.221 | 5.221 | 1.101 | 5.478 | 0.979 | 7.740*** |
| Academia Driven | 4.141 | 1.174 | 4.537 | 1.028 | 4.535 | 1.017 | 15.960*** |
| Society Driven | 3.739 | 1.184 | 4.517 | 1.018 | 4.365 | 1.018 | 55.030*** |
| Government | 4.026 | 1.329 | 4.014 | 1.478 | 3.807 | 1.487 | 0.139 |
***p < 0.001; **p < 0.01; *p < 0.05
Significant differences were shown across clusters for Divergence (F(2, 1,030) = 12.920, p < 0.001), with the Basic cluster having the lowest scores, followed by Applied and Balanced. Collaboration exhibited significant differences (F(2, 1,030) = 3.053, p < 0.05) in the omnibus ANOVA test, but the post− hoc test failed to identify specific pairs with significantly different scores. As such, the evidence for differences in Collaboration was inconclusive. Mentor Influence exhibited significant differences across clusters (F(2, 1,030) = 6.569, p < 0.01), with the Basic cluster having lower scores than the Applied and Balanced clusters, which did not differ from each other. Discovery also exhibited significant differences across clusters (F(2, 1,030) = 7.740, p < 0.001). Again, the Basic cluster had the lowest scores for Discovery, followed by Applied and then Balanced. Academia Driven (F(2, 1,030) = 15.960, p < 0.001) and Society Driven (F(2, 1,030) = 55.030, p < 0.001) also differed significantly across clusters, following the same pattern: Basic had the lowest scores, Applied had mid− level scores, and Balanced had the highest scores. Figure 2 shows the significant differences for specific pairs.
Fig. 2.
Cluster comparison, with Tukey’s HSD post− hoc comparisons (Tukey, 1953). ****p < 0.0001; ***p < 0.001; ** p < 0.01; *p < 0.05
Analysis 3—Multinominal regression
In our final analysis, we conducted a multinominal regression on the clustering membership variable, using the full suite of SRA dimension scores and several control variables. Three hundred and sixty one of the participants skipped the survey questions on career data, which were required to produce the control variables; they were therefore excluded, which reduced the working sample for this analysis to 799 participants. In this regression, the “Balanced” cluster was used as the baseline. The results are shown in Table 10.
Table 10.
Multinominal regression of clusters on agendas and controls
| Variables | Basic | Applied |
|---|---|---|
| Gender (male) | 0.195 | 0.193 |
| (0.218) | (0.208) | |
| FOS (Engineering & Technology) | 0.587 | 0.432 |
| (0.812) | (0.695) | |
| FOS (Humanities) | 0.414 | 0.812 |
| (0.940) | (0.810) | |
| FOS (Medical & Health Sciences) | 0.514 | 0.430 |
| (0.849) | (0.734) | |
| FOS (Natural Sciences) | 1.002 | 0.915 |
| (0.824) | (0.713) | |
| FOS (Social Sciences) | 0.903 | 1.066 |
| (0.823) | (0.708) | |
| External funding (yes) | − 0.216 | 0.045 |
| (0.220) | (0.208) | |
| Non− academic experience (yes) | − 0.014 | 0.887*** |
| (0.217) | (0.221) | |
| Mobility (silver− corded) | 0.918** | 0.484 |
| (0.377) | (0.364) | |
| Mobility (adherents) | 0.307 | 0.067 |
| (0.310) | (0.291) | |
| Mobility (mobile national) | 0.488 | 0.239 |
| (0.313) | (0.289) | |
| Mobility (mobile international) | 0.679** | 0.422 |
| (0.334) | (0.314) | |
| Full professor | 0.498* | 0.286 |
| (0.267) | (0.246) | |
| Ambition | 0.110 | 0.152 |
| (0.114) | (0.111) | |
| Divergence | − 0.164 | − 0.318** |
| (0.125) | (0.124) | |
| Collaboration | 0.027 | 0.102 |
| (0.133) | (0.134) | |
| Mentor influence | − 0.131* | − 0.108 |
| (0.076) | (0.073) | |
| TTLF | 0.072 | 0.075 |
| (0.084) | (0.080) | |
| Discovery | − 0.273** | − 0.395*** |
| (0.106) | (0.107) | |
| Academia driven | − 0.079 | − 0.155 |
| (0.117) | (0.115) | |
| Society driven | − 0.307*** | 0.404*** |
| (0.117) | (0.119) | |
| Government support | 0.116 | 0.102 |
| (0.077) | (0.073) | |
| Academic career duration | 0.003 | 0.012 |
| (0.009) | (0.008) | |
| Percentage teaching | − 0.004 | − 0.003 |
| (0.006) | (0.005) | |
| Percentage research | 0.007 | 0.000 |
| (0.005) | (0.005) | |
| Percentage knowledge transfer | − 0.068*** | − 0.010 |
| (0.016) | (0.007) | |
| Percentage admin | − 0.005 | 0.001 |
| (0.006) | (0.006) | |
| Percentage supervision | 0.002 | − 0.005 |
| (0.007) | (0.007) | |
| Observations | 799 | 799 |
***p < 0.01; **p < 0.05; *p < 0.1. Standard errors in parentheses
Among the SRA variables, a high Discovery score reduced the odds of placement4 in both the Basic (B = − 0.273, p < 0.05, OR 0.761) and Applied (B = − 395, p < 0.01, OR 0.673) clusters. Divergence only reduced the odds of placement in the Applied cluster (B = − 0.318, p < 0.05, OR 0.728), whereas higher Mentor Influence scores reduced the odds of being in the Basic cluster (B = − 0.131, p < 0.1, OR 0.877). Finally, high Society Driven scores led to a reduced propensity for Basic research (B = − 0.307, p < 0.01, OR 0.735) and a greater likelihood of belonging to the Applied cluster (B = 0.404, p < 0.01, OR 1.497). The main findings regarding SRA can be better visualized through a forest plot, shown as Fig. 3.
Fig. 3.
Forest plot of odds ratio for the various SRA variables
In terms of the control variables, most were not statistically significant. However, having non-academic experience increased the likelihood of belonging to the Applied cluster, relative to the Balanced cluster (B = 0.887, p < 0.01, OR 2.427). Being silver-corded rather than non-mobile increased the odds of membership in the Basic cluster relative to the Balanced cluster (B = 0.918, p < 0.05, OR 2.504). This was also the case for being a Mobile International (B = 0.679, p < 0.05, OR 1.972), but none of the other mobility types had a significant impact on placement in the Applied cluster. Finally, an increased percentage of time dedicated to Knowledge Transfer reduced the odds of placement in the Basic cluster (B = − 0.068, p < 0.01, OR 0.934). Contrary to the literature, we found that full professors were more engaged in basic research than associate and assistant professors (B = − 0.498, p < 0.1, OR 0.608), and there were no statistical differences between genders, recipients of external funding, or between fields of science, suggesting that the research dynamics of academics in developing scientific systems may be quite distinct from those in developed scientific systems.
Conclusion
This paper’s results regarding research preference clustering are very similar to those of Werker and Hopp (2020). Only a relatively small number of academics can synergistically pursue basic research, applied research, and experimental development. This group of academics was the smallest of the three research preference clusters in our sample. The other two clusters, although showing marginal levels of complementarity, were dominated by a single research preference, either basic or applied research. This is somewhat at odds with the findings of Gulbrandsen and Kyvik (2010) and Bentley et al. (2015), as our findings suggest relatively strong research specialization, as evidenced by low levels of complementarity and research focuses that are moderately dominated by a single preference (e.g., basic research). Nonetheless, similarly to Gulbrandsen and Kyvik (2010) and Bentley et al. (2015), we found that the number of academics who prefer applied research exceeds those preferring basic research and that external funding and national and institutional strategies had little or no effect on academics’ research focuses. Our findings may differ from the papers mentioned above for two reasons. First, those papers are not recent, and academia has recently endured substantial pressure that has transformed the way academics conceptualize research and how they act when doing research. Second, it is possible that in countries with developing scientific and academic systems, stronger specializations in basic and applied research may still exist either because academic knowledge production is still dominated by traditional, disciplinary, and hierarchical modes of knowledge production or because there are fewer opportunities for triple, quadruple, or quintuple helixes in the academic sector (Jaramillo et al., 2016).
Responding to the second research question driving this study, we demonstrated that four of academics’ SRAs were moderately associated with their individual preferences for basic research, applied research, or experimental development. Notably, high Discovery scores were associated with a lower preference for basic and applied research; higher Divergence scores were associated with a lower preference for applied research; higher Mentor Influence was associated with a lower preference for basic research; and higher Society Driven scores were associated with a lower preference for basic research but a higher preference for applied research.
Regarding the research questions driving this study, we made two other major findings.
The first important finding is related to the relationships between the individual SRA dimensions and research focus preferences. In particular, we find that academics of the balanced cluster have high scores on the Discovery dimension. This suggests that academics interested in research that has the potential to result in breakthroughs generally combine the three types of research. This may be because combining focuses results in a complex articulation of ideas, research approaches, and uses for the knowledge they acquire, leading to the creation of new knowledge, products, and services with the potential for added value. However, it may also relate to the high stakes, high risks, and high costs of the development of products and services that is typical of experimental development. Since the Divergence scores (i.e., multidisciplinarity) for academics in the balanced cluster are not statistically different from those in the basic research cluster,5 the latter explanation may be the most likely. However, there are no statistical differences between fields of science, suggesting that the higher Discovery score of academics who adopt a balanced research focus does not seem to have more to do with the riskiness of experimental development. Experimental development can be found in all fields of science, although it is riskier and costlier in some than others (see Olmos-Peñuela et al., 2014; Sandoz, 2021). This is a finding which explanation is hard to pinpoint and requires further research. The fact that there are no statistical differences in the research preferences of academics in different fields of science is also important per se; although some fields of science might be expected to be more applied than others (see Gulbrandsen & Kyvik, 2010), this does not seem to influence the research focus preferences of the academics in our sample. The same is true of the findings concerning gender: we do not find different research focus preferences between male and female academics, which is inconsistent with other studies indicating that male academics lean toward the basic sciences and female academics lean toward the applied sciences (Zhang et al., 2021). Furthermore, full professors in Mexican research universities do not seem to lean toward applied research, as other studies have found (e.g., Gulbrandsen & Smeby, 2005); instead, they prefer to focus on basic research. The explanation for the inconsistency of these findings when compared to the literature seems to be related to differences between academics working in developed and developing countries and are relevant for policy purposes, underlining the relevance of understanding national and developmental characteristics and dynamics. Our finding that academics who have worked outside academia tend to have a more applied research profile is consistent with the literature.
Our second main finding is related to the additional control variables, which have not been tested previously. Most of them have little effect on the research preferences of academics. Career mobility has limited effect on the research preferences of academics: academics who are currently working in the university where they obtained their Ph.D. after having worked somewhere else and academics with work experiences abroad are more inclined to prefer basic research to a more balanced approach than academics in the career immobile group. Work allocation also has a small impact on the research preferences of academics: academics dedicating more time to knowledge transfer activities are less likely to engage in basic research, which is consistent with the literature (e.g., Gulbrandsen & Thune, 2017). The number of years in academia has no influence on research preferences.
In addition to these findings, we test and validate the MDRAI-R in a new setting. We demonstrate strong psychometric properties, consistent with previous validation exercises. We also introduce a new dimension (i.e., Government), transforming the MDRAI-R into the MDRAI-R-S, which is a more optimized instrument, now available in both English (Appendix 1) and Spanish (Appendix 2). This will allow researchers to use the instrument in Spanish-speaking countries, particularly in Latin America, where it can be of important practical use for policymakers, research managers, academics, and researchers in or outside of academia.
This study has certain limitations. Two issues typically arise from non-probabilistic sampling: undercoverage, which occurs when members of the population have a zero chance of being selected, and the inability to accurately calculate the probability of a given member of the population being selected for the sample (Hirschauer et al., 2020). Undercoverage was not an issue because the entire population of interest was contacted. The second problem was not initially an issue because each member of the population had an ex-ante equal probability of being part of the sample: 100%. However, any response rate that falls short of 100% leads to the possibility of self-selection bias. Although we compared the sample to the population distribution of institutions, and it was nearly identical, potential confounding factors that could lead to self-exclusion from the survey, such as gender, age, or other socio-demographical characteristics, were not addressed. The literature has acknowledged the impossibility of accounting for all the potential confounding factors that can lead to self-exclusion (Hirschauer et al., 2020), and as such, while there is evidence in favor of the sample’s representativeness at least as far as the population’s institutions are concerned, the reader should be aware of the non-probabilistic nature of the sample when evaluating our findings. Additionally, this study focused specifically on Mexican institutions, and the findings may not be generalizable to other populations.
Appendix 1
Multi-Dimensional Research Agendas Inventory – Revised-S (MDRAI-R-S)
You will be asked a series of questions regarding your motivations and goals as an academic. Please read and determine your level of agreement with each statement. Then, check one of the seven boxes next to the corresponding item. If you do not know or a particular sentence does not apply to you, check the N/A box.
Some questions will ask about your field, and others will ask about your research topics. Please consider “field” to be the main theme of your research (for example, "higher education"), and “research topic” as a specific subject within the main theme (e.g., "doctoral education" and “access to higher education” would be research topics in the "higher education" theme). “Field community” is also a term that you will encounter while you complete the survey. “Field community” is defined as the research/scholarly community(ies) with which you identify. Keep these definitions in mind when you respond to the questions.
There are no right or wrong answers. Please read each statement and check the box that best applies to you. How much do you agree with the following statements?
| Completely disagree | Strongly disagree | Disagree | Neither agree nor disagree | Agree | Strongly agree | Completely agree | N/A | ||
|---|---|---|---|---|---|---|---|---|---|
| A1 | I aim to one day be one of the most respected experts in my field | ||||||||
| A2 | Being a highly regarded expert is one of my career goals | ||||||||
| A3 | I aim to be recognized by my peers | ||||||||
| A5 | I feel the need to constantly publish new and interesting papers | ||||||||
| A6 | I am constantly striving to publish new papers | ||||||||
| DV1 | I look forward to diversifying into other fields | ||||||||
| DV2 | I would be interested in pursuing research in other fields | ||||||||
| DV4 | I would like to publish in different fields | ||||||||
| DV5 | I enjoy multi-disciplinary research more than single-disciplinary research | ||||||||
| DV6 | Multi-disciplinary research is more interesting than single-disciplinary research | ||||||||
| COL2 | My publications are enhanced by collaboration with other authors | ||||||||
| COL5 | I enjoy conducting collaborative research with my peers | ||||||||
| COL7 | My peers often seek to collaborate with me in their publications | ||||||||
| COL8 | I am often invited to collaborate with my peers | ||||||||
| COL12 | I am frequently invited to participate in research collaborations due to my reputation | ||||||||
| M2 | Part of my work is largely due to my PhD mentor | ||||||||
| M3 | My research choices are highly influenced by my PhD mentor’s opinion | ||||||||
| M4 | My PhD mentor is responsible for a large part of my work | ||||||||
| M6 | My PhD mentor largely determines my research topics | ||||||||
| TTLF1 | Limited funding does not constrain my choice of topic | ||||||||
| TTLF3 | The availability of research funding for a certain topic does not influence my decision to conduct research on that topic | ||||||||
| TTLF4 | I am not discouraged by the lack of funding on a certain topic | ||||||||
| D3 | I prefer "innovative" research to “safe” research, even when the odds of success are much lower | ||||||||
| D4 | I would rather engage in new research endeavors, even when success is unlikely, than safe research that contributes little to the field | ||||||||
| D9 | I am driven by innovative research | ||||||||
| O1 | My choice of topics is determined by my field community | ||||||||
| O9 | I often decide my research agenda in collaboration with my field community | ||||||||
| O6 | I adjust my research agenda based on my institution’s demands | ||||||||
| O7 | My research agenda is aligned with my institution's research strategies | ||||||||
| S1 | I decide my research topic based on societal challenges | ||||||||
| S4 | Societal challenges drive my research choices | ||||||||
| S2 | I choose my research topics based on my interactions with my non-academic peers | ||||||||
| S3 | I consider my research topics myself, but this consideration often occurs after I hear what my non-academic peers have to say about these topics | ||||||||
| S6 | I consider the opinions of my non-academic peers when I choose my research topics | ||||||||
| G1 | The government supports my research field | ||||||||
| G2 | The government supports academic development in general | ||||||||
| G3 | The government supports with incentives to develop science and technology |
Appendix 2
Multi-Dimensional Research Agendas Inventory – Revised-S (MDRAI-R-S)
A continuación se le harán una serie de preguntas acerca de sus motivaciones y metas como académico.
Por favor haga click en solo una de las siete opciones a la derecha de cada planteamiento dependiendo de su nivel de acuerdo o desacuerdo. siendo por ejemplo el número 1 el nivel de mayor desacuerdo, así como el número 7 el número de mayor acuerdo para cada enunciado.
Algunas preguntas se harán de acuerdo a su campo de trabajo y otra respecto a su campo de investigación.
Favor de considerar “campo de trabajo” como el tema principal de su investigación (por ejemplo, “educación superior”) y su “campo de investigación” como un área específica de su tema principal (e.g. “educación doctoral” o “acceso a la educación superior”, serán considerados como temas de investigación entre los temas a abordar dentro del área de “educación superior”), “comunidad de investigación” es un término que se encontrará durante el transcurso de esta encuesta. “Comunidad de investigación” es definida como la comunidad escolar o académica con la cual usted se identifica. Es importante tener estas definiciones en mente al responder a las preguntas que enseguida se muestran.
| Completamente en desacuerdo | Fuertemente en desacuerdo | En desacuerdo | Opinión neutral | De acuerdo | Fuertemente de acuerdo | Completamente de acuerdo | N/A | ||
|---|---|---|---|---|---|---|---|---|---|
| A1 | Quisiera llegar a ser uno de los investigadores más respetados en mi área | ||||||||
| A2 | Una de mis metas es llegar a ser reconocido como experto en mi área de conocimiento | ||||||||
| A3 | Deseo obtener el reconocimiento de mis colegas | ||||||||
| A5 | Siento la necesidad de publicar constantemente artículos nuevos e interesantes | ||||||||
| A6 | Dedico gran esfuerzo a publicar nuevos artículos. Me motiva el publicar artículos | ||||||||
| DV1 | Busco diversificar mi área de investigación hacia otros temas | ||||||||
| DV2 | Estaría interesado en realizar investigación en otras áreas | ||||||||
| DV4 | Me gustaría publicar en diferentes campos de investigación | ||||||||
| DV5 | Disfruto más investigación multi-disciplinaria que investigación en una sola disciplina | ||||||||
| DV6 | La investigación multidisciplinaria me parece más interesante que la investigación mono-disciplinaria | ||||||||
| COL2 | Mis publicaciones tienden a mejorar cuando hago colaboraciones con otros autores | ||||||||
| COL5 | Me complace hacer colaboraciones de investigación con mis colegas | ||||||||
| COL7 | Mis colegas frecuentemente me invitan a colaborar en sus publicaciones | ||||||||
| COL8 | Frecuentemente soy invitado a colaborar con mis colegas | ||||||||
| COL12 | Debido a mi reputación frecuentemente me invitan a colaborar en proyectos o programas de investigación | ||||||||
| M2 |
Parte de mis proyectos de trabajo es influenciada en gran medida por mi mentor del doctorado |
||||||||
| M3 |
La elección de la temática de mi investigación se basa en las recomendaciones de mi mentor de doctorado |
||||||||
| M4 | Mi mentor del doctorado es responsable de una gran parte de mi trabajo | ||||||||
| M6 | Mi mentor del doctorado determina en gran medida la temática de mi investigación | ||||||||
| TTLF1 | Mis temas de investigación no se ven afectados por la falta de fondos | ||||||||
| TTLF3 |
La disponibilidad de fondos para investigación hacia ciertos temas no afecta mi decisión sobre la temática a investigar |
||||||||
| TTLF4 | La falta de fondos para ciertos temas de investigación no es algo que me desaliente | ||||||||
| D3 | Prefiero hacer investigación “innovadora” que investigación “tradicional” aun y cuando las posibilidades de éxito sean mucho menores | ||||||||
| D4 | Prefiero hacer investigación innovadora, aunque tenga bajas posibilidades de éxito que hacer investigación tradicional la cual aporta poco a mi área | ||||||||
| D9 | Mi motivación está orientada hacia la Investigación innovadora | ||||||||
| O1 | La temática de mi investigación es determinada en acuerdo con los miembros de mi comunidad científica | ||||||||
| O9 | Usualmente decido mi agenda de investigación en acuerdo con mi comunidad científica | ||||||||
| O6 | Adapto mi agenda de investigación a las necesidades de mi institución | ||||||||
| O7 | Mi agenda de investigación está alineada con los objetivos estratégicos de mi institución | ||||||||
| S1 | Decido mi temática de investigación en base a los desafíos sociales nacionales o internacionales. (alineados con los de mi gobierno) | ||||||||
| S4 | Los retos sociales definen mi temática de investigación | ||||||||
| S2 | Yo elijo la temática de mi investigación en base a la interacción con mis colegas no académicos | ||||||||
| S3 | Yo defino mi temática de investigación después de escuchar la opinión de mis colegas ajenos al área académica | ||||||||
| S6 | Valoro mucho las opiniones de mis colegas ajenos a la academia para elegir mi temática de investigación | ||||||||
| G1 | El gobierno apoya mi área de investigación | ||||||||
| G2 | El gobierno apoya el desarrollo académico en general | ||||||||
| G3 | El gobierno apoya con incentivos a académicos que desarrollan la ciencia y la tecnología |
Funding
Funding was provided by CONACYT (Grant No. 2018-000070-02EXTF-00077).
Footnotes
In some countries, such as China, the opposite trend is observed, with governments attempting to shift the focus from applied to basic research, although an overwhelming amount of government funding is still directed to the natural sciences and engineering (Huang et al., 2014).
SRAs are personal choices that result from a combination of factors related to individual and social goals and interests, influenced by scholarly communities and others, as well as by other considerations, including career perspectives and institutional pressures that are bound to influence topical research choices and engagement (Santos et al., 2020).
For Field of Science classification, the participants were manually classified by the authors using the OECD’s Frascati Manual classification scheme (OECD, 2015), under one of its six categories: Natural Sciences, Engineering & Technology, Medical & Health Sciences, Agricultural Sciences, Social Sciences, and Humanities.
The Odds Ratios are reported throughout this section as “OR”.
Some of the SRA variables that were shown to vary between clusters in the ANOVA analysis (Analyses 3), such as Divergence, Mentor Influence, and Academia Driven, lost statistical significance after the introduction of control variables in the multinominal regression (Table 4). Although Divergence and Mentor Influence retained statistical significance for some pairs, Academia Driven became completely insignificant.
References
- Abreu M, Grinevich V. Gender patterns in academic entrepreneurship. The Journal of Technology Transfer. 2016;42:763–794. doi: 10.1007/s10961-016-9543-y. [DOI] [Google Scholar]
- Almeida JAS, Pais AACC, Formosinho SJ. Science indicators and science patterns in Europe. Journal of Informetrics. 2009;3(2):134–142. doi: 10.1016/j.joi.2009.01.001. [DOI] [Google Scholar]
- Anastasi A, Urbina S. Psychological testing. Prentice Hall/Pearson Education; 1997. [Google Scholar]
- Arbuckle, J. (2007). Amos 16.0 user’s guide. SPSS, Chicago, IL.
- Barrett P. Structural equation modelling: Adjudging model fit. Personality and Individual Differences. 2007;42(5):815–824. doi: 10.1016/j.paid.2006.09.018. [DOI] [Google Scholar]
- Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
- Bentley PJ, Gulbrandsen M, Kyvik S. The relationship between basic and applied research in universities. Higher Education. 2015;70(4):689–709. doi: 10.1007/s10734-015-9861-2. [DOI] [Google Scholar]
- DiStefano C, Zhu M, Mindrila D. Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation. 2009;14(20):1–11. [Google Scholar]
- Enders CK, Bandalos DL. The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling. 2001;8(3):430–457. doi: 10.1207/S15328007SEM0803_5. [DOI] [Google Scholar]
- Fan X, Yang X, Yu Z. Effect of basic research and applied research on the universities’ innovation capabilities: The moderating role of private research funding. Scientometrics. 2021;126:5387–5411. doi: 10.1007/s11192-021-03998-9. [DOI] [Google Scholar]
- Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research. 1981;18:39–50. doi: 10.1177/002224378101800104. [DOI] [Google Scholar]
- Gaskin, J. (2016). Stats Tool Package. http://statwiki.kolobkreations.com
- Gläser J, Laudel G. Governing science: How science policy shapes research content. European Journal of Sociology. 2016;57(1):117–168. doi: 10.1017/S0003975616000047. [DOI] [Google Scholar]
- Godin B. Outline for a history of science measurement. Science, Technology, and Human Values. 2002;27(1):3–27. doi: 10.1177/016224390202700101. [DOI] [Google Scholar]
- Godin B. Research and development: How the ‘D’ got into R&D. Science and Public Policy. 2006;33(1):59–76. doi: 10.3152/147154306781779190. [DOI] [Google Scholar]
- Gulbrandsen M, Kyvik S. Are the concepts of basic research, applied research and experimental development still useful? An empirical investigation among Norwegian academics. Science and Public Policy. 2010;37(5):343–353. doi: 10.3152/030234210X501171. [DOI] [Google Scholar]
- Gulbrandsen M, Smeby J-C. Industry funding and university professors’ research performance. Research Policy. 2005;34(6):932–950. doi: 10.1016/j.respol.2005.05.004. [DOI] [Google Scholar]
- Gulbrandsen M, Thune T. The effects of non-academic work experience on external interaction and research performance. The Journal of Technology Transfer. 2017;42:795–813. doi: 10.1007/s10961-017-9556-1. [DOI] [Google Scholar]
- Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. Pearson Education Limited; 2014. [Google Scholar]
- Hirschauer N, Grüner S, Mußhoff O, Becker C, Jantsch A. Can p-values be meaningfully interpreted without random sampling? Statistics Surveys. 2020;14:71–91. doi: 10.1214/20-SS129. [DOI] [Google Scholar]
- Hirzel S, Hettesheimer T, Viebahn P, Fischedick M. A decision support system for public funding of experimental development in energy research. Energies. 2018;11(6):1357. doi: 10.3390/en11061357. [DOI] [Google Scholar]
- Horta H, Santos JM. The multidimensional research agendas inventory-revised (MDRAI-R): Factors shaping researchers’ research agendas in all fields of knowledge. Quantitative Science Studies. 2020;1(1):60–93. doi: 10.1162/qss_a_00017. [DOI] [Google Scholar]
- Horta H, Feng S, Santos JM. Homophily in higher education research: A perspective based on co-authorships. Scientometrics. 2021 doi: 10.1007/s11192-021-04227-z. [DOI] [Google Scholar]
- Horta H, Veloso FM, Grediaga R. Navel gazing: Academic inbreeding and scientific productivity. Management Science. 2010;56(3):414–429. doi: 10.1287/mnsc.1090.1109. [DOI] [Google Scholar]
- Huang C, Su J, Xie X, Li J. Basic research is overshadowed by applied research in China: A policy perspective. Scientometrics. 2014;99:689–694. doi: 10.1007/s11192-013-1199-x. [DOI] [Google Scholar]
- Jaramillo P, Estefania I, Güemes Castorena D. Identification of key factors of academia in the process of linking in the triple helix of innovation model in Mexico: A state of the art matrix. Nova Scientia. 2016;8(16):246–277. doi: 10.21640/ns.v8i16.354. [DOI] [Google Scholar]
- Jongbloed B, Enders J, Salerno C. Higher education and its communities: Interconnections, interdependencies and a research agenda. Higher Education. 2008;56:303–324. doi: 10.1007/s10734-008-9128-2. [DOI] [Google Scholar]
- Kaufman L, Rousseeuw PJ. Finding groups in data: An introduction to cluster analysis. Wiley; 2009. [Google Scholar]
- Kline RB. Principles and practice of structural equation modeling. Guilford Press; 2016. [Google Scholar]
- Kowalczewska K, Behagel J. How policymakers’ demands for usable knowledge shape science-policy relations in environmental policy in Poland. Science and Public Policy. 2019;46(3):381–390. doi: 10.1093/scipol/scy065. [DOI] [Google Scholar]
- Larivière V, Macaluso B, Mongeon P, Siler K, Sugimoto CR. Vanishing industries and the rising monopoly of universities in published research. PLoS ONE. 2018;13(8):e0202120. doi: 10.1371/journal.pone.0202120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mallon M, Duberley J, Cohen L. Careers in public sector science: Orientations and implications. R&D Management. 2005;35:395–407. doi: 10.1111/j.1467-9310.2005.00397.x. [DOI] [Google Scholar]
- Marôco J. Análise estatística: Com utilização do SPSS. Edições Sílabo; 2003. [Google Scholar]
- Marôco, J. (2010). Análise de equações estruturais: Fundamentos teóricos, software & aplicações. ReportNumber, Lda.
- Marques M, Powell JJW, Zapp M, Biesta G. How does research evaluation impact educational research? Exploring intended and unintended consequences of research assessment in the United Kingdom, 1986–2014. European Educational Research Journal. 2017;6(6):820–842. doi: 10.1177/1474904117730159. [DOI] [Google Scholar]
- Mittermeir R, Knorr KD. Scientific productivity and accumulative advantage: A thesis reassessed in the light of international data. R&D Management. 1979;9:235–239. doi: 10.1111/j.1467-9310.1979.tb01302.x. [DOI] [Google Scholar]
- Norusis MJ. IBM SPSS statistics 19 statistical procedures companion. Prentice Hall; 2012. [Google Scholar]
- OECD . Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development. OECD Publishing; 2015. [Google Scholar]
- Olmos-Peñuela J, Benneworth P, Castro-Martinez E. Are sciences essential and humanities elective? Disentangling competing claims for humanities’ research public value. Arts and Humanities in Higher Education. 2014;14(1):61–78. doi: 10.1177/1474022214534081. [DOI] [Google Scholar]
- Overland I, Sovacool BK. The misallocation of climate research funding. Energy Research & Social Science. 2020;62:101349. doi: 10.1016/j.erss.2019.101349. [DOI] [Google Scholar]
- Ranga L, Debackere K, Tunzelmann N. Entrepreneurial universities and the dynamics of academic knowledge production: A case study of basic vs. applied research in Belgium. Scientometrics. 2003;58:301–320. doi: 10.1023/A:1026288611013. [DOI] [Google Scholar]
- Raynor K. Participatory action research and early career researchers: The structural barriers to engagement and why we should do it anyway. Planning Theory & Practice. 2019;20(1):130–136. doi: 10.1080/14649357.2018.1556501. [DOI] [Google Scholar]
- Roll-Hansen N. A historical perspective on the distinction between basic and applied science. Journal of General Philosophy of Science. 2017;48:535–551. doi: 10.1007/s10838-017-9362-3. [DOI] [Google Scholar]
- Rousseeuw PJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics. 1987;20:53–65. doi: 10.1016/0377-0427(87)90125-7. [DOI] [Google Scholar]
- Salter AJ, Martin BR. The economic benefits of publicly funded basic research: A critical review. Research Policy. 2001;30(3):509–532. doi: 10.1016/S0048-7333(00)00091-3. [DOI] [Google Scholar]
- Sandoz R. Thematic reclassifications and emerging sciences. Journal for General Philosophy of Science. 2021;52:63–85. doi: 10.1007/s10838-020-09526-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Santos JM, Horta H. The generational gap of science: A dynamic cluster analysis of doctorates in an evolving scientific system. Scientometrics. 2015;104(1):381–406. doi: 10.1007/s11192-015-1558-x. [DOI] [Google Scholar]
- Santos JM, Horta H. The research agenda setting of higher education researchers. Higher Education. 2018;76:649–668. doi: 10.1007/s10734-018-0230-9. [DOI] [Google Scholar]
- Santos JM, Horta H. The association between researchers’ conceptions of research and their strategic research agendas. Journal of Data and Information Science. 2020;5(4):56–74. doi: 10.2478/jdis-2020-0032. [DOI] [Google Scholar]
- Santos JM, Horta H, Amâncio L. Research agendas of female and male academics: A new perspective on gender disparities in academia. Gender and Education. 2021;33(5):625–643. doi: 10.1080/09540253.2020.1792844. [DOI] [Google Scholar]
- Santos JM, Horta H, Li H. Are the strategic research agendas of researchers in the social sciences determinants of research productivity? Scientometrics. 2022 doi: 10.1007/s11192-022-04324-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Santos JM, Horta H, Zhang L-F. The association of thinking styles with research agendas among academics in the social sciences. Higher Education Quarterly. 2020;74(2):193–210. doi: 10.1111/hequ.12240. [DOI] [Google Scholar]
- Sapir A. Protecting the purity of pure research: Organizational boundary work at an institute of basic research. Minerva. 2017;55:65–91. doi: 10.1007/s11024-016-9309-6. [DOI] [Google Scholar]
- Schauz D. What is basic research? Insights from historical semantics. Minerva. 2014;52:273–328. doi: 10.1007/s11024-014-9255-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steiger JH, Shapiro A, Browne MW. On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika. 1985;50(3):253–263. doi: 10.1007/BF02294104. [DOI] [Google Scholar]
- Tukey J. Multiple comparisons. Journal of the American Statistical Association. 1953;48(263):624–625. [Google Scholar]
- Valentin F, Norn MT, Alkaersig L. Orientations and outcome of interdisciplinary research: The case of research behavior in translational medical science. Scientometrics. 2016;106:67–90. doi: 10.1007/s11192-015-1784-2. [DOI] [Google Scholar]
- Werker C, Hopp C. Balancing act between research and application: How research orientation and networks affect scholars’ academic and commercial output. Journal of Business Economics. 2020;90:1171–1197. doi: 10.1007/s11573-020-00979-x. [DOI] [Google Scholar]
- Zapp M, Powell JJ. Moving towards Mode 2? Evidence-based policy-making and the changing conditions for educational research in Germany. Science and Public Policy. 2017;44(5):645–655. [Google Scholar]
- Zhang L, Sivertsen G, Duj H, Huang Y, Glanzel W. Gender differences in the aims and impacts of research. Scientometrics. 2021;126:8861–8886. doi: 10.1007/s11192-021-04171-y. [DOI] [Google Scholar]
- Zhang Z. Missing data imputation: Focusing on single imputation. Annals of Translational Medicine. 2016;4(1):9. doi: 10.3978/j.issn.2305-5839.2015.12.38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang, T., Ramakrishnon, R., & Livny, M. (1996). BIRCH: An efficient data clustering method for very large data bases (pp. 103–114). Montreal, Canada



