Framework development of the social impact of medical research utilization: Fuzzy Delphi approach

Abstract

Background

Medical research plays an important role in increasing and improving the health of society. Utilizing the results of research will enable the provision of both material and spiritual support for research centres. It is essential to establish mechanisms for effective communication between researchers and officials. It is of great importance to pay attention to tools that facilitate the process of conducting this research. Therefore, the purpose of this study is to develop the framework of the social impact of medical research utilization (SIMRU).

Methods

The study is applied in purpose and qualitative in analysis, utilizing a fuzzy Delphi approach with a panel of 25 experts. A purposeful and snowball sampling method was employed. The questionnaire consisted of two rounds of Delphi assessments, divided into three sections: Demographics, Core Variables (measured on a 5-point Likert scale) and Open-Ended Responses. It was developed using a meta-synthesis approach and expert consultations, incorporating feedback for clarity. A pilot test with 13 participants confirmed the reliability (Cronbach’s alpha: 0.987 and 0.974) and content validity through expert reviews.

Results

From the initial set of 147 basic themes identified during the meta-synthesis, the conceptual framework of the social impact of medical research utilization – comprising 6 contextual themes, 11 global themes, 20 organizing themes and 103 basic themes – represents the layers of the social impact of medical research utilization and was approved by experts. These contextual themes include knowledge governance and research, knowledge production and exchange, knowledge dissemination and transfer, knowledge mobilization, knowledge popularization and knowledge utilization.

Conclusions

The results of this research support policy-makers in higher education by improving research management and policy-making processes through consideration of the social impact of research and by informing the development of necessary strategies to enhance community health. Additionally, these finding can help universities and research centres measure research activities and knowledge popularization dissemination more effectively while strengthening the connection between researchers and their target audiences. This research fills a significant gap by introducing social impact indicators and science promotion in medical research for the first time, demonstrating innovation in both the chosen topic and research methodology.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-025-01419-4.

Background

The impact of research is a complex process that, in addition to generating academic knowledge, produces benefits (health, economic, and cultural) and reflects the various ways in which knowledge is generated and utilized [1]. The concept of social impact has received serious attention since the late 1990s [2] and the National Science Foundation of the USA has emphasized the importance of measuring the social impacts of research [3]. This concept can be summarized in three approaches: social impact [4–6], social impact of research [2] and evaluation of social impact [7].

These three concepts have different dimensions in three approaches: traditional [2, 8–11], quantitative [12] and new [13]. In the traditional approach, the following dimensions are considered: social impact, economic impact, cultural impact and environmental impact. In contrast, the quantitative approach includes the dimensions of scientific impact, technological impact, policy impact and organizational impact. However, the new approach encompasses the dimensions of educational impact, social impact (altmetric approach), impact on arts and humanities, health impact, commercial impact and symbolic impact (Fig. 1).

Fig. 1.

The concept of social impact (in three approaches)

The social impact of research is recognized as a key concept in community health development [14], as medical research plays an important role in increasing and improving the health levels of society, particularly in developed countries [15]. It is expected that this research will lead to improvements in health and treatment in the medium and long term. Therefore, it is crucial to pay attention to the tools that facilitate the process of conducting this research [16]. These tools can include the design of conceptual framework, quantitative and qualitative indicators and appropriate analytical methods to measure and evaluate the social impacts of medical research. Using such tools can help increase the quality and quantity of research in the field of medicine and improve the health of society.

The main challenges in measuring the impact of research include the two dimensions of time and attribution [15, 17, 18], which can be addressed through the concepts of Altmetric and social media. Barriers to the utilization of research, particularly during the knowledge transfer stage, require special attention and targeted solutions [19]. From a methodological perspective, the evaluation of social impact faces several limitations [17, 20, 21], as most studies have been conducted using case study, survey and qualitative methods, often in a limited manner focusing separately on either research utilization or social impact [22]. Additionally, most of the studies have focused on research utilization (the utilization of research findings [23–29], knowledge translation [30–35], investigation of the transfer and utilization of research results [36–41]), social impact [the social impact of research [17, 42–46], and the evaluation of social impact [47–51]). This indicates the attention given to and the role of research issues in previous studies. However, no study has addressed the specific issue of the social impact of medical research utilization (SIMRU).

However, according to Bakker’s [52] wave model, the history of knowledge utilization in the USA shows that, in the period from 1920 to 1960, activities focused more on agricultural innovations and farmers. After the Second World War, the transfer of agricultural knowledge and technologies advanced significantly at the international level. In the second period (1960–1980), the second wave of knowledge utilization emphasized the adoption of technological innovations by organizations. During this period, social improvement and program evaluation emerged as a new area of scientific research. The third wave (the 1990s to the present) has focused primarily on how the utilization of knowledge can lead to improvements in the areas of health, education and human services [53].

Knowledge utilization refers to the production, exchange and proper utilization of knowledge, which, from an ethical standpoint, occurs through a complex process of interactions between researchers and research stakeholders. This process leads to the delivery of more effective products and services and strengthens research systems [54], aligning with the outcomes of various research studies [23, 28, 31, 32, 37]. Different approaches to research utilization include knowledge management, knowledge production, knowledge transfer, knowledge outsourcing and knowledge utilization, each comprising its own specific subsets [23, 28, 31–33, 37]. Models and frameworks related to knowledge use encompass instrumental use, conceptual use, symbolic use and process use [55]. Studies by Kabiri [26] and Thomas and Bussières [56] have emphasized the importance of these frameworks. The analysis and comparison of frameworks and models indicated that the Canadian Institutes of Health Research Framework (CIHRF) and Research Excellence Framework (REF), or a combination of both, can be utilized as a foundational model for assessment SIMRU [15, 17, 51].

In planning and policy-making, one of the important factors in developing and strengthening research centres is the utilization of research results. This not only helps improve health but also attracts the support and attention of officials to health research. Utilizing the results of research will enable both material and spiritual support to focus on research centres. This focus will ensure the continuity of their activities and ultimately lead to self-sufficiency in the future. Failure to utilize research results leads to aimless research and alienates researchers from the country’s priorities. Therefore, it is essential to establish mechanisms for effective communication between researchers and officials. This will help build the necessary trust and enable the utilization of research results for planning and development [57].

The Delphi method is used as a structured process for collecting and classifying existing knowledge among a group of experts and specialists, by distributing questionnaires and providing controlled feedback [58]. This method has different types [59–61]. While most surveys seek to answer the question “What is?”, the Delphi method seeks to answer the question “What could be?” or “What should be?”. The purpose of this study is to develop the conceptual framework of SIMRU from the perspective of experts.

One of the important focuses of this research is to identify the interaction of social influence in the field of medical research. By closely examining this subject, researchers are determining its role in the mobilization of knowledge and the application of medical research. This subject is important in promoting science in this field. It also plays a key role in determining social impact indicators. The issue of science promotion and social impact indicators in medical research has been addressed for the first time in this study. This demonstrates the innovation and creativity of researchers in choosing topic and research approach.

The results of this study can be used by policy-makers and planners to improve the utilization of medical research. In other words, the evaluation conducted in this study can provide guidance to decision-makers for enhancing the effectiveness and application of research findings in the field of health and hygiene. Finally, the results of this research can serve as a model for evaluating science promotion activities in other scientific fields. This can lead to the development and enhancement of evaluation systems across various scientific fields, significantly contributing to the advancement of knowledge and technology in the country.

Methods

Research design

Given the complex implications of medical research, it is essential to involve individuals with expertise in the economic, social and political aspects of this field. The current study is designed as a fuzzy Delphi study, which closely resembles the classical Delphi method. However, the key distinction lies in the fuzzification of responses, allowing for qualitative definitions of variables. This characteristic is regarded as an advantage of the fuzzy method [62]. Typically, this procedure is conducted over two to three rounds. The first round focuses on generating new proposals and ideas, while the information gathered is utilized to formulate questions for subsequent rounds of research [63]. In this study, the fuzzy Delphi method was implemented over two rounds.

Data collection: Protocol, advisory board and expert panel

The fuzzy Delphi method is a structured group communication process designed to gather the opinions of a panel of experts on a specific topic or to address a central question [64, 65]. There are four main activities involved in planning and implementing the fuzzy Delphi method [66]: defining the research problem, determining the panel size, selecting the panel and conducting Delphi rounds. The research team include the researcher and three subject matter experts with extensive experience in research and statistical methods. The team is responsible for guiding and supervising the implementation of the Delphi method. Their expertise encompasses knowledge and information sciences, knowledge organization, scientometrics, bibliometrics, information representation, information and communication sciences and information retrieval; information management, data management, digital resource management and human interaction with information; and methodologies related to futures studies in health and health policy. None of them participated in the scoring exercises. The quality of the results obtained from the fuzzy Delphi method largely depends on the knowledge and expertise of the participating experts [67]. In addition to their ability, interest and commitment to the topic, continuous engagement throughout all rounds is also essential [68]. The process of identifying and selecting experts, as well as inviting them to participate in the fuzzy Delphi method, was carried out in the following stages [66].

1. Preparation for selection [Knowledge Resource Nomination Worksheet (KRNW)]: Ministry of Health and Medical Education1 (4 experts), National Research Institute for Science Policy (NRISP)2 (8 experts), Iranian Research Institute for Information Science and Technology (IranDoc) 3 (five experts), universities under the Ministry of Science, Research and Technology (MSRT)4: (16 experts) and universities affiliated with the Ministry of Health and Medical Education (30 specialists).

2. Determining the population: A total of 63 experts were selected for the study. Of these, 44 were identified through referrals from other experts, while the remaining 19 were recruited using the methods described below.

3. Introducing other experts: For the ranking of the experts, the Iranian Scientometrics Information Database from the Ministry of Health and Medical Education’s Deputy of Research and Technology was utilized, along with the Scientific Publications and Information Development Center,5 the Research Profile of the University of Tehran6 and the research works and the records of the experts.

4. Ranking and inviting experts: A total of 63 experts were selected for participation. Due to email access restrictions (some email addresses were inactive), invitations were sent to 58 experts [Electronic Supplementary Material (ESM) 1]. Of these, 41 experts responded positively to the invitation to join the Delphi panel. Ultimately, 25 experts participated in the first round of the Delphi panel.

One of the key considerations in forming a Delphi group is determining the appropriate number of experts. While increasing the number of experts can reduce error, an excessively large group can complicate data analysis and hinder consensus and agreement [61]. Selecting the right sample size requires a balance between the breadth and depth of the research [69]. As a general guideline, Martino suggests that a sample size of 15–30 individuals is suitable for heterogeneous populations of experts [70]. Consequently, the research community consists of 25 specialists across the following fields: knowledge and information sciences, medical librarianship and information sciences, scientometrics, information and communication sciences, epidemiology, medical informatics, healthcare services management and health policy.

According to the information in Table 3, the breadth of the topic of social impact and the importance of utilizing research results and knowledge translation in the universities under the Ministry of Science and the Ministry of Health indicate that the number of social impact experts is limited, while the number of knowledge utilization experts is heterogeneous. This scarcity is primarily due to the emerging nature of these interdisciplinary fields and the relatively recent institutional emphasis on knowledge translation and social impact in academic research. The experts came from a variety of medical fields and had backgrounds in practice rather than research. Consequently, this challenge was carefully considered during the selection of experts.

Table 3.

Descriptive statistics of respondents (demographic information of panel experts)

Question title	Indicator	First round frequency	First round frequency percentage (%)	Second round frequency	Second round frequency percentage (%)
Sex	Male	18	72	18	78
	Female	7	28	5	22
Academic rank	Professor	6	24	6	26
	Associate professor	8	32	7	30
	Assistant professor	10	40	9	39
	Employee	1	4	1	4
Activity or experience (years)	5–10	7	28	7	30
	11–15	9	36	7	30
	16–20	4	16	4	17
	21–25	3	12	3	13
	26 and above	2	8	2	9
Specialization expert panel	Knowledge and information sciences	11	44	9	39
	Medical librarianship and information sciences	3	12	3	13
	Scientometrics	1	4	1	4
	Information and communication sciences	1	4	1	4
	Epidemiology	3	12	3	13
	Medical informatics	2	8	2	9
	Healthcare services management	2	8	2	9
	Health policy	2	8	2	9
Total		25	100	23	92

Three sampling methods can be employed in the Delphi method: targeted, continuous and criteria-based sampling. In this research, the experts of the fuzzy Delphi panel were selected using purposive and chain sampling (snowball sampling). In the second round, the number of panel experts was reduced to 23.

Research implementation steps

The Delphi method is implemented through a questionnaire. In the first round, a variety of questions may be posed; however, some of these may have limited relevance to the research topic. Therefore, before designing the questions, it is essential to clearly define the nature, dimensions and scope of the research topic [71]. To identify the research topic and formulate design questions in the initial phase, researchers must first review the relevant literature and enhance the questionnaire items through engagement with focus groups and interviews with experts [72]. In the first round of the fuzzy Delphi panel, the researcher analysed the research question using the themes derived from the meta-synthesis stage [73]. At this stage, the researcher, in collaboration with the research team and three experts – with two PhDs in Information Science and Knowledge Management and a PhD in Information Retrieval – reviewed the themes that had been gathered. In developing the questionnaire, careful attention was given to several important factors, including the wording of the questions (to avoid ambiguity, bias and negativity), the logical sequence of the questions (from general to specific and from simple to complex), the overall appearance of the questionnaire and the clarity of the instructions for completion [74].

The questions in the first round of the fuzzy Delphi method were divided into three parts (ESM 2). The first part included questions related to demographic information (age, gender, education and executive experience), utilizing nominal and interval scales. The second part comprised questions that employed a 5-point Likert scale (1 = “entirely disagree”, 2 = “somewhat disagree”, 3 = “neutral”, 4 = “somewhat agree” and 5 = “entirely agree”) [75], as well as open-ended questions. The research team calibrated the survey for content and clarity before administering it. Three days after the initial distribution, we followed up confirm receipt of the questionnaire. After 10 days, reminders were sent to optimize participation. Reminders were issued twice during the second round of the questionnaire (Table 1).

Table 1.

Distribution and collection dates of questionnaires

Questionnaire round	Distribution of questionnaires		Collection of questionnaires			Average follow-up per expert
	Distribution date	Number of experts	Deadline	Number of experts	Percentage (%)
First	25 October 2023	25	21 November 2023	25	100	2–3
Second	22 December 2023	25	20 January 2024	23	92	3–4

To encourage and motivate experts to participate in the Delphi method, we implemented several strategies to maximize response rates and reduce preparation bias. Follow-up was conducted at least three times to ensure engagement. The questionnaire was meaningfully presented with images of the conceptual framework, and explanations of each theme were included in the footnotes. Additionally, emails were personalized to enhance the sense of importance among the experts.

In the second round of the fuzzy Delphi method, the questionnaire from the initial phase was revised, incorporating the indicators derived from the experts’ feedback.

Survey analysis

Based on the experts’ opinions, in addition to removing eight basic themes that did not receive Delphi points, several changes were made to the questionnaire for the second round. These changes were made to improve the accuracy and validity of the results and to better align with the experts’ opinions (ESM 3).

In this step, the average opinions from the first round of the fuzzy Delphi questionnaire, along with the previous views of each panel expert, were included in the second-round questionnaire (ESM 4). Experts were asked to indicate their opinions on the categories and items of the second-round questionnaire using a 5-point Likert scale, based on their previous opinions and the average opinions of the experts from the first round.

Tools and techniques

The research tool is a questionnaire. The questionnaire measurement tool in the Delphi method must have the necessary validity. There are several methods for determining the validity of a questionnaire, including: construct validity, concurrent validity, content validity, external validity and internal validity [66, 76]. To determine the content validity (face validity) of the questionnaire, the judgment of the experts and professors of the research team and the extent of coverage and suitability of the questions with the objectives of the research subject were used. To measure reliability, the reliability coefficient index was used. A reliability coefficient of zero indicates no reliability, while a reliability coefficient of one indicates complete reliability. The most important measure of reliability is Cronbach’s alpha. If it is greater than 0.7, it is considered acceptable [76]. The Delphi method cannot achieve 100% reliability [66, 77]. If the experts participating in the Delphi have the time and patience to complete the questionnaire again, the test–retest method can be used to measure the reliability of the questionnaire. Reliability between the first and second questionnaires can be estimated by examining the level of agreement between respondents. If sufficient consensus is achieved among the experts on the options in the second questionnaire, it can be said that the researcher has adequately extracted the experts’ meaning from the answers to the first questionnaire [66, 78]. The reliability of the questionnaire was examined in two rounds of fuzzy Delphi using Cronbach’s alpha coefficient with SPSS software. The reliability of the questionnaire in the first and second rounds was reported to be 0.987 and 0.974, respectively (Table 2).

Table 2.

Reliability statistics

Delphi round	Number (N) of expert	Percentage (%)	Cronbach’s alpha	Cronbach’s alpha based on standardized items	Number (N) of basic themes
First round	13	100.0	0.986	0.987	147
Second round	13	100.0	0.974	0.974	103

Data analysis

To analyse the responses received in the first round, triangular fuzzy numbers (equivalent to a 5-point Likert scale) were used: very low (0, 0, 0.25), low (0, 0.25, 0.5), medium (0.25, 0.5, 075), high (0.5, 0.75, 1) and very high (0.75, 1, 1). For this purpose, first, triangular fuzzy values of experts’ opinions were calculated. Then, the fuzzy mean of the opinions of n respondents was calculated. The calculation of the fuzzy number τ for each of the indicators was performed using the following relations:

$${\tilde{\tau }}_{\mathit{ij}}=\left(a_{\mathit{ij}},b_{\mathit{ij}},c_{\mathit{ij}}\right),i=1,2,\dots ,nj=1,2,\dots ,m$$ 1

$$a_j=\sum \frac{a_{\mathit{ij}}}{n}$$ 2

$$b_j=\sum \frac{b_{\mathit{ij}}}{n}$$ 3

$$c_j=\sum \frac{c_{\mathit{ij}}}{n}$$ 4

In the above relations, index i refers to the expert, and index j refers to the decision-making index. Also, the defuzzified value of the average fuzzy number is obtained from the following relation [79]:

$$\text{Crisp}=\frac{a+b+c}{3}$$ 5

First, on the basis of triangular fuzzy numbers, the numbers were fuzzified. Then, on the basis of relations 2–4, the fuzzy average of the scores was obtained. Finally, using relation 5, the fuzzy average was converted into a definite number. For Row 1 of the fuzzy Delphi calculations, 0 experts gave a high score, and 0 experts gave a very high score. Therefore, the fuzzy and non-fuzzy (crisp) scores are as follows:

$$\begin{array}{cc}\hfill \text{Fuzzy}\text{score}& =\frac{3\times \left(0,0,0.25\right)+2\times \left(0,0.25,0.5\right)+3\times \left(0.25,0.5,0.75\right)+1\times \left(0.5,0.75,1\right)+3\times \left(0.75,1,1\right)}{12}\hfill \\ \hfill & =\left(6.73,8.45,9.5\right)\hfill \\ \hfill \end{array}$$

$$\text{Crisp}\text{score}=\frac{6.73+8.45+9.5}{3}=8.23$$

In this study, the threshold number 0.7 was considered. To determine the level of consensus among experts in the Delphi method, the Kendall’s coefficient of concordance can be used. This coefficient ranges from zero to one, where zero indicates no consensus and one indicates complete consensus. A value of 0.7 or higher indicates satisfactory agreement. If the coefficient is less than 0.7, the questionnaire should be resent to the experts [66]. Respondents’ confidence in their opinions can be increased by reducing the variance of responses, especially when Delphi participants are heterogeneous and possess different areas of expertise [72]. A significant increase in the variance of panelists’ responses to questionnaire items indicates a decrease in the level of agreement among members regarding those items [80].

Ethical considerations

In the first stage, participants’ privacy was fully ensured. To promote transparency in the research process, we sent an email to participants before the Delphi stages began, outlining the research purpose, methods and data usage. This enabled participants to engage in the process with a clear understanding of the research details, should they choose to participate (ESM 1). (Fig. 2).

Fig. 2.

Methods flow chart

Results

Results from the first round of the fuzzy Delphi

In this research, the majority of panel experts are from the fields of knowledge and information sciences, medical librarianship and library and information sciences. The second group consists of experts in medical informatics, healthcare services management and health policy. Additionally, experts in scientometrics and information and communication sciences comprise the subsequent groups of panel experts.

At this stage, a questionnaire including six contextual themes of SIMRU was presented to the fuzzy Delphi panel experts. They were asked to express their opinions on each criterion in the form of verbal variables included in the questionnaire. The initial results of the experts’ opinions are shown in ESM 5. In this part of the study, the threshold number of the first round of the fuzzy Delphi questionnaire was considered to be 0.7.

According to ESM 2, 8 basic themes out of 147 proposed basic themes were eliminated due to not obtaining Delphi points. In addition, 36 basic themes were merged on the basis of the experts’ open-ended responses. Finally, 44 basic themes out of 147 basic themes remained.

• The basic theme “conducting a workshop for research users”, with a Delphi score of 0.670 and an average score of 3.76 for the first round, and the basic theme “research user oversight board team”, with a Delphi score of 0.637 and an average score of 3.64 for the first round, were eliminated due to not obtaining a Delphi score.

• The basic theme “membership of researchers and managers in associations”, with a Delphi score of 0.677 and an average score of 3.76 for the first round, and the basic theme “participation in knowledge-based conferences”, with a Delphi score of 0.683 and an average score of 3.80 for the first round, were also eliminated due to the lack of Delphi scores.

• The basic theme “dissemination of knowledge from health research”, with a Delphi score of 0.553 and an average score of 4.16 for the first round, was removed because this basic theme was not given a score by seven experts in the first-round questionnaire.

• The basic theme “shared responsibility for research dissemination”, with a fuzzy Delphi score of 0.667 and the average score of 3.76 for the first round, and the basic theme “peer review of similar research by academic peers”, with a fuzzy Delphi score of 0.683 and the average score of 3.84 for the first round, were also eliminated due to the lack of Delphi scores.

• The basic theme “psychological factors of research”, with a fuzzy Delphi score of 0.700 and the average score of 3.88 for the first round, was also removed.

• Basic themes with a mean higher than 4 and a standard deviation higher than 1 were eliminated due to the dispersion of responses relative to the mean and lower agreement among panel experts. For the two basic themes “challenges in measuring the social impact of research” and “social impact on scientometric fields”, which had a mean higher than 4 and a standard deviation higher than 1, the “mode” was calculated. Despite the high dispersion of responses relative to the mean and lower consensus among panel experts, these themes were not excluded from the Delphi round. The convergence index was 5, indicating a tendency towards agreement in the Delphi reports [81].

• The highest averages belonged to the basic themes “feeling of the need for research in decision-making” (4.760), “impact on service delivery” (4.720), “encouragement of the use of research evidence in the decision-making system” (4.680) and “link between social needs and the university” and “improving disease treatment outcomes” (both 4.640).

• The lowest averages belonged to the following basic themes: “approaches based on research processes” (3.960), “presentation in workshops, seminars, conferences, posters, lectures” (3.920), “multi-stage process of knowledge transfer” (3.880), “psychological factors of research” (3.880), “peer review of similar research by academic peers” (3.840), “participation in knowledge-based conferences” (3.800), “conducting a workshop for research users” (3.760), “membership of researchers and managers in associations” (3.760), “shared responsibility for research dissemination” (3.760), and “research user oversight board team” (3.640).

The results showed that the panel experts in the first round of fuzzy Delphi identified 22 basic themes out of 23 basic themes for knowledge governance and research (KGR); 29 basic themes out of 31 basic themes for knowledge production and exchange (KPE); 18 basic themes out of 23 basic themes for knowledge dissemination and transfer (KDT); 21 basic themes for knowledge mobilization (KM); 22 basic themes for knowledge popularization (KP); and 27 basic themes for knowledge utilization (KU), which were presented in previous studies, as having a high and very high impact in the conceptual framework of SIMRU. Therefore, respondents selected 103 basic themes from a total of 147 basic themes provided by previous research meta-synthesis [73].

Results from the second round of fuzzy Delphi

In the second round of the Delphi questionnaire, demographic information was very similar to the first round, except for a decrease in the percentage of women from 27 to 22%. The academic rank of assistant professor, with a decrease of 1%, still maintained the highest frequency. The frequency of work experience in the ranges of 11–15 years and 5–10 years was equal in this period. In the field of specializations, despite the decrease in the percentage of knowledge and information sciences, the order of frequency of the specialization ranks of the panel experts did not change (Table 3). In this part of the study, the threshold number of the second round of the fuzzy Delphi questionnaire was also considered to be 0.7. In the second round of the fuzzy Delphi questionnaire, the panel experts evaluated the various basic themes (ESM 6).

• The mean7 responses of the experts regarding the importance of KGR increased from 4.24 in the first round to 4.36 in the second round. This increase was also observed in the importance of KPE (from 4.28 to 4.50), in the importance of KDT (from 4.16 to 4.48), in the importance of KM (from 4.35 to 4.45), in the importance of KP (from 4.29 to 4.41) and in the importance of KU (from 4.38 to 4.44). This increase meant that the experts’ opinions on the six contextual themes of the conceptual framework were getting closer. In fact, increasing agreement and positive views lead to improved credibility and greater usability of the conceptual framework.

• The standard deviation of the experts’ responses regarding the importance of KGR decreased from 0.73 in the first round to 0.65 in the second round. This decrease was also observed in the importance of KPE (from 0.75 to 0.56), in the importance of KDT (from 0.76 to 0.56), in the importance of KM (from 0.70 to 0.60), in the importance of KP (from 0.80 to 0.69) and in the importance of KU (from 0.69 to 0.63). This decrease indicates the dispersion of the responses and their approach to the mean.

• The increase in the average findings indicates that the experts’ opinions are getting closer to each other. This increase also reflects the increase in agreement and positive views of experts on the issues under review. The increase in expert agreement indicates improved validity and greater usability of the conceptual framework of SIMRU. The average of most basic themes increased. Also, a decrease in the standard deviation indicates a decrease in the dispersion of the responses and their approach to the mean.

• The highest average belongs to the following basic themes: “feeling of the need for research in decision-making”, with a score of 4.826; “link between social needs and the university”, with a score of 4.826; “impact on service delivery”, with a score of 4.783; “role policy implementers”, with a score of 4.739; and “impact on health status”, with a score of 4.739.

• The lowest averages belong to the basic themes “pathways to research commercialization”, with a score of 4.174; “health technology evaluation indicators”, with a score of 4.174; “social return on research investment” and “identification of research dimensions”, each with a score of 4.087; and “social impact of research in the fields of scientometrics”, with a score of 4.00.

The results showed that 29 KGR basic themes, 17 out of 29 KPE basic themes; 12 out of 18 KDT basic themes, 11 out of 21 KM basic themes, 10 out of 22 KP basic themes and 24 out of 27 KU basic themes presented by the experts were identified as having high and very high impact in the conceptual framework of SIMRU.

Level of agreement

The results of the desired level of agreement between experts of the fuzzy Delphi panel in two rounds are shown in ESM 7. If the difference between the two polling stages was less than a very small threshold (0.1), the polling process stopped, meaning we reached consensus [82]. If we did not reach a consensus, a questionnaire was to be created from the indicators that had not reached consensus and was to be made available to the experts again. The difference between the averages of all indicators in the second and first stages of the fuzzy Delphi was less than 0.1, which shows that we reached consensus in this section as well. All research indicators that were confirmed in the first and second rounds of fuzzy Delphi had an average above the very low range. This range is considered 0.3 [83]. The results of this section indicated that, for all indicators extracted from the meta-synthesis stages, the mean difference was less than 0.1, which means it was significantly below the threshold of 0.3. Therefore, consensus was achieved, eliminating the need for a third round of Delphi polling. All indicators have been officially approved and selected as influential indicators for the SIMRU conceptual framework.

At this stage of distributing the questionnaires, all fundamental themes received approval from experts without any deletions. There were no standard deviations exceeding 1 and no means below 4, except for in two themes, indicating that the data were consistent and aligned with expectations, reflecting a strong consensus among the Delphi panel experts. Consequently, there was no need to eliminate any themes. The results confirm that the conceptual framework was validated by experts and possessed the necessary credibility. The qualitative feedback from the Delphi panel primarily focused on modifications to enhance the quality and structure of the questionnaire. Additionally, the absence of differing opinions regarding the addition, removal, or alteration of themes signifies the panel experts’ satisfaction with the structure and content of the second-round questionnaire.

Therefore, the survey process was stopped, and 29 basic themes remained in KGR, 17 basic themes in KPE, 12 basic themes in KDT, 11 basic themes in KM, 10 basic themes in KP and 24 basic themes in KU. Finally, all indicators were finally approved and selected as influential indicators of conceptual framework for SIMRU.

Discussion

Comparing the results of the two rounds of the Delphi panel shows that, compared with previous rounds, in addition to the increase in the average, the basic themes related to “feeling of the need for research in decision-making”, “impact on service delivery” and “link between social needs and the university” in particular attracted the attention of experts. This may be due to the experts’ deep experience and knowledge in the aforementioned fields, which has given these aspects special importance in the experts’ eyes.

It seems that past experiences and the unique perspectives of experts in this field have led to special focus and attention on these issues. Therefore, it can be said that these basic themes have been recognized as strengths and vital issues in the field under study. And probably due to their high effectiveness in decision-making and service delivery, experts have given them more importance. These results demonstrate the impact and value of analysing and reviewing the Delphi panel rounds on the decision-making process and improving activities in the field under study. These changes represent a careful and systematic process in refining and improving the conceptual framework of SIMRU. They also indicate the consistency and validity of the basic themes selected in this conceptual framework. And these can be used as a basis for future research in this area.

The findings indicate that some of the indicators and basic themes of the extracted conceptual framework align with the results of the research studies conducted [23, 28, 31–35]. The importance of communication and knowledge exchange between researchers and decision-makers in solving problems is highlighted within the second layer of global themes in the conceptual framework, which aligned with the findings of study [84], and confirms researchers’ efforts to engage with social challenges. Additionally, the indicators and basic themes of the outer layer of the framework align with the findings of studies [25, 30, 56].

The timely dissemination of research results, the lack of delays in the publication and application of findings, the improper use of evidence-based medicine and the lack of awareness regarding the barriers to implementing evidence-based care indicate that the impact of research in health service organizations requires not only attention to organizational, social, economic and cultural contexts but also another key element for the successful implementation of research [38, 41, 85, 86]. These issues have been addressed in the contextual themes of KU and KGR, aligning with the findings of Abedi et al. [27]. Additionally, studies emphasize the importance of avoiding delays in research [15, 17, 18]. These challenges have been defined in the contextual theme of KGR at intermediate and macro levels, in the process of measuring the social impact of research and the pathway from initiation to impact. The timely presentation of research results has been considered in the contextual theme of KU in behaviours leading to the application of research.

In response to the identified needs and barriers in the studies [29, 36, 87], the contextual theme of KM, along with KGR activities at micro, meso and macro levels, can be instrumental in this regard. To address the challenges of knowledge transfer and sharing [19, 39, 88], activities related to knowledge dissemination and mobilization within the research framework can help solve these issues. Also, popularization of science at the general community level is very important and requires the participation of all stakeholders [40, 89–93]. These topics have been addressed in the basic themes introduced in the contextual theme of KP. Various studies have been conducted to evaluate the social impact of research [42, 44, 45, 49, 94, 95]. This topic has been considered in different sections of the research’s conceptual framework, such as the role of stakeholders (knowledge users), mechanisms for knowledge exchange, activities based on active and effective knowledge dissemination, extensive stakeholder collaboration, health technology assessment indicators and more. The role of social media [96] and communication methods [97] in social-network-based activities has been highlighted in the concepts of knowledge distribution from the perspective of KM. Overall, the conceptual framework of the research is a blend of introduced models that can be applied to most educational, research and academic institutions [26].

Limitations

With the increase in the production of research works [98, 99], the importance of evaluations is essential, especially to demonstrate the effectiveness and social impact of medical research in developing countries such as Iran. Iran ranks in the top tier of publications related to social impact and research utilization; however, it is not represented in the common communications of this field [22]. The limited number of Persian works on the topic of SIMRU can be attributed to the fact that this subject is new and has not yet been extensively studied by Iranian researchers [73]. Therefore, the expert community studied was from Iran, and the specific conditions of developing countries have influenced their opinions and the research environment. Multiple studies indicate that obstacles to the utilization research results in Iran stem from the university environment, knowledge production, the knowledge transfer process and the users of research results [25, 36, 86, 87, 100]. Experts from the Ministry of Health lack familiarity with social impact, scientometrics and altmetrics, and their integration into scientometric system indicators for medical university faculty is concerning. Unclear definitions and the implementation of research utilization activities, such as knowledge translation, by the Ministry of Science experts, alongside differing perspectives between these fields, further limit research application. In Iran, initiatives such as the Ministry of Health and Medical Education’s Deputy of Research and Technology, the Scientific Publications and Information Development Center (SPIDC)8 and the Society and Industry Communication System with the University (SAJID)9 emphasize knowledge mobilization and practical applications. Other activities include the National Health Research Dissemination Database10 and scientific discourses and performance monitoring of universities. Additionally, the Novel System for Medical Research Information in Iran (NOPA) was developed to address these issues and includes 12 scientific research information systems that provide 12 new research services to medical researchers in the country. However, challenges remain, requiring further action.

Conclusions

To answer the research question and develop the conceptual framework of SIMRU from the experts’ perspective, first, the components of the conceptual framework designed as a result of the meta-synthesis steps were confirmed using the fuzzy Delphi method. Then, the indicators were confirmed and screened. The fuzzy Delphi panel was conducted in two rounds. The conceptual framework of the present study, with 6 contextual themes, 11 global themes, 20 organizing themes and 103 basic themes that represent the layers of SIMRU, was approved by experts (Fig. 3). Finally, to assess the social impact of medical research using established indicators, policy-makers should strengthen knowledge governance, develop legal frameworks, formulate national policies and allocate sufficient funding, while organizations should adopt research utilization approaches, establish effective structures, incentive systems and user needs assessment mechanisms to improve research management and knowledge transfer. Researchers play a key role by managing knowledge, engaging with stakeholders and expanding knowledge transfer channels to facilitate its application.

Fig. 3.

Conceptual framework SIMRU using the fuzzy Delphi method

Abbreviations

KGR

Knowledge governance and research

KPE

Knowledge production and exchange

KDT

Knowledge dissemination and transfer

KM

Knowledge mobilization

KP

Knowledge popularization

KU

Knowledge utilization

SIMRU

Social impact of medical research utilization

Author contributions

N.N., M.H.G. and A.N. contributed to data collection. N.N. and M.H.G. contributed to data analysis and the drafting and revising of the manuscript. N.N., M.H.G., A.N. and A.S.H. contributed to the design of the stakeholder meetings, attended the meetings and commented on and approved the final manuscript.

Funding

This research was conducted without any external funding. All work was carried out personally by the authors of this study, who are members of the PhD dissertation team.

Availability of data and materials

The data and materials used in this study are detailed in the Electronic Supplementary Materials (ESM) section.

Declarations

Ethics approval and consent to participate

In the research stages, informed consent and a complete explanation of the research process, confidentiality of information and the method of publishing results were provided to the participants. The study was conducted under the supervision of the Ethics Committee in Research at the University of Tehran in 2021, with code 157690.

Consent for publication

The stakeholder group were contacted for consent for their names to be included within the collaboration author group.

Competing interests

The authors declare no competing interests.