Abstract
To comprehensively assess the impact of K-12 engineering education research (K-12 EER) and provide insights to stakeholders and policymakers, this study uses the scientometric analysis software CiteSpace to evaluate recent advances in K-12 EER. A search of 885 articles from the Web of Science Core Collection was conducted. This study analyzed publication trends, authors, countries, and research institutions to determine the trajectory of the K-12 EER. In addition, keyword co-occurrence and clustering maps were generated to identify major hotspots, and keyword burst detection was used to demonstrate development trends. The analysis reveals that global interest in the K-12 EER is growing, and the results are accumulating rapidly. However, cooperation between universities, institutions, experts, and scholars must be strengthened. Current significant topics in K-12 EER focus on the practical form of engineering education, its impact on learners, the centrality of engineering design, and teachers' professional development. Research frontiers primarily revolve around the nature of engineering. Future research efforts should promote the systematic integration of K-12 engineering education through the learning process, develop comprehensive measurement tools to assess its impact on learners, and expand research on teachers’ professional development in K-12 engineering education.
Keywords: K-12, Engineering education, Bibliometric analysis, CiteSpace
1. Introduction and background
With rapid innovation in science and technology, the information age is progressing at an unprecedented speed. The development of high-quality, knowledge-intensive work, and innovative and creative enterprises has made the invention of new technologies and the design of new projects an essential manifestation of the strength of a nation. Countries are becoming increasingly aware of the importance of engineering education. Barak et al. [1] believe that this primarily includes the following three reasons: first, engineering thinking, such as problem solving, systems thinking, and innovation capabilities, is of significant value to the sustainable development of modern society; second, engineering literacy is essential for living, learning, and working in a technology-driven world; finally, the importance of comprehensive engineering talents in social construction, economic growth, and national strength has become increasingly prominent, although there is a persistent shortage of professional talent in the field of engineering. Against this background, engineering education worldwide has indicated a distinct tendency to advance in school segments, and countries are paying increasing attention to the causes of engineering education in primary and secondary schools as essential support for cultivating reserve engineering talents. K-12 engineering education has become increasingly critical. It has received widespread attention from scholars in science and engineering. Considering the gradual emergence of K-12 engineering education research (K-12 EER), there is a need for more transparent research maps and guidelines to help conduct research in this area effectively. Simultaneously, educational policymakers are interested in systematic reviews of qualitative research, secondary data analysis, and meta-studies, and these analytical studies help them plan educational policies [2]. Therefore, this study aims to reveal the current state of K-12 EER and its pointers and indications for future research development to provide a more comprehensive research panorama and reference for educational policymakers to formulate sustainable K-12 engineering education policies.
1.1. K-12 engineering education
Engineering education has sparked keen interest in the twenty-first century. However, by reviewing the literature, we determined that engineering design, a practical form of engineering education, is a concept that was introduced previously in science education. This is because engineering design has been referred to as technological design in previous reform efforts, such as the National Science Education Standards [3] and Benchmarks for Science Literacy [4]. In response to the demand for STEM (abbreviation for science, technology, engineering, mathematics) talent driven by economic development, the United States successively introduced The New Framework for K-12 Science Education (Framework) [5] and the Next Generation Science Standards (NGSS) [6]. These new reforms in science education emphasize integrating engineering design into K-12 science education. Such advancements foster connections between engineering and science education, enhance students' understanding of scientific concepts, and promote the development of systematic thinking and spatial reasoning [7]. Owing to the significant value of K-12 engineering education, its scope extends beyond the realms of science and technology to include broader educational domains, such as mathematics [8], teachers’ professional development [9], early childhood education [10], and robotics programming education [11]. These interdisciplinary efforts aim to cultivate collaborative communication and problem-solving skills among various educational stakeholders, ranging from students to educators, elevate awareness of engineering-related careers, and provide pathways for diversity in the STEM field. Moreover, they create many challenges for teachers, because engineering education is yet to be part of their formal training [12]. Thus, in the face of ambitious demands for reform, there is an urgent need for adequate educational research findings to support improvements in the curriculum, assessment, and teacher education to adapt to the new vision.
1.2. Current research on K-12 EER trends
Three articles have been reviewed and analyzed in the K-12 EER (see Table 1). All the scholars, Lancaster et al. [13], Mendoza Díaz et al. [10], and Sneider et al. [14], used an inductive, traditional approach to review the literature on K-12 engineering education research; Lancaster et al. and Mendoza Díaz et al. reviewed the literature before 2011, and because of the rapid development of K-12 EER in the last decade or so, it is not easy to provide an updated picture of the research frontiers for the academic community. Although the studies by Sneider et al. and selected literature spanned an extended period, the amount of data were small. Hence, reviews of the overall development of K-12 EER are relatively limited. Sneider et al. primarily focused on empirical research and paid less attention to exploring K-12 EER-related theories. Although these literature reviews have presented specific trends in K-12 EER, they cannot outline previous work or provide clear guidance for future research owing to time or scope constraints. Regarding research methods, previous studies used traditional literature reviews or mixed bibliometric methods, and the coverage of the sample articles was not sufficiently extensive. K-12 engineering education is a broad and diverse field that requires a high degree of objectivity and comprehensiveness, and an unbiased and comprehensive review based on current research findings would be of significant importance; therefore, this study seeks to conduct a comprehensive and systematic review through bibliometric analysis.
Table 1.
Research on K-12 EER trends.
| Authors | Database | Number of documents | Year range | Methods used |
|---|---|---|---|---|
| L ancaster et al. [13] | Web of Science | 761 articles | 1980–2010.2 | inductive |
| Mendoza Díaz et al. [10] | Search K-12 related journals, websites, and library databases | over 50 empirical research studies | 2001–2011 | inductive |
| Sneider et al. [14] | ERIC | 263 empirical research studies | 2000.1–2021.6 | inductive |
Bibliometric analyses differ from systematic literature reviews. For example, in systematic literature reviews, content and thematic analyses are typically conducted manually. Furthermore, as the systematic literature review is undertaken qualitatively, interpretation bias by scholars from different academic backgrounds may obscure the analysis results [15]. Bibliometric analysis is a quantitative analytical method that allows for a comprehensive exploration of K-12 EER on a global scale by systematically collecting, organizing, and summarizing a large body of relevant research literature, ensuring both breadth and depth of review. In addition, as an objective and quantifiable research methodology, bibliometric methods can provide evidence on the topic using machine algorithms through scientometric analysis software such as CiteSpace and VosViewer. Analyzing metrics such as changes in subject area, citation status, author influence, literature impact, and publication age makes it possible to assess the quality and reliability of research while minimizing subjective bias [16]. Therefore, to comprehensively understand the K-12 EER, we conducted a systematic analysis using bibliometric analysis methods.
This study aims to address the following research questions.
-
(1)
What is the primary distribution of the K-12 EER and who are the prominent contributing authors, countries, and institutions? What are the characteristics of their distribution?
-
(2)
What are the hotspots for K-12 EER?
-
(3)
What are the future research trends in K-12 EER?
2. Methods
The following sub-sections present details of the data sources, search criteria, and analysis methods.
2.1. Software and tools used in the analysis
The first step was to identify a suitable source of information for the analysis. Currently, three chief options are available to obtain citation data: Web of Science Core Collection (WoSCC), Scopus, and Google Scholar. We selected the WoSCC because it encompasses a wide range of interdisciplinary peer-reviewed research publications, including education-focused and STEM discipline publications [17]. WoSCC is the longest-standing database for tracking citation information and provides widely recognized subject classification for research journals. Moreover, it allows the convenient download of citation sources for further analysis, as opposed to Google Scholar, which, while tracking citation counts, does not support the download of large volumes of raw citation data.
The bibliometric analysis tool used in this study was CiteSpace (version 6.1. R6), a software application developed by Professor Chaomei Chen of Drexel University in the United States. CiteSpace is a versatile, real-time, dynamic information visualization software based on the co-citation analysis theory and Pathfinder network algorithms [18]. It quantifies specific domains of literature to explore the critical pathways of discipline evolution and knowledge turning points (represented by pivotal papers). The software generates a series of visualizations, analyzes the potential driving mechanisms for the evolution of disciplines, and detects the forefront of disciplinary development. It is particularly suitable for explaining the current state of research, evolution of themes, and predicting prospects in a given field [16]. Compared with other bibliometric analysis software (such as VOSviewer and Bibliometrics), CiteSpace was selected as the preferred analytical tool for this study because of its comprehensiveness, analytical consistency, and user-friendliness.
2.2. Data exploration and plan
This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [19], employing a comprehensive report based on the PRISMA template. Readers can assess the applicability of these methods to evaluate the accuracy of research conclusions. The PRISMA template involves four processes: identification, screening, eligibility, and determination of studies to be included in the review. Fig. 1 illustrates the details of this process.
Fig. 1.
Flow diagram of PRISMA methodology.
To retrieve articles relevant to the theme of this study, we defined search terms for the K-12 EER. Initially, we conducted a search using the keywords TS = (“engineering education”) AND TS = (K-12), which yielded only 906 literature records. To ensure that the collected literature covered all stages of K-12 education as comprehensively as possible, we developed a broader search strategy based on reviews by Lancaster [13], Mendoza Díaz [10], and Sneider [14] on the K-12 EER. The revised search strategy was as follows: TS = (“engineer* education”) AND TS = (K-12 or “secondary school” or “primary school” or “elementary school” or “middle school” or “high school” or “kindergarten”). The asterisk represents a wildcard representing any number of characters. To comprehensively reflect the changes in the K-12 EER, we did not set a start time for the search. Data retrieval was extended until October 31, 2023, resulting in 3008 records.
To ensure the accuracy and consistency of the dataset, we conducted the following data-cleaning steps: (a) Retaining only journal articles and excluding other types of literature, such as conference papers, books, and book chapters. As suggested by Freire et al. [20] and Wang et al. [21], journal articles hold the highest academic value. (b) Retaining only entries with English as the language of the literature. Considering that CiteSpace typically assumes uniform language usage across all literature and that differences in vocabulary and citation habits may exist between different languages, retaining only English literature helps ensure consistency in the dataset language. (c) Two scholars carefully reviewed the literature abstracts to exclude those that did not match the research topic, such as literature primarily focusing on higher education, as they did not align with our definition of K-12 engineering education. Finally, the built-in program of CiteSpace was used to remove duplicate studies.
2.3. Data analysis
The data analysis process comprised the following two steps.
-
(1)
Document Selection and Export: A total of 885 finalized documents were exported in “plain text” format. The exported literature was in the format of “complete records and cited references,” capturing information such as title, authors, affiliations, abstracts, keywords, publication date, journal of publication, and references.
-
(2)
Data were imported into CiteSpace (version 6.1. R6) and its built-in program was used to remove duplicate entries and visually analyze the documents. According to the search results, the first document about K-12 EER was recorded in 1997, therefore, the period was set to “January 1997–October 2023,” the time slice was set to 1 year, and the selected nodes included authors, institutions, country, keywords; the topics analyzed in the document included title, abstract and keywords. “PathFinder” was selected as the cropping method and the default settings were retained for the other options.
After running CiteSpace, we generated visual network maps that displayed collaborations between various institutions, countries, and authors. Furthermore, we created visual representations of the co-cited keywords, clustered the keywords, and identified burst keywords. A comparative analysis of knowledge network maps enables a rapid understanding of the current research status, the overall evolution in the field in recent years, and potential future development trends.
3. Results
3.1. Analysis of basic information on K-12 EER
3.1.1. Annual distribution of publications
One method of gauging the development of a research discipline is to assess the quantity of academic literature in that field [22]. Plotting the temporal distribution of literature can effectively evaluate the research status of a subject and provide insights into its dynamic development trends. Fig. 2 illustrates the core research set used for analysis in the WoSCC from 1997 to 2023.
Fig. 2.
Annual distribution of the number of articles issued.
The orange line in Fig. 2 represents the annual count of literature records, whereas the blue dashed line is a third-order polynomial fit curve based on yearly publications guided by peer-reviewed studies. The fit curve achieves an R-squared value of 89.14 %, indicating an exponential growth trend in K-12 EER research in recent years. Three distinct developmental stages are evident in Fig. 2: the initiation stage (1997–2006), the growth period (2007–2018), and the rapid stage (2019–2023).
Initiation Stage (1997–2006): During this phase, from 1997 to 2006, the annual publication count was consistently below ten articles, indicating relative stability in research output.
Growth Period (2007–2018): Literature counts experienced fluctuating growth during this stage, reaching a peak in 2017, with 68 publications.
Rapid Stage (2019–2023): The literature count rose sharply, with an average annual number of publications exceeding 100 articles since 2019. This surge suggests that the K-12 EER is undergoing rapid development.
In summary, the overall increase in literature reflects an exponential growth trend. The exponential growth in annual publications may indicate that the field's development trend has not yet saturated, signifying ongoing rapid advancement [23]. Consequently, K-12 EER has significant potential for future development, exerting an increasingly influential role in international science education. This suggests that more scholars are likely to engage in discussions and exchanges within this field, thereby fostering diverse developments in theoretical constructs and practical educational applications.
3.1.2. Active authors and their collaboration in K-12 EER
The lead author drives academic development, fosters innovation, and establishes academic discourse [24]. The educational community evaluates primary authors based on their publication and citation counts. According to Price's classic theory [25], the number of published papers significantly manifests as scientific research activities, directly reflecting an author's academic involvement; a positive correlation between citation indices and author influence [26]. As illustrated in Table 2, Portsmore, Rogers, and Brophy are among the most cited scholars. This indicates that their research laid an essential foundation for subsequent studies. Their focal points predominantly revolve around how design-based learning aids learners in organizing their learning experiences, thereby supporting their learning objectives. Additionally, scholars such as Capobianco and Margot focus on the role of engineering design in supporting teachers' professional development. Guzey, Siddika Selcen, Moore, Tamara, Roehrig, and Gillian demonstrate prolificacy, suggesting significant knowledge accumulation in the field. A closer examination of their research revealed a focus on applying the engineering design process to STEM and science education classrooms. They explore the impact of the engineering design process on learners' academic performance, attitudes, and interests. An in-depth analysis of lead authors indicates that these scholars propel K-12 EER in more profound and comprehensive directions. This involves a thorough analysis of the intrinsic mechanisms of engineering education, a continuous exploration of teaching practices, and a shift in focus toward teacher preparation. Such comprehensive research directions will provide solid theoretical support and practical guidance for the future development of K-12 engineering education.
Table 2.
Top 10 most prolific and cited authors.
| Authors ranked by number of publications | Authors ranked by number of citations | ||||
|---|---|---|---|---|---|
| Authors | Articles | Citations | Authors | Articles | Citations |
| Guzey, Siddika Selcen | 16 | 213 | Portsmore, M | 3 | 410 |
| Moore, Tamara J. | 9 | 211 | Rogers, C | 2 | 475 |
| Roehrig, Gillian H | 6 | 55 | Brophy, S | 2 | 384 |
| Chiang, Feng-Kuang | 6 | 25 | Adams, RS | 2 | 283 |
| De vries, M J | 6 | 62 | Crismond, DP | 1 | 280 |
| Chou, Pao-Nan | 6 | 74 | Capobianco, Brenda M | 6 | 255 |
| Capobianco, Brenda M | 6 | 255 | Doppelt, Y | 2 | 240 |
| English, Lyn D | 5 | 167 | Margot, KC | 1 | 229 |
| Dori, Yehudit Judy | 5 | 148 | Guzey, Siddika Selcen | 16 | 213 |
| Jong, Morris S Y | 5 | 38 | Moore, Tamara J. | 9 | 211 |
According to Price's law, core authors' minimum number of publications is M 0.749, where M refers to the number of papers, and is the number of documents published by the most prolific author in the statistical year; this means that when the number of papers published by an author is greater than M, that author is a core author in the field; when the total number of documents published by core authors reaches 50 % of all papers, the field forms a core group of authors [27]. As presented in Table 2, is 16, which, when included in the equation, gives M 2.996, indicating that authors with more than three publications are the core authors in the field. A literature analysis revealed that between 1997 and 2023, 67 scholars published three or more papers, totaling 284 papers, accounting for 32.09 % of the sample literature (885 papers). This falls short of 50 %, indicating that there is yet to be a consensus and concerted effort among authors in K-12 EER, presenting significant room for development and potential.
Fig. 3 illustrates the academic collaboration among the authors in the K-12 EER. This indicates that the 542 nodes represent 542 authors, including their collaborative partners. There are 396 connecting lines representing 396 instances of collaboration among authors in this field. Density indicates the strength of cooperation in the network, which was 0.0027. In general, higher density suggests a more vital partnership [24]. The graph demonstrates a relatively low level of collaboration among scholars, and the distribution and scale of the author networks are dispersed and relatively small. Overly independent, small-scale collaborative networks often hinder in-depth research. In conclusion, scholars in the K-12 EER field exhibit insufficiently close collaborative ties, and a mature academic system is yet to be fully formed. This is one of the reasons why the K-12 EER is in its early developmental stage.
Fig. 3.
Visual result of the authors with their collaborative links.
3.1.3. Distribution of country/region
Focusing on the authors’ countries or regions can measure the research output performance of a country or region, reflecting the geographical distribution of the research output [22]. To understand the distribution of the source countries of the K-12 EER articles, we obtained a network map based on author countries/regions using CiteSpace (see Fig. 4). The map contains 68 nodes and 65 connecting lines. Each node represents a country or region, and the node size indicates the number of articles published by that country or region. When two countries or regions collaborate, a line is formed, the thickness of which reflects the degree of collaboration between them [28]. The number of nodes and lines in the graph reveals academic collaborative connections between countries or regions in the K-12 EER (density = 0.0285). Table 3 presents the most published papers in the top 10 countries and regions. Regardless of the number of published papers, mean year, or betweenness centrality, the United States ranks first. This suggests that U.S. scholars were among the earliest to engage in research in the K-12 EER field and have conducted a substantial amount of research in this area. Compared with other countries, the United States has a more extensive body of research outcomes and experiences in K-12 EER. By examining the practices of K-12 engineering education in the United States, other countries can obtain valuable experience and perspectives for reference.
Fig. 4.
Visual mapping of publications by country/region.
Table 3.
Top 10 countries/regions in terms of number of articles issued.
| Number | Country/Region | Number of publications | Centrality | Mean year | Proportion(%) |
|---|---|---|---|---|---|
| 1 | USA | 464 | 0.78 | 1997 | 52.43 % |
| 2 | PEOPLES R CHINA | 74 | 0.08 | 2010 | 8.36 % |
| 3 | TAIWAN | 56 | 0.06 | 1999 | 6.33 % |
| 4 | TURKEY | 43 | 0.04 | 2010 | 4.86 % |
| 5 | ISRAEL | 40 | 0.17 | 2004 | 4.52 % |
| 6 | AUSTRALIA | 39 | 0.04 | 2002 | 4.41 % |
| 7 | SPAIN | 37 | 0.07 | 2010 | 4.18 % |
| 8 | ENGLAND | 29 | 0.14 | 2014 | 3.28 % |
| 9 | SOUTH KOREA | 18 | 0 | 2010 | 2.03 % |
| 10 | NETHERLANDS | 15 | 0.04 | 2006 | 1.7 % |
Notably, although the number of papers from China is far behind that of the United States, it is almost 1.5 times that of the country ranked third. This underscores China's activity in the K-12 EER field, reflecting growing recognition among Chinese scholars of the importance of implementing engineering education in the country's bare education stage. Education is one of the most crucial factors in determining economic, social, and political development [29]. Throughout human history, engineering, along with technology, has played a pivotal role in propelling the development of human society and is a powerful lever for industrial revolution, economic growth, and social progress [30]. The high correlation between K-12 engineering education and the cultivation of creativity, engineering, and technological talent has prompted many countries [1], particularly those leading industrialization, to focus on K-12 engineering education.
Chinese K-12 EER is in its early stages and faces numerous challenges in terms of teacher resources, curriculum development, and availability of resources. However, with the increasing prevalence of evidence-based research, more Chinese scholars focus on K-12 EER. These research outcomes have been reflected in the formulation of China's educational policies. For example, in the 2022 revision of China's compulsory education curriculum standards, several subjects emphasize cultivating students' technological and engineering practical abilities [31,32]. In 2018, Zhejiang Province initiated engineering education in primary and secondary schools. This involved project-based learning practices in primary schools focusing on “design thinking” and compulsory courses in engineering enlightenment education for one academic year in junior high school, along with a certain number of elective course modules. Simultaneously, engineering enlightenment education in compulsory education should coordinate with general technology courses in high schools. High-school engineering enlightenment education relies on general technology courses, offering selectively compulsory technical and engineering series modules.
3.1.4. Distribution of institution
The number of papers published by a research institution reflects its research capabilities, as well as its developmental trajectory, and research achievements [23]. Table 4 lists the institutions leading the development of K-12 EER. Of the 406 institutions in the database, only 10 have published 10 or more papers. Purdue University has the highest number of published papers in this field, and is the most cited institution. Most of the listed institutions are from the United States, highlighting the pioneering contributions of U.S. universities to the K-12 EER field. After using CiteSpace to conduct further analyses of research institutions and their collaboration in the K-12 EER field, a network map of institutional collaboration was created (see Fig. 5). The graph illustrates relatively loose connections among institutions, indicating the absence of a clearly defined collaborative network or center in the K-12 EER. Examining centrality, publication volume, and the graph reveals that the research cluster centered around Purdue University forms an initial collaborative network. Purdue University holds a pioneering position in K-12 EER in the United States, with its School of Engineering Education being the first department in U.S. universities explicitly dedicated to engineering education. One of the departments’ key responsibilities is researching K-12 engineering curricula, standards, and teacher education [33]. Therefore, a comprehensive analysis of the K-12 EER at this university may assist scholars in understanding its development and specific research focal points within K-12 EER.
Table 4.
Top 10 institutions for publications in K-12 EER.
| Number | Institution | Number of Publications | Centrality | Country |
|---|---|---|---|---|
| 1 | Purdue University | 61 | 0.15 | USA |
| 2 | Technion Israel Institute of Technology | 20 | 0.04 | Israel |
| 3 | National Taiwan Normal University | 18 | 0.06 | China |
| 4 | Pennsylvania State University | 15 | 0.02 | USA |
| 5 | Education University of Hong Kong | 14 | 0.02 | China |
| 6 | Tufts University | 14 | 0.03 | USA |
| 7 | Beijing Normal University | 12 | 0.09 | China |
| 8 | Chinese University of Hong Kong | 11 | 0.01 | China |
| 9 | Iowa State University | 10 | 0.03 | USA |
| 10 | University Pennsylvania | 10 | 0.00 | USA |
Fig. 5.
Visual result of the productive with their collaborative links.
3.2. Analysis of research hotspots in K-12 EER
The co-occurrence network of keywords reflects a particular field's hotspots and core research content [18]. Fig. 6 illustrates a keyword co-occurrence network with 463 nodes and 1589 lines, where each node represents a keyword, and the font size of the keyword is proportional to its co-occurrence frequency. In Fig. 6, keywords such as “science,” “education,” “student,” and “STEM education” are associated with larger nodes and closely connected, indicating a shared focus among scholars on these keywords. To demonstrate the interactions among keywords, we employed CiteSpace's log-likelihood ratio algorithm to automatically extract cluster labels through co-citation relationships in the literature, using keywords or noun phrases from cited documents. These clusters help summarize research hotspots, with each cluster considered an independent cohesive research area [18]. CiteSpace employs two metrics to assess clustering effectiveness: Modularity (Q) and Silhouette (S). Q is an evaluation index of network modularity, where a higher Q value indicates better network clustering. Q ranges from 0 to 1, with Q > 0.3 indicating a significant network structure. S measures the network homogeneity, with a higher value indicating greater homogeneity. When S exceeds 0.7, the clustering results are considered highly reliable. If the value is above 0.5, the clustering results are deemed reasonable [18]. It is evident from Fig. 7 that Q = 0.7561 and S = 0.8648 indicate well-structured and highly reliable clusters.
Fig. 6.
Keyword co-occurrence analysis.
Fig. 7.
Clustering of co-words for co-citation clustering.
To maintain clustering clarity, only the top six clusters with high citation counts and homogeneity are displayed in the figure. Table 5 summarizes the information for these six clusters, including cluster ID, cluster label, size, S-value, mean year, and alternative labels. The data in the table indicate that all clusters had S values greater than 0.8, suggesting good clustering consistency. To better derive the research frontiers of the K-12 EER, we carefully read and analyzed the relevant academic literature in each cluster's alternative labels. Based on the associations between the labels of these literature topics, secondary clustering was performed, ultimately identifying four hot topics in the K-12 EER (see Table 6; labels reflecting the development characteristics of EER, such as “k-12 education” and “k-12 engineering education,” are not included in the secondary clustering in Table 6).
Table 5.
Summary of identified co-citation clusters.
| Cluster ID | Cluster label | Size | Silhouette | Mean | Alternative label |
|---|---|---|---|---|---|
| 0 | achievement | 45 | 0.86 | 2013 | Achievement, k-12 engineering education, design, technology, science, engineering curricula |
| 1 | engineering design | 43 | 0.906 | 2014 | engineering design, model, design-based research, design-based learning, design process, classroom, motivation |
| 2 | k-12 education | 43 | 0.878 | 2016 | k-12 education, conception, inquiry, science education, engineering design |
| 3 | 21st century skills | 40 | 0.842 | 2014 | stem integration, interdisciplinary projects, attitude, engineering, student |
| 4 | stem education | 35 | 0.859 | 2014 | stem education, education, student, mathematics, stem integration, literacy |
| 5 | teaching | 29 | 0.837 | 2016 | pedagogical content knowledge, teacher preparation, teaching methods, impact, edp |
Table 6.
Quadratic clustering.
| Cluster ID | Cluster title | Include tags |
|---|---|---|
| 1 | Practical Forms of K-12 Engineering Education | technology, science, mathematics, stem education, stem integration, interdisciplinary, projects engineering curricula |
| 2 | K-12 Engineering Education Centers around Engineering Design | design process, edp, design-based science, design-based learning, inquiry, conception, model |
| 3 | Impact of K-12 Engineering Education on Learners | achievement, 21st century skills, motivation, attitude, engagement, impact, literacy, student |
| 4 | K-12 Engineering Education and Teachers' Professional Development | pedagogical content knowledge, teacher preparation, teaching methods, classroom |
3.2.1. Cluster#1: practical forms of K-12 engineering education
Cluster#1 reflects two forms of engineering education in K-12 education: (a) integrated engineering education and (b) independent engineering education. Initially organized as a systematic process of hands-on and minds-on learning, engineering entered the K-12 curriculum in an integrated form, relying on the content of science and technology courses. However, with global economic development and the recognition of a STEM-trained workforce as a significant driver of economic growth in each country [34], engineering is considered beneficial for workforce preparation, student learning, and life success. Therefore, as nations increasingly emphasize the development of engineering talent, it compels a rethinking of how to fully and effectively implement engineering education requirements in the NGSS. Additionally, it requires a reconsideration of how science, technology, and mathematics teachers apply engineering. Therefore, independent engineering education has begun to emerge. They are independent of the national curriculum system through various STEM and maker competitions, college summer camps, and research activities, providing students with more concentrated engineering education opportunities and time.
-
(a)
Integrated Engineering Education: Engineering courses differ from other subject-based courses, however, have yet to gain formal recognition in school classrooms. Therefore, initially, engineering was included in K-12 education in an integrated manner. After analyzing the literature, we can classify this integration into three chief types: integration with technology education, integration with science education, and integrated STEM education (iSTEM). This is evident as engineering is inherently interdisciplinary [35]. As Cajas [36] mentioned, engineering occurs at the intersection of multiple disciplines (Fig. 8), and solutions to engineering problems involve various aspects of science, technology, society, and economics.
Fig. 8.
Technical and social dimensions of engineering [47].
Regarding integration with technology, the release of the “Standards for Technological Literacy” [37] in 2000 provided a crucial opportunity for reshaping the technology education curriculum to collaborate with engineering and technology education. Similar to engineering education, technology education emphasizes open-ended problem solving and design processes [38]. The National Science Foundation in the United States funded and established the National Center for Engineering and Technology Education (NCETE), which collaborated with nine partner institutions and aimed to integrate engineering design into technology education, reflecting a high level of national attention. Therefore, most of the research purposes in the literature reviewed in this area align with the goals of NCETE, which aim to prepare teachers and students for technology from the perspective of engineering design in K-12 and teacher education.
Concerning integration with science education, the Framework and NGSS released in the United States unprecedentedly integrated the content of the engineering discipline into K-12 education, establishing a new talent development path that combines science and engineering practices. Therefore, some studies have focused on exploring the incorporation of engineering design into K-12 science teaching to support science learning, indicating that teaching models that integrate science and engineering effectively enhance learners' understanding of scientific concepts [1,8]. However, combining science and engineering poses challenges for science teachers, as most do not receive much preparation in engineering. Thus, scholars are beginning to address the integration of engineering design into science teacher education programs to develop teachers’ engineering design knowledge and guide them in incorporating it into science teaching [39,40].
With the advancement of STEM education, scholars have determined that schools often emphasize the integration of “S” (science) and “M” (mathematics), whereas the business and industrial sectors often emphasize the practical skills related to “T” (technology) and “E” (engineering). The contradiction between academic and career needs has prompted governments, scholars, teachers, and business institutions to rethink and reposition STEM education, proposing the concept of “integration,” leading to the emergence of iSTEM. Regarding the U.S. curriculum reform, breaking down subject boundaries, STEM courses with pre-engineering embody the educational philosophy of iSTEM, representing the most suitable form for enhancing students' STEM literacy [41]. Current research in this area includes the development of curriculum design patterns, such as Learning by DesignTM [42,43] and iSTEM PjBL [11,44], and some studies have focused on the impact of iSTEM on students’ career interests, gender, and other aspects [45,46].
-
(b)
Independent Engineering Education: With the development of the times and changes in educational approaches, K-12 engineering education has gradually entered people's vision and classroom practices with an independent identity. In the reviewed literature, some scholars attempt to create engineering-based curriculum materials and projects, making engineering concepts the core of the classroom, such as “Engineering is Elementary” [48], Program History and Overview: Project Lead the Way [49,50]. Another group of studies focuses on Engineering K-12 education outreach programs [51]. Many educators know that students must be exposed to engineering early, however, K-12 educators are not necessarily experts in this field. Therefore, many universities conduct outreach programs in various forms, sending students into K-12 classrooms to introduce engineering disciplines to young children.
However, it is essential to note that these projects may not adhere to standards. Bybee [52] believes that these projects provide a critical entry point for engineering in the school system, and the first step in realizing this opportunity is to clarify their purpose and establish standards. Opportunities to develop engineering education standards have matured from both societal and educational perspectives. However, many challenges remain to be addressed, such as the limited qualifications, number of teachers, and relatively insufficient experience in K-12 engineering education. Moreover, National Academy of Engineering [53] opposes the development of independent national engineering standards, preferring an approach that integrates engineering with other educational standards. Therefore, many states have explicitly defined or integrated engineering standards in science and technology. Scholars have also investigated this approach. For example, in the reviewed literature, Ronald et al. [54] extensively examined and coded the mathematics, science, technology, and career standards of all 50 states to identify instances of engineering content in existing standards. Moore et al. [55] used the Delphi method to propose an engineering education framework suitable for K-12.
3.2.2. Cluster#2: K-12 engineering education centers around engineering design
One of the core principles of engineering education is “design” [56]. The Engineering Design Process (EDP) is an anchor in K-12 education, linking crosscutting concepts in science and mathematics through natural learning environments [41,56]. Since 2011, EDP has become a crucial method for “effective” STEM education [46,57].
Reviewing the relevant literature on keywords such as “design-based science” and “design-based learning” in Cluster#2, we found that these studies focused on the interaction between scientific inquiry and EDP. This is related to the initial form of the EDP entering K-12 education. In the late 20th century, science research using design as a teaching method began to appear, and engineering design and scientific inquiry merged to form a new science teaching approach: design inquiry [35]. The core idea of this method is to consider design as a teaching method that promotes a deeper understanding of core ideas in science. In practice, “design-inquiry” takes various forms, with representative examples being design-based science, design-based learning, and learning by design. This phase laid the foundation for engineering design in K-12 education, providing students with more learning opportunities related to science and technology.
Subsequently, to address the long-standing absence of K-12 engineering education and cultivate engineering talent, the United States successively issued reports and documents [53,58] related to engineering education. These reports explicitly viewed engineering design as a disciplinary practice guiding the development of engineering education projects and courses. Design plays an important role in disciplinary practice and has become critical to STEM education. In 2012 and 2013, with the release of the Framework and NGSS, it was explicitly stated that “science and engineering practices” should be considered one of the three dimensions of K-12 science education and that engineering design should be considered a core disciplinary idea. The 2018 release of Science and Engineering for Grades 6–12: Investigation and Design at the Center [59] further emphasized that engineering education should be designed at its core. This development formally introduced engineering design into K-12 science education, endowing it with a dual identity of “disciplinary practice” and “core idea,” providing students with more opportunities for in-depth learning in science and technology.
Further examination of the literature in this cluster revealed that, in addition to entering classrooms as a teaching method, EDP could also serve as a domain for discovery and exploration before formal instruction in science and mathematics [56]. Most studies have used quasi-experimental designs. For example, Lin et al. [60] explored the integration of the engineering design process into STEM learning using a quasi-experimental design to examine its effects on the engineering design thinking of pre-service technology teachers. Using a quasi-experimental design and qualitative analysis, Xu et al. [61] determined the impact of EDP integration with a maker education intervention on elementary students' design thinking. However, the literature reveals a limited number of studies examining students’ understanding of EDP and assessing changes or improvements in design skills. In courses using EDP, teachers may only organize teaching using its process, overlooking whether students understand the process and whether there are changes in their design skills [56].
3.2.3. Cluster#3: impact of K-12 engineering education on learners
Current research generally recognizes that many teaching strategies and projects adopt a design environment when acquiring knowledge and engaging in practical activities, creating a learning atmosphere with depth and inspiration, leading to better learning outcomes for learners [57,62]. These methods emphasize providing guidance that directs students toward successful design outcomes, thereby profoundly stimulating process-oriented learning and significantly enhancing their skill levels and practical experience. Several methods have been developed for the classification of learning outcomes, among which one widely accepted and used classification targets three overlapping domains: a) the cognitive domain focuses on thinking abilities, including memory, understanding, application, analysis, synthesis, and evaluation; b) the affective domain encompasses sensory perceptions, interests, attitudes, emotions, and values; and c) the psychomotor domain involves imitation, manipulation, precision, articulation, and naturalization [63]. Based on this classification, a detailed analysis of the keywords and literature included in Cluster#3 reveals that current research on the impact of K-12 engineering education on learners primarily includes cognitive and affective aspects.
In the cognitive domain, multiple studies have indicated that engineering education can effectively motivate students to acquire scientific or mathematical knowledge. Some studies suggest that engineering activities can cultivate 21st-century skills in students, such as creativity [1], problem-solving abilities [64], teamwork, and communication [65,66]. In the affective domain, some studies argue that engineering activities can stimulate learners’ interest in learning and enhance their STEM motivation, shaping the correct values, particularly when students can complete the engineering process through prototype design, testing, and communicating their designs [[66], [67], [68]].
Furthermore, by reviewing the literature on the impact on learners, we found a relatively uniform research approach for assessing the effectiveness of K-12 engineering education. In the cognitive domain, most studies have adopted descriptive or quasi-experimental research designs that measure specific content knowledge or abilities through pre- and post-tests. More qualitative research methods are used in the emotional field, such as ethnographic methods based on classroom field notes and focus group interview methods based on interviews [69,70].
3.2.4. Cluster#4: teacher preparation and professional development in engineering education content
In the K-12 education system, standards and teacher guidance have been established for science, mathematics, and technology. However, despite integrating engineering into STEM education, the lack of clarity regarding its form and mode of operation, and clear academic standards, lead to insufficient guidance for teachers' professional development. With the Framework and NGSS emphasizing the integration of engineering design into K-12 science teaching, pre-service and in-service science teachers are tasked with integrating engineering into classroom practices. However, a comparative analysis revealed that this cluster's literature needs to be more extensive, indicating inadequate focus on this research direction. Reviewing the literature on teachers' professional development, this study primarily focuses on the pre-service science teacher field, addressing the organizational forms of teachers' professional development programs, their impact on teachers, and the factors facilitating program effectiveness.
Regarding the organizational forms of teachers' professional development programs, multiple studies have employed a theoretical framework in which teachers are considered learners. These measures included field visits, face-to-face seminars, and follow-up strategies [40]. These studies aimed to strengthen our understanding of the role of engineering in science and society through direct interactions between pre-service science teachers and engineers, further involving them in the engineering design process. This helps to clarify the similarities and differences between science and engineering, building confidence in integrating engineering into science education. Teachers experience vertical development through collaborative completion of engineering designs with students, interactions with experts, reflective discussions, and feedback from various sources. A recent study by Mumba et al. [40] introduced an “engineering design integrated science teaching model” (EDIS) that significantly influenced the development of EDIS teaching resources and practices for pre-service science teachers, successfully improving their understanding and learning outcomes of the engineering design process with middle school students. Other studies promote pre-service science teachers’ understanding of engineering design and integrate engineering practices into science education through engineering-design-based STEM courses [71,72].
Regarding the impact on teachers, most studies adopted small-scale case study methods and found positive changes in pre-service teachers' conceptual knowledge, teaching understanding, and emotional attitudes before and after the intervention. After specific courses or training, pre-service teachers can articulate the overall process and characteristics of engineering design, differentiate between science, technology, and scientific inquiry, enhance their understanding of engineering, and overcome stereotypical impressions of engineering and engineers [[73], [74], [75]]. Almost all studies emphasize that through specific teachers’ professional development programs, pre-service science teachers transform from initially lacking confidence to having positive expectations for design and obtaining confidence in the process.
Studies on factors promoting the effectiveness of teachers' professional development needs to be more extensive and is primarily scattered across these two aspects. These factors include context-based activities, peer assistance and collaboration, student interest, progress, and assessment of teaching improvements. Only a few studies have independently explored the impact of engineering teaching self-efficacy and external support on teachers’ professional development.
3.3. Future research trends in K-12 EER
The dynamic characteristics of research topics are reflected by a sudden increase in keywords or cited references [24]. Analyzing keywords that experience a sudden increase at a specific time can help identify research trends at different stages [18]. Fig. 9 illustrates the top 10 emerging keywords in the K-12 EER, including burst strength, start year, end year, and duration.
Fig. 9.
Top 10 keywords with the strongest citation bursts.
Analyzing the start and end times of the burst keywords provides insights into the development trajectory of the K-12 EER field. Keywords such as “science education,” “k-12 education,” “inquiry,” and “design” began to emerge before 2011, consistent with our previous analysis that engineering education initially used engineering design in conjunction with scientific inquiry in K-12 science education. This may be because scientific inquiry helps students understand scientific concepts and grasp their regularities. However, engineering design, through activities such as product design, manufacturing, and testing, continuously strengthens students’ understanding of scientific concepts and their application in the real world. From 2011 to 2020, the emergence of keywords such as “STEM integration,” “technology,” “motivation,” “performance,” and “gender” indicates that during this period, K-12 engineering education expanded beyond integration with science education. Scholars have begun to focus on integrating engineering education with STEM or technology education, and have conducted studies on the effectiveness of engineering education.
When a keyword continues to experience a burst period, it usually represents the forefront of the research and latest research hotspots. The graph in Fig. 9 demonstrates that the “nature of engineering (NOE)” began to appear in 2020 and continues to remain in the burst stage, indicating that research related to NOE will become a research hotspot in current and future periods.
A review of NOE-related literature identified that NOE originated from a series of studies investigating students' and teachers' perceptions of engineers and engineering [66,76]; these studies raise concerns about how students currently understand the engineering discipline. For example, in its 2008 report Changing the Conversation [77], the NAE concluded that most students had a “limited understanding of what engineers do.” Similarly, Fralick et al. [78] concluded that “students’ views of engineering are inaccurate,” and this situation “may be a factor in students choosing not to consider engineering as a career option.” The most common misconception highlighted by these studies is that people tend to view engineers primarily as manual laborers (such as auto mechanics and construction workers). Many scholars advocate that K-12 students and teachers better understand the structure of the engineering discipline [66,76,79].
Therefore, as the ongoing discussion surrounds the content that should be included in university preparatory engineering education, an increasingly interesting area is the role of NOE in STEM teaching. Similar to the nature of science (NOS), NOE is a collection of related concepts involving what engineering is, how engineering work is conducted, the status of engineering in society, and the relationship between engineering and other related fields such as science [76]. Understanding NOE is essential to engineering literacy, as the NOS is crucial to scientific literacy [80,81]. Moreover, it is helpful for teachers because understanding the NOE can help them create authentic student engineering experiences [82].
In recent years, several educational scholars have attempted to describe and clarify the various dimensions of NOE [1,66,83]. Broadly, these frameworks provide similar explanations for NOE, although there are differences in the details, focus areas, and concept categories. Tools related to NOE have been developed based on these different frameworks [[83], [84], [85]], and more tools will continue to emerge.
4. Discussion
Although some literature reviews have identified research trends in K-12 engineering education and highlighted the practical approach and value of engineering education, they have yet to provide an up-to-date representation of the latest research frontiers in this area. This is primarily owing to the limited scope and number of studies examined in these reviews. For this purpose, this study provides an overview of the latest research structure and content of K-12 EER based on practical results, provides a high-level overview of the relationship between the research field and other related fields through cluster analysis, and clarifies topics that can be further explored in future research.
4.1. Learning progression promotes systemic integration of K-12 engineering education
In Cluster#1, we observed both independent and integrated forms of K-12 engineering education. Despite enthusiasm for independent engineering education, we must critically assess the constraints and influences of practical conditions on its effectiveness. K-12 engineering education is in its early stages, with nascent developments in terms of quantity, form, and quality. Concurrently, K-12 education faces curriculum “overflow” situations [52]. Independent engineering education activities outside the national curriculum system, from government-led and commercially driven to those initiated by universities or K-12 schools, present a diverse but uneven landscape. Therefore, current K-12 engineering education projects require systematic integration to infuse engineering and design concepts into other subjects, forming convenient integrated STEM courses. The learning progression (LP) theory provides opportunities for systematic integration. LP are statements focusing on progressive and incremental learning in a subject area [86], providing a framework for the core concepts of K-12 engineering education and facilitating the integration of teaching and learning objectives into existing teaching frameworks. However, progress in K-12 engineering education LP research could be faster, lacking a universal definition [86], potentially leading to the marginalization of engineering in core subjects. Thus, further academic research and curriculum development are required to promote the systematic integration of K-12 engineering education at all stages.
4.2. Developing measurement tools to comprehensively assess the impact of K-12 engineering education on learners
Through the examination of Cluster#3, we determined that research on the impact of engineering education on learners has focused on cognitive and affective aspects, however, is yet to progress in the psychomotor domain. This may be owing to the need for more suitable measurement tools, as developing multiple-choice tests to assess knowledge and understanding is relatively easy. In contrast, tests measuring higher-order thinking skills pose significant challenges. Psychomotor assessment aims to evaluate students' ability to actively apply knowledge while performing specific tasks [87]. K-12 engineering education requires students to actively identify or formulate engineering problems; prepare, plan, and evaluate solutions to problems; and engage in practical activities centered on engineering design. The literature suggests that although many science teachers integrate design into science teaching through engineering projects, they may overlook providing students with experiences of the critical elements of design, as most teachers view engineering only as a learning process or use engineering challenges to create learning environments, focusing less on students' involvement in design and design reasoning discourses [88]. Therefore, there is an urgent need to assess students' psychomotor skills. In the current practice context, we need to find effective methods to support students’ more profound understanding of design and develop new measurement tools to comprehensively assess the impact of K-12 engineering education on learners. This effort will help ensure the sustained effectiveness of K-12 engineering education, providing students with better preparation to meet the evolving challenges of 21st-century society and professions.
4.3. Expanding the research horizons of teachers’ professional development in K-12 engineering education
Teachers' understanding of engineering is critical in enhancing students' interest in science, mathematics, engineering, and academic achievement [58,65]. With increasing emphasis on integrating engineering into K-12 teaching, teachers' professional development programs focused on engineering are becoming increasingly important [66]. Cluster#2 results indicate that teaching strategies based on EDP can systematically organize learning resources to achieve engineering education goals. Cluster#3 indicates that engineering education can be an effective form of education that comprehensively affects students' cognition, behavior, and emotions. Cluster#4 results demonstrate that teachers' professional development programs can support in-service science, mathematics, and technology teachers in integrating engineering into classroom teaching, which is consistent with the findings of Moore et al. [55]. However, the results highlight development opportunities for implementing K-12 EER in teacher education. Current research focuses more on pre-service teachers, whereas in-service teachers have more teaching knowledge and experience and more opportunities to implement engineering education in actual educational practices. Therefore, for the long-term success of implementing engineering in formal school settings, teachers must understand the role of engineering in education, and the methods required to integrate engineering into existing teaching. Implementing K-12 engineering education teachers’ professional development for in-service teachers is essential and urgent.
Moreover, to strengthen the understanding of teachers' professional development, further development of tools to describe the changes in pre-service teachers before and after participating in projects is required. Based on various data collection tools such as students' comprehensive performance and changes in teachers' knowledge and practice, triangulation can provide scholars with a more thorough understanding of the long-term impact of projects on teachers' professional development, reveal the interactions between teachers and projects at different stages, and discover the possibilities of suitable models for post-service teachers to develop engineering education professionally. Through these in-depth and expanded research directions, we expect to comprehensively understand the process and impact of teachers’ professional development in K-12 engineering education, thus providing a more in-depth and comprehensive perspective for future educational research.
5. Conclusion
Since the 1970s, engineering education has been considered an essential component of K-12 education in many countries [89]. The significant increase in the literature on the K-12 EER within the past decade indicates the close attention paid to the field. At this critical intersection between theoretical research and educational practice, we must conduct a comprehensive tracking and visual analysis of K-12 EER to help relevant scholars and practitioners understand the latest research results on K-12 EER more comprehensively and promptly. In this study, CiteSpace (6.1. R6) was used to conduct a visual analysis of 885 K-12 EER-related journal papers included in WoSCC, and a K-12 EER visualization map was drawn to examine the essential characteristics, research hotspots, and trend evolution in this field. This study yielded the following findings.
-
(1)
By analyzing the primary contributing authors, countries, and institutions in the K-12 EER, we clarified the primary distribution of K-12 EER. The K-12 EER is rapidly growing, indicating sustained global interest and rapid accumulation of achievements in K-12 EER. The results revealed that the U.S. is the major contributor to this field, with publication output far surpassing that of other countries, highlighting its significant role in advancing the K-12 EER. Moreover, we observed dispersed characteristics of the K-12 EER among countries, institutions, and authors, indicating the need for interdisciplinary collaboration. Therefore, strengthening cooperation among regions and authors is critical for future development, fostering stronger partnerships between scholars and institutions to enhance knowledge exchange and collaborative research, and exploring factors supporting the development of the K-12 EER in the U.S., which can provide valuable insights for other countries with similar concerns regarding engineering talent development and promoting the development of the K-12 EER.
-
(2)
Using keyword and cluster analysis, we determined that the current hotspots in K-12 EER primarily focus on four aspects: practical forms of K-12 engineering education, the central role of engineering design, the impact on learners, and teachers' professional development. These four aspects have led to considerable research, and research methods have gradually transitioned from descriptive to quantitative. Initially, research primarily comprised descriptive studies. However, with the increasing demand for evidence-based educational practices, scholars are paying more attention to empirical research on the learning outcomes of K-12 EER. Although the research methods primarily involve quasi-experimental studies, there are also some qualitative research designs. This trend reflects the K-12 EER shift toward data-driven decision making.
-
(3)
Through burst detection, we revealed future research trends in K-12 EER, with more scholars exploring NOE. Studies will continue to advance in developing and refining NOE frameworks and evaluating NOE tools and teaching methods. The theoretical conceptualization of NOE is crucial for developing engineering and scientific literacy, particularly for exploring the overall conceptualization of NOE frameworks, clarifying the various dimensions of NOE, and promoting student and teacher understanding of NOE.
6. Limitation and future research
Although this study provides valuable insights, it has certain limitations. The bibliometric analysis provides a macro-level assessment and does not examine specific areas of K −12 EER research. Subsequent research should explore an integrated approach that incorporates a systematic review to evaluate the literature comprehensively. Previous scholars have successfully used meta-analysis methods to conduct related studies in engineering education [17,90], which provide us with beneficial guidance. Meta-analysis methods contribute to obtaining precise results at the micro level, focusing on deriving the actual relationships between research objects by integrating and analyzing data from similar studies. Literature reviews and meta-analyses complement each other, proving to be highly beneficial for the in-depth exploration of a problem, evaluating existing research, and proposing future research directions. Additionally, cluster analysis was limited to examining the initial six cluster samples, indicating a certain degree of subjectivity. Future studies should conduct more in-depth literature analyses and research reviews based on this foundation to better understand the research trajectory. Finally, our study only considered literature results from the WOSCC, which restricts further exploration of other databases. Therefore, future research should expand the scope of the databases to enhance the comprehensiveness of the study by increasing the sample size.
Declaration of several items of PRISMA2020 were not applicable in our study
-
1.
The authors only followed the steps in scrutiny and do not conduct systematic literature review as per PRISMA methodology. That is why items 2&24&25 was not applicable.
-
2.
The method section of this study describes how we conducted a systematic review with reference to the PRISMA guidelines. The flow chart (see Fig. 1) shows our review steps more intuitively and clearly. The method used is bibliometric analysis, using the CiteSpace quantitative analysis tool. Due to the limitations of the research method, our analysis of the included literature was mainly a macro-evaluation, which did not allow an assessment of the individual value of each article. That is why items 10–15&17-22 were not applicable.
Ethical statement
Review and/or approval by an ethics committee was not needed for this study because no animal or human participants were involved.
Data availability statement
All data associated with this study wasn't deposited into a publicly available repository, because it will be made available on request from the corresponding author.
Funding
This work was supported by the Social Science Fund Project of Jilin Province in 2022 (Doctoral and Youth Support Project) ‘Practical Research on Cultivating Interdisciplinary Teaching Literacy of Normal Science Students under the Background of New Curriculum Reform’ (2022C90).
CRediT authorship contribution statement
Dongxue Jin: Writing – review & editing, Writing – original draft, Conceptualization. Min Jian: Writing – original draft, Visualization, Validation, Software, Formal analysis, Conceptualization. Siqi Liu: Writing – original draft.
Declaration of competing interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:Dongxue Jin reports financial support was provided by The Social Science Fund Project of Jilin Province in 2022 (Doctoral and Youth Support Project) Practical Research on Cultivating Interdisciplinary Teaching Literacy of Normal Science Students under the Background of New Curriculum Reform (2022C90). If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
Not available.
Contributor Information
Dongxue Jin, Email: jindongxue1108@163.com.
Min Jian, Email: jianmin06062021@163.com.
Siqi Liu, Email: sqiliu2024@163.com.
References
- 1.Barak M., Ginzburg T., Erduran S. Nature of engineering. Sci. Educ. 2022 doi: 10.1007/s11191-022-00402-7. [DOI] [Google Scholar]
- 2.Rossman G., Yore L., Hand B., Shelley M. Stitching the pieces together to reveal generalised patterns: systematic research reviews, secondary Re-analyses, case-to-case comparisons, and meta-syntheses of qualitative research studies. Qual. Res. Lit. Sci. Educ. Int. Perspect. Gold Stand. 2009:575–601. [Google Scholar]
- 3.National Research Council[NRC] National Academies Press; Washington, DC: 1996. National Science Education Standards. [DOI] [Google Scholar]
- 4.American Association for the Advancement of Science[AAAS] Oxford University Press; New York: 1993. Benchmarks for Science Literacy.http://www.project2061.org/publications/bsl/online/index.php [Google Scholar]
- 5.National Research Council . Illustrated edition. The National Academies Press; Washington, D.C: 2012. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. [Google Scholar]
- 6.NGSS Lead States . The National Academies Press; Washington, DC: 2013. Next Generation Science Standards: for States, by States. [Google Scholar]
- 7.Gunckel K.L., Tolbert S. The imperative to move toward a dimension of care in engineering education. J. Res. Sci. Teach. 2018;55:938–961. doi: 10.1002/tea.21458. [DOI] [Google Scholar]
- 8.Driessen E.P., Wilhelm J., Cole M., Dunn A., Anderson K. The impacts of two curricula on middle-level students' engineering understanding. J. Educ. Res. 2023;116:1–16. doi: 10.1080/00220671.2022.2158156. [DOI] [Google Scholar]
- 9.Christian K.B., Kelly A.M., Bugallo M.F. NGSS-based teacher professional development to implement engineering practices in STEM instruction. Int. J. STEM Educ. 2021;8:21. doi: 10.1186/s40594-021-00284-1. [DOI] [Google Scholar]
- 10.Mendoza Díaz N.V., Cox M.F. An overview of the literature: research in P-12 engineering education. Adv. Eng. Educ. 2012;3 [Google Scholar]
- 11.Chung C.-C., Huang S.-L., Cheng Y.-M., Lou S.-J. Using an iSTEAM project-based learning model for technology senior high school students: design, development, and evaluation. Int. J. Technol. Des. Educ. 2022;32:905–941. doi: 10.1007/s10798-020-09643-5. [DOI] [Google Scholar]
- 12.Watkins J., McCormick M., Wendell K.B., Spencer K., Milto E., Portsmore M., Hammer D. Data-based conjectures for supporting responsive teaching in engineering design with elementary teachers. Sci. Educ. 2018;102:548–570. doi: 10.1002/sce.21334. [DOI] [Google Scholar]
- 13.Lancaster M., Luo Y., Strobel J., Keyword Field. San Antonio, Texas. 2012. Social network analysis trends for K-12 engineering education research. 25.872.1-25.872.16. [DOI] [Google Scholar]
- 14.Sneider C., Ravel M. Insights from two decades of P-12 engineering education research. J. Pre-Coll. Eng. Educ. Res. J-PEER. 2021;11 doi: 10.7771/2157-9288.1277. [DOI] [Google Scholar]
- 15.MacCoun R.J. Biases in the interpretation and use of research results. Annu. Rev. Psychol. 1998;49:259–287. doi: 10.1146/annurev.psych.49.1.259. [DOI] [PubMed] [Google Scholar]
- 16.Chen C. CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature. J. Am. Soc. Inf. Sci. Technol. 2006;57:359–377. doi: 10.1002/asi.20317. [DOI] [Google Scholar]
- 17.Marchand Martella A., Yatcilla J.K., Park H., Marchand-Martella N.E., Martella R.C. Investigating the active learning research landscape through a bibliometric analysis of an influential meta-analysis on active learning. SN Soc. Sci. 2021;1:1–24. doi: 10.1007/s43545-021-00235-1. [DOI] [Google Scholar]
- 18.Chen C. Nova Science Pub Inc; New York: 2016. CiteSpace: a Practical Guide for Mapping Scientific Literature. [Google Scholar]
- 19.Page M.J., McKenzie J.E., Bossuyt P.M., Boutron I., Hoffmann T.C., Mulrow C.D., Shamseer L., Tetzlaff J.M., Akl E.A., Brennan S.E., Chou R., Glanville J., Grimshaw J.M., Hróbjartsson A., Lalu M.M., Li T., Loder E.W., Mayo-Wilson E., McDonald S., McGuinness L.A., Stewart L.A., Thomas J., Tricco A.C., Welch V.A., Whiting P., Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi: 10.1136/bmj.n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Freire R., Nicol C.J. A bibliometric analysis of past and emergent trends in animal welfare science. Anim. Welf. 2019;28:465–485. doi: 10.7120/09627286.28.4.465. [DOI] [Google Scholar]
- 21.Wang B., Pan S.-Y., Ke R.-Y., Wang K., Wei Y.-M. An overview of climate change vulnerability: a bibliometric analysis based on Web of Science database. Nat. Hazards. 2014;74:1649–1666. doi: 10.1007/s11069-014-1260-y. [DOI] [Google Scholar]
- 22.Bicheng D., Adnan N., Harji M.B., Ravindran L. Evolution and hotspots of peer instruction: a visualized analysis using CiteSpace. Educ. Inf. Technol. 2023;28:2245–2262. doi: 10.1007/s10639-022-11218-x. [DOI] [Google Scholar]
- 23.Wang W., Lu C. Visualization analysis of big data research based on Citespace. Soft Comput. 2020;24:8173–8186. doi: 10.1007/s00500-019-04384-7. [DOI] [Google Scholar]
- 24.Wang S., Chen Y., Lv X., Xu J. Hot topics and fontier evolution of science education research: a bibliometric mapping from 2001 to 2020. Sci. Educ. 2023;32:845–869. doi: 10.1007/s11191-022-00337-z. [DOI] [Google Scholar]
- 25.de Solla Price D.J. In: Commun. Sci. Eng., Heath Lexington Book, Lexingto. Pollock D.K., Nelson C.E., editors. 1970. Citation measures of hard science, soft science, technology, and nonscience; pp. 3–22. [Google Scholar]
- 26.Small H. Characterizing highly cited method and non-method papers using citation contexts: the role of uncertainty. J. Informetr. 2018;12:461–480. doi: 10.1016/j.joi.2018.03.007. [DOI] [Google Scholar]
- 27.Price D.J.D.S. Columbia University Press; New York: 1963. Little Science, Big Science. [DOI] [Google Scholar]
- 28.Li Y., Xiao Y. Authorship and topic trends in STEM education research. Int. J. STEM Educ. 2022;9:62. doi: 10.1186/s40594-022-00378-4. [DOI] [Google Scholar]
- 29.Brown P., Lauder H. In: Deregulierung Arb. — Plur. Bild. Bolder A., Heinz W.R., Kutscha G., editors. VS Verlag für Sozialwissenschaften; Wiesbaden: 2001. Education, globalization and economic development; pp. 31–53. [DOI] [Google Scholar]
- 30.Fremdling R. Industrial revolution and scientific and technological progress. Jahrb. Für Wirtsch. Econ. Hist. Yearb. 1997;38:147–168. doi: 10.1524/jbwg.1997.38.2.147. [DOI] [Google Scholar]
- 31.Ministry of Education of the People’s Republic of China[MOE] Beijing Normal University Press; 2022. Compulsory Education Physics Curriculum Standards. [Google Scholar]
- 32.Ministry of Education of the People’s Republic of China[MOE] Beijing Normal University Press; 2022. Compulsory Education Science Curriculum Standards. [Google Scholar]
- 33.Welty K., Katehi L., Pearson G., Feder M. Analysis of K 12 engineering education curricula in the United States—a preliminary report. 2008. https://peer.asee.org/analysis-of-k-12-engineering-education-curricula-in-the-united-states-a-preliminary-report 13.202.1-13.202.11.
- 34.National Academy of Engineering[NAE] National Academies Press; Washington, D.C.: 2014. Making a World of Difference: Engineering Ideas into Reality. [DOI] [Google Scholar]
- 35.Purzer S., Quintana-Cifuentes J.P. Integrating engineering in K-12 science education: spelling out the pedagogical, epistemological, and methodological arguments. Discip. Interdiscip. Sci. Educ. Res. 2019;1:13. doi: 10.1186/s43031-019-0010-0. [DOI] [Google Scholar]
- 36.Cajas F. The science/technology interaction: implications for science literacy. J. Res. Sci. Teach. 2001;38:715–729. doi: 10.1002/tea.1028. [DOI] [Google Scholar]
- 37.International Technology Education Association . Author; Reston, VA: 2000. Standards for Technological Literacy: Content for the Study of Technology. [Google Scholar]
- 38.Hailey C., Becker K., Thomas M., Erekson T. National Center For Engineering And Technology Education. 2005:10.957.1–10.957.13. https://peer.asee.org/national-center-for-engineering-and-technology-education [Google Scholar]
- 39.Mesutoglu C., Baran E. Integration of engineering into K-12 education: a systematic review of teacher professional development programs. Res. Sci. Technol. Educ. 2021;39:328–346. doi: 10.1080/02635143.2020.1740669. [DOI] [Google Scholar]
- 40.Mumba F., Rutt A., Bailey R., Pottmeyer L., van Aswegen R., Chiu J., Ojeogwu J. A model for integrating engineering design into science teacher education. J. Sci. Educ. Technol. 2023 doi: 10.1007/s10956-023-10055-y. [DOI] [Google Scholar]
- 41.Tech Virginia, Grubbs M., Strimel G. West Virginia university, engineering design: the great integrator, J. STEM teach. Educ. Next. 2015;50 doi: 10.30707/JSTE50.1Grubbs. [DOI] [Google Scholar]
- 42.Fan S.-C., Yu K.-C., Lin K.-Y. A framework for implementing an engineering-focused STEM curriculum. Int. J. Sci. Math. Educ. 2020;19:1–19. doi: 10.1007/s10763-020-10129-y. [DOI] [Google Scholar]
- 43.Wahono B., Chang C.-Y., Khuyen N.T.T. Teaching socio-scientific issues through integrated STEM education: an effective practical averment from Indonesian science lessons. Int. J. Sci. Educ. 2021;43:2663–2683. doi: 10.1080/09500693.2021.1983226. [DOI] [Google Scholar]
- 44.Chang C.-C., Chen Y.-K. Educational values and challenges of i-STEM project-based learning: a mixed-methods study with data-transformation design. Front. Psychol. 2022;13 doi: 10.3389/fpsyg.2022.976724. https://www.frontiersin.org/articles/10.3389/fpsyg.2022.976724 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Hermans S., Gijsen M., Mombaers T., van Petegem P. Gendered patterns in students' motivation profiles regarding iSTEM and STEM test scores: a cluster analysis. Int. J. STEM Educ. 2022;9:67. doi: 10.1186/s40594-022-00379-3. [DOI] [Google Scholar]
- 46.McLure F.I., Tang K.-S., Williams P.J. What do integrated STEM projects look like in middle school and high school classrooms? A systematic literature review of empirical studies of iSTEM projects. Int. J. STEM Educ. 2022;9:73. doi: 10.1186/s40594-022-00390-8. [DOI] [Google Scholar]
- 47.Dixon J.R. McGraw-Hill; New York: 1966. Design Engineering: Inventiveness, Analysis and Decision Making.http://archive.org/details/designengineerin0000dixo [Google Scholar]
- 48.Kelly G.J., Cunningham C.M. Epistemic tools in engineering design for K-12 education. Sci. Educ. 2019;103:1080–1111. doi: 10.1002/sce.21513. [DOI] [Google Scholar]
- 49.Dixon R.A., Brown R.A. Transfer of learning: connecting concepts during problem solving. J. Technol. Educ. 2012;24:2–17. doi: 10.21061/jte.v24i1.a.1. [DOI] [Google Scholar]
- 50.Merrill C., Custer R.L., Daugherty J., Westrick M., Zeng Y. Delivering core engineering concepts to secondary level students, J. Technol. Educ. 2008;20 [Google Scholar]
- 51.Miel K., Swanson R.D., Portsmore M., Paul K.M., Moison E.A., Kim J., Maltese A.V. Characterizing engineering outreach educators' talk moves: an exploratory framework. J. Eng. Educ. 2023;112:337–364. doi: 10.1002/jee.20514. [DOI] [Google Scholar]
- 52.Rogers C.B., Wendell K., Foster J. A review of the NAE report, engineering in K‐12 education. J. Eng. Educ. 2010;99:179–181. doi: 10.1002/j.2168-9830.2010.tb01054.x. [DOI] [Google Scholar]
- 53.National Academy of Engineering [NAE] National Academies Press; Washington, D.C.: 2010. Standards for K-12 Engineering Education?Committee on Standards for K-12 Engineering Education. [DOI] [Google Scholar]
- 54.Carr R.L., Bennett I.V.L.D., Strobel J. Engineering in the K-12 STEM standards of the 50 U.S. States: an analysis of presence and extent. J. Eng. Educ. 2012;101:539–564. doi: 10.1002/j.2168-9830.2012.tb00061.x. [DOI] [Google Scholar]
- 55.Moore T.J., Glancy A.W., Tank K.M., Kersten J.A., Smith K.A., Stohlmann M.S. A framework for quality K-12 engineering education: research and development. J. Pre-Coll. Eng. Educ. Res. J-PEER. 2014;4 doi: 10.7771/2157-9288.1069. [DOI] [Google Scholar]
- 56.Arık M., Topçu M.S. Implementation of engineering design process in the K-12 science classrooms: trends and issues. Res. Sci. Educ. 2022;52:21–43. doi: 10.1007/s11165-019-09912-x. [DOI] [Google Scholar]
- 57.Cunningham C.M., Lachapelle C.P., Brennan R.T., Kelly G.J., Tunis C.S.A., Gentry C.A. The impact of engineering curriculum design principles on elementary students' engineering and science learning. J. Res. Sci. Teach. 2020;57:423–453. doi: 10.1002/tea.21601. [DOI] [Google Scholar]
- 58.National Research Council [NRC] The National Academies Press; 2009. Engineering in K-12 Education: Understanding the Status and Improving the Prospects. [DOI] [Google Scholar]
- 59.Moulding B., Songer N., Brenner K., editors. Science and Engineering for Grades 6-12: Investigation and Design at the Center. National Academies Press; Washington, D.C.: 2019. [DOI] [Google Scholar]
- 60.Lin K.-Y., Wu Y.-T., Hsu Y.-T., Williams P.J. Effects of infusing the engineering design process into STEM project-based learning to develop preservice technology teachers' engineering design thinking. Int. J. STEM Educ. 2021;8:1. doi: 10.1186/s40594-020-00258-9. [DOI] [Google Scholar]
- 61.Xu W., Chen J.-C., Lou Y., Chen H. Impacts of maker education-design thinking integration on knowledge, creative tendencies, and perceptions of the engineering profession. Int. J. Technol. Des. Educ. 2023 doi: 10.1007/s10798-023-09810-4. [DOI] [Google Scholar]
- 62.Van Breukelen D., Van Meel A., De Vries M. Teaching strategies to promote concept learning by design challenges. Res. Sci. Technol. Educ. 2017;35:368–390. doi: 10.1080/02635143.2017.1336707. [DOI] [Google Scholar]
- 63.Rovai A.P., Wighting M.J., Baker J.D., Grooms L.D. Development of an instrument to measure perceived cognitive, affective, and psychomotor learning in traditional and virtual classroom higher education settings. Internet High Educ. 2009;12:7–13. doi: 10.1016/j.iheduc.2008.10.002. [DOI] [Google Scholar]
- 64.Yu K.-C., Wu P.-H., Fan S.-C. Structural relationships among high school students' scientific knowledge, critical thinking, engineering design process, and design product. Int. J. Sci. Math. Educ. 2020;18:1001–1022. doi: 10.1007/s10763-019-10007-2. [DOI] [Google Scholar]
- 65.Johnston A.C., Akarsu M., Moore T.J., Guzey S.S. Engineering as the integrator: a case study of one middle school science teacher's talk. J. Eng. Educ. 2019;108:418–440. doi: 10.1002/jee.20286. [DOI] [Google Scholar]
- 66.Pleasants J., Olson J.K. What is engineering? Elaborating the nature of engineering for K‐12 education. Sci. Educ. 2019;103:145–166. doi: 10.1002/sce.21483. [DOI] [Google Scholar]
- 67.Brophy S., Klein S., Portsmore M., Rogers C. Advancing engineering education in P-12 classrooms. J. Eng. Educ. 2008;97:369–387. doi: 10.1002/j.2168-9830.2008.tb00985.x. [DOI] [Google Scholar]
- 68.Vennix J., den Brok P., Taconis R. Do outreach activities in secondary STEM education motivate students and improve their attitudes towards STEM? Int. J. Sci. Educ. 2018;40:1263–1283. doi: 10.1080/09500693.2018.1473659. [DOI] [Google Scholar]
- 69.Denson C.D. The MESA study. J. Technol. Educ. 2017;29:66–94. doi: 10.21061/jte.v29i1.a.4. [DOI] [Google Scholar]
- 70.Penuel W.R. Studying science and engineering learning in practice, Cult. Stud. Sci. Educ. 2016;11:89–104. doi: 10.1007/s11422-014-9632-x. [DOI] [Google Scholar]
- 71.Aydin Gunbatar S., Oztay E.S., Ekiz Kiran B. Supporting pre-service teachers' integration of engineering into STEM lessons throughout engineering-infused training. Res. Sci. Technol. Educ. 2022;0:1–21. doi: 10.1080/02635143.2022.2121691. [DOI] [Google Scholar]
- 72.Ozkizilcik M., Cebesoy U.B. The influence of an engineering design-based STEM course on pre-service science teachers' understanding of STEM disciplines and engineering design process. Int. J. Technol. Des. Educ. 2023 doi: 10.1007/s10798-023-09837-7. [DOI] [Google Scholar]
- 73.Hanson J.R., Hardman S., Luke S., Lucas B. Developing pre-service primary teachers' understanding of engineering through engineering habits of mind and engagement with engineers. Int. J. Technol. Des. Educ. 2022;32:1469–1494. doi: 10.1007/s10798-021-09662-w. [DOI] [Google Scholar]
- 74.Kuvac M., Koc I. Enhancing preservice science teachers' perceptions of engineer and engineering through STEM education: a focus on drawings as evidence. Res. Sci. Technol. Educ. 2023;41:1539–1559. doi: 10.1080/02635143.2022.2052038. [DOI] [Google Scholar]
- 75.Capobianco B.M., Radloff J., Clingerman J. Facilitating preservice elementary science teachers' shift from learner to teacher of engineering design-based science teaching. Int. J. Sci. Math. Educ. 2022;20:747–767. doi: 10.1007/s10763-021-10193-y. [DOI] [Google Scholar]
- 76.Pleasants J. Rethinking the nature of engineering: attending to the social context of engineering. Sci. Educ. 2023:1–18. doi: 10.1007/s11191-023-00445-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.National Academy of Engineering [NAE] NATIONAL ACADEMIES PRESS; 2008. Changing the Conversation: Messages for Improving Public Understanding of Engineering. [DOI] [Google Scholar]
- 78.Fralick B., Kearn J., Thompson S., Lyons J. How middle schoolers draw engineers and scientists. J. Sci. Educ. Technol. 2009;18:60–73. doi: 10.1007/s10956-008-9133-3. [DOI] [Google Scholar]
- 79.Cheung K.K.C. Exploring the inclusion of nature of science in biology curriculum and high-stakes assessments in Hong Kong. Sci. Educ. 2020;29:491–512. doi: 10.1007/s11191-020-00113-x. [DOI] [Google Scholar]
- 80.Antink-Meyer A., Brown R.A. Nature of engineering knowledge. Sci. Educ. 2019;28:539–559. doi: 10.1007/s11191-019-00038-0. [DOI] [Google Scholar]
- 81.Lederman N.G., Antink A., Bartos S. Nature of science, scientific inquiry, and socio-scientific issues arising from genetics: a pathway to developing a scientifically literate citizenry. Sci. Educ. 2014;23:285–302. doi: 10.1007/s11191-012-9503-3. [DOI] [Google Scholar]
- 82.Antink-Meyer A., Arias A.M. Teachers' incorporation of epistemic practices in K-8 engineering and their views about the nature of engineering knowledge. Sci. Educ. 2022;31:357–382. doi: 10.1007/s11191-021-00265-4. [DOI] [Google Scholar]
- 83.Deniz H., Kaya E., Yesilyurt E., Trabia M. The influence of an engineering design experience on elementary teachers' nature of engineering views. Int. J. Technol. Des. Educ. 2020;30:635–656. doi: 10.1007/s10798-019-09518-4. [DOI] [Google Scholar]
- 84.Antink-Meyer A., Brown R.A. Development of views about the nature of engineering knowledge questionnaire. 2020. https://peer.asee.org/development-of-views-about-the-nature-of-engineering-knowledge-questionnaire
- 85.Kaya E., Deniz H., Yesilyurt E. Toward developing a valid and reliable assessment of adults' nature of engineering views. J. Eng. Educ. 2023;112:634–673. doi: 10.1002/jee.20524. [DOI] [Google Scholar]
- 86.Bonsall A., Bianchi L., Hanson J. A scoping literature review of learning progressions of engineering education at primary and secondary school level. Res. Sci. Technol. Educ. 2022;40:407–430. doi: 10.1080/02635143.2020.1799780. [DOI] [Google Scholar]
- 87.Rahayu W.P., Munadhiroh S. Atlantis Press; 2020. Project-Based Psychomotor Assessment Instruments to Increase Student Competencies in the 21st Century; pp. 342–355. [DOI] [Google Scholar]
- 88.Lie R., Aranda M.L., Guzey S.S., Moore T.J. Students' views of design in an engineering design-based science curricular unit. Res. Sci. Educ. 2021;51:663–683. doi: 10.1007/s11165-018-9813-9. [DOI] [Google Scholar]
- 89.Purzer Ş., Goldstein M.H., Adams R.S., Xie C., Nourian S. An exploratory study of informed engineering design behaviors associated with scientific explanations. Int. J. STEM Educ. 2015;2:9. doi: 10.1186/s40594-015-0019-7. [DOI] [Google Scholar]
- 90.Delen I., Sen S. Effect of design-based learning on achievement in K-12 education: a meta-analysis. J. Res. Sci. Teach. 2023;60:330–356. doi: 10.1002/tea.21800. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All data associated with this study wasn't deposited into a publicly available repository, because it will be made available on request from the corresponding author.









