Abstract
The objective of this investigation was to analyze scientific production assessed by indexed doctoral theses in the Ordered Spanish Theses (TESEO) database, on the topic of the sport of handball in Spain. Productivity was analyzed on the basis of variables grouped by contextual information, methodologies and procedures. Seventy-two indexed theses from between 1976 and 2021 were analyzed. A progressive increase was identified in scientific production based on these theses during this period. The scientific disciplines that presented the highest number of theses were Sport Sciences (n = 33) and Sport Pedagogy (n = 19). The main results show that quantitative approaches are the most common (n = 49), doctoral theses using descriptive studies based on systematic observation represent the majority (73%), the predominant type of data collection was the use of cross-sectional studies (70.8%) versus longitudinal studies (26.4%), and the most used sampling method was that of convenience (n = 65). The results make it possible to ascertain the reality of this research topic, the methodological positioning and research tendencies, and to draw the basic lines for development.
Keywords: handball, review, methodology, doctoral theses, Spain
1. Introduction
At present, the need to transmit scientific knowledge in our society has gained significant importance in different environments (universities, congresses, symposia, etc.), with different academic forms (undergraduate dissertations, master’s theses, and doctoral dissertations), or for dissemination (articles, manuals, and books). The same aspects also exist in sport science research [1].
In Spain, there are institutions that sponsor specialist research groups in different scientific disciplines, develop courses and seminars, and establish programs for the training of young researchers. These institutions that collaborate with research are public and can be national through the Spanish Ministry of Universities by the Order of 6 November 2020, regional, such as those of the Ministry of Economy and Finance of the Government of Extremadura through Decree 56/2021 of 4 June, or through each university’s own grants, also from the Spanish Ministry of Universities through Royal Decree 289/2021 of 20 April. All of them can be accessed through competitive public calls for applications. It is undoubtedly in the institutional context of the university par excellence that the generation of knowledge and the training of individuals at the different academic levels of the degree, graduate studies, master’s degree and doctorate, are encouraged. The doctoral training process culminates with the defense of a doctoral thesis, which implies completing an original scientific project with the aim of showing the scientific community that the doctoral student is capacitated to develop quality research. This is the highest academic level.
One of the indicators that makes it possible to assess a scientific field in general is the production of doctoral theses [2]. It is also one of the indicators of quality for the establishment of rankings of researchers, universities, departments, etc. [3,4]. The completion of a doctoral thesis as well as being a requisite for obtaining the highest academic level, contributes important scientific findings [5]. The analysis of doctoral theses makes it possible to identify the reality of a scientific discipline, its evolution, methodological positioning and research tendencies [1]. Researchers in the different scientific fields develop studies with the aim of updating the progress made in research topics with reviews derived from other studies. There are three formats for these reviews [6]: (i) narrative reviews (a subjective theoretical review of primary studies on a research topic, without empiric contributions from the researcher); (ii) systematic reviews (reviews of primary studies, with a systematic development of the data collection process, where statistical procedures are not used to summarize the studies; (iii) quantitative systematic reviews or meta-analyses (overview of the primary studies with quantitative methodology that contains both a systematic development of the data collection process and the use of statistical methods to summarize the studies).
To answer a concrete research question, systematic reviews collect the main evidence that comply with the previously specified eligibility criteria [7]; thus, they limit the studies to different scientific disciplines and research topics [8], as well as the reality and context of each country [9,10,11], or to a concrete sport discipline [12,13]. It is imperative to delimit the studies in a systematic review to precisely understand the current situation regarding a research topic in the local and international context, generating the development of comparative studies among countries [1]. In this regard, the study by Ibáñez and Feu [14], which presents a comparative review of the scientific literature on the sport of basketball between Spain and Portugal, should be highlighted. Moreover, there are studies that develop reviews on research in the field of game analysis [15,16]. Similarly, other analyses focus on systematic reviews on a sport discipline such as volleyball [17], soccer [18,19,20], or basketball [21].
In handball, systematic reviews have been identified that specifically address some game action, such as match analysis [22]; the analysis of throwing speed [23]; injury profiles in handball players [24]; or physical and psychological performance factors [25].
The majority of systematic reviews in sport sciences are focused on analyses of topics of scientific interest disseminated in research articles with quality indicators. The searches are based on indexed articles in reference databases (WOS, Scopus, and Pubmed). It cannot be said that there are a large number of systematic reviews of doctoral theses in sport science topics. Yaman and Atay [11] analyzed the development of the production of doctoral theses on physical education and sport. In Spain, we can find reviews on the incidence of women completing and directing doctoral theses in sport sciences [10], with reference to the history of physical education and sport [26], or about the state of research in basketball [12] or the topic of the sport coach [1]. Publications can also be found related to reviews on the production of doctoral theses in other areas of knowledge such as Education Theory [27], Psychology [5], or Psychiatry [28]. The scarcity of systematic reviews in doctoral theses is due to the difficulties that researchers have to access this type of primary documents. However, the scientific knowledge produced ends up being transmitted to society because many Ph.D.s take advantage of the results of their theses to disseminate them partially or in toto in scientific journals.
Spain, as an active member of the processes leading to the creation and development of a European knowledge area, has been incorporating the legislative reforms that have enabled it to consolidate a range of courses in accordance with the principles of the EHEA. Thus, official doctoral studies are regulated by Royal Decree 99/2011 of January 28. A doctoral thesis is an original research work related with one of the fields of knowledge (scientific, humanist, biomedical, social and technical) that the doctoral student will defend in front of a panel in public once they have completed their training in a doctoral program.
In sport sciences, the bibliometric study of doctoral theses is of great interest for analyzing and observing the state of the discipline on which researchers are working, especially if it is desired to consolidate knowledge as it shows a specific development, as well as to include in the body of knowledge, if desired, new constructs and theories that help to better understand the different ambits of the different disciplines [29,30].
In this respect, it must be pointed out that the study of handball, in particular, has seen significant growth in the last two decades. Not only has there been a marked increase in the number of doctoral theses in Spain that have this discipline as their object of study, but also in the organized academic events (congresses, seminars, workshops, forums, etc.), and in the books and research articles published with special mention of the specific scientific journal E-Balonmano com, indexed in the Emerging Sources Citation Index and recently in SCOPUS.
Faced with the scarcity of research that analyzes the reality of a scientific discipline or research topic in a systematic review of the production of doctoral theses, it is imperative to increase this type of study as in other sport disciplines [1]. Thus, the objective of the present study was to analyze the production of doctoral theses in Spain that are centered on handball as their main object of study, bearing in mind the research classification, procedures and relations. The study has therefore been carried out using the analysis of documents [31] included in the Ordered Spanish Theses Database TESEO.
2. Methodology
2.1. Design
The compilation of the reports on progress in research on handball using doctoral theses presented in Spain uses a theoretical research design [6]. It is a systematic review with a methodological development to obtain the data (compilation of studies, specification of variables, cataloging, analysis, etc.). The research followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [7] guidelines determined as: (i) definition of the objectives with explicit and reproducible methodology; (ii) systematic search for evidence following eligibility criteria; (iii) assessment of the validity of the findings; and (iv) systematic presentation and synthesis of the characteristics and findings of the included studies.
2.2. Sample
Data were collected from the TESEO database (Tesis Españolas Ordenadas, https://www.educacion.es/teseo, accessed on 31 January 2021), which is generated by the Spanish Ministry of Education, Culture and Sport, and indexes all the doctoral theses completed and defended in public and private universities since 1976. It is compulsory for newly created Ph.D.s to have the general data on their theses indexed, for the general knowledge of the whole scientific community. This database makes it possible to search using five fields, title, author, identity card number, university and academic year.
2.3. Procedure
The systematic review of scientific literature demands the use of precise procedures and tools that make it possible to present evident scientific contributions [32]. The methodology of this study followed the PRISMA guidelines for systematic reviews [7] (Figure 1), divided into four phases: (I) identification; (II) screening; (III) eligibility; (IV) included.
Figure 1.
Flow diagram based on the PRISMA guidelines [7] of the methodology used in the search for doctoral theses on handball.
During phase I, a total of 60 doctoral theses were initially identified in the database including the term “handball” and limiting the search to the field of “title”, making it possible to identify the object of this review generically. It was found that the number increased to 121 if the abstract was included in the search, and 123 if title and abstract were included. Due to the possibility of defending doctoral theses in Spain in different languages, it was decided to include the term handball in the globally recognized languages of English, German, Portuguese and French, and the languages recognized in the Spanish constitution, Galician, Basque and Catalan, identifying 13 doctoral theses and bringing the number to 135. Two theses were found in Google Scholar that were not indexed in the TESEO database such that the total increased to 137. Doctoral theses are not indexed immediately, such that the database may not be completely up to date. For this reason, the search was made at two time points, on 31 January 2021 and on 30 June 2021, when the search was terminated.
In Phase II, the titles, abstracts and UNESCO descriptors included in each thesis found were read, leading to their screening to eliminate those that were not on the topic of handball, selecting only those in which the authors included handball in their research, and discriminating between the theses that exclusively studied handball and those that included it with other sports. In certain cases, the term “handball” appeared in the abstract as secondary to the study or simply in a reference; thus, precise inclusion and exclusion criteria were determined.
Four inclusion criteria were defined: (i) doctoral theses included in the TESEO database with the term “handball” in the title in globally recognized languages (Spanish, English; German, French and Portuguese) and in the languages recognized by the Spanish Constitution; (ii) handball as the main object of the study, handball as the only sample as the object of study, handball as a sample among other sports; (iii) accessibility to the document in the TESEO database or to the complete thesis if necessary; (iv) produced in the period between 1976 and 2021.
The two exclusion criteria were: (i) the sport of handball was not the object of study and/or the term handball was not included in the thesis title; (ii) failure to access the document in the TESEO database or the complete thesis if necessary.
After applying the inclusion criteria and coding all the doctoral theses, the sample was reduced to 72 documents in which the term handball was included in the thesis title.
In Phase III, the documents retrieved from the TESEO database were analyzed. On occasions, the summary in the database did not contain sufficient information to identify all the variables of the study. In this case, the complete document of the doctoral thesis was sought in Google Scholar and the archive of doctoral theses in the university where it was publicly defended. Once the complete theses had been located, none had to be excluded in this phase.
Finally, in Phase IV, the identified theses were analyzed, according to 13 previously defined variables.
2.4. Variables
Thirteen variables were established divided into contextual, methodological and procedural (Figure 2).
Figure 2.
Diagram of the grouping of the study variables.
Seven contextual variables were coded for each doctoral thesis: (a) author’s name; (b) author’s sex; (c) year of defense; (d) title of thesis; (e) name of director/s; (f) sex of director/; (g) university where defended.
Three variables were used to code the methodological variables: (h) classification of the scientific disciplines defined by Borms [33], in the catalogue in the Directory of Sport Science (5th edition) which are: (i) Adapted Physical Activity, (ii) Biomechanics of Sport, (iii) Coaching Sciences, (iv) Kinanthropometry, (v) Motor behavior, Motor development, Motor control and Motor learning, (vi) Philosophy of sport, (vii) Sociology of sport, (viii) Sport and Exercise Physiology, (ix) Sport and Exercise Psychology, (x) Sport and Leisure Facilities, (xi) Sport History, (xii) Sport Information, (xiii) Sport Law, (xiv) Sport Management, (xv) Sports Medicine, (xvi), Sport Pedagogy; (j) research approaches suggested by Ibáñez et al. [1]: (i) qualitative, (ii) quantitative, (iii) mixed, (iv) intervention; and (k) research methods proposed by Montero and León [34]: theoretical study, (i) classic, (ii) meta- analysis; empirical study with quantitative methodology, (iii) descriptive study using systematic observation, (iv) descriptive study of populations via research by surveys, (v) experiments, (vi) quasi-experiments, (vii) ex post facto studies, (viii) experiments with a single subject, (ix) instrumental studies; empirical qualitative studies: (x) ethnography, (xi) case studies, and (xii) research-action. This variable was coded as having multiple answers as different research methods can be applied.
Finally, three variables were used to code the research procedures: (l) a method of data gathering based on the proposals of Nelson and Silverman [35], Hernández et al. [36] and Polit and Hungler [37]. This variable was coded as having multiple answers, as a thesis can be carried out using different data collection methods. The categories into which this variable was divided were: (i) questionnaire/scale, (ii) interview, (iii) systematic observation, (iv) qualitative observation/field notes, (v) documents, (vi) tests, (vii) discussion groups, (viii) self-report and others identified by the authors, (ix) training program, (x) teaching units, (xi) and electronic devices; (m) for the type of sampling the categorization by Cubo was used [38]: probability: (i) simple random, (ii) stratified random, (iii) systematic random, (iv) random cluster, (v) multistage; and non-probability: (vi) convenience, (vii) intentional, (viii) quasi-probability, (ix) hard to reach populations, (x) by quotas; and (n) depending on the type of information or data that are obtained as proposed by Cubo [39]: (i) cross-sectional, (ii) longitudinal l, or (iii) both if the two possibilities are used in the same study.
A formula for coding was created for each doctoral thesis and stored by date of defense/reading.
2.5. Reliability
Three researchers who knew the document analysis system performed the coding. The reliability of the coding was tested with the free-marginal multirater kappa [40]. Ten per cent of the doctoral theses were analyzed to calculate the agreement among the coders. The score obtained for each variable can be considered “almost perfect” [41], with an average value of 0.95. Four variables were identified with 100% agreement (author’s sex, type of sampling, research methods, type of information) and three variables between 83.33% and 91.67% agreement (research procedure [35,36,37], classification of scientific disciplines following Borms [33] and research approach [1]).
2.6. Data Analysis
A descriptive analysis was made of all the variables recorded in the research (frequency and percentage). An analysis was also made of the multiple answers in the following variables: director/s, sex of director/s; research method; data compilation method. These variables in the contingency tables may surpass 100%, as there are variables with multiple answers. The analysis was completed with contingency tables to identify the relations among the study variables.
3. Results
Seventy-two doctoral theses were identified for this systematic review that mainly focused on handball (Annex 1). Eighty-four point seven percent of the authors were men (n = 61), and 15.3% women (n = 11). This difference is maintained regarding the directors as 82.9% were men and 17.1% were women.
The first doctoral thesis that fulfilled the inclusion criteria coincided with the first one devoted to handball in 1991. The production of doctoral theses on handball shows growing interest in the Spanish university context, as there has been a linear progression in production (R2 = 0.2899), increasing by 28.99% in the period analyzed.
Doctoral theses on handball have only been defended in 39% (n = 34) of Spanish universities. Those that have developed the most are the University of Granada (UGR) and the University of Coruña (UdC) with 9.7% each (n = 7), followed by the University of Lérida (UdL) with 7% (n = 5) and the Universities of Barcelona (UB), Vigo (UVIGO) and Valencia (UV), with 5.5% each (n = 4).
3.1. Methodological Variables
According to the classification of scientific disciplines by Borms [33] as the criterion for classifying doctoral theses on the topic of handball, it can be seen that Coaching Sciences (45.8%) and Sport Pedagogy (26.4%) are the most commonly studied disciplines (Figure 3), dominating the rest.
Figure 3.
Distribution of doctoral theses on handball by scientific discipline.
There are a large number of doctoral theses that use a quantitative research approach (70.84%; n = 51) regarding the topic of handball. This number increases if it is considered that 11.1% (n = 8) use a mixed research approach (qualitative and quantitative). The rest of the investigations employed a qualitative approach (5.6%; n = 4) or included an intervention (12.5%; n = 9).
Following the classification proposed by Montero and León [34], the most commonly used research methods are descriptive with systematic observation (48.1%). They are followed by four less usual methods: quasi-experimental studies (12.3%), descriptive studies of populations via surveys (11.1%); instrumental studies (9.9%), case studies (7.4%) and ex post facto studies (6.2%) (Figure 4).
Figure 4.
Distribution of the research methods used in the doctoral theses on handball.
Figure 5 shows the relations in the research methodological variables among the classification of scientific disciplines, the research approaches and the research methods. The main finding is that quantitative approaches are the most common (n = 49), doctoral theses that use descriptive studies based on systematic observation represent the majority (73%), and Sport Sciences is the discipline that includes the most studies on handball (55.1%).
Figure 5.
Classification of research in doctoral theses on handball.
3.2. Procedural Research Variables
The doctoral theses analyzed mainly used convenience sampling (91.7%; n = 66), one case used simple random sampling and another intentional sampling with identical percentages (1.4%; n = 1). In the remaining cases, the type of sampling was not specified (5%; n = 4).
The doctoral students used different instruments to obtain their data (Figure 6), and sometimes several instruments were used in the same investigation. The most commonly used instrument was systematic observation (27.4%), followed by questionnaires/scales and tests (15.9%), electronic devices (13.3%), interviews (10.6%) and training programs (8%), leaving documents (4.4%,), field notes and discussion groups (1.8%), and self-report (0.9%).
Figure 6.
Distribution of the instruments used in doctoral theses on handball.
The predominant type of data collection was using cross-sectional studies (70.8%; n = 51) compared to longitudinal studies (26.4%). Only one study used both types of data.
Figure 7 shows the relations in the research procedures among methods for data collection, type of sampling and type of information or data. There is a strong predominance of convenience sampling (n = 65) over the rest, with cross-sectional theses being the most frequent (n = 51). In these studies, systematic observation is the most commonly used method for collecting data (n = 18), followed by questionnaires and tests (n = 14 in each case) and electronic devices (n = 11). Theses with longitudinal data collection are also evident although fewer in number (n = 19), with systematic observation being the data gathering method most commonly used (n = 12).
Figure 7.
Cross-matching of research procedures.
3.3. Interaction of the Methodological Variables with the Procedural Variables
Table 1 shows the results relating the methodological and procedural variables bearing in mind the search methods, data collection methods and type of sampling. It should be mentioned that there is a clear predominance of convenience sampling in practically all the search methods, except the classic one, the most frequent studies being descriptive using systematic observation. A small percentage used simple random and intentional sampling.
Table 1.
Interaction of data between classification and research procedures relating the variables of search methods, data collection methods and type of sampling.
| Research Methods | Type of Sampling | Data Collection Instruments | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Questionnaire/Scale | Interview | Systematic Observation | Qualitative Observation/Field Notes | Document | Test | Training Program | Electronic Devices | Discussion Group | Self-Report | |||
| Classic | Not specified | n | 1 | 3 | ||||||||
| % | 33.3% | 100.0% | ||||||||||
| Descriptive study using systematic observation | Convenience | n | 5 | 4 | 26 | 1 | 5 | 8 | 1 | |||
| % | 12.8% | 10.3% | 66.7% | 2.6% | 12.8% | 20.5% | 2.6% | |||||
| Intentional | n | 1 | 1 | |||||||||
| % | 2.6% | 2.6% | ||||||||||
| Not specified | n | 1 | ||||||||||
| % | 2.6% | |||||||||||
| Descriptive study of populations through survey research | Convenience | n | 6 | 3 | 2 | 1 | 1 | 1 | ||||
| % | 66.7% | 33.3% | 22.2% | 11.1% | 11.1% | 11.1% | ||||||
| Intentional | n | 1 | 1 | |||||||||
| % | 11.1% | 11.1% | ||||||||||
| Quasi experiments | Convenience | n | 2 | 1 | 4 | 6 | 2 | |||||
| % | 20.0% | 10.0% | 40.0% | 60.0% | 20.0% | |||||||
| Ex post facto studies | Convenience | n | 1 | 1 | 3 | 2 | 2 | |||||
| % | 20.0% | 20.0% | 60.0% | 40.0% | 40.0% | |||||||
| Instrumental studies | Convenience | n | 4 | 2 | 3 | 2 | 1 | 1 | 1 | |||
| % | 50.0% | 25.0% | 37.5% | 25.0% | 12.5% | 12.5% | 12.5% | |||||
| Not specified | n | 1 | ||||||||||
| % | 12.5% | |||||||||||
| Ethnography | Simple random | n | 1 | 1 | ||||||||
| % | 100.0% | 100.0% | ||||||||||
| Case study method | Convenience | n | 2 | 3 | 2 | 4 | 1 | 3 | ||||
| % | 33.3% | 50.0% | 33.3% | 66.7% | 16.7% | 50.0% | ||||||
Table 2 presents the interaction of data between methodologies and research procedures bearing in mind the variables of type of information, the Borms classification [33] and the data collection instruments. Regarding cross-sectional information, studies in Coaching Sciences are the most prolific (64%, n = 23), followed by studies in Sport Pedagogy (26%, n = 13). Only a few studies correspond to Sport Psychology (8%, n = 4), Sports Medicine and Kinanthropometry (6%, n = 3 in both cases), Sport Biomechanics (4%, n = 2) and Motor Behavior (2%, n = 1).
Table 2.
Interaction of the data between classification and research procedures relating the variables of type of information, the Borms classification [33] and the data collection instruments.
| Type of Information | Borms Classification | Data Collection Instruments | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Questionnaire/Scale | Interview | Systematic Observation | Qualitative Observation/Field Notes | Document | Test | Training Program | Electronic Devices | Discussion Group | Self-Report | |||
| Cross-sectional | Biomechanics of Sport | n | 1 | 2 | ||||||||
| % | 2.0% | 4.0% | ||||||||||
| Coaching Sciences | n | 3 | 3 | 12 | 5 | 3 | 5 | |||||
| % | 6.0% | 6.0% | 24.0% | 10.0% | 6.0% | 10.0% | ||||||
| Motor Behavior | n | 1 | 1 | |||||||||
| % | 2.0% | 2.0% | ||||||||||
| Sport and Exercise Physiology | n | 1 | 1 | |||||||||
| % | 2.0% | 2.0% | ||||||||||
| Kinanthropometry | n | 2 | 3 | |||||||||
| % | 4.0% | 6.0% | ||||||||||
| Sports Medicine | n | 2 | 2 | 2 | ||||||||
| % | 4.0% | 4.0% | 4.0% | |||||||||
| Sport pedagogy | n | 6 | 4 | 4 | 1 | 2 | 1 | 1 | 1 | |||
| % | 12.0% | 8.0% | 8.0% | 2.0% | 4.0% | 2.0% | 2.0% | 2.0% | ||||
| Sport and Exercise Psychology | n | 3 | 1 | 1 | 1 | 1 | ||||||
| % | 6.0% | 2.0% | 2.0% | 2.0% | 2.0% | |||||||
| Total Cross sectional | n | 14 | 8 | 18 | 1 | 14 | 8 | 11 | 1 | 1 | ||
| % | 28.0% | 16.0% | 36.0% | 2.0% | 28.0% | 16.0% | 22.0% | 2.0% | 2.0% | |||
| Longitudinal | Coaching Sciences | n | 1 | 2 | 9 | 1 | 1 | |||||
| % | 5.6% | 11.1% | 50.0% | 5.6% | 5.6% | |||||||
| Sports Medicine | n | 1 | 1 | 1 | 1 | |||||||
| % | 5.6% | 5.6% | 5.6% | 5.6% | ||||||||
| Sport pedagogy | n | 2 | 2 | 2 | 2 | 1 | 1 | 1 | ||||
| % | 11.1% | 11.1% | 11.1% | 11.1% | 5.6% | 5.6% | 5.6% | |||||
| Total Longitudinal | n | 4 | 2 | 12 | 2 | 2 | 2 | 1 | 3 | 1 | ||
| % | 22.2% | 11.1% | 66.7% | 11.1% | 11.1% | 11.1% | 5.6% | 16.7% | 5.6% | |||
| Both | Sport and Exercise Physiology | n | 1 | |||||||||
| % | 100.0% | |||||||||||
| Total both | n | 1 | ||||||||||
| % | 100.0% | |||||||||||
Longitudinal studies are found exclusively in three types of scientific disciplines, with again Coaching Sciences the most prolific (55%, n = 10), followed by Sport Pedagogy (33.3%, n = 6) and finally Sports Medicine (11%, n = 2). There is only one thesis that used both types of information, and it is in the scientific discipline of Sport Psychology.
4. Discussion
The purpose of this research was to analyze the production of doctoral theses in Spain from 1976 to 2021, recorded in the TESEO database, that focus on the study topic of handball, tendencies, methodological positioning according to the research designs and procedures used, and their evolution.
Agreement among coders was almost perfect [41], showing that that the definition of the variables was adequate. Similar variables were used in the studies by Ibáñez et al. [1] and Gamonales et al. [42]. Agreement in this investigation obtained similar values to those of the studies by Gamonales et al. [42], Gómez-Carmona et al. [43] and Reina et al. [44], which were higher than 0.90 in the different Kappa coefficients. Thus, the information used for this review is reliable.
Doctoral theses on handball are relatively recent in Spain. To date, 72 theses have been identified since the first thesis defended in 1991 by Gil, Ph.D. in the University of Valencia, the title of which included the term “handball”. In spite of being a relatively new research topic, it has seen a progressive increase over the last thirty years reaching a volume of scientific production in doctoral theses that is still well below that of other more general research themes in the field of Sport Sciences, such as the prevalence of women in the completion and direction of doctoral theses in Sport Sciences (333 theses) [10], referring to the History of Physical Education and Sport (88 theses) [26], but similar to the situation of other sport disciplines such as basketball (75 theses) [12], and higher than other more specific themes such as the sport coach (60) [1]. Considering the presence and relevance of handball in the ambit of Spanish sport, it can be stated that from a scientific perspective the evolution in research via doctoral theses on handball is becoming established compared to the other research topics mentioned. This is due to the increase in sport science faculties, with specific subjects on handball in the Physical Activity and Sport Sciences degree, and thus specialized staff in handball in the faculties, and research master’s degrees and doctoral programs, which have increased the number of Ph.D.s in the context of this sport. Research in sport sciences has stopped being dependent on other scientific disciplines, and at present, it is much easier to research a sport discipline from any viewpoint. Some of the Ph.D.s who completed their doctoral theses on handball have in turn become the directors of other doctoral students, thus consolidating their own lines of research.
Handball is studied mainly from two scientific disciplines [33]. The main one is Coaching Science, in which the research lines are directed toward the analysis of sport performance relating this sport discipline with an analysis of the coach’s behavior, technique, performance indicators or the game itself. Although less predominant, it is also featured in Sport Pedagogy, with the studies based on this discipline investigating coaching, teaching and training, including coaches’ training. These investigations deal both with the ways in which players learn and the knowledge and pedagogical abilities that the coaches need to teach more effectively [45]. The predominance of theses that situate their research in these two scientific disciplines coincide with the studies on doctoral theses on coaches [1] and basketball [12]. They are therefore investigations applied to the study of sports modalities.
Regarding research methods, the data analyzed show the predominance of descriptive studies with systematic observation using observational instruments as the most common method, due to its capacity to provide strong data on the development of the activity as occurs in other sport disciplines [46]. Similarly, the research approach most frequently used by the researchers is quantitative, far above qualitative research, and especially, interventions. Mixed approaches (qualitative/quantitative) are used more frequently than these last two as a consequence of the need to tackle the complexity of the research problems considered in handball with the interdisciplinary research processes that require such approaches. Research using mixed methods is currently a new tendency in scientific methodology that represents the integration of qualitative and quantitative research, which can provide more complete knowledge in sport sciences [47]. This information has to be interpreted from two viewpoints. First, historically, researchers who use a quantitative method are considered to be better [48], particularly researchers into sport value more the tradition of quantitative research [47], and moreover, students in doctoral programs are hardly trained in other research methods, as most of them, or even the training manuals, are mainly quantitative, with great importance given to statistics and their application in training courses [48]. This leads to the accumulation of more descriptive knowledge, a more basic level of research. Qualitative research will make possible better knowledge of complex experiments in the sports field [49] but this is not the tendency that has been observed. Research methods used by doctoral students reflect that the research on this sport discipline is still in its infancy, often involving cases of descriptive research, and should evolve to studies that make it possible to diagnose, predict and prescribe, starting from the results obtained, that is, develop from knowledge of the past to being able to anticipate the future.
The research procedure most used in handball has been descriptive studies, with systematic observation using surveys with questionnaires and mainly convenience sampling. Systematic observation has been one of the most commonly used methods in research on sport training and competition and the coach’s intervention [50]. Instruments generated ad hoc to analyze the coach’s intervention, players’ participation and the development of the game, have made it possible to describe their behavior, enhancing the description of the sport context [1]. These are instruments that are tedious to generate and validate, but easy to use for the researcher [51], being mainly used during training and competition. These data are similar to those identified when analyzing Spanish doctoral theses on basketball, with a predominant use of questionnaires and systematic observation [12]. This is a tendency in research topics in the process of development [1]. Using descriptive studies hinders an in-depth understanding of the coach’s intervention and the participation of the players in the game. Both players and coaches have little knowledge of their behavior, and there is an epistemological gap between knowledge and practice [52], again making it necessary to study one’s own actions. The analysis of the demands of training and the competition, as well as the coach’s intervention, require instruments that are applied ecologically to obtain information. This is one of the interests manifested by the researchers, which is why the use of ad hoc instruments using an observational methodology predominate.
The specific procedures for collecting data on players’ performance are varied. There are tests that are characterized by being of a general nature or specific to the respective sport, presenting results with greater ecological validity and reliability, as used in other studies on team sports such as basketball [53]. Conversely, data collection using a great variety of electronic devices makes it possible to monitor training and competition providing objective and powerful data that, with the advances made in the technology, are progressively increasing and becoming more reliable. Pino-Ortega et al. [54] stated that the technology used is attractive for monitoring performance indicators in team sports given its accuracy in collecting the data; although it is true, access is limited due to high cost, and thus it is difficult to obtain the devices. Finally, training programs that provide data on differences in learning tasks or performance, are mostly designed with specific objectives, the validation of which will help researchers to compare their effects with other studies [55]. The objectives of the investigations condition the instruments used to obtain information. These three groups of instruments allow researchers to tackle the issues that are considered in the investigation with great accuracy.
In the majority of studies, the representative population is obtained with convenience sampling choosing available subjects [38], because of the ease with which researchers can have access to, and monitor, the studied population. It has been shown that, with few exceptions, samples are not random in this field of study [56]. Moreover, there is no indication of uniformity of criteria when specifying the volume of representative data to guarantee their validity. All of this complicates generalizing the results, making it increasingly important to incorporate the sampling error for the study population and the effect size in the statistical analyses [57]. Researchers carry out their investigations on populations that are easier to monitor to guarantee the success of their study. The evolution of research in this sport discipline should occur when researchers introduce samples that make it possible to generalize the results.
Cross-sectional studies are the most frequently used to collect data in this context. Compared with longitudinal studies, they are cheaper, making it possible to work with more individuals, produce immediate results, obtain cooperation from the study subjects more easily, and are less affected by the measuring effect [58]. For these reasons, cross-sectional studies are better adapted to handball research, contributing to a simplicity bias if the data provided in the remaining variables are considered. Having reached this level in research on handball, it would be interesting to carry out longitudinal studies to provide information on the evolution of different aspects related to this sport.
5. Conclusions
The results of the analysis of the doctoral theses indexed on the TESEO database have made it possible to characterize the present state of research on this sport discipline, research tendencies, and the methodological positioning through the research designs and procedures used. This analysis makes it possible to appreciate the evolution of research on handball and advance in its development.
Research on handball, as represented in doctoral theses, is based on the study of the past, intent on discovering what happened and why the analyzed events occurred, and should evolve toward the future, investigating in order to predict and try to discover what can happen.
Maturity in research on handball in Spain will be reached when there is evolution from descriptive and correlational studies toward a significant increase in intervention and quasi-experimental studies. Thus, it is necessary to evolve from description to the study of players’ behaviors and thoughts in game actions and the coach’s intervention with more qualitative and mixed designs.
An increase in longitudinal designs will make possible a repeated and ordered observation of coaches and players, and their contribution to the development of the game will provide information to identify the processes and causes that are produced in the evolution of handball in their multiple possibilities.
Regarding future perspectives, it is considered that the adaptation of doctoral programs in Spain, extending and guiding the training of doctoral students toward more complex and reflective research methodologies will favor the completion of studies that provide more complete information for sport disciplines in general and handball in particular.
It is considered important to increase predoctoral grants and aid for research on the part of the competent Spanish authorities to obtain technological equipment that will foment a great improvement in data collection at the qualitative and quantitative level in the study of sport disciplines.
This first article reviewing the state of the art on research through doctoral theses on handball in Spain should have continuity over time by updating this scientific production periodically (5 or 10 years). Likewise, comparative review research should be carried out between different sports modalities, as well as research in doctoral theses on the sport of handball between different countries.
Finally, we have found the following limitations to this research. It is possible that our study is incomplete at the closing date of data collection as there is a delay between the defense of the doctoral thesis and its indexing in the TESEO database. Conversely, the selection of the doctoral theses was carried out exclusively using the search criterion of the word handball in the title of the doctoral thesis. It is possible that there are other doctoral theses on handball in which the authors did not decide to include this word in the title.
Author Contributions
Conceptualization, A.A., S.F. and S.J.I.; methodology, A.A., S.F. and S.J.I.; formal analysis, A.A. and S.J.I.; reviewers, A.A., S.F. and S.J.I.; writing—original draft preparation, A.A., S.F. and S.J.I.; writing—review and editing, A.A., S.F. and S.J.I.; visualization, A.A.; supervision, A.A., S.F. and S.J.I. All authors have read and agreed to the published version of the manuscript.
Funding
This study has been partially subsidized by the Aid for Research Groups (GR18170) from the Regional Government of Extremadura (Department of Employment, Companies and Innovation), with a contribution from the European Union from the European Funds for Regional Development.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Ibáñez S.J., García-Rubio J., Antúnez A., Feu S. Coaching in Spain Research on the Sport Coach in Spain: A Systematic Review of Doctoral Theses. Int. Sport Coach. J. 2019;6:110–125. doi: 10.1123/iscj.2018-0096. [DOI] [Google Scholar]
- 2.Escolar P., Medina F., Montilla J., Jimeno F.J., Oliveira S.L., Lillo M.C. Experiencia en la Formación de Doctorandos en Programas Vinculados a Fisioterapia. In: Castro A., Guillen-Riquelme A., editors. Sobre Evaluación de la Calidad de la Investigación y de la Educación Superior. Asociación Española de Piscología Conductual; Granada, Spain: 2010. pp. 372–376. [Google Scholar]
- 3.Buela-Casal G., Guillén-Riquelme A., Ramiro-Sánchez T., Quevedo-Blasco R. Ranking de investigación de las universidades públicas españolas. Rev. Iberoam. Psicol. Salud. 2017;8:21–35. doi: 10.23923/j.rips.2017.08.003. [DOI] [Google Scholar]
- 4.Sanz-Casado E., García-Zorita C., Serrano-López A.E., Efraín-García P., De Filippo D. Rankings nacionales elaborados a partir de múltiples indicadores frente a los de índices sintéticos. Rev. Española Científica. 2013;36:12. doi: 10.3989/redc.2013.3.1.023. [DOI] [Google Scholar]
- 5.Agudelo D., Bretón López J., Ortiz Recio G., Poveda Vera J., Teva I., Valor Segura I., Vico C. Análisis de la productividad científica de la Psicología española a través de las Tesis Doctorales. Psicothema. 2003;15:595–609. [Google Scholar]
- 6.Ato M., López-García J.J., Benavente A. Un sistema de clasificación de los diseños de investigación en psicología. An. Psicol. 2013;29:1038–1059. doi: 10.6018/analesps.29.3.178511. [DOI] [Google Scholar]
- 7.Moher D., Shamseer L., Clarke M., Ghersi D., Liberati A., Petticrew M., Shekelle P., Stewart L.A., Group P.-P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015;4:1–9. doi: 10.1186/2046-4053-4-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gilbert W.D., Trudel P. Analysis of Coaching Science Research published from 1970–2001. Res. Q. Exerc. Sport. 2004;75:388–399. doi: 10.1080/02701367.2004.10609172. [DOI] [PubMed] [Google Scholar]
- 9.Galatti L., Bettega O.B., Brasil V.Z., de Souza Sobrinho A.E.P., Bertram R., Tozetto A.V.B., Rodrigues H.A., Collet C., Nascimento J., Milistetd M. Coaching in Brazil Sport Coaching as a Profession in Brazil: An Analysis of the Coaching Literature in Brazil From 2000-2015. Int. Sport Coach. J. 2016;3:316–331. doi: 10.1123/iscj.2015-0071. [DOI] [Google Scholar]
- 10.Ortega E., Valdivia P., Olmedilla A., Martinez M., Villarejo D. A bibliometric study of the role of women in Doctoral theses in sports science. J. Sport Health Res. 2015;7:139–148. [Google Scholar]
- 11.Yaman H., Atay E. PhD theses in Turkish sports sciences: A study covering the years 1988–2002. Scientometrics. 2007;71:415–421. doi: 10.1007/s11192-007-1679-y. [DOI] [Google Scholar]
- 12.Ibáñez S.J. Pasado, Presente y Futuro del Congreso Ibérico de Baloncesto. In: Antúnez A., Ibáñez S.J., editors. El Camino Hacia la Excelencia en Baloncesto. Wanceulen; Sevilla, Spain: 2012. pp. 17–41. [Google Scholar]
- 13.Pérez-Gutiérrez M., Valdés-Badilla P., Gutiérrez-García C., Herrera-Valenzuela T. Produção científica de taekwondo publicada na web of science (1988–2016): Colaboração e temas. Movimento. 2017;23:1325. doi: 10.22456/1982-8918.75386. [DOI] [Google Scholar]
- 14.Ibáñez S.J., Feu S. La Literatura Científica Sobre Baloncesto en España y Portugal en las Revistas Indexadas en la Web of Science (2003–2013) In: Ibáñez S.J., García J., editors. Avances Científicos en Baloncesto. Estudios Ibéricos. Universidad de Extremadura; Cáceres, Spain: 2016. pp. 13–24. [Google Scholar]
- 15.Marcelino R., Sampaio J., Mesquita I. Investigação centrada na análise do jogo. Rev. Port. Cienc. Deporte. 2011;11:481–499. [Google Scholar]
- 16.Lord F., Pyne D.B., Welvaert M., Mara J.K. Methods of performance analysis in team invasion sports: A systematic review. J. Sports Sci. 2020;38:2338–2349. doi: 10.1080/02640414.2020.1785185. [DOI] [PubMed] [Google Scholar]
- 17.Silva M., Marcelino R., Lacerda D., Joao P.V. Match Analysis in Volleyball: A systematic review. Montenegrin J. Sports Sci. Med. 2016;5:35–46. [Google Scholar]
- 18.Aquino R., Puggina E.F., Alves I.S., Garganta J. Skill-related performance in soccer: A systematic review. Hum. Mov. 2017;18:3–24. doi: 10.1515/humo-2017-0042. [DOI] [Google Scholar]
- 19.Sarmento H., Marcelino R., Anguera M.T., Campaniço J., Matos N., Leitao J.C. Match analysis in football: A systematic review. J. Sports Sci. 2014;32:1831–1843. doi: 10.1080/02640414.2014.898852. [DOI] [PubMed] [Google Scholar]
- 20.Sarmento H., Clemente F.M., Harper L.D., Costa I.T.D., Owen A., Figueiredo A.J. Small sided games in soccer—A systematic review. Int. J. Perform. Anal. Sport. 2018;18:693–749. doi: 10.1080/24748668.2018.1517288. [DOI] [Google Scholar]
- 21.Courel-Ibáñez J., McRobert A.P., Ortega E., Cárdenas D. Collective behaviour in basketball: A systematic review. Int. J. Perform. Anal. Sport. 2017;17:44–64. doi: 10.1080/24748668.2017.1303982. [DOI] [Google Scholar]
- 22.Ferrari W.R., Sarmento H., Vaz V. Match analysis in handball: A systematic review. Montenegrin J. Sports Sci. Med. 2019;8:63–76. doi: 10.26773/mjssm.190909. [DOI] [Google Scholar]
- 23.Vila H., Ferragut C. Throwing speed in team handball: A systematic review. Int. J. Perform. Anal. Sport. 2019;19:724–736. doi: 10.1080/24748668.2019.1649344. [DOI] [Google Scholar]
- 24.Raya-González J., Clemente F.M., Beato M., Castillo D. Injury profile of male and female senior and youth handball players: A systematic review. Int. J. Environ. Res. Public Health. 2020;17:3925. doi: 10.3390/ijerph17113925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Manchado C., Tortosa-Martínez J., Vila H., Ferragut C., Platen P. Performance factors in women’s team handball: Physical and physiological aspects—A review. J. Strength Cond. Res. 2013;27:1708–1719. doi: 10.1519/JSC.0b013e3182891535. [DOI] [PubMed] [Google Scholar]
- 26.García J.M. Evolution in Doctorals Theses about Physical Activity and Sport in Spain (1990–2013) Apunt. Educ. Física Deportes. 2016;125:21–34. [Google Scholar]
- 27.Ramos-Pardo F.J., Sánchez-Antolín P. Production of educational theory Doctoral theses in Spain (2001–2015) Scientometrics. 2017;112:1615–1630. doi: 10.1007/s11192-017-2435-6. [DOI] [Google Scholar]
- 28.Moyano M., Delgado Domínguez C.J., Buela Casal G. Análisis de la productividad científica de la Psiquiatría española a través de las Tesis Doctorales en la base de datos TESEO (1993–2002) Int. J. Psychol. Psychol. Ther. 2006;6:111–120. [Google Scholar]
- 29.Fernández A.B., Delgado M.A., Ortega G., Pérez A.J. Las Áreas de Educación Física y Deportiva y Expresión Corporal en ISI. Editorial Fernández-Revelles; Granada, Spain: 2008. [Google Scholar]
- 30.Olmedilla A., Ortega E., Garcés de los Fayos E.J., Jara P., y Ortín F.J. Evolución de la investigación y de la aplicación en Psicología del Deporte, a través del análisis de los Congresos Nacionales de Psicología del Deporte. Cienc. Cult. Deporte. 2009;5:15–23. doi: 10.12800/ccd.v4i10.130. [DOI] [Google Scholar]
- 31.Thomas R.J., Nelson J.K., Silverman S.J. Research Methods in Physical Activity. 5th ed. Human Kinetics; Champaign, IL, USA: 2015. [Google Scholar]
- 32.Bennie A., Apoifis N., Caron J., Falcão W., Marlin D., Bengoechea E.G., Koh K.T., Macmillan F., George E. A Guide to Conducting Systematic Reviews of Coaching Science Research. Int. Sport Coach. J. 2017;4:191–205. doi: 10.1123/iscj.2017-0025. [DOI] [Google Scholar]
- 33.Borms J. Directory of Sport Science. 5th ed. Human Kinetics; Champaign, IL, USA: 2008. [Google Scholar]
- 34.Montero I., León O.G. A guide for naming research studies in Psychology. Int. J. Clin. Health Psychol. 2007;7:847–862. [Google Scholar]
- 35.Nelson L., Groom R., Potrac P. Research Methods in Sports Coaching. Routledge; London, UK: 2014. [Google Scholar]
- 36.Hernández-Sampieri R., Fernández C., Baptista P. Metodología y Técnica de la Investigación. 4th ed. McGraw-Hill; Mexico City, Mexico: 2006. [Google Scholar]
- 37.Polit D.F., Hungler B.P. Investigación Científica en Ciencias de la Salud. 6th ed. McGraw-Hill Interamericana; Mexico City, Mexico: 2000. [Google Scholar]
- 38.Cubo S. Muestreo. In: Cubo S., Marín B.M., Sanchez J.L.R., editors. Métodos de Investigación y Análisis de Datos en Ciencias Sociales y de la Salud. Ediciones Piramide; Madrid, Spain: 2011. pp. 109–135. [Google Scholar]
- 39.Cubo S. La Investigación Experimental. In: Cubo S., Martín B.M., Sanchez J.L.R., editors. Métodos de Investigación y Análisis de Datos en Ciencias Sociales y de la Salud. Ediciones Piramide; Madrid, Spain: 2011. pp. 235–328. [Google Scholar]
- 40.Randolph J.J. Free-Marginal Multirater Kappa (Multirater Κfree): An Alternative to Fleiss’ Fixed-Marginal Multirater Kappa; Proceedings of the Joensuu Learning and Instruction Symposium; Joensuu, Finland. 14–15 October 2005. [Google Scholar]
- 41.Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
- 42.Gamonales J.M., Muñoz-Jiménez J., León-Guzmán K., Ibáñez S.J. Football 5-a-side for individuals with visual impairments: A review of the literature. Eur. J. Adapt. Phys. Act. 2018;11:1–19. doi: 10.5507/euj.2018.004. [DOI] [Google Scholar]
- 43.Gómez-Carmona C.D., Pino-Ortega J., Ibáñez S.J. Diseño y validación de una batería de pruebas de campo para la valoración del perfil multi-ubicación de carga externa en deportes de invasión [Design and validity of a field test battery for assessing multi-location external load profile in invasion team sports] E-Balonmano Com Rev. Cienc. Deporte. 2020;16:23–48. [Google Scholar]
- 44.Reina M., García-Rubio J., Ibáñez S.J. Training and Competition Load in Female Basketball: A Systematic Review. Int. J. Environ. Res. Public Health. 2020;17:2639. doi: 10.3390/ijerph17082639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Armour K. Sport Pedagogy and Introduction for Teaching and Coaching. Routledge; London, UK: 2011. [Google Scholar]
- 46.McKenzie T.L., Van der Mars H. Top 10 Research Questions Related to Assessing Physical, Activity and Its Contexts Using Systematic Observation. Res. Q. Exerc. Sport. 2015;89:13–29. doi: 10.1080/02701367.2015.991264. [DOI] [PubMed] [Google Scholar]
- 47.Petrovic A., Koprivica V., Bokan B. Quantitative, qualitative and mixed research in Sport Science: A Methodological Report. S. Afr. J. Res. Sport Phys. Educ. Recreat. 2017;39:181–197. [Google Scholar]
- 48.Trudel P., Culver D., Gilbert W. Publishing Coaching Research. In: Nelson L., Groom R., Potrac P., editors. Research Methods in Sports Coaching. Routledge; London, UK: 2014. pp. 251–260. [Google Scholar]
- 49.Culver D.M., Gilbert W., Sparkes A. Qualitative research in sport psychology journals: The next decade 2000–2009 and beyond. Sport Psychol. 2012;26:261–281. doi: 10.1123/tsp.26.2.261. [DOI] [Google Scholar]
- 50.Cope E., Partington M., Harvey S. A review of the use of a systematic observation method in coaching research between 1997 and 2016. J. Sports Sci. 2017;35:2042–2050. doi: 10.1080/02640414.2016.1252463. [DOI] [PubMed] [Google Scholar]
- 51.Rhind D., Davis L., Jouwett S. Questionaires. In: Nelson L., Groom R., Potrac P., editors. Research Methods in Sports Coaching. Routledge; London, UK: 2014. pp. 111–122. [Google Scholar]
- 52.Partington M., Cushion C. An investigation of the practice activities and coaching behaviors of professional top-level youth soccer coaches. Scand. J. Med. Sci. Sports. 2013;23:374–382. doi: 10.1111/j.1600-0838.2011.01383.x. [DOI] [PubMed] [Google Scholar]
- 53.Mancha-Triguero D., García Rubio J., Calleja-González J., Ibáñez S.J. Physical Fitness in basketball players: A systematic review. J. Sports Med. Phys. Fit. 2019;59:1513–1525. doi: 10.23736/S0022-4707.19.09180-1. [DOI] [PubMed] [Google Scholar]
- 54.Pino-Ortega J., Oliva-Lozano J.M., Gantois P., Nakamura F.Y., Rico-González M. Comparision of the validity and reliability of local positioning system against other tracking technologies in team sport: A systematic review. Proc. Inst. Mech. Eng. Part P J. Sports Eng. Technol. 2021 doi: 10.1177/1754337120988236. [DOI] [Google Scholar]
- 55.García-Ceberino J.M., Antúnez A., Feu S., Ibañez S.J. Validation of two intervention programs for teaching school soccer. Rev. Int. Med. Cienc. Act. Fis. Deporte. 2020;20:257–274. [Google Scholar]
- 56.Kahan D. Coaching behavior: A review of the systematic observation research literature. Appl. Res. Coach. Athl. Ann. 1999;14:17–58. [Google Scholar]
- 57.Feu S., Ibáñez S.J., García-Rubio J., Antúnez A. La investigación sobre la enseñanza de los deportes de invasión y su transferencia al contexto escolar. Rev. Port. Ciências Desporto. 2017;17((Suppl. 1A)):50–58. doi: 10.5628/rpcd.17.S1A.50. [DOI] [Google Scholar]
- 58.Martín B. In: La Investigación Descriptiva. Métodos de Investigación y Análisis de Datos en Ciencias Sociales y de la Salud. Cubo S., Martín B.M., Sanchez J.L.R., editors. Ediciones Piramide; Madrid, Spain: 2011. pp. 373–385. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Not applicable.