Internal Validity in Resistance Training Research: A Systematic Review

Abstract

Ensuring internal validity is the key procedure when planning the study design. Numerous systematic reviews have demonstrated that considerations for internal validity do not receive adequate attention in the primary research in sport sciences. Therefore, the purpose of this study was to review methodological procedures in current literature where the effects of resistance training on strength, speed, and endurance performance in athletes were analyzed. A computer-based literature searches of SPORTDiscus, Scopus, Medline, and Web of Science was conducted. The internal validity of individual studies was assessed using the PEDro scale. Peer-reviewed studies were accepted only if they met all the following eligibility criteria: (a) healthy male and female athletes between the ages of 18-65 years; (b) training program based on resistance exercises; (c) training program lasted for at least 4 weeks or 12 training sessions, with at least two sessions per week; (d) the study reported maximum strength, speed, or endurance outcomes; and (e) systematic reviews, cohort studies, case-control studies, cross-sectional studies were excluded. Of the 6,516 articles identified, 133 studies were selected for rating by the PEDro scale. Sixty-eight percent of the included studies used random allocation to groups, but only one reported concealed allocation. Baseline data are presented in almost 69% of the studies. Thirty-eight percent of studies demonstrated adequate follow-up of participants. The plan to follow the intention-to-treat or stating that all participants received training intervention or control conditions as allocated were reported in only 1.5% of studies. The procedure of blinding of assessors was also satisfied in only 1.5% of the studies. The current study highlights the gaps in designing and reporting research in the field of strength and conditioning. Randomization, blinding of assessors, reporting of attrition, and intention-to-treat analysis should be more fully addressed to reduce threats to internal validity in primary research.

Key points.

• The implementation of internal validity procedures is often not satisfied in resistance training research.

• A high risk of bias in resistance training studies was identified in the following criteria: concealed allocations, assessor blinding, and intention-to-treat.

• Follow-up and eligibility criteria should be widely implemented and reported for future studies.

• The PEDro scale items may be used to improve the quality of future investigations involving resistance training.

Introduction

Resistance training, commonly referred to as strength training or weight training, is considered an essential intervention for improving sports performance in most sports. There are many resistance training modalities, including free weights, machine weights, isokinetic devices, elastic bands, resisted running, and plyometrics. Resistance training has been repeatedly shown to be highly effective at increasing strength, speed, endurance, and sports performance in a variety of athletic populations (Seitz et al., 2014; Balsalobre-Fernández et al., 2016; Davies et al., 2017; Kwok et al., 2021). Therefore, it is not surprising that coaches are increasingly interested in resistance training methods whose effectiveness has been demonstrated by rigorous evaluation in high-quality studies. Evidence-based practice in sport sciences is a paradigm that promotes the integration of the practical experience of coaches and practitioners, athletes’ values, and high-quality studies into the decision-making process for athlete care. Practitioners utilizing resistance training as a modality should apply the best scientific evidence to their training programs that are well-designed and well-reported (Amonette et al., 2016).

One approach to meeting these quality challenges is to take the appropriate steps in design, execution and reporting of research to increase the internal validity of investigations. Specifically, internal validity in experimental and quasi-experimental research designs indicates the degree to which changes in a dependent variable can be attributed to changes in an independent variable (Taylor and Asmundson, 2008).

Several basic methodological procedures are required to ensure high internal validity. Random assignment of participants to experimental and control groups (randomization) is a gold standard for experimental research, as it decreases the risk of selection bias by equally distributing the study participants with particular characteristics among all groups. Simple randomization, block randomization, and stratified randomization are the most common randomization techniques (Lin et al., 2015). It is interesting to note that estimates of intervention effects tend to be smaller in publications where random allocation is clearly reported, compared to works when non-random or quasi-random methods are used (Savović et al., 2012). A randomized experiment is an essential tool for testing the effectiveness of the intervention.

The second important feature for randomization procedures, therefore to internal validity is allocation concealment. Concealed allocation is adopted when the researcher is unaware of the sequence for group allocation. When an allocation is not concealed, the researcher (recruiter) may consciously or unconsciously influence allocation to a particular group, increasing the potential for selection bias which may taint the data. Schulz and Grimes (2002) determined that experimental research with inadequate or unclear allocation concealment tended to overestimate treatment effects, up to 40%, compared with those where concealment was used adequately. In addition, Schulz et al. (1995) reported that studies where allocation concealment was poor showed greater heterogeneity in results, due to more extensive fluctuation between above and below the estimates, relative to studies where allocation was both clear and adequate.

Examining similarity of the experimental and control groups at baseline is another important criterion for internal validity. This procedure allows the researcher to compare the groups on variables of interest following randomization to determine whether the groups are equivalent. Because it is still possible that some differences will occur, the knowledge about the magnitude of potential differences between groups at baseline is important for reliable interpretation of data from an investigation. Typically, demographic (e.g., age, sex) and anthropometric (e.g., height, body mass) variables, as well as key outcome measures for the study, are included in this analysis. Note that using the statistical control for baseline data across the randomized groups is often discouraged (Schulz et al., 2010). For example, according to the Consolidated Standards of Reporting Trials (CONSORT) guidelines for individually randomized trials, baseline statistical testing should not be applied, because based on the assumption of randomization, it is known that any baseline differences are caused by chance (Schulz et al., 2010). On the other hand, some researchers claim that baseline testing allows them to check that the randomization process has not been subverted (Berger, 2005).

Blinding is a procedure that reduces the risk of detection and performance bias by preventing subjects and researchers who are involved in the study from knowing to which group a participant was allocated. However, in strength and conditioning research, where differing exercises or training programs of often evaluated, both participants and supervisors (training instructors) are actively engaged in an intervention. In such cases, blinding is not completely possible. This limitation appears in these types of studies because participants and supervisors may consciously or unconsciously influence study results. For example, researchers may more efficiently motivate experimental group participants to confirm the hypothesis they set out to test. This is concerning for correct application of evidence-based practice in strength and conditioning, as there is evidence that lack of blinding leads to overestimated intervention effects (Jüni et al., 2001). While it is impossible to eliminate the risk of performance bias in studies of this type, reducing the risk of detection bias is possible by blinding the researchers who measure outcomes along with those who collect and analyze data.

Another methodological common challenge is attrition of participants during study. This threat to internal validity refers to the differential and systematic loss of participants from experimental and control groups. When attrition occurs, the characteristics of randomized groups may change from the initial allocation, and these changes may affect the study outcomes in an uncontrolled manner (Beutler and Martin, 1999; Dumville et al., 2006). One study that analyzed the effects of treatment in 235 randomized control trials (RCT) published in leading medical journals found that different assumptions about outcomes of participants who withdraw from the study could change interpretation of results of up to 58% of RCTs (Akl et al., 2012). Attrition may occur for numerous reasons, including diseases, participant loss of interest or poor tolerance for an intervention. In strength and conditioning research, there is also the potential for participants who are athletes to be injured in practice or competition outside of the intervention being used in the study. Even if the intervention is highly tolerable, some participants may not adhere to the allocated intervention due to the intervention is perceived as ineffective or they are dissatisfied with their allocated intervention. An athlete, for example, might fear loss of a competitive advantage if they are placed in a group which receives an intervention they believe may be less effective, causing them to withdraw from the study.

To mitigate attrition bias, different statistical techniques have been applied. Intention-to-treat (ITT) analysis is considered the gold standard method for dealing with attrition in RCT studies. According to the basic assumption of ITT, all randomized participants are included in the analysis in accordance with group assignment, regardless of their adherence, intervention duration or change of intervention regimen. As a result, data is analyzed irrespective of the planned study protocol. If the attrition rate is relatively high (e.g. >20 %), ITT analysis tends to underestimate the intervention effects in participants who complete the study (Armijo-Olivo et al., 2009).

Study quality assessment is often performed with the use of an assessment tool. No formal recommendation regarding such tool for strength and conditioning research exists, leading researchers to sometimes adapt tools from other, related, disciplines (Smart et al., 2015). One of the most common used assessment tool for rating the internal validity of primary articles included in systematic reviews is the Physiotherapy Evidence Database (PEDro) scale (Maher et al., 2003). The PEDro scale is based on the Delphi list developed by Verhagen et al. (1998) and, as its name suggests, was initially designed for physiotherapy-based studies. Although some criteria of the PEDro scale are redundant for strength and conditioning studies, and others relevant for training intervention-effectiveness (e.g., adverse events, training frequency, volume and intensity of training) are not addressed, the PEDro scale does allow for the assessment of essential features of internal validity for strength and conditioning research.

The authors of this paper take a view that the PEDro scale items may also provide distinct advantages for primary research like RCT studies (outside the context of a systematic reviews) (Albanese et al., 2020). For example, by using them as guidelines during the planning stage of study design, researchers may minimize the influence of confounding variables, improve methodology, and receive more reliable information about experimental interventions. Numerous studies (Harries et al., 2015; Blagrove et al., 2018; Thiele et al., 2020; Trowell et al., 2020) suggest that internal validity of strength and conditioning research could be higher. However, because the results are not conclusive, we decided to review methodological quality of strength conditioning research which examined the effects of resistance training on strength, speed, and endurance in athletes. Our goal was not to provide a review of the total quality assessment of studies as previous systematic reviews have done, but rather focus on internal validity criteria to provide more attention to methodological procedures which should be used during planning and reporting of experimental research in the field of strength and conditioning. In addition, we believe that a greater depth of understanding of research quality by practitioners may support the process of using the best evidence in practice.

Methods

Search strategy

The PRISMA guidelines for reporting a systematic review were adopted (Moher et al., 2009). As this study did not involve human subjects, institutional review board approval was not required. Four relevant electronic databases (SPORTDiscus, Scopus, Medline, and Web of Science) were comprehensively searched for studies examining the effects of resistance training on strength, speed, and endurance in athletes. The identified terms with Boolean operators with different expressions are presented in Table 1. All publications listed prior to April 15, 2021 were considered for inclusion without language restrictions.

Table 1.

Search strategy

1.	athlete* OR player* OR elite OR "highly trained" OR "highly skilled" OR "well-trained"
2.	“strength training” OR “weight training” OR “resistance training” OR “power training” OR “eccentric training” OR “strength exercise*” OR “weight exercise*” OR “resistance exercise*” OR “power exercise*” OR “eccentric exercise*” OR “isokinetic exercise*” OR “heavy load*” OR hypertrophy OR bodybuilding OR plyometric* OR “Olympic lift*” OR “muscular endurance” OR crossfit OR calisthenics OR “free weight*” OR “machine exercise*” OR “machine weight*” OR “elastic bands” OR “weight vest” OR “weights belts” OR “medicine ball*” OR kettlebell* OR “resisted speed” OR “resisted sprint*” OR “resisted run*” OR “sled towing” OR “resisted sled” OR “uphill run*” OR “muscle strength”
3.	"1 RM" OR "1RM" OR "rep* max*" OR "max* strength" OR "max* strength" OR squat OR "clean and jerk" OR "power clean" OR snatch OR deadlift OR "bench press" OR "leg press" OR "strength performance" OR "strength outcome*" OR sprint OR "speed run*" OR "run* time" OR "run* speed" OR "run* performance" OR "endurance run*" OR "run* endurance" OR "distance run*" OR "long distance run*" OR "run* economy" OR "run* distance" OR run* outcome* OR "sprint time" OR “1 repetition maximum”
4.	#1 AND #2 AND #3

Study selection and data extraction

Two independent investigators (HM, MS) screened titles, abstracts, and full-text articles against the PICO criteria (Table 2). Following extraction, duplicate articles were removed automatically using EndNote X9.3.3 (Clarivate Analytics). Any remaining duplicates were deleted manually. In any disagreement regarding inclusion/exclusion, ambiguous issues were discussed, and a consensus was reached before proceeding.

Table 2.

Selection criteria for muscular strength studies, running speed studies and running endurance studies

Category	Inclusion criteria	Exclusion criteria
Population	Healthy male and female athletes (defined as participants who engaged in organized sports training and competition) with a mean age ≥ 18 years	Older adults (65 and over), disabled athletes, injured athletes
Intervention	Training program needed to last for a minimum of 4 weeks (or 12 training sessions) with at least 2 sessions per week Resistance training interventions including free weights, machine weights, isokinetic devices, elastic bands, resisted running and plyometrics	Combined interventions containing no resistance exercises (e.g. sprint or endurance running, balance exercises) or nutritional, pharmacological, physiological and psychological aids
Comparator	No restriction
Outcome	Studies that tested maximum strength performance (in kg or Ib) or maximum running speed performance distance or maximum running endurance performance (in units of time) as a dependent variable	Athletic performance was tested by isokinetic condition, VO2max, power measurements (e.g. in Watts), and jumping tests
Study design	Randomised and non-randomised controlled trials	Systematic reviews, cohort studies, case-control studies, cross sectional studies

Internal validity analysis

The internal validity of each study included in this review was assessed using the PEDro quality scale. This appraisal tool was chosen because it demonstrates high reliability and validity for randomized control trials (Maher et al., 2003). In addition, the PEDro scale makes it possible to assess the clarity of statistical methods and report external validity criteria. A given PEDro item was scored as a “yes” if the criterion was met and the item scored as a “no” if the criterion was not met. Points were awarded only when a given criterion was satisfied according to the PEDro guidelines (please see https://pedro.org.au/english/resources/pedro-scale/; Maher et al., 2003). Note that the PEDro scale was adapted to the methodological and reporting requirements of strength and conditioning filed in this study. Criterion 1 “eligibility criteria were specified” was satisfied if the study reported the source of subjects and a list of criteria used to determine who was eligible to participate in the study. Criterion 2 “subjects were randomly allocated to groups” was satisfied if study stated that allocation was random. The precise method of randomization was not needed to specified in order to satisfy this criterion. However, quasi-randomization research did not satisfy this criterion. A point was awarded for criterion 3 “allocation was concealed” when authors of a study stated that group allocation was concealed or, when it was stated that the researcher who determined if a subject was eligible for inclusion in the study was unaware, when this decision was made, to which group the subject would be allocated. Criterion 4 “the groups were similar at baseline regarding the most important prognostic indicators” was considered to have been met if a study reported at least one key outcome (primary measure of the effectiveness of the training intervention) and anthropometric variables such as body mass, body height or one repetition maximum test at baseline. Measure of the severity of the condition being treated was not applicable in this study. Criterion 4 was met if baseline data were presented by group allocation, and when there was no difference between prognostic indicators. Criterion 5 “there was blinding of all subjects” and criterion 6 “blinding of all therapists who administered the therapy” were not applicable in this study. A noted earlier, in strength and conditioning research, both participants and those applying a particular intervention are aware of which intervention is being applied. Criterion 7 “there was blinding of all assessors who measured at least one key outcome” was met when it is stated that the assessor of the primary outcome was blinded to group allocation. Criterion 8 “measures of at least one key outcome were obtained from more than 85% of the subjects initially allocated to groups” was satisfied only if the study explicitly reported both the number of subjects initially allocated to groups as well as the number of subjects from whom key outcome measures were obtained. In studies in which outcomes are measured at several points in time, a key outcome must have been measured in more than 85% of subjects at one of those points in time. Criterion 9 “all subjects for whom outcome measures were available received the treatment or control condition as allocated or, where this was not the case, data for at least one key outcome was analyzed by intention-to-treat” was met if intention-to-treat analysis was performed or, if the study explicitly demonstrated that all subjects received the training intervention or control condition as allocated. Criterion 10 “the results of between-group statistical comparisons are reported for at least one key outcome” was satisfied when comparison of two or more training interventions, or comparison of training intervention with a control condition was applied and when a factorial analysis of variance or hypothesis testing, describing the probability that the groups differed only by chance or in the form of an estimate (e.g., the mean or median difference) and its confidence interval have been used to analyze the data. Criterion 11 “the study provides both point measures and measures of variability for at least one key outcome” was met when a point measure was a measure of the size of the treatment effect (e.g., described or presented as a difference in group outcomes), or as the outcome in (each of) all groups and measures of variability included standard deviations, standard errors, confidence intervals, interquartile ranges (or other quantile ranges). Two qualified PEDro raters independently evaluated all included studies (HM, MS). All differences of opinion regarding the PEDro rating were discussed with a third independent qualified PEDro rater (MP).

Results

Study selection

Figure 1 shows the flow chart of the study selection process. The systematic literature search identified 6,516 records, of which 2,270 duplicates were removed. Screening for title and abstract identified 211 possibly relevant studies. After the full-text screening, 133 studies were selected for rating by the PEDro scale.

Figure 1.

The flow chart of the study selection process.

Characteristics of the included studies and participants

The characteristics of the included studies are summarized in Table S1 (see Supplemental File Table S1, the references of the included studies are also listed in this Supplemental File). The total sample size was 3,117 subjects (n = 2,532 men, 81%; n = 585 women, 19%). The subjects were athletes between the ages of 18-65 years who engaged in organized sports training and competition. Across all studies, 74% of participants represented team sports (e.g., American football, basketball, handball, rugby, soccer), and 26% of participants represented individual sports (e.g., cross country skiing, cycling, track and field, tennis). Overall, 54 studies reported sports experience (range: 0 to 14 years) with the remaining 79 not explicitly describing subjects’ level of experience.

Table S1.

Overview of the studies included in the review

Study	Sport, sports expertise	Participants, gender (n), mean age (in years)	Type and duration of intervention	Test used to measure outcomes (study design)
Abade et al. (2019)	Handball, semi-professional players (sport experience = 12.2 ± 1.8 y)	M=20, 25 y	Compound strength training, complex strength training, 12 wks	10-m and 20-m linear sprint (BS)
Alcarez et al. (2014)	Track and field (sprinters, long jumpers, decathletes), national level athletes	F=8, M=14, 21 y	Sled towing training, control, 4 wks	50-m sprint (BS)
Ali et al. (2019)	Soccer, university players (sport experience > 4 y)	M=36, 21 y	Complex training, contrast training, control, 6 wks	20-m sprint (BS)
Aloui et al. (2019)	Handball, national cadet and junior team (sport experience = 7.0 ± 1.1 y)	M=30, 19 y	Elastic band training, control, 8 wks	5-m and 30-m sprint, 1RM half back squat (BS)
Alvarez et al. (2012)	Golfers, athletes (sport experience = 10.2 ± 4.5 and 9.7 ± 6.1 y)	M=10, 24 y	Strength training (maximal, explosive, and specific exercises), control, 18 wks	1RM squat, 1RM bench press, 1RM seated row machine, 1RM triceps push-down, 1RM seated calf extension, 1RM seated barbell military (BS)
Anderson et al. (2008)	Basketball, wrestling, hockey, athletes (sport experience = 3.6 and 3.7 y)	F=22, M=22, 20 y	Elastic and free weight resistance training, free weight resistance training alone, 7 wks	1RM bench press, 1RM back squat (BS)
Appleby et al. (2019)	Rugby, union academy players	M=33, 22 y, range: 18-29 y	Bilateral resistance training, unilateral resistance training, control, 18 wks	1RM squat, 1RM step-up (BS)
Arazi and Asadi (2011)	Basketball, semi-professional athletes (sport experience = 4.8 ± 2.5 y)	M=18, 19 y	Aquatic plyometric training, land plyometric training, control, 8 wks	36.5-m and 60-m sprints, 1RM leg press (BS)
Arazi et al. (2018)	Volleyball, players (sport experience = 5.2 ± 1.7 y)	F=30, 19 y	Resistance training (cluster sets), resistance training (traditional set), control, 8 wks	20-m sprint, 1RM back squat, 1RM bench press, 1RM military press, 1RM deadlift (BS)
Arede et al. (2021)	Basketball, group form U18 to senior amateur level athletes	M=20, 20 y, range: 15-34 y	Strength training, control, 10 wks	10-m and 25-m sprint (BS)
Ataee et al. (2014)	Kung-Fu, wrestling, trained athletes	M=24, 21 y	Accommodation resistance training, constant resistance training, 4 wks	1 RM squat, 1RM bench press (BS)
Ayers et al. (2016)	Volleyball, softball, NCAA Division I athletes	F=23, 20 y, range: 18-22 y	Hang clean training, hang snatch training, 6 wks	40-yard sprint, 1RM back squat (BS)
Bachero-Mena et al. (2021)	Track and field (800-m runners), national and international levels athletes	M=13, 22 y, range: 17-35 y	High-speed resistance training, circuit training, 25 wks	20-m and 200-m sprints, 800-m run (BS)
Balsalobre-Fernández et al. (2013)	Track and field (400-meter hurdles), highly competitive athletes	M=7, 22 y	Power training, 10 wks	30-m sprint, 1RM half squat (WS)
Bartolomei et al. (2014)	Track and field (throwing events), rugby, American football, experienced athletes	M=24, 24 y	Block periodization training, traditional periodization training, 15 wks	1RM isometric half-squat, 1RM bench press (BS)
Bartolomei et al. (2016)	Track and field (throwing events), rugby, American football, wrestling, experienced athletes	M=18, 25 y	Block periodization training, weekly undulating training, 15 wks	1RM isometric half-squat, 1RM bench press (BS)
Beattie et al. (2017)	Track and field (1,500–10,000 m runners), competitive collegiate and national-level athletes	M=20, 28 y	Strength training, control, 40 wks	1RM back squat (BS)
Ben Brahim et al. (2021)	Soccer, elite players (Tunisian national U- 19 team members)	M=34, 19 y	Combined muscular strength and resisted sprint training, control, 6 wks	5-m and 20-m sprints, 1RM half-back squat (BS)
Berryman et al. (2010)	Track and field (endurance runners), from moderately to well-trained athletes	M=35, 30 y	Plyometric training, dynamic weight training, control, 8 wks	3,000-m run (BS)
Berryman et al. (2021)	Track and field (endurance runners), well-trained athletes	M=8, 33 y	Plyometric training, dynamic weight training, 8 wks	3,000-m run (BS)
Blazevich and Jenkins (2002)	Track and field (sprint runners), elite athletes (nationally ranked) (sport experience > 5 y)	M=9, 19 y	High-velocity resistance training, low-velocity resistance training, 7 wks	20-m sprint, 1-RM squat (BS)
Brito et al. (2014)	Soccer, local soccer club players	M=57, 20 y	Resistance training, plyometric training, complex training, control, 9 wks	5-m and 20-m sprints, 1-RM squat (BS)
Burnham et al. (2010)	Volleyball and basketball, collegiate athletes (NCAA Division II)	F=19, 20 y	Traditional training, chain training, 16 sessions	1 RM bench press (BS)
Campos-Vazquez et al. (2015)	Soccer, youth team that competes in the top Spanish U-19 category (sport experience > 5 y)	M=21, 18 y, range: 16-19 y	Squat training, take-off training, 8 wks	RSA (BS)
Chelly et al. (2010)	Soccer, regional team (sport experience = 7.2 ± 1.2 y)	M=21, 19 y	Plyometric training, control, 8 wks	40-m sprint (BS)
Cherif et al. (2016)	Handball, elite players recruited from a team ranked among the better of the Tunisian first league (sport experience > 12 y)	M=22, 22 y	Strength training, control, 12 wks	1RM half-back squat, 1RM bench press, 1RM developed neck, 1RM print, 1RM pull-over (BS)
Cherni et al. (2021)	Basketball, elite players (sport experience = 10.8 ± 3.2 y and 10.8 ± 4.8 y)	F=27, 21 y	Plyometric training, control, 8 wks	10-m, 20-m, and 30-m sprints (BS)
Coratella et al. (2019)	Soccer, fourth-division (Serie-D) (sport experience > 5 y)	M=40, 23 y	Negative work-based training, weight training in the change of direction, 8 wks	10 m and 30-m sprints, 1RM squat (BS)
Crewther et al. (2016)	Rugby, premier club players in New Zealand	M=24, 30 y	Full-body training, split-body training, 4 wks	1RM back squat, 1RM bench press (WS)
Cross et al. (2018)	Soccer, rugby, club-level athletes	F=12, M=24, 27 y	Resisted sprint training (optimal load), resisted sprint training (high load), 12 weeks	5-m, 10-m, and 30-m sprints (BS)
Cummings et al. (2018)	Golf, Mississippi State University team players	M=10, 21 y, range: 18-22 y	Fat grip training, control, 8 wks	1RM deadlift (BS)
Dolezal et al. (2016)	Track and field (throwers, jumpers, sprinters), Division III	F=9, M=11, 20 y	Combining velocity-based training with eccentric focus, velocity-based training, 12 wks	1RM squat, 1RM bench press (BS)
Douglas et al. (2018)	Rugby, trained in academy	M=14, 19 y	Resistance training incorporating accentuated eccentric loading, traditional resistance training, 8 wks	10-m, 20-m and 30-m sprints, 1RM back squat (BS)
El-Ashker et al. (2019)	Track and field (long jumpers), regional level athletes (sport experience = 4.9 ± 2.1 y and 4.4 ± 1.9 y)	M=28, 19 y	Plyometric training, control, 8 wks	30 m sprint (flying start) (BS)
Enoksen et al. (2013)	Soccer, well-trained elite junior players	M=24, 19 y	Supervised strength training, unsupervised strength training, control, 10 wks	10-m and 40-m sprints, 1RM leg press (BS)
Escobar-Alvarez et al. (2020)	Rugby, amateur union players (mean sport experience = 8.0 ± 1.7 y)	F=31, 24 y	Resisted sled training (2 groups), control, 8 wks	5-m and 20-m sprints (BS)
Faude et al. (2013)	Football, high-level amateur players (sport experience > 10 y)	M=16, 22 y	Strength training, control, 7 wks	10-m and 30-m sprints, 1RM half squat (BS)
Franchini et al. (2015)	Judo, athletes	M=13, age not provided	Linear undulating periodized resistance training, daily undulating periodized resistance training, 8 wks	1RM bench press, 1RM squat, 1RM row (BS)
Freitas et al. (2019)	Basketball, semi-professional players	M=18, 21 y	Optimal load training, modified complex training, 6 wks	10-m sprint, 1RM half-squat, 1RM bench press, 1RM hip thrust (BS)
Ghigiarelli et al. (2009)	Football, Division 1-AA players	M=36, 20 y	Elastic bands training, weighted chain training, control, 7 wks	1RM bench press (BS)
Gil-Cabrera et al. (2021)	Cycling, professional athletes	M=22, 19 y	Optimum power load training, traditional resistance training, 8 wks	1RM squat, 1RM hip thrust, 1RM lunge (BS)
Gjinovci et al. (2017)	Volleyball, players participating at the highest competitive level in Kosovo (i.e., first division players)	F=41, 22 y	Plyometric skill based training, volleyball skill based training, 12 wks	20-m sprint (BS)
Grazioli et al. (2020)	Soccer, professional players	M=17, 26 y	Moderate load sled training, heavy load sled training, 11 wks	10-m and 20-m sprints (BS)
Guglielmo et al. (2009)	Track and field, regional and national level middle- and long-distance runners (3000 m to half-marathon)	M=16, 27 y	Explosive strength training, heavy weight strength training, 4 wks	1RM leg press (BS)
Hansen et al. (2011)	Rugby, elite union players	M=18, 27 y	Traditional strength training, cluster training, 8 wks	1RM back squat (BS)
Harris et al. (2008)	Rugby, well-trained league players	M=18, 22 y	Machine squat jump training (80% 1RM), machine squat jump training (individual load for peak power), 7 wks	10-m and 30-m sprints, 1RM machine hack-squat (BS)
Harrison and Bourke (2009)	Rugby, professional or semiprofessional players	M=15, 21 y	Resisted sprint training, control, 6 wks	30-m sprint (BS)
Hermassi et al. (2010)	Handball, elite players (top National Handball League) (sport experience = 8.2 ± 0.6 y)	M=26, 20 y	Heavy resistance training, moderate resistance training, 10 wks	1RM bench press, 1RM pull-over (BS)
Hermassi et al. (2011)	Handball, elite, national level players (sport experience = 9.1 ± 0.2 y and 8.7 ± 0.6 y)	M=24, 20 y	Heavy resistance training, control, 8 wks	30-m sprint, 1RM back half squat, 1RM pull-over, 1RM bench press (BS)
Hermassi et al. (2014)	Handball, team that played at the highest (Tunisia) national level (sport experience = 12.4 ± 2.1 y)	M=24, 20 y	Plyometric training, control, 8 wks	RSA (BS)
Hermassi et al. (2015)	Handball, elite players (top national handball league) (sport experience = 7.5 ± 0.5 y)	M=34, 18 y	Resistance training, regular throwing training, control, 8 wks	1RM bench press, 1RM pullover (BS)
Hermassi et al. (2017)	Handball, national-level Tunisian youth team (U19) (sport experience = 9.1 ± 0.3 y)	M=22, 19 y	Strength training, control, 10 wks	1RM half squat, RSA (BS)
Hermassi et al. (2019a)	Handball, first and second league handball players (sport experience = 8.2 ± 0.8 y)	M=22, 21 y	Combined resistance training, resistance training, 10 wks	RSA, 1RM bench press, 1RM back half-squat (BS)
Hermassi et al. (2019b)	Handball, elite players (first national league)	M=20, 21 y	Weightlifting training, control, 12 wks	30-m sprint, 1RM bench press, 1RM snatch, 1RM clean and jerk, 1RM back half-squat (BS)
Hermassi et al. (2019c)	Handball, elite players (first national league) (sport experience = 9.2 ± 0.7 y)	M=22, 20 y	Weightlifting training, control, 12 wks	30-m sprint, 1RM bench press, 1RM snatch, 1RM clean and jerk, 1RM back half-squat (BS)
Hermassi et al. (2020)	Handball, elite players (sport experience = 7.2 ± 1.1 y)	M=19, 19 y	Circuit strength training, control, 12 wks	15-m and 30-m sprints, 1RM bench press, 1RM pull-over, 1RM half-squat (BS)
Hertzog et al. (2020)	Soccer, players playing with the reserve (U21) and U19 teams of a French first division club (sport experience > 5 y)	M=28, 18 y	Upper-body resistance training, control, 30 wks	1RM bench press, 1RM bench pull (BS)
Hoff and Almasbakk (1995)	Handball, Norwegian second division	F=16, 20 y, range: 17-26 y	Strength training, control, 10 wks	1RM bench press (BS)
Hoff et al. (2002)	Cross-country skiing, well-trained skiers	M=19, 20 y	Strength training, control, 8 wks	1RM at the modified cable pulley apparatus (BS)
Hoffman et al. (1990)	American football, NCAA division IAA team	M=61, 19 y	Resistance training (four groups: 3, 4, 5, 6 days per week), 10 wks	40-yard sprint, 2-mile run, 1RM squat, 1RM bench press (BS)
Hoffman et al. (1991a)	Basketball, NCAA division I	M=9, 19 y	Resistance training, 25 wks	27-m sprint, 1.5 mile run, 1RM squat, 1RM bench press (WS)
Hoffman et al. (1991b)	Basketball, NCAA division I	M=22, 19 y	Resistance training (three groups), 25 wks	27-m sprint, 1.5 mile run, 1RM squat, 1RM bench press (BS)
Hoffman et al. (2004)	American football, NCAA division III	M=20, 19 y	Olympic lifting training, power lifting training, 15 wks	40-yard sprint, 1RM squat, 1RM bench press (BS)
Hoffman et al. (2005)	American football, NCAA division III	M=47, 20 y	Loaded jump squat training, unloaded jump squat training, control, 5 wks	40-yard sprint, 1RM squat, 1RM power clean (BS)
Hoffman et al. (2009)	American football, NCAA division III	M=51, 20 y	Non-periodized training, traditional periodized linear training, nonlinear periodized training, 15 wks	1RM squat, 1RM bench press (BS)
Hong-Sun et al. (2009)	Swimming, national team athletes	M=10, 19 y	Periodized strength training, 40 wks	50-m sprint, 100-m run, 1RM squat, 1RM bench press, 1RM power clean, 1RM deadlift (WS)
Horwath et al. (2019)	Ice hockey, Swedish and Norwegian junior hockey leagues	M=22, 18 y	Isokinetic resistance training, eccentric overload training, traditional resistance training, 8 wks	30-m sprint, 1RM back squat (BS)
Iacono et al. (2017)	Handball, elite players (sport experience > 8 y)	M=18, 23 y	Vertical drop jump training, horizontal drop jump training, 10 wks	10-m and 25-m sprints (BS)
Impellizzeri et al. (2008)	Soccer, amateur players	M=44, 25 y	Plyometric grass training, plyometric sand training, 4 wks	10-m and 20-m sprints
Iodice et al. (2020)	Futsal, elite players	M=30, 24 y	Slow-speed resistance training with low intensity, traditional resistance training, 8 wks	30-m and 60-m sprint (BS)
Izquierdo-Gabarren et al. (2010)	Rowing, trained athletes (sport experience = 12.1 ± 5 y)	M=43, 26 y	Resistance training (four exercises to failure), resistance training (four exercises not to failure), resistance training (two exercises not to failure), control, 8 wks	1RM prone bench pull (BS)
Jones et al. (1999)	Football, collegiate NCAA Division 1AA players	M=30, 20 y	Maximum concentric acceleration training, control (traditional upper-body training), 14 wks	1RM bench press (BS)
Joy et al. (2016)	Basketball, National Collegiate Athletic Association division II players	M=14, age not provided	Variable resistance training, control, 5 wks	40-y sprint, 1RM back squat, 1RM bench press, 1RM deadlift (BS)
Kale (2016)	Handball, Super League Team players (sport experience > 4 y)	F=19, 20 y	Plyometric training, control, 6 wks	10-m, 20-m, and 30-m sprint (BS)
Katushabe and Kramer (2020)	Soccer, collegiate players (sport experience > 1 y)	M=17, 20 y	Power-band resistance training, conventional resistance training, 6 wks	40-m sprint, 1RM squat (BS)
Kostikiadis et al. (2018)	Combat sport, professional, national level Mixed Martial Arts experienced fighters	M=17, 27 y	“Sport specific” strength training, “regular” strength training, 4 wks	1RM back squat, 1RM bench press, 1RM deadlift (BS)
Kraemer et al. (2003)	Tennis, collegiate players (sport experience = 8.1 ± 3.5 y)	F=30, 19 y	Resistance training (nonlinear periodized), resistance training (non-periodized), control, 9-month	20-m sprint, 1RM leg press and bench press (BS)
Lago-Fuentes et al. (2018)	Futsal, professional players, Spanish First Division Professional Futsal League	F=14, 24 y	Core strength training (stable surface), core strength training (unstable surface), 6 wks	10-m sprint (BS)
Lahti et al. (2020)	Soccer, professional players, premier division in Finland	M=32, 24 y	Resisted sled training (60% velocity decrement from maximal velocity), resisted sled training (50% velocity decrement from maximal velocity), control, 9 wks	30-m sprint (BS)
Li et al. (2019)	Track and field (long distance runners), collegiate well-trained athletes (sport experience > 4 y)	M=28, 21 y	Complex training with endurance training, heavy resistance training with endurance training, control (strength-endurance training and endurance training), 8 wks	50-m sprint and 5-km time trial, 1RM squat (BS)
Losnegard et al. (2011)	Cross country skiing, competitive athletes	F=8, M=11, 21 y	Strength training, control, 12 wks	1RM half-squat, 1RM seated pull-down (BS)
Loturco et al. (2013)	Soccer, professional players (sport experience > 10 y)	M=32, 19 y	Strength/power training (velocity-based), strength/power training (intensity-based), 6 wks	10-m and 30-m sprint, 1RM squat (BS)
Loturco et al. (2015)	Soccer, elite players (sport experience > 6 y)	M=24, 18 y	Resistance training (increased bar velocity group), resistance training (reduced bar velocity group), 6 wks	20-m sprint, 1RM leg-press (BS)
Loturco et al. (2017)	Soccer, professional players	M=18, 22 y	Resistance training (optimum power load + resisted sprints), resistance training (optimum power load + vertical/horizontal plyometrics), 5 wks	30-m sprint (BS)
Manouras et al. (2016)	Soccer, players (sport experience > 3 y)	M=30, 20 y	Horizontal plyometric training, vertical plyometric training, control, 8 wks	10-m and 30-m sprint (BS)
Maroto-Izquierdo et al. (2017)	Handball, professional players, Spanish first division handball league (ASOBAL)	M=29, 22 y	Resistance training (flywheel), control (traditional resistance training with weight-stack machine), 6 wks	20-m sprint, 1RM leg-press (BS)
Marques et al. (2006)	Handball, high level professional players (sport experience = 9.8 ± 1.9 y)	M=16, 23 y, range: 18–29 y	Resistance training, 12 wks	30-m sprint, 1RM bench press (WS)
Marques et al. (2019)	Futsal, players (sport experience = 5.7 ± 2.8 y)	M=21, 18 y	Resistance training, control, 6 wks	20-m sprint (BS)
McCurdy et al. (2009)	Baseball, Division II players	M=28, 21 y	Resistance training (chain-loaded bench press), resistance training (plate-loaded bench press), 9 wks	1RM bench press (plate- and chain-loaded) (BS)
McMaster et al. (2014)	Ruby, semi-professional trained players	M=14, 21 y	Complex resistance training (strength + heavy ballistic), complex resistance training (strength + light ballistic), 2 x 5 wks	30-m sprint, 1RM back squat, 1RM bench press (WS)
McMorrow et al. (2019)	Soccer, professional players, Irish top division	M=18, 25 y	Resisted sled training, unresisted sprint training, 6 wks	20-m sprint (BS)
Mirzak (2015)	Basketball, Ataturk University players	M=28, 22 y	Whole body resistance training, control, 12 wks	1RM bench press (BS)
Mohanta et al. (2019)	Lawn tennis, players (sport experience > 2 y)	M=40, 22 y, range: 18-25 y	Plyometric training, circuit training, 8 wks	50-m sprint, 1RM chess press (BS)
Moore et al. (2005)	Soccer, collegiate athletes (sport experience = 12.5 y)	F=10, M=5, 20 y	Resistance training (Olympic-style lifts), resistance training (traditional),12 wks	25-m sprint (BS)
Morin et al. (2022)	Track and Field (sprinters), trained experienced athletes (sport experience = 7.6 ± 2.0 y)	F=9, 22 y; M=13, 22 y	20-m resisted sprints, 12 wks	5-m and 30-m sprint (WS)
Nonnato et al. (2020)	Soccer, professional players	F=16, 23 y, range: 18-29 y	Plyometric training, control, 12 wks	10-m and 30-m sprint (BS)
Oberacker et al. (2012)	Soccer, National Collegiate Athletic Association Division II players	F=19, 19 y	Resistance training (stable surface), resistance training (unstable surface), 5 wks	30-m sprint (WS)
Oranchuk et al. (2020)	Softball, University players	F=28, 20 y	Resistance training (sport-specific exercises), resistance training (general-training exercises), 8 wks	1RM chop-test (BS)
Ozban (2015)	Soccer, University Sports Club female soccer team, Women First League (sport experience > 5 y)	F=20, 19 y	Plyometric training, control, 10 wks	30-m sprint (BS)
Ozban et al. (2014)	Soccer, University Sports Club female soccer team, Women Second League (sport experience > 4 y)	F=18, 18 y, range: 15-22 y	Plyometric training, control, 8 wks	20-m sprint (BS)
Paavolainen et al. (1999)	Cross country runners (orienteers), elite athletes (sport experience > 8 y)	M=22, 24 y	Explosive-strength training, control, 9 wks	5-km time trial (BS)
Pacholek and Zemkova (2020)	Football, professional players, Slovakia national U19 and senior team members (sport experience = 6.3 ± 1.9 y)	F=13, 20 y	Complex training (intermittent load), combined strength training (maximal strength and dynamic method), 9 wks	1RM bench press, 1RM full squat (WS)
Pareja-Blanco et al. (2017)	Soccer, professional players	M=20, 24 y	Squat training (velocity loss 15%), squat training (velocity loss 30%), 6 wks	30-m sprint (BS)
Paz-Franco et al. (2017)	Futsal, professional players	M=35, 24 y	Lower body resistance training (2 sessions/wk), Lower body resistance training (1 session/wk), Lower body resistance training (1 session every second week), 6 wks	5-m, 10-m, and 15-m sprint (BS)
Pearson et al. (2009)	Sailors, elite-level	M=14, 34 y	Weight training, control, 6 wks	1RM bench pull (BS)
Pedersen et al. (2019)	Football, level two and three in Norway	F=46, 19 y, range: 15-26 y	Maximal strength free-barbell squat training, control, 5 wks	5-m, 10-m, and 15-m sprint, 1RM squat (90° knee angle) (BS)
Prokopy et al. (2008)	Softball, NCAA Division I players, (sport experience range: 0-5 y)	F=14, 21 y	Resistance training (closed–kinetic chain exercises), resistance training (open–kinetic chain exercises), 12 wks	1RM bench press (BS)
Ramirez-Campillo et al. (2014)	Middle- and long-distance runners, highly competitive	F=14, M=22, 24 y	Plyometric training, control, 6 wks	20-m sprint, 2.4 km running endurance (BS)
Ramos-Veliz et al. (2014)	Water polo, elite national-level athletes (sport experience = 8.5 ± 4.1 y)	M=27, 20 y	High-intensity strength training, control, 18 wks	1RM bench press, 1RM full squat (BS)
Randell et al. (2011)	Rugby, professional players (sport experience > 3 y)	M=13, 25 y	Resistance training (real time feedback), resistance training (no feedback), 6 wks	30-m sprint (BS)
Rey et al. (2017)	Soccer, amateur experienced players (sport experience = 14.7 ± 4.1 y)	M=19, 24 y	Resisted sprint training (weighted vest), unresisted sprint training, 6 wks	10-m and 30-m sprint (BS)
Richard and Dawson (2009)	Lower limb power athletes (i.e., sprinter/hurdler, 100 to 800 m or a jumper, long jump, high jump, triple jump).	F=14, range: 18-30 y	Traditional strength training, multidirectional strength training, 6 wks	1RM shoulder flexion and shoulder abduction
Rodriguez-Rosell et al. (2017)	Soccer, semi-professional Spanish third division players	M=30, 25 y	Full squat training, full squat with plyometrics, control, 6 wks	20-m sprint, 1RM squat (estimated) (BS)
Ronnestad et al. (2008)	Soccer, professional players, Norwegian Premier League	M=21, 24 y	Heavy strength training, heavy strength and plyometrics, control, 7 wks	40-m sprint, 1RM half-squat (BS)
Ronnestad et al. (2011)	Soccer, professional players	M=19, 24 y	Strength training (1/wk), strength training (1/every second wk), 10 wks overall +12 wks experimental	40-m sprint, 1RM half-squat (BS)
Ronnestad et al. (2012)	Nordic Combined, well-trained, international athletes	M=17, 20 y	Heavy strength training with usual Nordic combined training, control, 12 wks	1RM deep squat, 1RM seated pull-down (BS)
Sabido et al. (2017)	Handball, first National Handball Division players (sport experience = 10.3 ± 3.4 y)	M=18, 24 y	Eccentric-overload training (flyweel device), control, 7 wks	20-m sprint, 1RM half-squat (BS)
Saez de Villareal et al. (2015)	Water polo, professional athletes, Spanish first division (sport experience = 7.8 ± 3.1 y)	M=30, 23 y	Dryland and in-water specific strength training, in-water specific training, upper and lower dryland plyometric training, 6 wks	1RM bench press, 1RM full squat (BS)
Sedano et al. (2013)	Endurance runners, well-trained (sport experience > 4 y)	M=18, 24 y	General strength with plyometrics and endurance training, endurance strength training, control, 12 wks	5-km time trial, 1RM squat (BS)
Shalfawi et al. (2013)	Soccer, elite well-trained players, second division level in Norway	F=20, 19 y	Strength training, resisted running and repeated sprint training, 10 wks	20-m and 40-m sprint (BS)
Singh et al. (2018)	Field hockey, elite players	F=6, M=11, 23 y	Low-to-high drop jump training, high-to-low drop jump training, 6 wks	10-m and 20-m sprint (BS)
Speirs et al. (2016)	Rugby, academy players	M=18, 18 y	Bilateral strength training, unilateral strength training, 5 wks	10-m and 40-m sprint, 1RM squat (BS)
Spineti et al. (2019)	Soccer, professional players, Brazilian league division	M=22, 18 y	Traditional strength training, complex contrast training, 8 wks	1RM squat (BS)
Storen et al. (2008)	Endurance runners, well-trained	F=8, M=9, 29 y	Maximal strength training, control, 8 wks	1RM squat (90° knee angle) (BS)
Styles et al. (2016)	Soccer, elite professional players	M=17, 18 y	Individualized strength training, 6 wks	5-m, 10-m, and 20-m sprint, 1RM squat (90° knee angle) (WS)
Taher et al. (2021)	Track and field (long jumpers), professional athletes	M=20, 23 y	Vertical and horizontal plyometric training, control, 8 wks	30-m sprint (BS)
Torres-Torrelo et al. (2017)	Futsal, Spanish third division players (sport experience > 10 y)	M=36, 24 y	Full squat training, full squat and COD training, control, 6 wks	20-m sprint, RSA (BS)
Torres-Torrelo et al. (2018)	Futsal, Spanish third division players (sport experience > 10 y)	M=36, 24 y	Full squat training, full squat and COD training, control, 6 wks	20-m sprint, RSA (BS)
Veliz et al. (2014)	Water polo, national level players (sport experience = 8.5 ± 4.1 y)	M=27, 20 y	Strength and power training, control, 16 wks	1RM full squat, 1RM bench press (BS)
Veliz et al. (2015)	Water polo, Spanish first national division players (sport experience = 10.6 ± 4.1 y)	F=21, 26 y	Strength and power training, control, 16 wks	1RM full squat (BS)
Watkins et al. (2021)	Rugby, semiprofessional players	M=32, 20 y	Horizontal plyometric training, vertical plyometric training, control, 6 wks	10-m, 20-m, 30-m sprint (WS)
Zabaloy et al. (2020)	Rugby, highly trained (sport experience > 10 y)	M=33, 22 y, range: 21-24 y	Strength with plyometrics and sprint training, control, 7 wks	5-m, 10-m, 20-m, 30-m sprint, 1RM squat (BS)
Zaferanieh et al. (2021)	Table tennis, elite athletes (sport experience = 5 y)	M=30, 24 y	Power training, ballistic training, control, 8 wks	1RM bench press, 1RM knee extension (BS)

Characteristics of the interventions

Training program durations varied from 4 to 40 weeks. The training interventions consisted of different types of resistance training, including traditional heavy resistance training (e.g., weight training, free weight training, squat training, eccentric exercises), resistance power training (e.g., Olympic lifting, explosive strength training, high-velocity resistance training), plyometrics (e.g., horizontal, vertical jumps, aquatic plyometrics), assistance exercise (e.g., core exercise, elastic bands), machine weights (e.g., machine squat jump training, exercise with isokinetic device) and resisted running (e.g., sled towing).

Characteristics of the tests used to measure outcomes

Most studies evaluated running speed with short linear sprints ranging from 5 to 50 m, including repeated sprint ability tests. Strength evaluation was mainly based on the one-repetition maximum (1RM) test of a back squat (full or half squat variations), bench press, and/or deadlift exercises. The included studies also assessed strength performance with 1RM tests for the pull-over, leg-press, power clean, snatch, clean and jerk, hip thrust, seated lat pull-down, chop-test, cable pulley, step-up, military press, lunge, shoulder flexion, and shoulder abduction. Six studies evaluated resistance training interventions on the improvement in endurance performance with running-based time trials. These outcomes were evaluated at distances ranging from 800 to 5000 m.

Internal validity PEDro items

The evaluation of internal validity items across the 133 studies is illustrated in Table 3 (for complete data, see Supplemental File Table S2). Although 68% (n = 90) of the included studies used random allocation to groups (Criterion 2), only 0.8% (n = 1) reported concealed allocation (Criterion 3). Baseline data (Criterion 4) are presented in 69% (n = 92) of the studies included in this review. Thirty-eight percent (n = 51) of studies showed adequacy of follow-up of tested athletes (Criterion 8). The plan to follow intention-to-treat (Criterion 9) or, stating that all participants received training intervention or control conditions as allocated, were reported in 1.5% (n = 2) of the studies we reviewed. Blinding of assessors (Criterion 7) was satisfied in 1.5% (n = 2) of the studies. None of the studies met the blinding of subjects and training supervisors’ criteria.

Table 3.

Summary of rating for the included studies (n = 133)

PEDro scale item		Rates of meeting criteria
		n	%
1.	Eligibility criteria specified*	28	21.1
2.	Random allocation to groups	89	67.6
3.	Concealed allocation	1	0.8
4.	Groups similar at baseline	92	69.2
5.	Blinding of subjects (athletes)	0	0.0
6.	Blinding of therapists (training supervisors)	0	0.0
7.	Blinding of assessors	2	1.5
8.	Adequacy of follow-up	51	38.3
9.	Intention-to-treat analysis	2	1.5
10.	Between group comparison	113	85.0
11.	Point measures and measures of variability	128	96.2

*criterion not included in the final score

Table S2.

PEDro rating (Criterion,CR) for the included studies (n = 133).

Study	CR 1	CR 2	CR3	CR 4	CR 5	CR 6	CR 7	CR 8	CR 9	CR 10	CR 11	Total
Abade et al. (2019)	y	n	n	n	n	n	n	y	n	n	y	3
Alcarez et al. (2014)	y	y	n	y	n	n	n	n	n	y	y	5
Ali et al. (2019)	y	y	n	y	n	n	n	n	n	y	y	5
Aloui et al. (2019)	n	y	n	y	n	n	n	y	n	y	y	5
Alvarez et al. (2012)	n	y	n	y	n	n	n	n	n	y	y	4
Anderson et al. (2008)	n	y	n	y	n	n	n	y	n	y	y	5
Appleby et al. (2019)	n	n	n	y	n	n	n	y	n	y	y	4
Arazi and Asadi (2011)	n	y	n	y	n	n	n	n	n	n	y	3
Arazi et al. (2018)	y	y	n	y	n	n	n	y	n	y	y	6
Arede et al. (2021)	n	y	n	y	n	n	n	n	n	y	y	4
Ataee et al. (2014)	n	y	n	y	n	n	n	n	n	n	n	2
Ayers et al. (2016)	n	y	n	n	n	n	n	n	n	y	n	2
Bachero-Mena et al. (2019)	n	n	n	y	n	n	n	n	n	y	y	3
Balsalobre-Fernández et al. (2013)	n	n	n	y	n	n	n	n	n	n	y	2
Bartolomei et al. (2014)	n	y	n	y	n	n	n	n	n	y	y	4
Bartolomei et al. (2016)	n	y	n	y	n	n	n	n	n	y	y	4
Beattie et al. (2017)	n	n	n	y	n	n	n	n	n	y	y	3
Ben Brahim et al. (2021)	n	n	n	n	n	n	n	n	n	y	y	2
Berryman et al. (2010)	n	y	n	y	n	n	n	n	n	y	y	4
Berryman et al. (2021)	n	n	n	y	n	n	n	y	n	n	y	3
Blazevich and Jenkins (2002)	n	n	n	n	n	n	n	y	n	y	y	3
Brito et al. (2014)	n	y	n	y	n	n	n	y	n	y	y	5
Burnham et al. (2010)	y	y	n	y	n	n	n	y	n	y	y	6
Campos-Vazquez et al. (2015)	n	y	n	y	n	n	n	n	n	y	y	4
Chelly et al. (2010)	n	y	n	y	n	n	n	y	n	y	y	5
Cherif et al. (2016)	n	n	n	n	n	n	n	n	n	y	y	2
Cherni et al. (2021)	y	n	n	y	n	n	n	y	n	y	y	5
Coratella et al. (2019)	y	y	n	n	n	n	n	n	n	y	y	4
Crewther et al. (2016)	n	y	n	y	n	n	n	y	n	y	y	5
Cross et al. (2018)	n	y	n	y	n	n	n	y	n	y	y	5
Cummings et al. (2018)	y	y	n	y	n	n	n	y	n	n	y	5
Dolezal et al. (2016)	n	y	n	n	n	n	n	y	n	y	y	4
Douglas et al. (2018)	n	y	n	n	n	n	n	y	n	y	y	4
El-Ashker et al. (2019)	n	n	n	y	n	n	n	y	n	y	y	4
Enoksen et al. (2013)	n	n	n	n	n	n	n	n	n	n	y	1
Escobar-Alvarez et al. (2020)	n	n	n	y	n	n	n	n	n	y	y	3
Faude et al. (2013)	n	y	n	n	n	n	n	n	n	y	y	3
Franchini et al. (2015)	y	y	n	y	n	n	n	n	n	y	y	5
Freitas et al. (2019)	n	y	n	y	n	n	n	n	n	y	y	4
Ghigiarelli et al. (2009)	n	y	n	y	n	n	n	n	n	y	y	4
Gil-Cabrera et al. (2018)	n	y	n	n	n	n	n	y	n	y	y	4
Gjinovci et al. (2017)	n	y	n	n	n	n	n	n	n	y	y	3
Grazioli et al. (2020)	n	y	y	y	n	n	y	n	n	y	y	6
Guglielmo et al. (2009)	n	y	n	y	n	n	n	y	n	y	y	5
Hansen et al. (2011)	y	y	n	y	n	n	n	n	n	y	y	5
Hariss et al. (2008)	n	y	n	y	n	n	n	n	n	y	y	4
Harrison and Bourke (2009)	n	y	n	n	n	n	n	n	n	y	n	2
Hermassi et al. (2010)	n	y	n	y	n	n	n	n	n	y	y	4
Hermassi et al. (2011)	n	y	n	y	n	n	n	n	n	n	y	3
Hermassi et al. (2014)	n	n	n	y	n	n	n	n	n	y	y	3
Hermassi et al. (2015)	n	y	n	y	n	n	n	n	n	y	y	4
Hermassi et al. (2017)	n	n	n	y	n	n	n	y	n	y	y	4
Hermassi et al. (2019a)	n	n	n	n	n	n	n	y	n	y	y	3
Hermassi et al. (2019b)	n	y	n	y	n	n	n	n	n	y	y	4
Hermassi et al. (2019c)	n	n	n	n	n	n	n	y	n	y	y	3
Hermassi et al. (2020)	n	y	n	y	n	n	n	n	n	y	y	4
Hertzog et al. (2020)	n	n	n	y	n	n	n	n	n	y	y	3
Hoff and Almasbakk (1995)	n	y	n	y	n	n	n	n	n	y	y	4
Hoff et al. (2002)	n	y	n	y	n	n	n	y	n	y	y	5
Hoffman et al. (1990)	n	n	n	n	n	n	n	n	n	n	y	1
Hoffman et al. (1991a)	n	n	n	n	n	n	n	n	n	n	y	1
Hoffman et al. (1991b)	n	n	n	y	n	n	n	n	n	n	y	2
Hoffman et al. (2004)	n	n	n	y	n	n	n	y	n	y	y	4
Hoffman et al. (2005)	n	y	n	y	n	n	n	n	n	y	y	4
Hoffman et al. (2009)	n	y	n	y	n	n	n	n	n	y	y	4
Hong-Sun et al. (2009)	n	n	n	y	n	n	n	n	n	n	y	2
Horwath et al. (2019)	n	y	n	y	n	n	n	y	n	y	y	5
Iacono et al. (2017)	y	y	n	n	n	n	n	n	n	y	y	4
Impellizzeri et al. (2008)	n	y	n	n	n	n	n	n	n	y	y	3
Iodice et al. (2020)	n	y	n	y	n	n	n	n	n	y	y	4
Izquierdo-Gabarren et al. (2010)	n	y	n	y	n	n	n	n	n	y	y	4
Jones et al. (1999)	n	y	n	y	n	n	n	n	n	n	y	3
Joy et al. (2016)	n	n	n	y	n	n	n	n	n	n	n	1
Kale (2016)	n	n	n	y	n	n	n	n	n	n	y	2
Katushabe and Kramer (2020)	y	y	n	y	n	n	n	n	n	y	y	5
Kostikiadis et al. (2018)	y	n	n	y	n	n	n	n	n	y	y	4
Kraemer et al. (2003)	n	y	n	y	n	n	n	y	n	y	y	5
Lago-Fuentes et al. (2018)	y	y	n	y	n	n	n	y	n	y	y	6
Lahti et al. (2020)	y	n	n	n	n	n	n	y	n	y	y	4
Li et al. (2019)	n	n	n	y	n	n	n	y	n	y	y	4
Losnegard et al. (2011)	y	n	n	y	n	n	n	n	n	y	y	4
Loturco et al. (2013)	n	y	n	y	n	n	n	n	n	y	y	4
Loturco et al. (2015)	n	y	n	y	n	n	n	n	n	y	y	4
Loturco et al. (2017)	n	y	n	y	n	n	n	n	n	y	y	4
Manouras et al. (2016)	n	y	n	y	n	n	n	n	n	y	y	4
Maroto-Izquierdo et al. (2017)	y	y	n	y	n	n	n	n	n	y	y	5
Marques et al. (2006)	n	n	n	n	n	n	n	y	n	n	y	2
Marques et al. (2019)	n	n	n	y	n	n	n	n	n	y	y	3
McCurdy et al. (2009)	n	n	n	n	n	n	n	n	n	y	y	2
McMaster et al. (2014)	n	n	n	n	n	n	n	n	n	y	y	2
McMorrow et al. (2019)	n	y	n	y	n	n	n	n	n	y	y	4
Mirzak (2015)	n	n	n	y	n	n	n	n	n	y	y	3
Mohanta et al. (2019)	y	y	n	y	n	n	n	n	n	y	y	5
Moore et al. (2005)	n	n	n	y	n	n	n	n	n	y	y	3
Morin et al. (2022)	n	n	n	y	n	n	n	n	n	n	y	2
Nonnato et al. (2020)	n	y	n	n	n	n	n	y	y	y	y	5
Oberacker et al. (2012)	n	n	n	y	n	n	n	n	n	y	y	3
Oranchuk et al. (2020)	n	y	n	n	n	n	n	y	n	n	y	3
Ozban (2015)	y	n	n	y	n	n	n	y	n	y	y	5
Ozban et al. (2014)	n	y	n	y	n	n	n	y	n	y	y	5
Paavolainen et al. (1999)	n	n	n	y	n	n	n	n	n	y	y	3
Pacholek and Zemkova (2020)	n	n	n	n	n	n	n	n	n	y	y	2
Pareja-Blanco et al. (2017)	n	y	n	n	n	n	n	n	n	y	y	3
Paz-Franco et al. (2017)	n	y	y	y	n	n	n	y	n	y	y	6
Pearson et al. (2009)	n	y	n	y	n	n	n	y	n	y	y	5
Pedersen et al. (2019)	y	y	n	y	n	n	n	n	n	y	y	5
Prokopy et al. (2008)	n	y	n	n	n	n	n	y	n	y	y	4
Ramirez-Campillo et al. (2014)	y	y	n	y	n	n	n	y	n	y	y	6
Ramos-Veliz et al. (2014)	n	y	n	n	n	n	n	n	n	y	y	3
Randell et al. (2011)	n	y	n	y	n	n	n	n	n	y	y	4
Rey et al. (2017)	n	y	n	y	n	n	n	y	n	y	y	5
Richard and Dawson (2009)	y	y	n	y	n	n	y	y	y	y	y	8
Rodriguez-Rosell et al. (2017)	n	y	n	n	n	n	n	n	n	y	y	3
Ronnestad et al. (2008)	n	y	n	y	n	n	n	n	n	y	y	4
Ronnestad et al. (2011)	n	y	n	n	n	n	n	n	n	n	y	2
Ronnestad et al. (2012)	n	n	n	y	n	n	n	y	n	y	y	4
Sabido et al. (2017)	n	n	n	n	n	n	n	y	n	y	y	3
Saez de Villareal et al. (2015)	y	y	n	n	n	n	n	n	n	y	y	4
Sedano et al. (2013)	y	y	n	y	n	n	n	y	n	y	y	6
Shalfawi et al. (2013)	n	y	n	n	n	n	n	n	n	y	y	3
Singh et al. (2018)	n	y	n	n	n	n	n	n	n	y	y	3
Speirs et al. (2016)	y	y	n	y	n	n	n	n	n	y	y	5
Spineti et al. (2019)	y	y	n	n	n	n	n	y	n	y	n	4
Storen et al. (2008)	n	y	n	y	n	n	n	y	n	y	y	5
Styles et al. (2016)	y	n	n	n	n	n	n	n	n	n	y	2
Taher et al. (2021)	n	n	n	y	n	n	n	n	n	y	y	3
Torres-Torrelo et al. (2017)	n	y	n	y	n	n	n	y	n	y	y	5
Torres-Torrelo et al. (2018)	n	y	n	y	n	n	n	y	n	y	y	5
Veliz et al. (2014)	n	y	n	n	n	n	n	y	n	y	y	4
Veliz et al. (2015)	n	y	n	n	n	n	n	y	n	y	y	4
Watkins et al. (2021)	n	y	n	y	n	n	n	y	n	y	y	5
Zabaloy et al. (2020)	y	n	n	n	n	n	n	y	n	y	y	4
Zaferanieh et al. (2021)	n	y	n	n	n	n	n	n	n	y	y	3

Non-internal validity PEDro items

The eligibility criteria (Criterion 1) were specified in 22% (n = 28) of studies. Reporting of results of between-group statistical comparisons (Criterion 10) and point measures and measures of variability (Criterion 11) were included in 85% (n = 113) and 96% (n = 128) of studies, respectively.

Discussion

This systematic review aimed to provide a comprehensive overview of the procedures that were used to ensure internal validity in resistance training research in athletes. This review revealed strengths and limitations of study designs and reporting procedures in strength and conditioning research. The main findings of the current study suggest that internal validity, as assessed by particular PEDro scale items, varied from moderate to very low. Accordingly, there are methodological safeguards which should be widely adopted in experimental studies in strength and conditioning to improve internal validity.

Randomization is fundamental to the design and conduct of experimental research, the current analysis of resistance training research showed there is still room for improvement. Thirty-two percent of analyzed studies used less rigorous, often quasi-experimental design, in which there is only one group or in which randomization to more than one group is lacking (Balsalobre-Fernandez et al., 2013; Grazioli et al., 2020; Bachero-Mena et al., 2021). Quasi-experimental design does not control for unwanted systematic differences (selection bias) between groups, therefore it is a potential a threat to internal validity. This limitation occurs quite often in strength and conditioning research due to athletes’ or coaches’ training preferences. For example, coaches may be concerned about a reduction in preparedness for their athletes who are assigned to an intervention that they perceive to have less efficacy. There is also an ethical problem, if participant in the control group miss out taking part in experimental intervention which usually are considered as more beneficial than the intervention (or the lack of intervention) assigned to control group. To deal with these constraints, alternatively, researchers may consider the use of crossover or within-subject designs, where each participants of the study receive a series of interventions in random order and the outcomes are uniquely associated with each intervention.

Sixty-eight percent of the included studies were considered to have a random allocation, but only a few of them provide information about the techniques of randomization utilized (Anderson et al., 2008; Impellizzeri et al., 2008; Ali et al., 2019). From a methodological assessment of randomization procedure perspective, providing information about which technique was utilized in the study is important. For example, it is known that simple randomization is suitable for large samples of participants (n > 100), but not appropriate when the size of sample is small, which is very common for strength and conditioning research – on average, 23 participants took part in each study in the present review. In such cases, utilizing of stratified randomization is recommended. This method allows for control and balancing the influence of covariates. The prerequisite for stratified randomization is that researchers should be able to identify each of the covariates which potentially influence key outcomes prior to group allocation. Common covariates for evaluation of effectiveness of resistance training interventions are age, sex, body mass, and training status. Considering above arguments, we believe that providing details whether, and how randomization was performed is important for internal validity of studies in strength and conditioning.

Despite data which suggests that lack of allocation concealment may improperly lead to a larger effect for a particular intervention (Odgaard-Jensen et al., 2011), only one of the included studies reported concealed allocation (Grazioli et al., 2020). This is consistent with data presented on the Physiotherapy Evidence Database (PEDro) (https://pedro.org.au/wp-content/uploads/11Jan2021.pdf). However, given the ease of implementation and reporting of concealed allocation, which does not seem to present the challenges in strength and conditioning research which have been noted in other criterion, the lack of intention to this is concerning. Concealed allocation is a safeguard against researchers consciously or subconsciously introducing systematic differences in groups (Elkins, 2013). For example, when the effectiveness of a particular training method is being tested, athletes who are anticipated to have low adherence, or perhaps those athletes who are considered to be of a lower level by coaches or investigators, may be delayed until the probability of allocation to a control group is greater. Therefore, performing and reporting of concealed allocation in strength and conditioning research are required.

Randomized allocation to intervention and control groups does not guarantee that the groups are similar at baseline. Therefore, it is recommended to compare a few variables before the intervention to investigate the whether the groups are comparable. According to the PEDro scale, an article should provide data for each group for at least two variables, one measure of severity, and at least one (different) key outcome, which provides a measure of effectiveness or lack of effectiveness of the intervention. This criterion is only satisfied if baseline values are presented. Since strength and conditioning research typically includes healthy participants, the authors of this review replaced the "measure of severity of condition treated" with measures more related to the specifics of strength and conditioning studies, namely body mass and height. There is large of body research that indicate that body characteristics may influence resistance training outcomes (Twist et al, 2021). Almost 70% of the included studies reported these data, mainly in a table or figure. However, reporting of information on other baseline variables which may potentially influence the outcomes of resistance training intervention, such as results of one repetition maximum (1RM) testing, was much less common (Grgic et. al, 2020). This is concerning, as the training status of athletes would seem to clearly influence the potential for adaptation to a given intervention.

Interestingly, although using statistical significance tests for baseline data is not recommended (Schulz et al. 2010), statistical testing for judging the baseline comparability was common in the included studies. According to recommendations published in a top medical journal, The Lancet, it is sufficient for baseline characteristics to be reported and compared using descriptive statistics with a mean and a measure of variability (Schulz and Grimes, 2002). In addition, Schulz and Grimes (2002) proposed that continuous variables (e.g., age, body mass) may be reported as a mean and standard deviation. However, when data distribute asymmetrically, a median and percentile range (interquartile range) are more appropriate (Schulz and Grimes, 2002).

Blinding of participants and training instructors who involved in resistance training research is not realistic. However, those who are assessing and interpreting outcomes, including: data collectors, judicial assessors of outcomes and data analysts should be blinded to increase internal validity. Only 1.5% (n=2) of the studies included reported this procedure. These results are consistent with those of other reviews of resistance training interventions, which suggest that this criterion of internal validity is often not met (Thiele et al., 2020; Trowell et al., 2020). When assessors are blinded, they may be less likely to have conscious or unconscious biases affect outcome assessments due to their expectations or beliefs about the effectiveness of the intervention. There are several strategies aimed at blinding assessors in strength and conditioning training (e.g. preventing assessors from accessing data which has the potential to compromise blinding).

Withdrawal and dropout of participants provide various challenges to investigators due to changing the baseline characteristics between the randomized groups. High rate of attrition and uncontrolled attrition have high potential to harm internal validity of outcomes. According to the criterion of the PEDro scale, a high risk of attrition appears when the publication does not explicitly state both the number of subjects who were initially allocated to a group and when key outcomes have not been measured in more than 85% of subjects. It should be noted that authors of the included studies in strength and conditioning often did not explicitly report the number of participants who completed the study. Adequacy of follow-up was fulfilled only by 40% of the studies. As this is an issue of reporting rather than design, this is simple to improve in future investigations. Attrition from research training programs in athletes may be due to several potential reasons, such as injury, concern for injury, loss of motivation to participate in research, or scheduling conflicts with their training program. To prevent high attrition rate and increase compliance with the protocol for the assigned groups, the research should be attractive and beneficial for participants and coaches of these athletes. It is good practice to include coach into research team.

The other method of reducing attrition bias is to implement an intention-to-treat analysis (Moseley et al., 2011). Employing the intention-to-treat procedures is not difficult. Firstly, the subjects whose participation in training intervention has been interrupted should be encouraged to participate in the remaining outcome measurements, if possible. Secondly, statistical analysis should reflect the allocation design and consider all obtained data (Elkins and Moseley 2015). Note that several methods have been identified to fill in the missing data (Nakai et al., 2014; Smart et al., 2015). In our review, intention-to-treat was undertaken only in 1.5% (n=2) of all included studies (Nonnato et al., 2020; Richards and Dawson, 2009).

The PEDro scale items also assess if statistical information reported in a study are interpretable. To fulfil this criterion, a study needs to report between-group comparisons. The second criterion is satisfied when a study reports both a measure of the size of the intervention effect and a measure of variability for at least one key outcome. As noted earlier, our findings showed that most of the reviewed studies demonstrated sufficient information to make them interpretable, 85% for criterion 10 and 96% for criterion 11.

The third, outside internal validity, criterion relates to external validity. This criterion is considered to have been met if a publication describes the source of the subject pool and when a list of inclusion or exclusion criteria of participants in the study is reported. Although reporting the source of participants of the study and specified eligibility criteria is an important attribute of generalizability of the study findings and is easy to apply in resistance training research, only one in five articles in the current review included this information. These results are consistent with those of other systematic reviews which investigated the effects of resistance training on the performance of athletes (Thiele et al., 2020; Trowell et al., 2020). For example, Trowell et al. (2020) found that 75% of included studies did not list eligibility criteria for participants. To address this, and the threats mentioned above to internal validity, researchers may wish to consider the use of CONSORT (Consolidated Standards of Reporting Trials) guidelines (Schulz et al., 2010).

This systematic review provides an overview of the literature regarding internal validity procedures in resistance training studies. Nonetheless, some limitations should be considered. An investigation of internal validity was based only on criteria included in the PEDro scale. Other variables may need to be considered when validity in resistance training research is investigated. From the present study, it is impossible to determine if internal validity limitations in the included studies were caused by shortcomings in the study design or reporting procedures. The sample used in the present review was limited to athletes; therefore, these findings may not be generalizable to studies involving non-athletes.

Although the PEDro scale is used to evaluate the quality of randomized controlled trials in systematic reviews, our findings showed that particular items of the PEDro scale may be used as a guideline in study design and conducting experimental research in strength and conditioning. The largest threats to internal validity in the studies we analyzed were associated with concealed allocation, intention-to-treat, and blinding of assessors of the main outcomes. Further improvement in the quality of studies should also involve random allocation to groups, ensuring that groups are similar at baseline, measures of key outcomes from the highest number of the subjects according to their initial allocation to groups, as well as one external validity variable (specification of eligibility criteria).

Conclusion

Because the current review showed that internal validity items like concealed allocation, intention to treat, and blinding of assessors of the main outcomes are often not reported in resistance training studies, the internal validity of future studies should be improved. Well-designed, well-conducted, and well-reported experimental research studies are essential to confirm whether training interventions improve outcomes as poorly designed and reported studies can mislead decision making in professional practice.

Biographies

Hubert MAKARUK

Employment

Assistant Professor at Józef Pilsudski University of Physical Education in Warsaw, Faculty of Physical Education and Health, Poland

Degree

PhD

Research interests

plyometrics, strength and conditioning, athletic performance, motor learning

E-mail: hubert.makaruk@awf.edu.pl

Marcin STARZAK

Employment

Assistant at Józef Pilsudski University of Physical Education in Warsaw, Faculty of Physical Education and Health, Poland

Degree

PhD

Research interests

sports science, strength and conditioning, athletic performance

E-mail: marcin.starzak@awf.edu.pl

Maciej PŁASZEWSKI

Employment

Assistant Professor at Józef Pilsudski University of Physical Education in Warsaw, Faculty of Physical Education and Health, Poland

Degree

PhD

Research interests

Evidence-Based practice, physiotherapy, systematic reviews

E-mail: maciej.plaszewski@awf.edu.pl

Jason B. WINCHESTER

Employment

Associate Professor: Division of Health Sciences & Human Performance, Concordia University Chicago, USA

Degree

PhD

Research interests

human performance, injury prevention, strength and conditioning

E-mail: jwinchester@outlook.com

Supplements