Validity and Reliability of the Turkish Version of the Medial Tibial Stress Syndrome Score

Abstract

Background:

Medial tibial stress syndrome (MTSS) is a common leg injury in military personnel and athletes and is especially related to running and jumping. A patient-reported outcome measure, the MTSS score, was developed to determine the severity of MTSS.

Purpose:

To translate, culturally adapt, and validate the MTSS score for the Turkish language.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Methods:

Established guidelines were used for translation and adaptation. The Turkish version of the MTSS (MTSS-Tr) score was completed twice with a 1-week interval between assessments. In the first assessment, patients also completed the Turkish version of the 36-Item Short-Form Survey (SF-36-Tr) and a visual analog scale (VAS) for pain. Test-retest reliability and internal consistency of the MTSS-Tr were measured with the intraclass correlation coefficient (ICC) and Cronbach α coefficient, respectively. The construct validity was demonstrated with the Spearman correlation coefficient (r_S).

Results:

A total of 48 participants were included in the study. The test-retest reliability was good and internal consistency was good (ICC, 0.9; Cronbach α, 0.884). The MTSS-Tr score was highly negatively correlated with the physical component score of the SF-36-Tr (r_S = −0.716; P < .001). There was a moderate correlation between the MTSS-Tr score and the VAS pain score (r_S = 0.465; P = .001).

Conclusion:

The translated MTSS-Tr score has good internal consistency and good reliability and validity. Therefore, the MTSS-Tr score is useful to evaluate symptoms in patients with MTSS.

Registration:

NCT05400668 (ClinicalTrials.gov identifier).

Medial tibial stress syndrome (MTSS) is a common overuse injury in military personnel and athletes. ¹¹ MTSS is most common in runners, and it is also common in other sports such as soccer, basketball, and dance.^13,24 MTSS is defined as exercise-induced pain along the posteromedial border of the tibia, and recognizable pain is provoked on palpation of this posteromedial border over a length of at least 5 cm. ²⁴

Patient-reported outcome measures have been developed that can quantitatively assess injury severity from the patient's perspective and can accurately measure change in the patient's clinical status.^6,17 Various outcome measures such as recovery time, visual analog scales (VASs), Likert scales, and numeric rating scales have been used in the evaluation of patients with MTSS^7,10; however, until 2016 there had been no MTSS-specific standardized assessment instrument to evaluate the severity of injury. ²² In 2016, Winters et al ²³ published a new patient-reported outcome measure to help evaluate the severity of MTSS and named it the MTSS score. The score consists of 4 items, each with 4 response options, for a possible total of 10 points: item 1 assesses current limitations in sports activities (0-3 points), item 2 assesses pain while performing sports activities (0-3 points), item 3 assesses pain while walking (0-2 points), and item 4 assesses pain at rest (0-2 points). Higher scores indicate more severe pain and limitations and hence more severe symptoms. The MTSS score is a valid, reliable, and responsive measure for assessing injury severity in patients with MTSS. ²³

The aim of the present study was to translate and culturally adapt the MTSS score into the Turkish language using an internationally recognized methodology and to validate the Turkish version of the MTSS (MTSS-Tr) score in patients diagnosed with MTSS.

Methods

Study Design

The study consisted of cross-culturally translating the original Dutch MTSS score into the Turkish language and then testing the MTSS-Tr score for its validity and reliability. The medical ethics committee of our hospital provided approval of the study protocol, and the project was registered on ClinicalTrials.gov (registration No. NCT05400668). We followed the Consensus-based Standards for Selection of Health Measurement Instruments guidelines while validating the MTSS-Tr score. ¹²

Translation of Original Dutch MTSS Score Into the Turkish Language

Information about the original MTSS score was obtained from the developers. The translation of the MTSS score was conducted according to the guidelines published by Beaton et al ¹ as well as through slight modifications to the backward translation method described by Werner and Campbell. ²⁰ The translation process consisted of the following 4 phases:

1. Forward translation by 2 independent Dutch-Turkish bilingual translators;

2. Initial assessment of the comprehensibility of the translated score by an expert committee;

3. Backward translation into Dutch by 2 native Dutch speakers blinded to the original version with no medical background; and

4. Discussion by the expert committee and agreement on a final version.

The first phase was performed by 2 bilingual Dutch translators whose native language was Turkish. The expert committee included all translators and the research team (5 sports medicine physicians, 1 public health specialist, and 1 physical therapist). Backward translation of the Turkish version into Dutch was performed by 2 other bilingual Dutch translators who did not know the purpose of study and were completely blind to the original version of the score. The forward and backward translations were then compared, and final corrections were made for the penultimate version. The research team reviewed and corrected any incomprehensible or incomplete content. During this process, the face validity, ease of application, and order and readability of the items were evaluated by the expert panel.

In the last stage, the questionnaire was tested for comprehensibility on 30 randomly selected patients with MTSS. The patients were asked 3 questions upon completion of the questionnaire: (1) Do you think the survey questions included questions about your pain/complaint? (2) Was it difficult to complete the survey? (3) Would you like to add or change any questions in the survey? After this procedure, there was no need for revisions. The final form of the MTSS-Tr score was established and is shown in Appendix Table A1.

Cross-Cultural Validation of the MTSS-Tr Score

Participants

The cross-cultural validation portion of the study was conducted between June and October 2022 at the sports medicine outpatient clinics of 3 hospitals (Bozyaka Education and Research Hospital, Tepecik Education and Research Hospital, and Ege University Hospital). Sports medicine physicians at each center evaluated the inclusion and exclusion criteria. Individuals aged ≥18 years who were diagnosed with MTSS and who were native to Turkey and literate in the Turkish language were included in the study. Participants with a tibial fracture history, suspected chronic compartment syndrome, and clinical suspicion of stress fracture or concomitant injuries were excluded from the study. Included participants were verbally informed about the purpose of the study and signed informed consent forms.

Sample Size Estimation and Final Study Sample

We performed an a priori analysis of the required sample size for test-retest reliability analysis and confirmatory factor analysis. The size of the smallest sample necessary for test-retest reliability of the MTSS-Tr score and the confirmatory factor analysis was determined to be 48 patients, comprising at least 10 patients per calculated item in the score and a 20% nonresponse rate. ²

Of 64 patients admitted to the clinic for pain in the lower leg, 53 patients were diagnosed with MTSS. All patients diagnosed with MTSS volunteered to participate in the study. Three individuals were excluded, as they were <18 years of age, and 2 patients did not respond to repeated telephone calls for follow-up. Thus, 48 people were included in the analysis.

Assessment Procedure

The participants were interviewed and the MTSS-Tr score was completed twice, with a 1-week interval between assessments. During the first visit, participants completed the MTSS-Tr score as well as the Turkish version of the 36-Item Short-Form Survey (SF-36-Tr) and a VAS for pain and provided information relating to their demographic characteristics and sports activities. During the second visit, the MTSS score was completed again. In addition, participants completed a transition scale in which they indicated their condition as “worsened,”“improved,” or “unchanged” compared with the visit, to evaluate the patient's perception of change in injury status. Participants who selected “unchanged” were considered stable. ²³

Outcome Measures

36-Item Short-Form Survey

The SF-36 measures impairments in physical, mental, and social functioning and is widely used with many translations to assess the health status burden of people worldwide. The questionnaire consists of 36 questions that are clustered into 8 areas of health status, which are represented by 2 domains, a physical component summary (PCS) and a mental component summary (MCS). The combined score for the PCS and MCS ranges from 0 to 100, with higher scores indicating higher levels of function and/or better health-related quality of life. ¹⁸ The SF-36 was translated and validated in Turkish (SF-36-Tr) in 1999. ⁸

VAS for Pain

The VAS is a simple and frequently used rating tool to measure subjective pain. The VAS uses an analog format in which the pain level represents a continuous range of values, with “no pain” at 0 points and “worst pain” at 10 points. The patient is asked to give a pain score appropriate to the level of pain they feel. ²¹

Statistical Analysis

Statistical analysis was performed using SPSS for Windows Version 21.0 (IBM Corp). Continuous variables are reported as mean ± standard deviation or median [interquartile range], depending on normality, while categorical variables are reported as numbers and proportions. Normal distribution of the continuous variables was investigated using the Shapiro-Wilk test. After the cross-cultural translation process, the MTSS-Tr score was assessed for its reliability and validity. Structural validity, internal consistency, test-retest reliability, and construct validity were examined. The threshold for statistical significance was set at a P value of <.05.

Structural Validity and Internal Consistency

The structural validity was examined using confirmatory factor analysis. Because the MTSS score does not have any subscores, it was expected to have a single-factor structure. The Kaiser-Meyer-Olkin (KMO) test was used to measure sample adequacy. The Bartlett test of sphericity was used to measure the equality of variances between groups against the alternative that the variances are unequal for at least 2 groups. A KMO value of >0.5 and a significant result for the Bartlett test (P < .05) indicate that there is substantial correlation in the data. KMO measures were also calculated for each variable, in which values >0.5 were deemed acceptable.

Internal consistency of the MTSS-Tr score was determined by calculating the Cronbach α coefficient. A Cronbach α of 0.6 was considered acceptable and >0.75 was considered good.^4,14

Test-Retest Reliability, Measurement Error, and Smallest Detectable Change

Test-retest reliability was evaluated in those participants whose symptoms were considered stable as determined by the transition scale. Test-retest reliability was assessed using a 2-way random-effects, consistent, single-measure intraclass correlation coefficient (ICC) for all items. ICC values were regarded as insufficient if <0.50, acceptable if between 0.50 and 0.75, and good if >0.75. ¹⁵ Test-retest reliability was measured for each of the 4 items as well as the total MTSS-Tr score.

We expressed measurement error by the standard error of measurement (SEM) and by calculating the smallest detectable change (SDC) at both the individual and group levels. The SEM was calculated as SEM = SD_{measurement 1+2}× (√1 – ICC). ¹⁴ The SDC was calculated at the individual level as SEM × 1.96 ×√2 and at the group level as (SEM × 1.96 ×√2)/√n.^5,19

Construct Validity

Construct validity was evaluated using the Spearman correlation coefficient (r_S) for the association of the MTSS-Tr score with the SF-36-Tr PCS domain and VAS pain scores. We accepted r_S values of 0.1 as small, 0.3 as moderate, and ≥0.5 as large. ³

Results

The characteristics of the 48 study participants are shown in Table 1. The mean MTSS symptom duration was 53.4 ± 38.2 months (range, 3-96 months), and the mean duration of participation in sports activities was 9.2 ± 4.3 years (range, 5-15 years).

Table 1.

Characteristics of the Included Patients (N = 48) ^a

Variable	Value
Sex
Male	19 (39.6)
Female	29 (60.4)
Age, y	24 [22-25]
Height, cm	170 [165-180]
Weight, kg	68 ± 12.1
BMI, kg/m2	22.4 ± 2.5
Occupation
Athlete	32 (66.7)
Military personnel	6 (12.5)
Trainee preparing for police work	10 (20.8)
Hours of exercise per week at the first assessment	8 [5-10]
Duration of symptoms, mo	52 [14-90]
Side of symptoms
Bilateral	38 (79.2)
Left	5 (10.4)
Right	5 (10.4)

^aData are presented as n (%), median [interquartile range], or mean ± SD. BMI, body mass index.

Cross-Cultural Validation of the MTSS-Tr Score

Structural Validity and Internal Consistency

As the original Dutch MTSS score was found to be unidimensional, it was expected that the MTSS-Tr score would also exhibit a single-factor structure. Factor loadings for the MTSS-Tr score are presented in Table 2. All items (1-4) loaded on “item 1,” which was the only factor of the MTSS-Tr score. This result was consistent with the original MTSS score. The KMO value was 0.753, indicating sufficient sampling, and the significance level of the Bartlett test was P < .001, indicating suitability for factor analysis. The Cronbach α for the MTSS-Tr score was 0.884, indicating a good level of internal consistency.

Table 2.

Confirmatory Factor Analysis and Internal Consistency Analysis ^a

Variable	Factor Analysis	Internal Consistency Analysis (Cronbach α)
MTSS-Tr score		0.884
Item 1: Current sports activities	0.757
Item 2: Pain while performing sports activities	0.791
Item 3: Pain while walking	0.758
Item 4: Pain at rest	0.637

^aMTSS-Tr, Turkish version of the medial tibial stress syndrome.

Test-Retest Reliability, SEM, and SDC

In 17 participants, the injury status had remained unchanged from the first assessment to the second assessment and was considered stable. The test-retest reliability of the MTSS-Tr score was between 0.58 and 0.81 for each item (Table 3). The MTSS-Tr score exhibited good test-retest reliability in the stable patients (ICC, 0.901). The SEM was 1.16. The SDC on the individual level was 3.21. The SDC at the group level was 0.78.

Table 3.

Test-Retest Reliability for the MTSS-Tr Score ^a

Variable	Visit 1	Visit 2	Test-Retest Reliability (ICC)	P
Item 1: Current sports activities	1.83 ± 0.69	1.70 ± 0.68	0.812	<.001
Item 2: Pain while performing sports activities	2.33 ± 0.69	2.17 ± 0.80	0.538	<.001
Item 3: Pain while walking	1.56 ± 0.68	1.63 ± 0.87	0.685	<.001
Item 4: Pain at rest	1.67 ± 0.75	1.69 ± 0.69	0.660	<.001
MTSS-Tr score (n = 48)	3.40 ± 1.91	2.94 ± 1.74	0.768	<.001
MTSS-Tr score for stable patients (n = 17)	2.65 ± 1.90	2.65 ± 1.77	0.901	<.001

^aData are presented as mean ± SD. ICC, intraclass correlation coefficient; MTSS-Tr, Turkish version of medial tibial stress syndrome.

Construct Validity

The MTSS-Tr score and the PCS and MCS domains of the SF-36-Tr were negatively correlated (r_S = −0.716 [P < .001] and r_S = −0.348 [P = .015], respectively). The PCS domain of the SF-36-Tr was strongly and positively correlated with the MTSS Tr score (r_S > 0.5). These findings were as we expected. The VAS pain values were moderately to strongly positively correlated with the MTSS-Tr score (r_S = 0.465; P = .001).

Discussion

The MTSS-Tr score was found to be a valid, reliable, and responsive instrument to measure pain injury severity of MTSS. To the best of our knowledge, this is the first cross-culturally translated and validated version of the MTSS score in a different language. Our study showed that the Turkish and Dutch versions of the MTSS score assess the same aspects of clinical severity in this group of patients. The MTSS-Tr score is a simple-to-use 4-item scale that addresses pain at rest, pain while performing activities of daily living, limitations in sporting activities, and pain while performing sporting activities in Turkish populations with MTSS.

Commonalities and Differences Between the Dutch and Turkish MTSS Scores

The construct validity of the MTSS-Tr score was evaluated to determine its correlation with the PCS domain of the SF-36-Tr. The SF-36-Tr is a frequently used, valid, and reliable scale that has been used in many studies in Turkey. ⁸ This finding of significant correlations between the MTSS-Tr and the SF-36-Tr gives us valuable information. In the original study, 2 questions in the SF-36, one assessing walking a few hundred meters and the other walking >1 km and an additional 2 questions evaluating sports intensity and volume change, were compared with each question of the MTSS. ²³ The SF-36 questionnaire does not have a total score; after the calculation of different parameters, scores evaluating the physical and mental components (PCS and MCS) can be obtained. ¹⁸ Of the 2 domains, we chose to compare the MTSS-Tr score with only the SF-36-Tr PCS score because when the SF-36-Tr questionnaire was examined, the items on the MTSS-Tr score that could be calculated were more closely related to questions about the physical aspects of patients’ quality of life. Consequently, significant correlations were found between the MTSS-Tr score and the SF-36-Tr PCS.

According to our reproducibility results, good test-retest reliability was found for the MTSS-Tr score (ICC, 0.901). This result was consistent with that of the original study, ²³ all items were suitable to remain in the MTSS-Tr score, and the reliability of the items was reconfirmed by this study. For test-retest analysis, the interval between the first and second assessments should be 2 to 4 weeks. ⁹ However, the memory factor, the individual's sensitization to the subject, and changes that may occur over time should also be taken into account. Considering that the level of complaints for diseases may change with the effect of treatment in the long term, based on the original version, we needed to include a change criterion for patients. Thus, the 17 patients in the unchanged (stable condition) classification (35% of the study sample) were subject to test-retest analysis. Ten percent of the total sample size is reported to be sufficient for test-retest reliability analysis. ¹⁶

The mean Cronbach α value of the MTSS-Tr score was 0.884, indicating good internal consistency. In the original version of the scale, the Cronbach α internal consistency coefficient was 0.58. ²³ The Cronbach α value of the MTSS-Tr score was higher but was consistent with the original MTSS score. At the same time, low measurement error is important for the usability of the MTSS-Tr score. A fairly large SDC (3.21) was found for individual differences, but at the group level the SDC was smaller (0.78).

Of the study participants, 66.7% were athletes, and the rest were soldiers and trainees preparing for police work. The rates are similar to those in the original version, in which 65% were athletes and 35% were military personnel. ²³ In our study, the median age was 24 years. In the study by Winters et al, ²³ the mean age was 24.2 ± 7.9 years. Participants were younger patients, and both studies are similar in this respect.

Clinical Utility of the MTSS-Tr Score

Approximately one-third of the patients who started treatment during the study recovered fully. This low rate shows the necessity of new treatment methods in patients with MTSS. This rate was only evaluated for a 1-week period. Because MTSS is an overuse injury, 1 week is too short a time to heal. We think that a longer period is necessary to evaluate the course of the disease, but our data should still be evaluated and presented to make a comparison with the original study. Additionally, in this study, we diagnosed MTSS based on patient history and physical examination and did not use any imaging methods. Although it is seen at a low frequency, we could not exclude a stress reaction/fracture in the tibia, so the recovery rate may have been lower.

The MTSS-Tr score is suitable for use in clinical studies and can also be used in the future to evaluate the results of existing treatment and new treatment methods. The MTSS-Tr score is not a tool used to make a definitive diagnosis, as was the MTSS score in the original study. It is suitable for use by athletes and nonathletes alike.

Strengths and Limitations

Face validity and comprehensibility were evaluated in a cross-cultural translation process, involving rigorous forward and backward translations from Dutch to Turkish. Consequently, the MTSS-Tr score was evaluated for its validity and reliability in 48 native Turkish individuals with MTSS. Structural analysis confirmed the unidimensionality of the MTSS-Tr score, consistent with the original Dutch MTSS score. It exhibited good structural validity, as demonstrated by moderate to large correlations with the SF-36-Tr PCS and VAS pain scores. Because the majority of the participants were athletes, the MTSS-Tr score can be used on athletes.

Test-retest analysis was performed with an interval of 1 week between assessments, and ethically, treatment had to be started during this period. There were patients who recovered during this process, and this was a limitation of this study, asking about improvement by asking about the recovery status of the patients. In terms of test-retest reliability, a sample size of 17 seems a little small; it is recommended that this number be larger in future studies. Finally, we calculated the SDC value, but we did not calculate the minimal important change. Future studies should also take this measure into consideration.

Conclusion

The translated MTSS-Tr score was found to have good internal consistency and good reliability and validity. The MTSS-Tr score is useful to evaluate symptoms in Turkish-speaking patients with MTSS.

Appendix

APPENDIX Table A1.

Turkish Version of the Medial Tibial Stress Syndrome Score

Soru No.	Skor
(1) Şu anki spor aktiviteleri:
○ Her zamanki spor aktivitelerimin hepsini yapıyorum	0
○ Alt bacağımdaki ağrıdan dolayı her zamanki spor aktivitelerimin daha azını yapmak zorunda kalıyorum	1
○ Alt bacağımdaki ağrıdan dolayıalternatif spor aktivitelerini yapmak zorunda kalıyorum	2
○ Alt bacağımdaki ağrıdan dolayı hiçbir spor aktivitesi yapamıyorum	3
(2) Spor aktivitelerini gerçekleştirirken:
○ Alt bacağımda hiçağrı yok	0
○ Alt bacağımda biraz ağrı var	1
○ Alt bacağımda çok ağrı var	2
○ Alt bacağımdaki ağrıdan dolayı hiçbir spor aktivitesi yapamıyorum	3
(3) Yürürken:
○ Alt bacağımda ağrı yok	0
○ Alt bacağımda biraz ağrı var	1
○ Alt bacağımda çok ağrı var	2
○ Alt bacağımdaki ağrıdan dolayıyürüyemiyorum	2
(4) Dinlenme sırasında, alt bacağım:
○ Ağrılı değil	0
○ Hassas	1
○ Ağrılı	2
○Çok ağrılı	2