Abstract
AIM
To translate the Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire into the Dutch language (VISA-A-NL), and to assess its reliability, validity, and applicability to non-athletes.
METHODS
After translation according to a forward-backward protocol, 101 patients with complaints of Achilles tendinopathy were asked to fill out the VISA-A-NL at two time points together with visual analogue scale, the Foot and Ankle Outcome Score, and the Short Form-36 questionnaires. Reliability, internal consistency, construct validity, and content validity were tested.
RESULTS
The VISA-A-NL showed high reliability (0.97, 95%CI: 0.95-0.98). Cronbach’s alpha (internal consistency) was 0.80. It increased to 0.88 without activity domain. Correlation with other questionnaires was moderate or poorer.
CONCLUSION
The VISA-A-NL proved to be an excellent evaluation instrument for the Dutch physician. If applied to non-athletes, using a modified score (questions 1-6) should be considered.
Keywords: Achilles tendon, Victorian Institute of Sports Assessment-Achilles, Validity, Patient reported outcome measurement
Core tip: This manuscript shows the validity and reliability of the Dutch version of the Victorian Institute of Sports Assessment-Achilles (VISA-A) in patients with Achilles tendinopathy. The most important finding is that the athletes and non-athletes cannot be compared. The effect of treatment, when using the VISA-A score to measure outcome, is underestimated in non-athletes. If applied to non-athletes, using a modified score (questions 1-6) should be considered.
INTRODUCTION
Achilles tendinopathy is a major cause of chronic pain and disability, which may lead to suboptimal overall health as physical inactivity is a risk factor for cardiovascular disease[1]. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. One of the factors limiting the quality of research may be the absence of standardised measures to evaluate the outcome of treatment[2]. A patient’s subjective assessment of treatment outcome such as pain, functional ability, and satisfaction fulfils the criteria of being valid, reliable, and sensitive to change if gathered by a correctly designed and tested patient-centred questionnaire[3]. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self- administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity. Subjective scoring systems can be used in countries other than the ones in which they were developed if translated and validated for a specific language and population[4-6]. The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages[1,7,8]. The aim of this study was to translate the VISA-A questionnaire into the Dutch language (VISA-A-NL) and assess its reliability and validity to provide a valid questionnaire for the Dutch population. Moreover, the questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle[9], applicability to non-athletes was also evaluated in this study.
MATERIALS AND METHODS
Translation procedure
A Dutch translation was made using a forward-backward translation protocol according to the guidelines of Guillemin et al[4,5]. Three people independently translated the English version of the VISA-A questionnaire to Dutch. All three were in the medical field, and spoke English as a second language. One independent native speaker who was not active in the medical field translated this Dutch version back into English. Discrepancies were discussed and adjusted for the final Dutch questionnaire (VISA-A-NL). It was assumed that no major cultural differences in lifestyle exist between the Dutch and Canadian populations, and therefore cultural adaptation of the questionnaire was not required[4,10].
Patients
The local accredited ethics committee (Dutch acronym: METC) reviewed this study in an expedited manner and determined, based on the Dutch Medical Research Involving Human Subjects Act (Dutch acronym: WMO), that the research activities described meet the requirements for exemption from METC review under the WMO. According to the Consensus-Based Standards for the Selection of Health Measurement Instruments criteria, we decided to choose a sample size of at least 100 patients. Actually, 104 consecutive patients from the outpatient clinic of three participating Dutch hospitals were included, of whom 47% were female; 79 (76%) were athletes and 25 (24%) were non-athletes. Their mean age was 48.5 years (SD, 11.6 years).
All patients had complaints of Achilles tendinopathy (including mid-portion and insertional tendinopathy, paratendinopathy, and retrocalcaneal bursitis)[11].
They were clinically assessed and an American Orthopaedic Foot and Ankle Society (AOFAS) questionnaire was taken by the consulting physician. All patients were asked to fill out two sets of questionnaires; the first [A = VISA-A-NL, the Visual Analogue Scale (VAS) pain, VAS function, the Foot and Ankle Outcome Score (FAOS), and the Short Form-36 (SF-36)] was completed on the day of consultation, and the second (B = VISA-A-NL) was completed 5 d later[12]. Additionally, they were asked whether their complaints had changed since the first assessment.
Questionnaires
The original English version of the VISA-A questionnaire as designed by Robinson and co-workers[2] contains eight questions that cover three domains; pain (questions 1-3), function (questions 4-6), and activity (questions 7-8). Scores are summed to yield a total of 100 points in an asymptomatic subject: questions 1-7 score a maximum of 10 points each; question 8, on sporting activity, carries a maximum of 30. Pain on undertaking sports will automatically lead to a loss of 10-20 points.
Since questions 7 and 8 refer to sport activities (accounting for 40% of points) and the study population contained both athletes and non-athletes, we modified the VISA-A score by deleting both questions and investigated the psychometric properties of both the VISA and the modified VISA.
FAOS is a 42-item questionnaire divided into five subscales: pain (9 items), other symptoms (7 items), activities of daily living (17 items), sport and recreation function (5 items), and foot and ankle related quality of life (4 items). Each question can be scored on a 5-point Likert scale (0-4) and each of the five subscale scores is calculated as the sum of the items included. Raw scores are then transformed to 0-100, worst to best score[13].
SF-36 is a self-administered, generic HRQL (health related quality of life) instrument[14-17]. It comprises 36 items across eight dimensions (physical functioning, role limitation due to physical problems, bodily pain, perception of general health, energy and vitality, social functioning, role limitation due to emotional problems, and mental health). The eight dimensions of the SF-36 score are calculated on a 0-100 worst to best scale[18].
The VAS is a 100 mm visual analogue scale and is used to determine the seriousness of pain and functional problems[19].
In 1994, the AOFAS developed a questionnaire to provide a standard method for reporting clinical status of the ankle and foot. The AOFAS-ankle and hindfoot clinical rating system combines both subjective and objective factors into numerical scales to describe function, alignment, and pain. Since objective aspects are incorporated, this questionnaire has to be completed by the investigator[20].
Testing
When a questionnaire is developed or translated, the most important consideration is that it must be able to accurately measure that for which it is designed. To evaluate the psychometric properties of the VISA-A-NL, both reliability and validity were assessed.
Reliability
Reliability is defined as the extent to which patients can be distinguished from each other, despite measurement errors[21].
Test-retest reliability: Test-retest reliability refers to the repeatability of the test and measures the extent to which the same results are obtained on repeated administration when no change in physical functioning has occurred[22]. To determine the test-retest reliability, a second VISA-A-NL (B) questionnaire was given to all patients; 67 patients responded. In 15 patients, complaints had changed at re-test measured by an anchor question (7 item Likert). Test-retest reliability was therefore assessed in 52 patients, using the intra-class coefficient (ICCagreement, two-way random effects model). An ICC > 0.75 was considered good[23]. A t-test was performed to determine the presence of a systematic difference between the first and second assessment. Additionally, standard error of measurement (SEM) was calculated as the square root of the within-subject variance. The smallest detectable change (SDC) was calculated as 1.96 × √2 × SEM. The SDC is the smallest measurement change that can be interpreted as real change[24].
Internal consistency: Internal consistency of the scale is the extent to which the items are inter-correlated and cover the same construct (homogeneity of the scale). To evaluate the internal consistency of the VISA-A, Cronbach’s alpha was calculated. A Cronbach’s alpha of 0.7 was considered to represent an acceptable degree of internal consistency, 0.8 was considered as good, and 0.9 as excellent internal consistency[25].
Validity
Validity relates to the ability of a questionnaire to measure outcome parameter of interest.
Construct validity: Construct validity was tested by determining the association between the VISA-A-NL questionnaire and the FAOS, SF-36, VAS scores for pain and function, and the AOFAS, using Pearson correlation coefficients. We evaluated construct validity by hypothesizing that correlation coefficients between the VISA-A-NL (with and without the activity questions) and VAS pain, FAOS pain, symptoms, and sport and recreation, and SF-36 bodily pain and physical functioning would be higher than correlations with the other domains.
Content validity: Content validity examines the extent to which all concepts of interest are adequately represented by the items in the questionnaire[12]. It was evaluated by assessing distribution and floor and ceiling effects of the VISA-A-NL. These are considered to be present if more than 15% of responders achieve the lowest or highest possible score[12].
Statistical analysis
Statistical analyses were performed using PASW statistics 18.0 software (SPSS Inc., Chicago, IL, United States). A P-value < 0.05 was considered statistically significant.
RESULTS
Of 104 participants, 11 returned questionnaires that were filled out incompletely or erroneously and were therefore excluded from analysis. Thus, 93 patients were finally included in the analysis.
Reliability
Of 93 patients, 52 (56%) returned the second set of questionnaires. The ICCagreement of the questionnaire was 0.97 (95%CI: 0.95-0.98) and Cronbach’s alpha was 0.78 for the entire study population (Table 1). A statistically significant difference between the two assessments was not observed for both versions of the VISA-A-NL in athletes, nor in non-athletes (0.29 < P < 0.67).
Table 1.
The ICCagreement and Cronbach’s alpha of the questionnaire
| Athletes (n = 39) | Non-athletes (n = 13) | Total (n = 52) | |
| ICCVISA total (95%CI) | 0.95 (0.91-0.97) | - | 0.97 (0.95-0.98) |
| ICCVISA modified (95%CI) | 0.96 (0.92-0.98) | 0.98 (0.93-0.99) | 0.97 (0.95-0.98) |
| SEMVISA total | 4.41 (4.4%) | - | 4.07 (4.1%) |
| SEMVISA modified | 2.64 (4.4%) | 2.42 (4.0%) | 2.68 (4.5%) |
| SDCVISA total | 12.21 (12.2%) | - | 11.28 (11.3%) |
| SDCVISA modified | 7.32 (12.2%) | 6.70 (11.2%) | 7.44 (12.4%) |
| Cronbach’s alphaVISA total | 0.72 | 0.82 | 0.78 |
| Cronbach’s alphaVISA modified | 0.83 | 0.86 | 0.86 |
ICC: Intra-class correlation coefficient; SEM: Standard error of measurement; SDC: Smallest detectable change.
Validity
Pearson correlation coefficients of the VISA-A-NL with the other questionnaires are shown in Table 2. The subscale “SF-36 physical functioning” correlated well with the VISA-A-NL questionnaire, both with and without activity domain. Most other physical domains correlated moderately, but the subscales of FAOS quality of life showed a poor correlation with VISA-A. A poor correlation was also observed for the psychological domains (SF-36 social functioning, mental health, role emotional, vitality, and general health perception).
Table 2.
Pearson correlation coefficients of the Victorian Institute of Sports Assessment-Achilles-NL with the other questionnaires (with and without activity domain)
|
VISA-ATotal |
VISA-AModified |
VISA-AModified |
VISA-ATotal |
VISA-AModified |
|
| Athletes (n = 71) | Athletes (n = 71) | Non-athletes (n = 22) | Entire population (n = 93) | Entire population (n = 93) | |
| VAS pain | -0.54a | -0.58a | -0.39 | -0.54a | -0.57a |
| VAS function | 0.52a | 0.51a | 0.44a | 0.50a | 0.52a |
| AOFAS | 0.48a | 0.46a | 0.31 | 0.56a | 0.50a |
| FAOS symptoms | 0.45a | 0.52a | 0.53a | 0.58a | 0.60a |
| FAOS pain | 0.53a | 0.56a | 0.52a | 0.58a | 0.60a |
| FAOS ADL | 0.56a | 0.55a | 0.47a | 0.59a | 0.58a |
| FAOS sport | 0.56a | 0.61a | 0.43a | 0.55a | 0.59a |
| FAOS QOL | 0.29a | 0.33a | 0.14 | 0.38a | 0.37a |
| SF-36 physical functioning | 0.55a | 0.63a | 0.66a | 0.70a | 0.71a |
| SF-36 role physical | 0.13 | 0.12 | 0.58a | 0.31a | 0.32a |
| SF-36 bodily pain | 0.31 | 0.40a | 0.46a | 0.49a | 0.51a |
| SF-36 social functioning | 0.01 | -0.04 | 0.29 | 0.27a | 0.21 |
| SF-36 mental health | -0.11 | -0.07 | 0.39 | 0.21 | 0.20 |
| SF-36 role emotional | -0.06 | 0.02 | 0.65a | 0.37a | 0.39a |
| SF-36 vitality | -0.26 | -0.25 | 0.26 | -0.05 | -0.09 |
| SF-36 general health perception | -0.01 | -0.02 | 0.28 | 0.25a | 0.21 |
| SF-36 physical component scale | 0.43a | 0.47a | 0.36 | 0.52a | 0.51a |
| SF-36 mental component scale | -0.34a | -0.32a | 0.49a | 0.04 | 0.03 |
P < 0.05. VISA-A: Victorian Institute of Sports Assessment-Achilles; VAS: Visual analogue scale; AOFAS: American Orthopaedic Foot and Ankle Society; FAOS: Foot and ankle outcome score; SF-36: Short form-36.
The mean scores of the VISA-A-NL questionnaire were 52.4 (SD 19.7) and 22.0 (SD 15.7) for athletes and non-athletes, respectively. The mean scores of the modified VISA-A-NL questionnaire were 36.2 (SD 13.9) and again 22.0 (SD 15.7) for athletes and non-athletes, respectively. Floor and ceiling effects were not observed in both versions of the questionnaire, as only one (1%) subject scored 0 points, nobody scored 100 points, and one patient scored the maximum of 60 points in the modified VISA-A score.
DISCUSSION
The aim of this study was to translate the original VISA-A questionnaire on the subjective complaints of patients with Achilles tendinopathy into the Dutch language, to validate it, and to assess its applicability to non-athletes.
The translation procedure did not create any problems, since the items are universal and there is no large cultural difference between Dutch and Canadian patients.
Reliability of the translation was excellent, with a statistically non-significant difference between assessments. This outcome may be explained by the fact that, as the procedure for the Dutch Oxford 12-item knee questionnaire taught us, we introduced a question if complaints had changed between assessments[26]. Fifteen (22%) of sixty-seven patients answered this question with “yes”, and therefore they were excluded from reliability testing.
The SDC in this study indicated that under a stable condition, the VISA-A score can vary up to 12 points. For clinical studies, this implies that clinical changes can only be detected if they exceed the 12 points.
The study questionnaire showed good internal consistency (Cronbach’s alpha = 0.80), but there indeed was a negative effect of the activity domain. Sub-analysis without these questions showed an increase of Cronbach’s alpha to 0.88. However, when measuring the effect of a treatment in a mixed group of athletes and non-athletes, the effect of treatment could be underestimated. For example, an athlete can score 0 points with questions 7 and 8 before treatment as complaints withhold him/her from being sports active. After treatment, the athlete is complaint-free and scores 100 points. The non-athlete also scores 0 points for questions 7 and 8 before treatment, and is also complaint-free after treatment. However, he/she will never score higher than 60 points as questions 7 and 8 will not be answered differently between assessments. The effect of treatment, when using the VISA-A score to measure outcome, is therefore underestimated in non-athletes.
When choosing the VISA-A questionnaire for a mixed population of athletes and non-athletes, deleting questions 7 and 8 can be considered, since the psychometric properties of the modified VISA-A were comparable with those of the original version.
Generally, Pearson correlation coefficients were higher for physical than psychological components. Convergent and divergent validity is confirmed as correlation coefficients were higher for the physical domains of the questionnaires. However, for non-athletes, correlation with socio-emotional components was also higher. This could imply that physical restrictions in this subgroup with chronic Achilles tendinopathy have greater emotional and social consequences than in athletes. Noticeable is the moderate correlation of VISA-A with VAS, which is a validated subjective outcome measure frequently used for scientific means.
Low correlations can be explained by the fact that none of the questionnaires except for the VISA-A were validated for Achilles tendinopathy. Initially it was intended to do so, but this study aim was departed to not further enlarge patient burden as many of these questionnaires are extensive. Given the laborious inclusion of 104 patients in 3.5 years and a 56% response rate to both assessments, this was well decided. This was also why responsiveness was not tested.
In conclusion, the VISA-A-NL questionnaire seems suitable for use in athletes. However, in a combined population of athletes and non-athletes, results will become incomparable as the highest possible score for non-athletes is 40 points lower than that for non-athletes. Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered a mixed population of patients with Achilles tendinopathy, meaning a version with only questions 1-6 and the complete questionnaire (questions 1-8) is reserved for athletes only.
ARTICLE HIGHLIGHTS
Background
Achilles tendinopathy is a major cause of chronic pain and disability. Many studies have been published on a multitude of treatments for Achilles tendinopathy, but prospective series on the outcome are lacking. The Victorian Institute of Sports Assessment-Achilles (VISA-A) questionnaire was created in 2001 to assess clinical severity for patients with Achilles tendinopathy. It is a self-administered questionnaire evaluating symptoms and their effect on physical activity, and displayed reliability and construct validity.
Research frontiers
The VISA-A has been translated into Swedish, Italian, and German and proved to be able to determine the clinical severity and provide information about the effect of the management of patients with Achilles tendinopathy who speak these languages.
Innovations and breakthroughs
This work translated the VISA-A questionnaire into the Dutch language (VISA-A-NL), assessed its validity and reliability, and provided a valid questionnaire for the Dutch population. The questionnaire seems to be designed only for athletes as 40% of the points account for activity. As approximately 30% of patients with complaints of Achilles tendinopathy have a sedentary lifestyle, applicability to non-athletes was also evaluated.
Applications
Psychometric properties of the VISA-A-NL, without questions 7 and 8, are satisfactory for both athletes and non-athletes. It is therefore proposed that the modified VISA-A-NL questionnaire is considered for a mixed population of patients with Achilles tendinopathy.
Footnotes
Institutional review board statement: The local accredited ethics committee (Dutch acronym: METC) reviewed this study in an expedited manner and determined, based on the Dutch Medical Research Involving Human Subjects Act (Dutch acronym: WMO), that the research activities described meet the requirements for exemption from METC review under the WMO.
Informed consent statement: All patients received a patient information brochure.
Conflict-of-interest statement: The authors do not report a conflict of interest.
Data sharing statement: None.
Manuscript source: Invited manuscript
Specialty type: Orthopedics
Country of origin: Netherlands
Peer-review report classification
Grade A (Excellent): 0
Grade B (Very good): B
Grade C (Good): 0
Grade D (Fair): 0
Grade E (Poor): 0
Peer-review started: February 2, 2017
First decision: June 26, 2017
Article in press: December 1, 2017
P- Reviewer: Hernandez-Sanchez S S- Editor: Kong JX L- Editor: Wang TQ E- Editor: Li RF
Contributor Information
Inger Sierevelt, Department of Orthopaedic Surgery, Slotervaart Hospital, Amsterdam 1066 EC, The Netherlands.
Maayke van Sterkenburg, Department of Orthopedic Surgery, Academic Medical Center, Amsterdam 1105 AZ, The Netherlands.
Hans Tol, Academic Center for Evidence Based Sports Medicine, Academic Medical Center Amsterdam, Amsterdam 1105 AZ, The Netherlands.
Bella van Dalen, Bergmanclinics te Naarden, Naarden 1411 GE, The Netherlands.
Niek van Dijk, Department of Orthopedic Surgery, Academic Medical Center, Amsterdam 1105 AZ, The Netherlands.
Daniel Haverkamp, Department of Orthopaedic Surgery, Slotervaart Hospital, Amsterdam 1066 EC, The Netherlands.
References
- 1.Maffulli N, Longo UG, Testa V, Oliva F, Capasso G, Denaro V. Italian translation of the VISA-A score for tendinopathy of the main body of the Achilles tendon. Disabil Rehabil. 2008;30:1635–1639. doi: 10.1080/09638280701785965. [DOI] [PubMed] [Google Scholar]
- 2.Robinson JM, Cook JL, Purdam C, Visentini PJ, Ross J, Maffulli N, Taunton JE, Khan KM; Victorian Institute Of Sport Tendon Study Group. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med. 2001;35:335–341. doi: 10.1136/bjsm.35.5.335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dawson J, Carr A. Outcomes evaluation in orthopaedics. J Bone Joint Surg Br. 2001;83:313–315. doi: 10.1302/0301-620x.83b3.12148. [DOI] [PubMed] [Google Scholar]
- 4.Guillemin F. Cross-cultural adaptation and validation of health status measures. Scand J Rheumatol. 1995;24:61–63. doi: 10.3109/03009749509099285. [DOI] [PubMed] [Google Scholar]
- 5.Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417–1432. doi: 10.1016/0895-4356(93)90142-n. [DOI] [PubMed] [Google Scholar]
- 6.Mathias SD, Fifer SK, Patrick DL. Rapid translation of quality of life measures for international clinical trials: avoiding errors in the minimalist approach. Qual Life Res. 1994;3:403–412. doi: 10.1007/BF00435392. [DOI] [PubMed] [Google Scholar]
- 7.Lohrer H, Nauck T. Cross-cultural adaptation and validation of the VISA-A questionnaire for German-speaking achilles tendinopathy patients. BMC Musculoskelet Disord. 2009;10:134. doi: 10.1186/1471-2474-10-134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Silbernagel KG, Thomeé R, Karlsson J. Cross-cultural adaptation of the VISA-A questionnaire, an index of clinical severity for patients with Achilles tendinopathy, with reliability, validity and structure evaluations. BMC Musculoskelet Disord. 2005;6:12. doi: 10.1186/1471-2474-6-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ames PR, Longo UG, Denaro V, Maffulli N. Achilles tendon problems: not just an orthopaedic issue. Disabil Rehabil. 2008;30:1646–1650. doi: 10.1080/09638280701785882. [DOI] [PubMed] [Google Scholar]
- 10.Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000;25:3186–3191. doi: 10.1097/00007632-200012150-00014. [DOI] [PubMed] [Google Scholar]
- 11.van Dijk CN, van Sterkenburg MN, Wiegerinck JI, Karlsson J, Maffulli N. Terminology for Achilles tendon related disorders. Knee Surg Sports Traumatol Arthrosc. 2011;19:835–841. doi: 10.1007/s00167-010-1374-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60:34–42. doi: 10.1016/j.jclinepi.2006.03.012. [DOI] [PubMed] [Google Scholar]
- 13.Roos EM, Brandsson S, Karlsson J. Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int. 2001;22:788–794. doi: 10.1177/107110070102201004. [DOI] [PubMed] [Google Scholar]
- 14.Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, te Velde A, Verrips E. Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol. 1998;51:1055–1068. doi: 10.1016/s0895-4356(98)00097-3. [DOI] [PubMed] [Google Scholar]
- 15.Brazier JE, Harper R, Jones NM, O’Cathain A, Thomas KJ, Usherwood T, Westlake L. Validating the SF-36 health survey questionnaire: new outcome measure for primary care. BMJ. 1992;305:160–164. doi: 10.1136/bmj.305.6846.160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Sullivan LM, Dukes KA, Harris L, Dittus RS, Greenfield S, Kaplan SH. A comparison of various methods of collecting self-reported health outcomes data among low-income and minority patients. Med Care. 1995;33:AS183–AS194. [PubMed] [Google Scholar]
- 17.Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–483. [PubMed] [Google Scholar]
- 18.Insall JN, Dorr LD, Scott RD, Scott WN. Rationale of the Knee Society clinical rating system. Clin Orthop Relat Res. 1989;(248):13–14. [PubMed] [Google Scholar]
- 19.Price DD, McGrath PA, Rafii A, Buckingham B. The validation of visual analogue scales as ratio scale measures for chronic and experimental pain. Pain. 1983;17:45–56. doi: 10.1016/0304-3959(83)90126-4. [DOI] [PubMed] [Google Scholar]
- 20.Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M. Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int. 1994;15:349–353. doi: 10.1177/107110079401500701. [DOI] [PubMed] [Google Scholar]
- 21.Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003;80:99–103. doi: 10.1207/S15327752JPA8001_18. [DOI] [PubMed] [Google Scholar]
- 22.Eechaute C, Vaes P, Duquet W, Van Gheluwe B. Test-retest reliability of sudden ankle inversion measurements in subjects with healthy ankle joints. J Athl Train. 2007;42:60–65. [PMC free article] [PubMed] [Google Scholar]
- 23.Fleiss JL. The design and analysis of clinical experiments. 1986. New York: Wiley. [Google Scholar]
- 24.Beckerman H, Roebroeck ME, Lankhorst GJ, Becher JG, Bezemer PD, Verbeek AL. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10:571–578. doi: 10.1023/a:1013138911638. [DOI] [PubMed] [Google Scholar]
- 25.Dunbar MJ, Robertsson O, Ryd L, Lidgren L. Translation and validation of the Oxford-12 item knee score for use in Sweden. Acta Orthop Scand. 2000;71:268–274. doi: 10.1080/000164700317411861. [DOI] [PubMed] [Google Scholar]
- 26.Haverkamp D, Breugem SJ, Sierevelt IN, Blankevoort L, van Dijk CN. Translation and validation of the Dutch version of the Oxford 12-item knee questionnaire for knee arthroplasty. Acta Orthop. 2005;76:347–352. [PubMed] [Google Scholar]