Translation, cross-cultural adaptation, reliability, and validation of the Italian version of the American Orthopaedic Foot and Ankle Society - MetaTarsoPhalangeal-InterPhalangeal Scale (AOFAS-MTP-IP) for the hallux

Abstract

Background and Aim of the work: An incorrect interpretation or patients’ misunderstanding of evaluation scales can induce a mistake; therefore the real applicability of an evaluation scale should be determined by procedures that take care of cultural adaptability and not only of scientific validity. Our purpose was to translate and culturally adapt into Italian the AOFAS-MTP-IP scale for hallux, and to check its reproducibility and validity. Methods: The AOFAS-MTP-IP scale was processed for translation and checked for medical part coherence. The scale was submitted to 10 patients to verify a correct cultural adaptation. Then, the scale was submitted to 50 randomized patients operated at their hallux. Intra and inter-observer reproducibility was checked by two interviewers and a repeated interview. Short-Form-36-questionnaire for Quality of Life and Visual-Analogue-Scale for pain were also administered to perform validation analysis. The Pearson’s-Correlation-Coefficient and the Intra-Class-Correlation coefficient were calculated to analyse the scale reproducibility and validation. Results: Cultural adaptation of the translated version of the scale resulted good in terms of understandability by patients. An optimal correlation of the inter and intra-observer reproducibility was obtained. The correlation with well-known validated scales as SF-36 and VAS has shown good correlation indicating success in the validation process. Conclusions: Validation of the Italian version of the AOFAS-MTP-IP evaluation scale for hallux has been performed successfully. Therefore its use can be considered appropriate and suggested in Italian clinical practice. (www.actabiomedica.it)

Introduction

Injuries and pathologies of the foot are frequently affected by a long recovering time, owing to continuous stimulation of the anatomical part during orthostatic posture and walking. In this contest it is important monitoring the status or the evolution of the injury: therefore several orthopedic scales for the evaluation of conservative or surgical treatments have been performed.

Questionnaires are constructed by several items that focus on the evaluation of different patients’ performances during their daily life, to assess the patient’s subjective condition. These scales allow physicians to have indications on real ameliorations or permanence of disabilities following the therapeutic intervention, to estimate the real impact on the performance of patients’ daily life activities (1).

Most part of these questionnaires was created in English-speaking regions; therefore, they are the result of the socio-linguistic tradition of these areas and their applicability in other areas, in which different socio-linguistic traditions exist, may generate interpretative difficulties.

This is a critical point in the use of these scales, because they were born and used with the intention to produce subjective response by patients, to evaluate the efficacy of the clinical intervention in relationships to individual patients’ satisfaction and wellness. A non-correct interpretation or patients’ misunderstanding can induce an evaluation mistake and make useless the application of these evaluation scales.

Guillemin (2) was the first to introduce the need of a careful translation procedure, in which the scale is not simply translated, but culturally adapted to the language to maintain the same evaluation properties; furthermore the translated scale needs to be statistically verified to give it a scientific validity for a proper use of its outcomes. Indeed, scientific validity of a scale outcomes is a point of great importance, because it allows to use homogeneous data about patients for allowing a more precise comparison among patients with a similar pathologic condition (3). In these terms, in the case of clinical conditions that in sè may not directly produce meaningful data, evaluation scales offer to scientists the availability to manage subjective data and related questions useful to address the analysis of clinical protocol in their impact on health-related quality of life (4).

The American Orthopedic Foot and Ankle Society Score (AOFAS) is one of the most widely used clinician-reporting tools for foot and ankle conditions. Developed in 1994, AOFAS is a clinician-based score that measures outcome for four different anatomic regions of the foot: the Ankle-Hindfoot, Midfoot, MetaTarsoPhalangeal (MTP)-InterPhalangeal (IP) for the Hallux, and MTP-IP for the Lesser Toes. The four anatomic regions of the AOFAS are all represented by a different version of the survey with each tool designed to be used independently (5).

Aim

In the present work we intended to perform a cross-cultural adaptation in Italian language of the American Orthopaedic Foot and Ankle Society - MetaTarsoPhalangeal (MTP)-InterPhalangeal (IP) Scale (AOFAS-MTP-IP) for the hallux and we also aimed to verify the reliability, and validity of the Italian version.

Materials and method

To assure that an italian translation of AOFAS-MTP-IP was not already in use, or recently proposed, we performed a Medline/PubMed search typing the keywords: ”AOFAS hallux score Italian validation” and the search didn’t find any previous Italian version.

We started the translation and the cultural adaptation following a process of three different stages as proposed by Guillemin (2, 6).

The First Stage consists in: primary translation of the AOFAS-MTP-IP questionnaire, from English into Italian, made by two translators aware of the study, namely an orthopaedic surgeon (M.L.), and a university student (P.L.) involved in non-medical disciplines; both translations were compared and discussed to obtain a unique version.

The Second Stage consists in: submission of the first Italian version to a native English translator, who was unaware of the study and of the original English version of the scale; the translator had to back-translate the AOFAS-MTP-IP scale from Italian to English. We gained a new English version from the native translator and we compared this one to the original to define a second correct Italian version; this step is important to verify and eventually change or shift of significance related to linguistic expression during translation procedure (7).

The Third Stage consists in: cultural adaptation of the translated questionnaire. We have enlisted randomly 10 patients with hallux orthopedic disease retrieved from the hospital DataBase “AcceWeb” (Hi.Tech S.p.A. Software Engineering, via di Campigliano 51, 50012 Bagno a Ripoli, Firenze, Italy); no particular conditions were required for enlistment like age, sex or nationality.

All patients gave their informed consent for participation in the research study.

To those who tested the second Italian version of AOFAS scale was added the question “difficult to understand?” to each sentence. We posed the limit of 90% of patients understanding the Italian questionnaire to indicate a good translation; otherwise we should have to restart from the first step of the process to try to improve the cultural adaptation.

We also submitted the AOFAS scale to 10 healthcare professionals (3 orthopaedists, 2 physioterapists, 2 medical specialists, 3 nurses), to check the comprehension of the medical part that consists in: MTP joint motion (dorsiflexion and plantarflexion), IP joint motion (plantarflexion), MTP-IP stability (all directions), callus related to hallux (MTP-IP) and alignment. The comprehension of the text by healthcare professionals had to be as for patients with a positive feedback of at least 90% to continue with the following steps, otherwise, even in these cases, we should have to restart from the first stage for searching a translation improvement.

Assessment of reproducibility and validity of the Italian version of the AOFAS-MetaTarsoPhalangeal - InterPhalangeal questionnaire for Hallux

The Italian AOFAS-MTP-IP was administered to a randomized group of 50 patients (with regular informed consent) who had undergone a surgical procedure at our institution for the treatment of hallux orthopedic disease. We considered a minimum follow-up of 3 months and thus included patients operated from 01/04/2017 to 31/03/2019. The 10 patients previously recruited to assess the cultural adaptation of the evaluation scale were also included in this group. Each patient of the group was submitted to three interviews made by two previously trained and independent interviewers (interviewers A and B). The first interview was made by A and the same day after 30 minutes it was made by B: this step was necessary to check the inter-observer reproducibility. Within 15 days, interviewer A had to reassess all the patients with the Italian AOFAS-MTP-IP questionnaires to check the intra-observer reproducibility. At the moment of the first interview, interviewer A also submitted the SF-36 questionnaire for Quality of Life and Visual Analogue Scale (VAS) to measure pain, in order to gain data to proceed to AOFAS hallux scale validation.

Statistical Analysis

Clinical and demographic data of the assessed patients were characterized.

The Pearson’s Correlation Coefficient (PCC) and the Intra-Class Correlation (ICC) coefficient were calculated to check the inter and intra-observer reproducibility for validation.

All statistical procedures were performed by STATA13.0 statistical program.

Results

Translation and cultural adaptation

The AOFAS specific questionnaire for the hallux (Table 1) consists of eight items: Pain, Activity limitation, Footwear requirements, MTP joint motion (dorsiflexion plus plantarflexion), IP joint motion (plantarflexion), MTP-IP stability (all directions), Callus related to hallux MTP-IP and Alignment. These items are distributed over three categories: Pain (40 points), Function (45 points) and Alignment (15 points).

Table 1.

Scala di valutazione AOFAS metatarsofalangea (MF) - Interfalangea (IF) dell’alluce (versione italiana validata da Leigheb et al.)

I Dolore (40 punti)
Nessuno		40
Lieve, occasionale		30
Moderato, quotidiano		20
Severo, quasi sempre presente		0
II Funzione (45 punti)
Limitazioni nelle attività quotidiane, lavorative e ricreazionali
Nessuna		10
Lieve		7
Moderata		4
Severa		0
Requisiti per le calzature
Scarpe alla moda, comuni, senza necessità di plantari/solette		10
Calzature comode, plantari/solette		5
Scarpe modificate o tutore ortopedico		0
Articolarità MF (dorsiflessione piùplantarflessione)
Normale o lieve limitazione (75° o più)		10
Limitazione moderata (30°-74°)		5
Limitazione severa (meno di 30°)		0
Articolarità IF (plantarflessione)
Nessuna limitazione		5
Severa limitazione (meno di 10°)		0
Stabilità MF-IF (tutte le direzioni)
Stabile		5
Decisamente instabile o lussabile		0
Callo relativo a MF-IF dell’alluce
Nessun callo o callo asintomatico		5
Callo sintomatico		0
III Allineamento (15 punti)
Buono, alluce ben allineato		15
Discreto, osservato qualche grado di malallineamento dell’alluce, asintomatico	8
Scarso, evidente malallineamento sintomatico		0
Cognome e nome pz:	Lato: Dx Sin Data compilazione:	Totale: /100

The maximum possible score is 100 points and consists in no pain (40 pt), no activity limitations (10 pt), no footwear requirements (10 pt), normal MTP joint motion (10 pt), no restriction of IP joint motion (5 pt), MTP-IP stability in all directions (5 pt), no callus or callus asymptomatic (5 pt) and hallux well aligned (15 pt).

The minimum score is 0 points and consists in the worst possible condition for our patients that will have a severe and almost always present pain, a severe limitation of daily and recreational activities, a modified shoes or brace, a severe restriction of MTP and IP joint motion, a MTP/IP articulation definitely unstable or able to dislocate, a symptomatic callus, a poor or obvious symptomatic malalignment of the hallux.

During the dispensing of the questionnaire for checking the cultural adaptation, six patients out of the ten, found difficulties to understand the second item (the Activity Limitation item), for which they required explanations, therefore this item of the scale, did invalidate the proposal of good comprehension level settled at 90% of patients. Because in any other items the patients have shown to have difficulties to understand and no improper translation was revealed inside the second item, the same was subject to a re-evaluation in its cultural adaptation. The re-evaluation process has brought to a proposal of a different item that gave no difficulty to understand (Table 2). All healthcare professionals interviewed for checking the medical part comprehension didn’t find any difficulty with the first translation nor with the second one.

Table 2.

Different translation of the second item to better perform the cultural adaptation of the item

First literal translation of the voice “activity limitation” of the AOFAS-MTP-IP
II Funzione (45 punti)
Limitazioni nelle attività
Nessuna limitazione	10
Nessuna limitazione delle attività quotidiane, come quelle lavorative	7
Attività quotidiane e ricreazionali limitate	4
Severa limitazione delle attività quotidiane e ricreazionali	0
Second translation culturally adapted of the voice “activity limitation” of the AOFAS-hallux scale.
II Funzione (45 punti)
Limitazioni nelle attività quotidiane, lavorative e ricreazionali
Nessuna	10
Lieve	7
Moderata	4
Severa	0

Statistical reproducibility and validity of the Italian version of the AOFAS-MPT-IP scale

To assess the validity of the scale, we randomly enlisted 50 patients, including 82% females and 18% males, with diagnoses of hallux valgus (72%, N=36) and hallux rigidus (28%, N=14): according to ICD-9 classification (International Classification of Diseases, 9th edition) respectively 735.0 and 735.2. Other demographic parameters are shown in Table 3.

Table 3.

Characteristics of the 50 patients included in the translation and validation process of the AOFAS-MTP-IP scale

Sociodemographic aspects	Values
Male gender	9
Female gender	41
Age (years): average	61±9
Age (years): range	45-80
Ethnicity: Caucasian	98%
Ethnicity: Non Caucasian	2%
Elapsed time after surgery (in months): average	3

The time elapsed between the two interviews performed by the interviewer A was not the same for each patient, but all were interviewed after a minimum interval of 7 days and a maximum of 21.

The data evaluation (PCC analysis) for every item of the AOFAS-MTP-IP scale collected by interviewer A, are detailed in Table 4, whereas the comparison of the total scores collected by the interviewer A in the first and in the second time and by the interviewer B are resumed in Table 5.

Table 4.

AOFAS-MTP-IP scores at the first interview

AOFAS-MTP-IP Questions/Items	Mean	SD	Maximum	Minimum
Pain	32.2	10.0	40	0
Activity limitation	8.4	2.5	10	3
Footwear requirements	3.8	1.3	5	0
MTP joint motion (dorsiflexion plus plantarflexion)	6.2	2.8	10	0
IP joint motion (plantarflexion)	5.0	0.0	5	5
MTP-IP stability (all directions)	5.0	0.0	5	5
Callus related to hallux MTP-IP	5.0	0.0	5	5
Alignment	13.7	2.7	15	8

Table 5.

AOFAS-MTP-IP total scores detected following the different interview performed on patients

Observer	Mean ± SD	CL 95 %
A	79.3 ± 13.3	75.6 - 83.0
Abis	78.8 ± 12.2	75.4 - 82.2
B	80.0 ± 12.2	76.6 - 83.4

PCC evaluation results, can be read in the following way: 0 < PCC < 0.3: weak correlation; 0.3 < PCC < 0.7: moderate correlation, 0.7 < PCC > 1.0: good correlation.

The analysis related to the reproducibility of scale outcomes, concerning inter- and intra-interviewer variability is resumed in Table 6.

Table 6.

Assessment of intra and inter-interviewer reproducibility of AOFAS-MTP-IP scale with Pearson correlation coefficient

AOFAS-MTP-IP Questions/Items	Pearson Correlation Coefficient
	Intra-Interviewer	Inter-Interviewer
Pain	0.8715	0.8799
Activity limitation	0.6269	0.6052
Footwear requirements	0.7844	0.6630
MTP joint motion (dorsiflexion plus plantarflexion)	0.6425	0.6070
IP joint motion (plantarflexion)	n.d.	n.d.
MTP-IP stability (all directions)	n.d.	n.d.
Callus related to hallux MTP-IP	n.d.	n.d.
Alignment	0.6309	0.6021

n.d. = not detectable, owing the occurrence of constant values report

The reproducibility evaluated by Pearson’s Correlation Coefficient (PCC) show a good correlation for two items (‘Pain’ and ‘Footwear requirement), a moderate correlation for three items, while the items 5 (IP Joint motion, plantar-flexion), 6 (MTP-IP stability, all directions) and 7 (Callus related to hallux, MTP-IP) do not display any PCC value, owing the occurrence of a binary response (0 or 5), that in the clinical cases examined have produced the same response (constant value = 5), this event makes PCC evaluation unable to produce results. The intra-interviewer coefficients are generally lightly higher than inter-interviewer coefficients, evidencing a high coherence of response obtainable with the scale (Table 6).

The Intra-Class Correlation (ICC) coefficient used to assess the reproducibility was compared with the Pearson’s correlation coefficient (Table 7); the analysis confirms the occurrence of a strong identity among the different scores detected by interviewer A vs. Abis and A vs. B, allowing us to judge optimal the correlation in terms of inter and intra- interviewer variability.

Table 7.

Analysis of the reproducibility by means of the Pearson’s correlation coefficient and of the intra-class correlation coefficient values for the total score of the AOFASMTP-IP assessment scale

	Intra-Interviewer	C.L. (95%)	Inter-Interviewer	C.L. (95%)
Pearson’s Coefficient	0.8788		0.8832
Intra-Class Coefficient, individual	0.8810	0.8000 - 0.9305	0.8809	0.7994 - 0.9306
Intra-Class Coefficient, average	0.9367	0.8889 - 0.9640	0.9367	0.8885 - 0.9640

In conclusion the analysis evidences an optimal level of reproducibility, indicating the Italian version of AOFAS-MTP-IP scale as adequate for the use by different interviewers.

The validation of the Italian version of AOFAS-MTP-IP scale was performed in comparison to the 8 domains of SF-36 health quality survey by the PCC (Table 8). The PCC coefficients show a general moderate correlation; in particular for the items ‘Role physical’ and ‘Bodily pain’; we observe values the are at limit between the weak and moderate correlation.

Table 8.

Correlation with Pearson’s coefficient, of the 8 domains of SF-36 with AOFAS-MTP-IP total score results obtained from interviewer A the first time

SF-36 domains	Pearson’s coefficient
Physical functions	0.4862
Role physical	0.3335
Bodily pain	0.3491
General health	0.5843
Vitality	0.5052
Social function	0.4704
Role emotion	0.5332
Mental health	0.4266

It is known that the occurrence of an item presenting binary response (as evidenced above) strongly reduces the intrinsic variability of the items, affecting the total scale value, that is used for the comparison with SF-36 scale items. This results in lower values than that we could obtain if all items of AOFAS-MTP-IP scale presented three or more choice.

For further control, we compared the AOFAS-MTP-IP Italian scale total values of the interviewer A with the VAS scale; similarly we compared the first item of the Italian version scale (Pain) with VAS scale (Table 9), being the first item related to the same concept of the VAS i.e. the pain. In both cases similar moderate correlation was obtained, indicating the repetition of the limits of AOFAS-MTP-IP scale as above evidenced. The negative sign in the PCC coefficient indicate that the value sequence of the two evaluation scales are displayed in opposite direction.

Table 9.

Correlation with Pearson’s coefficient of the VAS scale for pain AOFAS-MTP-IP total score results obtained from interviewer A the first time

AOFAS-MTP-IP scores	Pearson’s coefficient
Total score, interviewer A	- 0.5271
AOFAS-MTP-IP, first item score (pain)	- 0.5204

Discussion

Many questionnaires have been produced to investigate properly the different clinical or disease patterns in the musculoskeletal field, because a generic questionnaire cannot capture the full patients experience in a particular disease (8). Therefore, different questionnaires have been created to investigate specific orthopaedic diseases, giving to the clinicians the opportunity to obtain a correct evaluation on the impact of a specific disease and its related therapeutic intervention.

The AOFAS-MTP-IP scale for hallux is a specific tool investigating a focus orthopedic aspect of both non-operated patients and patients who have undergone a surgical operation, because it has been demonstrated that the degree of deformity, amount of correction, or type of operation did not influence the outcome (9). These evidences have brought to an increasing interest in the use of this scale among orthopedists, physiatrists and physiotherapists.

Using subjective scales, the major point to obtain a correct response by patients consists in the immediate intelligibility of the several items composing the scale, because a use of long and repeated explanation by healthcare workers can alter the result in terms of reliability of the response (7). Therefore, working with subjective evaluation scales, the cultural adaptation more than the translation is a critical point to obtain a useful tool, the outcome of which can be used properly. We have observed the importance of this concern in the cultural adaptation process of the second item. In spite of its intelligibility by professional workers, the immediate understanding of this item by the most part of patients is not good, and they required further explanations to respond adequately. Instead an adapted version of the point didn’t offer any incertitude by patients. Therefore, under the highlight of this result we further pointed out the importance of do not use evaluation scale that has not culturally validated.

The major point in the use of subjective scales is represented by their reliability, consisting into producing data that have an intrinsic objective validity and scientific consistence, to be used in comparison with other similar data, allowing to an interchange of experiences and therapeutical interventions among clinicians. The scientific validation of the subjective scales has a recognized protocol that has in ICC coefficient and in the PCC evaluation the statistical procedure, and in the SF-36 and VAS scale the comparison counterpart (10).

In our analysis we did not obtained a high level of positive correlation that might indicate a limited validity of the translated scale; indeed AOFAS-MTP-IP scale has weak points in the items ‘IP Joint motion, plantar-flexion’, ‘MTP-IP stability, all directions’, ‘Callus related to hallux, MTP-IP’, that are all characterized by a binary response. In fact, the binary response makes a reduced variability of the investigative response, performing a weakening of the validation procedure (PCC evaluation) in sè. In our analysis the constant response of some items further lowered the resolutive properties of the PCC evaluation. We may assume that the theoretic limit of the PCC test cannot be considered = 1, but should be considered lower, approximately 0.8 or 0.7. Following this consideration PCC evaluation outcomes such as 0.5 become values to be interpreted as good results.

In spite of the wide consensus that the AOFAS-MTP-IP scale for hallux have gained among clinicians, this scale offers some critical point in the variability of the outcomes that can report.

Curiously, patients treated with arthrodesis for hallux rigidus have a mandatory null item score on the MTP-joint mobility: this makes more rigid the evaluation potentiality of the scale. These limitations should induce clinicians to be aware in the use of the AOFAS-MTP-IP scale, and to verify its effective applicability.

In conclusion, validation and cross-cultural adaptation of the AOFAS-MTP-IP Italian version has been performed successfully and its use can be considered appropriate and suggested in Italian clinical practice.

Conflict of interest:

Each author declares that he or she has no commercial associations (e.g. consultancies, stock ownership, equity interest, patent/licensing arrangement etc.) that might pose a conflict of interest in connection with the submitted article