Validation of the Arabic version of the Teruel Orthorexia Scale (TOS) among Lebanese adolescents

Abstract

Objectives

This study aimed at examining the reliability and validity of the Teruel Orthorexia Scale (TOS) among a sample of Lebanese adolescents (15 to 18 years old).

Methods

This is a cross-sectional study that was conducted between May and June 2020 and had enrolled 555 Lebanese adolescents. To assess the internal structure of the TOS scale, we administered the confirmatory factor analysis based on polychoric correlation matrix using Weighted Least Squares with Means and Variance Adjusted estimation (WLSMV) method in Mplus v 7.2 as suggested in the original validation paper. To assess the degree to which the Lebanese adaptation converges with the original scale, we have conducted the Multigroup Confirmatory Factor Analysis (MGCFA; estimated as CFA) between the data reported in the current paper and from the original validation paper.

Results

The mean age of the participants was 16.66 ± 1.01 years, with 76.1% females. The bi-dimensional model fitted the data well (χ²₍₁₁₈₎ = 429.09; p < 0.001; CFI = 0.954; RMSEA = 0.069[0.062, 0.076]). The latent factors were highly correlated (ρ = 0.74; p < 0.001), and the strength of the standardized factor loadings was adequate on both factors (i.e., all > 0.60). The fit indices of the scalar model were at the boundary of the threshold and thus, with some pinch of caution, it might be interpreted as invariant (i.e., having equal item intercepts across groups). We have identified latent mean differences in orthorexia nervosa (0.30; p < 0.001), where Spanish individuals scored higher, but we did not find any differences in the healthy eating (0.03; p = 0.636).

Higher DOS scores were significantly correlated with higher scores on the TOS subscale OrNe (r = 0.715; p < 0.001) as well as with higher scores on the TOS subscale HeOR (r = 0.754; p < 0.001). Higher ORTO-R scores were significantly associated with less TOS OrNe (r = − 0.437; p < 0.001) and TOS HeOr (r = − 0.305; p < 0.001) scores, respectively.

Conclusion

The Arabic version of the TOS can be considered a reliable valuable instrument to assess the ON tendencies and behaviors in Lebanese adolescents, emphasizing the fine contrast between ON’s two dimensions: healthy vs. pathological.

Level V

Opinions of authorities, based on descriptive studies, narrative reviews, clinical experience, or reports of expert committees.

Introduction

For some time now, concerns about pathological healthful eating is of interest among researchers and clinicians. Termed “Orthorexia Nervosa (ON)” by Bratman and Knight in 1997 [1], the emerging eating behavior was described as the pathological obsession for proper nutrition through a “restrictive diet, a focus on food preparation, and ritualized patterns of eating” [2]. Not considered as a disease by the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), orthorexia nervosa can be conceptualized as a life-threatening behavior where in extreme situations, subjects with orthorexic tendencies would prefer starvation over eating anything they consider impure and poisonous to their health [3, 4].

A greater number of articles that have recently been published in the literature have referred to ON as a keyword through the past years [5, 6]. However, no universal definition of ON has been shared yet, since its diagnostic criteria were still debatable, as well as its screening and psychometric instruments [7]. Until recently, research in this field has been hindered by the dearth of validated ON measurement tools that may distinguish between healthy and pathological “healthful eating” [8, 9].

The Bratman’s Orthorexia Test (BOT) was the first created tool to specifically measure ON [5]. As far as we know, there were not any evidence about the internal structure and/or reliability of this informal self-assessment tool [3, 10, 11]. Based on the BOT and questions about psychological disorders and personality profiles that were drawn from the Minnesota Multiphasic Personality Inventory (MMPI) [12], the ORTO-15 questionnaire was developed. Initially created by Donini et al., in the Italian language [11], it was validated in many other languages: Arabic [13], Turkish [14], Polish [15], German [16], and Spanish [17]. Despite being the most used questionnaire for measuring ON, the ORTO-15 is known for its important limitations regarding its psychometric characteristics. It was criticized for instability in its internal consistency’s values; some studies have reported low values of Cronbach’s alphas, while others have reported higher values (range: 0.62–0.82 [18, 19]). Content validity issues were also described since the tool was used to identify the healthy eating patterns, even those without pathological aspects, what may explain the high rate of ON tendencies and behaviors that was reported in multiple studies [9, 19–21]. Doubts regarding the scoring techniques and interpretations [19, 22] were also documented. Consequently, these metric discrepancies led to the development of a range of new tools to assess ON, such as the Eating Habits Questionnaire (EHQ) [23], the Düsseldorf Orthorexia Scale (DOS) [24], and even a revised version of the ORTO-15 itself, the ORTO-R [25].

A propitious new instrument is the Teruel Orthorexia Scale (TOS) [10]. A major advantage of the TOS tool is that it differentiates between two dimensions of orthorexia: healthy orthorexia (HeOr) and orthorexia nervosa (OrNe) [18]. At first, the initial TOS was developed by Barrada and Roncero in a 31-item version measuring both HeOr and OrNe. Both authors have reviewed ON’s literature, have participated in the validation of the Spanish ORTO-15 version, and have published many other eating disorders studies, leading to a final shortened and refined version of the TOS, a 17-item questionnaire. In the latter, two separate factors were identifiable healthy orthorexia (HeOr) and orthorexia nervosa (OrNe), which were, respectively, assessed with nine and eight items [10]. In its original Spanish version, the TOS showed a satisfactory internal consistency with Cronbach’s alpha reaching 0.85 for HeOr and 0.81 for OrNe [10]. Also, the test–retest correlation with an 18-month time-lapse was higher than 0.70 for both HeOr and OrNe [10].

To note that the majority of the studies about ON were mostly conducted in adults’ populations, while few were the studies targeting ON among adolescent populations. Few were the published studies about ON in Arab countries, as well [26]; in Lebanon, most of them mainly targeted adults and university students [13, 27–32].

The period of adolescence poses a risk for developing distorted eating behaviors in general [33, 34], and ON in particular [7, 35–37]. A fine line exists between what may be considered a maladaptive versus a healthy eating behavior in such a population. In fact, Lebanese adolescents’ dietary styles would differ from other adolescents in Europe and the United States, regarding the food availability and cost [38]. While the Lebanese adolescents grew up on a Mediterranean food table where seasonal fruits and vegetables are widely available in the country, and extensively incorporated in most traditional ethnic dishes [39], not all of them showed an equal nutritional knowledge [38]. This heterogeneity in the source of nutritional knowledge in the Lebanese adolescents, weather from inequalities in the socio-economic classes, or the accessibility to media influences, or other factors [38], makes the Lebanese adolescents more prompt to serious ambiguities on their perception of the healthy food nuances, what may predispose them to highest healthy vs unhealthy orthorexic tendencies. Being torn between the Middle Eastern conflicts and the Western fitness tendencies [33] makes the Lebanese adolescents’ population a curious candidate for ON tendencies’ assessment, especially through a tool that distinguishes the healthy and pathological dimensions of ON, like the TOS. To our knowledge, in the Arabic-speaking populations, there has been no published studies reporting the validation of the TOS, while the ORTO-15 remains the only ON’s questionnaire to be validated in an Arabic version [13]. Hence, the Arabic version of the TOS may be of beneficial use for an appropriate instrument used to measure ON tendencies in the research field and clinical practice. The present study aims to examine the validity and reliability of an Arabic version of the TOS instrument among a sample of Lebanese adolescents.

Methods

Study design

This cross-sectional study enrolled 555 adolescents currently residing in Lebanon (15 to 18 years old), using a proportionate random sample from all Lebanese governorates (May–June 2020). The intention of the study was to distribute a hard copy of the questionnaire to a sample of adolescent with age ranging from 15 to 18 years, reaching to them through their schools. By this time, the coronavirus pandemic outbreak in Lebanon challenged this intended execution of data collection due to nationwide lockdown, imposed social distancing, and schools closing. In order to carry on with this study, snowball sampling and respondent-driven sampling techniques were implemented between May and June 2020. Consequently, a soft copy of the questionnaire was created using google forms software, and an online approach was conceived to proceed with the data collection. Prior to participation, study objectives and general instructions were delivered online for the individual subjects. No credits were received for participation.

Minimal sample size calculation

According to Comrey and Lee [40], the minimal sample size needed to perform a confirmatory factor analysis was 170 based on ten participants for each item scale (TOS scale composed of 17 items).

Questionnaire

The Arabic questionnaire was conceived with three sections: The first section was a written consent, confirming the approval of the adolescents and their parents to fill the questionnaire. The second section constitutes of questions assessing socio-demographic details (age, residency governorate, height, weight, etc.). The Body Mass Index (BMI) was consequently calculated as per the World Health Organization [41]. The household crowding index (HCI), reflecting the socioeconomic status (SES) of the family, was calculated by dividing the number of persons living in the house by the number of rooms in the house; higher HCI reflect a lower SES. The third section included the following scales:

Teruel Orthorexia Scale (TOS)

This 17-item instrument assesses ON with two separate dimensions [10]: 9 items for Healthy Orthorexia or “HeOr” and 8 items for Orthorexia Nervosa or “OrNe”. Answers are scored on a four-point Likert scale ranging from 0 (= strongly disagree) to 3 (= strongly agree). Each score by each dimension was computed by summing up the items’ responses. No specific timeframe was asked from the participants while responding. Higher scores reflect more orthorexia nervosa and more healthy orthorexia, respectively. In this study, the Cronbach’s alpha was 0.853 for the TOS OrNe and 0.829 for the TOS HeOr.

Dusseldorf Orthorexia Scale (DOS)

This scale is composed of ten items aiming at measuring orthorexic behavior [24]; the answers of those items range from 4 = this applies to me to 1 = this does not apply to me, with higher scores reflecting more orthorexic behavior. In this study, αCronbach = 0.849.

ORTO-R

This scale [25] is a revision of the ORTO-15 scale [11], previously validated in Lebanon [13]. It is composed of six items, scored on a four-point Likert scale. Lower scores reflect higher orthorexic tendencies and behaviors. In this study, αCronbach = 0.711.

Forward and back translation procedure

A forward and back translation procedure was implemented. At first, a single bilingual translator, natively Lebanese and fluent in English, conducted the forward translation. A committee of healthcare and language professionals verified the Arabic translated version. A native English speaker translator, fluent in Arabic and unfamiliar with the scales’ concepts, conducted the backward translation to English. Subsequently, the original English questionnaire was compared to the back-translated one by the expert committee. The process was continued until discrepancies and inconsistencies were resolved [42, 43].

Statistical analysis

The SPSS software version 23 was used to conduct data analysis. To assess the internal structure of the TOS scale, we administered the confirmatory factor analysis based on polychoric correlation matrix using Weighted Least Squares with Means and Variance Adjusted estimation (WLSMV) method in Mplus v 7.2 [44] as suggested in the original validation paper [10]. Scales/subscales internal consistency (Cronbach’s alpha) were also included.

The Student t and ANOVA tests were used to compare two and three or more means respectively. Pearson correlation was used to check for an association between continuous variables. Convergent validity of the TOS scale was assessed via the correlations with the DOS and ORTO-R scales scores. The effect size of the gender on TOS OrNe score was calculated; Cohen suggested that d = 0.2 be considered a small effect size, 0.5 represents a 'medium' effect size and 0.8 a 'large' effect size. p < 0.05 was considered significant.

Results

The mean age of the participants was 16.67 ± 1.00 years, with 75.7% females. The mean house crowding index was 0.95 ± 0.50. More details about the students can be found in Table 1. The mean TOS OrNe score was 5.23 ± 4.82 (median = 4; minimum = 0; maximum = 24), whereas the mean TOS HeOr score was 10.15 ± 5.83 (median = 9; minimum = 0; maximum = 27).

Table 1.

Sociodemographic and other characteristics of the participants (N = 555)

Variable	N (%)
Gender
Male	135 (24.3%)
Female	420 (75.7%)
District
Beirut	70 (12.6%)
Mount Lebanon	355 (64.0%)
North	84 (15.1%)
South	16 (2.9%)
Bekaa	30 (5.4%)

	Mean ± SD
Age (in years)	16.67 ± 1.00
Body Mass Index (kg/m2)	22.31 ± 4.07
Household crowding index	0.95 ± 0.50
TOS orthorexia nervosa score	5.23 ± 4.82
TOS healthy orthorexia score	10.15 ± 5.83
DOS score	19.64 ± 6.42
ORTO-R score	15.22 ± 3.97

Confirmatory factor analysis

The bi-dimensional model fitted the data well (χ²₍₁₁₈₎ = 429.09; p < 0.001; CFI = 0.954; RMSEA = 0.069[0.062, 0.076]). The latent factors were highly correlated (ρ = 0.74; p < 0.001), and the strength of the standardized factor loadings (Table 2) was adequate on both factors (i.e., all > 0.60).

Table 2.

Confirmatory factor analysis of the Teruel Orthorexia Scale items

Item	Orthorexia Nervosa	Healthy Orthorexia
1	0.60
2	0.65
3	0.61
6	0.81
7	0.71
8	0.77
11	0.63
13	0.69
15	0.70
4		0.83
5		0.61
9		0.72
10		0.61
12		0.90
14		0.68
16		0.77
17		0.62

We also assessed the measurement invariance across Lebanese adolescents and adults [30]. However, due to the poor model fit in the adult population and more complex nature of the problem, we were unable to present a detailed report on this issue

Convergent validity

Higher DOS scores were significantly correlated with higher scores on the TOS subscale OrNe (r = 0.715; p < 0.001) as well as with higher scores on the TOS subscale HeOR (r = 0.754; p < 0.001). Same applies for the ORTO-R score; higher ORTO-R scores were significantly associated with less TOS OrNe (r = − 0.437; p < 0.001) and TOS HeOr (r = − 0.305; p < 0.001) scores, respectively.

Factorial validity with the original TOS scale

To assess the degree to which the Lebanese adaptation converges with the original scale, we have conducted the Multigroup Confirmatory Factor Analysis (MGCFA; estimated as CFA) between the data reported in the current paper and from the original validation paper [10]. Results are reported in Table 3. Accordingly to the conventional criteria of evaluation of the measurement invariance models [45], we have provided evidence of full metric invariance (i.e., the strength of factor loadings is the same in Lebanese and Spanish populations). Furthermore, the fit indices of the scalar model was at the boundary of the threshold; thus, with some pinch of caution, it might be interpreted as invariant (i.e., having equal item intercepts across groups). We have identified latent mean differences in orthorexia nervosa (0.30; p < 0.001), where Spanish individuals scored higher, but we did not find any differences in the healthy eating (0.03; p = 0.636).

Table 3.

Multigroup Confirmatory Factor Analysis (MGCFA; estimated as CFA) between the data reported in the current paper and from the original validation paper

Model	χ2(df)	p	CFI	RMSEA
Configural	971.72(236)	0.001	0.941	0.078
Metric	1071.24(251)	0.001	0.934	0.080
Scalar	1248.81(283)	0.001	0.923	0.082
Metric vs Configural	99.52(15)	0.001	0.007	0.002
Scalar vs Metric	177.57(32)	0.001	0.011	0.002

Bivariate analysis

Higher age was significantly but weakly associated with lower TOS ON scores (r = − 0.122; p = 0.004), whereas higher BMI was significantly but weakly associated with higher TOS ON scores (r = 0.133; p = 0.002). It is noteworthy that no significant association was found between the household crowding index and TOS ON scores (r = − 0.007; p = 0.862). Moreover, higher TOS ON scores were found in females compared to males (5.58 ± 4.87 vs 4.12 ± 4.49; p = 0.002; Effect size Cohen’s d = 0.311).

Discussion

To the best of our knowledge, there are no previous published validation of an Arabic version of the TOS scale, targeting the Lebanese adolescent population. The psychometric properties of the TOS were rarely discussed in the literature, due to the lack of studies that used the TOS to assess ON tendencies and behaviors, so far [46]. However, we found a single study conducted in 2020 [47] that investigated the psychometric properties of the English version of the TOS and emphasized the superiority of the TOS’ psychometrics over other ON’s measuring instruments. In fact, the TOS is considered to be a highly promising instrument by its ability to distinguish between HeOr and OrNe [10, 18, 48]. Thus, the aim of the current study was to examine the validity and reliability of an Arabic version of the TOS instrument among a sample of Lebanese adolescents.

The confirmatory factor analysis (CFA) that we conducted yielded to a 17-item version of the TOS, for which the two-factor solutions showed satisfactory goodness-of-fit similar to the original scale [10]. Satisfactory values of internal consistency were found as well, with Cronbach’s alpha coefficients of 0.853 and 0.829 for the HeOr and OrNe subscales, respectively. The latter values are considered supportive of the Cronbach’s alpha values found in the original Spanish version of the TOS (above 0.8 for both HeOr and OrNe [10]). On a side note, these two coefficients emphasized the original, as well as, the current relatively high correlation between the sub-scales of the TOS, suggesting a consequent similarity in the measurement of HeOr and OrNe. This would draw the question of whether these two subscales are measuring the same aspect of ON; further studies and investigations are needed to tackle this question.

Furthermore, while assessing the degree to which the Lebanese adaptation of the TOS converges with the scale’s original Spanish version, the fit indices of the scalar model could be interpreted, to some extent, as invariant, in addition to the full metric invariance that we already found in the MGCFA. This evidence of invariance in terms of the bi-dimensional structure of the TOS and the factor loadings and intercepts highlights the strength of this Arabic version of the TOS, in its similitudes to the original Spanish version, even while comparing two different populations: Lebanese adolescents vs. Spanish adults [10]. However, this cross-cultural difference could be manifesting itself by a latent difference regarding results of the OrNe and not the HeOr, where Spanish adults from the original study, scored higher for OrNe and not for HeOr, in comparison to the current Lebanese adolescents sample [10]. From a sociodemographic perspective, many explanations would elicit the emergence of this difference. While both the Spanish and Lebanese food tables are thought to be inspired by the traditional Mediterranean diet, the adherence to this WHO-recommended diet has changed between these two countries [49]. In fact, to a lesser extent than the middle eastern countries, the European Mediterranean countries are succumbing to a westernization process of their diet [49, 50], affected by social, political and cultural changes. Southwestern European countries, such as Spain, Greece, and Italy [50], were the most prone to this decrease in Mediterranean diet adherence, due to changes in food availabilities and westernized dietary policies. On the other hand, a major common pattern that affects both the Lebanese and Spanish dietary models but on different terms is the socio-economic instability in both countries, where social classes inequalities [39, 51] make a profound gap in the adherence to the international dietary guidelines. Examples of recent and close socio-economic crises that affected both countries would be the 2008–2014 Spanish financial crisis [51], and the current 2019 Lebanese economic/financial crisis, where the perception and adherence to healthy dietary styles were profoundly shaken in both countries. Thus, all these diversified factors would explain the close similarities, and yet latent differences, between the Lebanese and Spanish populations’ perception of healthy food, and consequently orthorexic behaviors.

Moreover, the item-total correlations were high for most items, with a well-demonstrated convergent validity with the DOS scale. Hence, this makes the current Arabic version of the TOS a reliable instrument to assess ON’s tendencies and behaviors among Lebanese adolescents.

In terms of validity, the TOS OrNe subscale score significantly and positively correlated with the DOS score, supporting the convergent validity of the TOS. Nevertheless, the TOS HeOr subscale, known for its evaluation of non-pathological feature of healthy eating [52], correlated with DOS as well. Due to the lack of studies investigating the DOS–TOS association so far, conclusions must be drawn with caution. Future studies are need to clearly differentiate between the two constructs of “orthorexia nervosa” and “healthy orthorexia”.

In the current study, older age was weakly associated with lower TOS scores, thus less ON tendencies and behaviors. Another study as well [37] showed relatively similar results, where junior school-age youth was associated with higher ON tendencies compared to the senior school-age and university-age youth groups. This negative correlation between age and ON tendencies can be explained by the fact that younger adolescents, being at their early puberty period, are considered at their most vulnerable state facing the turbulent phase of dissatisfaction with their own body [3, 37]. They make frequent attempts to change their own appearance, implement diets without reliable knowledge about the principles of proper nutrition [37, 53].

Higher BMI was significantly but weakly associated with higher TOS scores, hence more ON tendencies. However, on the one side, orthorexia nervosa is thought to be related to food quality control rather than quantity control (unlike anorexia nervosa and other eating disorders), with previous findings having revealed little or no correlation between body mass index (BMI) and ON [10]. On the other side, other findings showed that higher BMI in adolescents was associated with higher orthorexic behaviors [54, 55]. Although orthorexia nervosa is thought to be related to food quality control rather than quantity control (unlike anorexia nervosa and other eating disorders), it is possible that overweight/obese subjects would adopt orthorexic strategies to achieve the accurate and healthy BMI, rather than the thin or good looking BMI, satisfying their obsession for wellness [55]. The weak correlations between ON and age and BMI require a cautious interpretation of the current results, as well as further studies to better explore these correlations.

Higher TOS scores were found in females compared to males. Despite the fact that no or small gender differences were found between male and female groups regarding ON tendency rates [30, 56, 57], many ON-related studies revealed the same problem as our sampling scheme, where females are more represented than males (76.1% of our participants are females) [58, 59]. However, aside from ON, disordered eating in general was found to be more susceptible in Lebanese female adolescents where concerns for healthy body image are of a higher grade than in male Lebanese adolescents [60]. However, further studies are needed to explore this relationship: gender–ON.

What is already known on this subject?

TOS is a promising instrument for assessing ON tendencies and behaviors, shedding the light on the distinction between healthy orthorexia and ON [7, 10], but it still lacks more validation and testing experiments for a broader and more reliable use, especially in the adolescent populations.

What does this study add?

In a sample of Lebanese adolescents, the TOS Arabic version seems to be a validated and reliable tool for ON assessment, what it may add to the literature what it lacked concerning the reliability and the validity of the TOS instrument.

Limitations and strengths

No causality can be inferred from this cross-sectional study. The DOS scale was not validated in the Arabic language yet. The orthorexic tendencies and behaviors were measured using a scoring instrument and not through a clinical patient–specialist interview; therefore, the accuracy of responses could not be confirmed. In fact, the TOS validity and use in the literature remain limited despite being a more promising instrument than any other, in terms of better internal consistency. Further assessment by a psychiatrist, a psychologist, and a qualified specialist may be needed to confirm the obtained results. The snowball sampling technique was used because of the challenging execution of data collection in Lebanon in the time of COVID-19 related lockdown; therefore, our data might not be generalizable. In our sample, female participants were more numerically represented than male participants. On another side, the used sample size is considerably large enough for the Lebanese high school population. Although, self-reported scales continue to be the most commonly used tools in epidemiological studies on disordered eating, the use of questionnaire can lead to information bias due to possible problems in understanding the questions and over-estimation / under-estimation of symptoms that may lead to inaccuracy. Obsessive–compulsive tendency and anorexic tendencies were not evaluated in the current study.

Conclusion

The values acquired from the validation and adaptation of the Arabic version of the TOS questionnaire were relatively acceptable. Hence, this Arabic version can be considered a valuable and reliable tool to assess the presence of orthorexic behaviors and tendencies in Lebanese adolescents, emphasizing the fine contrast between ON’s two dimensions: healthy vs. pathological. The evident numerous similarities, despite the few latent differences, that were found between the original Spanish version of the TOS and this current Arabic version, would draw the attention of the literature for the need of future studies that would re-interpret the ON’s measurement tools from the perspective of cross-cultural differences.

Funding

None.

Declarations

Conflict of interest

The authors declare not having any conflicts of interest to report in this study.

Ethical approval

The Ethics and Research Committee of the Psychiatric Hospital of the Cross approved this study protocol (HPC-035–2020). Students were asked to get their parents’ consent before filling the survey. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.