Skip to main content
NCBI home page
Clinical Neuropsychiatry logoLink to Clinical Neuropsychiatry
. 2023 Feb;20(1):55–60. doi: 10.36131/cnfioritieditore20230107

Psychometric Properties of the Italian Version of the Emiq-Hp (Exercise in Mental Illness Questionnaire-Health Professionals Version) to Investigate the Views of Health Professionals Regarding Exercise for Treatment of Mental Illness

Attilio Carraro 1,✉, Erica Gobbi 2, Robert Stanton 3,4, Giampaolo Santi 1, Simon Rosenbaum 5
PMCID: PMC10016097  PMID: 36936620

Abstract

Objective

People with mental illness experience a significantly increased mortality rate, partly attributable to high rates of premature metabolic disease. Exercise is increasingly recognized as an evidence-based component of treatment for improving both physical and mental illness. Ensuring health care staff have the necessary competence to support and promote exercise is critical to successful implementation. The Exercise in Mental Illness Questionnaire-Health Professionals Version (EMIQ-HP) was developed to determine the knowledge, attitudes, barriers, and behaviors of health practitioners regarding the role of exercise in the treatment for mental illness. This study aimed to translate and validate the questionnaire into Italian language (EMIQ-HP-IT) and to determine its test-retest reliability.

Method

The questionnaire was translated from English to Italian then back translated using an established protocol. To determine the test-retest reliability of the EMIQ-HP-IT, mental health professionals from an Italian psychiatric hospital completed the questionnaire on two separate occasions, seven days apart. Intra-class correlation coefficients (ICC) were calculated for each item. Twenty-five mental health professionals (4 psychiatrists, 9 psychologists, 10 nurses and 2 exercise specialists) completed the EMIQ-HP-IT.

Results

Except for two items, ICCs ranged from 0.48 to 0.92. The test-retest reliability of the EMIQ-HP-IT demonstrated comparable results to the English version.

Conclusions

The EMIQ-HP-IT is a reliable measure of practitioners’ knowledge, attitudes, beliefs, and behaviors regarding exercise interventions for people with mental illness.

Keywords: exercise, health professionals, physical activity, translation, validation

Introduction

The scientific evidence highlighting the efficacy of exercise in improving physical and mental health outcomes for people living with mental illness continues to increase at a significant rate. A comprehensive meta-review, coauthored by the European Psychiatric Association (EPA) and supported by the International Organization of Physical Therapists in Mental Health (IOPTMH) (Stubbs et al., 2018), found that moderate-vigorous physical activity supervised by an exercise specialist can reduce psychiatric symptoms, improve cognition and cardiorespiratory fitness in people with severe mental illness. Another meta-review from Czosnek and colleagues (2019) considered 33 separate systematic-reviews and meta-analyses (including 155 unique studies) on the health benefits, safety and cost of physical activity interventions for mental health conditions. The authors found consistent evidence that physical activity has a positive impact on symptom severity, quality of life and/or physical health outcomes (Czosnek et al., 2019).

Given the evidence supporting the role of exercise as an augmentative healthcare component, there is increasing focus worldwide on the implementation of clinical exercise programs and the integration of such programs into routine psychiatric care (Lederman et al., 2017). For instance, in Australia and in some European countries, university qualified exercise specialists and/or physiologists are increasingly being employed as allied health professionals and integrated within the standard multi-disciplinary mental health team to provide clinical exercise services for psychiatric patients (Furness et al., 2018; Lederman et al., 2016; Probst & Carraro, 2014; Vancampfort et al., 2015). In 2018, an international consensus statement (Rosembaum et al., 2018) was coauthored by four leading international sport and exercise organizations, collectively representing over 100,000 members from Australia (Exercise & Sports Science Australia - ESSA), the United States (American College of Sports Medicine - ACSM), the United Kingdom (British Association of Sports & Exercise Science - BASES) and New Zealand (Sport & Exercise Science New Zealand - SESNZ). In the statement, the organizations agreed on promoting participation in sport and exercise for people with mental illness, considering this as a global strategy for achieving a 50% reduction in the life expectancy gap faced by people with mental illness (Rosembaum et al., 2018). The statement identified three factors by which organizations can promote the integration of exercise practitioners into routine mental health care, specifically: knowledge, culture change, and infrastructure. Regarding increasing knowledge, the statement refers to the need to simultaneously increase the knowledge of mental health practitioners regarding exercise benefits, and to ensure exercise professionals receive adequate professional training in psychopathology. Both workforces must be upskilled with recent scientific advances in clinical care.

To promote the integration of exercise as a component for the treatment of people living with mental illness, Stanton and colleagues developed the Exercise in Mental Illness Questionnaire-Health Professionals version (EMIQ-HP; Stanton et al., 2014). The EMIQ-HP was designed to determine the knowledge, beliefs, attitudes, and behaviors of health professionals regarding the use of exercise in the treatment of mental illness. The questionnaire can also be used to determine the impact of specific education or training targeted for health professionals. Measuring and understanding the views of health professionals is crucial to changing practice, designing and delivering targeted initial and in-service professional training and facilitating culture change within psychiatric services (Fibbins et al., 2018; Negri et al., 2020; Stanton et al., 2014; 2018).

Given (i) the global problem of lack of exercise as part of usual psychiatric care; (ii) the clear importance of addressing health professionals’ views, knowledge and confidence at national level; and (iii) the need to provide appropriate educational programs for mental health professionals and to test their effectiveness, the aim of this paper was to determine the test-retest reliability of the EMIQ-HP Italian version (EMIQ-HP-IT).

Method

The development of the English version of the EM-IQ-HP has been reported elsewhere and encompassed a comprehensive literature review, and an expert-panel consultation and review performed by a multi-disciplinary panel of experts including clinicians, educators and researchers (Stanton et al., 2014). The questionnaire consists of six parts. The first four parts explore domains related to health professionals’ knowledge, beliefs and attitudes, behaviors, and perceived barriers to exercise. Part 1(items 1-10) examines knowledge of the benefits of exercise for people with mental illness, Part 2 (items 11-18) explores practitioners beliefs on the efficacy of exercise as a treatment for people with mental illness compared to other strategies, including attitudes towards exercise prescription,, Part 3 (items 19-25) identifies health professionals exercise prescription behaviors for people with mental illness, while Part 4 (items 26-48) identifies health practitioners barriers to prescribing exercise for people with mental illness. Additionally, the International Physical Activity Questionnaire – Short Form (IPAQ-SF; Craig et al., 2003; Mannocci et al., 2012) is included in Part 5 to investigate health professionals’ physical activity habits. Lastly, health practitioners’ demographic characteristics are gathered in Part 6.

The present study was conducted in accordance with the Declaration of Helsinki guidelines. Anonymity was ensured using participant codes, and demographic data was stored separately from the completed questionnaires.

Translation

The translation process followed an established protocol (WHO, 2007) and involved the following steps: (i) forward translation, (ii) reconciliation, (iii) back translation and review, (iv) harmonization, (v) cognitive debriefing and finalization. The translation process was led by authors AC and EG.

Instrument reliability

As per the original validation study of the EMIQHP, test-retest for each item was determined by calculating intraclass correlation coefficients using a two-factor mixed effects model (Shrout & Fleiss, 1979). Likert-type items are typically considered as a continuous variable and ICCs are the recommended method of assessing test-retest data for continuous data (Rankin & Stokes, 1998). Using ICCs can determine the proportion of total variance that occurs between time points (Shrout & Fleiss, 1979). This method is a suitable procedure with small sample sizes (see, e.g., de Sousa Magalhães et al., 2012).

Data were reviewed to ensure completeness and analyzed using SPSS version 28.0. ICCs were calculated for Part 1 (Exercise knowledge), Part 2 (Exercise beliefs) and Question 11 “Rank the value of exercise compared to other treatments”. As per the original validation study, items in Part 3 (Exercise prescription behaviors) were analyzed at an item level. For Part 4 (Barriers to exercise) the level of agreement with statements regarding barriers to prescribing exercise to people with mental illness was analyzed using the mean response to a series of Likert questions. ICCs with 95% confidence intervals (95% CI) were calculated for the total score, as well as each of the four domains, and interpreted based on Landis and Koch (1977), with ICCs of below 0.40 considered “poor to fair”, between 0.41 – 0.60 considered “moderate”, between 0.61 – 0.80 considered “excellent”, and between 0.81 – 1.00 considered “almost perfect” (Landis & Koch, 1977). Consistent with previous scale validation studies (Petzold et al., 2020; Stanton et al., 2014), Parts 5 and 6 did not undergo any statistical assessment, as they comprised an already validated scale (i.e., IPAQ; Craig et al., 2003; Mannocci et al., 2012) and demographic questions (e.g., gender, primary healthcare discipline).

Participants and Procedure

Twenty-five mental health professionals (4 psychiatrists, 9 psychologists, 10 nurses and 2 exercise specialists) from a private psychiatric hospital in Northeast Italy completed the EMIQ-HP-IT at Time 1 (T1) and, again seven days later at Time 2 (T2).

Thirty-five mental health professionals were initially invited to complete the questionnaire. Participants were informed of the aims of the study, that participation was voluntary and there were no consequences for not participating. There were no other exclusion criteria. Thirty-two participants completed the questionnaire on the first occasion (T1), then 7 participants were unable to complete the questionnaire at the second time point (T2), due to time constraints and clinical responsibilities. All participants provided written informed consent. The majority of the 25 professionals who completed the questionnaire on the two occasions were female (n = 18, 72%) and aged less than 50 years old (88%).

Results

Part 1 – Exercise knowledge items

Part 1 (Exercise knowledge) demonstrated high reliability, with excellent or almost perfect ICCs, ranging from 0.62 to 0.91 (see table 1 for full details).

Table 1.

Test-retest reliability of the exercise knowledge items (Part 1)

Item(s) Intraclass correlation coefficient 95% CI
1. Self-reported training in exercise prescription for people with mental illness 0.91 0.80 – 0.96
2. Details about formal training Open-ended question
3. Self-reported knowledge in exercise prescription for people with mental illness 0.62 0.10 – 0.84
4. Perceived confidence in exercise prescription for people with mental illness 0.90 0.77 – 0.96
5-10. Mean response to six level of agreement questions regarding the general benefits of exercise 0.89 0.76 – 0.95
Open in a new tab

Part 12-172 , – question Exercise 18 beliefs, question 11, questions

The ICCs for question 11 (perceived value of a range of treatments for mental illness compared to exercise) ranged from moderate to excellent (0.58 to 0.83) (See table 2). For the items 12-17, the ICCs for the mean item response ranged from 0.2 (item 14) to 0.76 (item 16). Except for the items 14 and 17, all other responses had excellent ICCs (See table 3). The last question in Part 2 (question 18) asked respondents to rank the importance of ten different treatment strategies used in the care of people with mental illness (i.e. medication, social support, electroconvulsive therapy, bright light therapy, family therapy, social skills training, cognitive behavioural therapy, vocational rehabilitation, exercise and hospitalisation), and was analyzed at an item level with the ICC for each item showing moderate to almost perfect test-retest reliability (ICC ranged from 0.48 to 0.91).

Table 2.

Test-retest reliability of question 11, perceived value of exercise compared to other forms of treatments (Part 2)

Item 11 Intraclass correlation coefficient 95% CI
a) Medication 0.69 0.31 – 0.87
b) Social support 0.77 0.48 – 0.90
c) Electroconvulsive therapy 0.83 0.62 – 0.93
d) Bright light therapy 0.77 0.48 – 0.90
e) Family therapy 0.58 0.10 – 0.82
f) Social skills training 0.83 0.62 – 0.93
g) Cognitive behavioral therapy 0.67 0.25 – 0.86
f) Vocational rehabilitation 0.79 0.53 – 0.91
Open in a new tab

Table 3.

Test-retest reliability of exercise beliefs items contained in questions 12 - 17 (Part 2)

Item Intraclass correlation coefficient 95% CI
12. People with a mental illness know that exercise is good for their physical health 0.67 0.26 – 0.86
13. People with a mental illness know that exercise is good for their mental health 0.71 0.35 – 0.87
14. People with a mental illness do not exercise because they don’t think they can 0.20 -0.82 – 0.65
15. Exercise is valuable for patients hospitalized with a mental illness in the same manner as outpatients 0.61 0.11 – 0.83
16. The physical and mental health benefits of exercise for people with a mental illness are not long lasting 0.76 0.46 – 0.90
17. People with a mental illness who are prescribed exercise will not adhere to it 0.22 -0.78 – 0.66
Open in a new tab

Part 3 – Exercise Prescription

The ICCs for the items in Part 3 (Exercise prescription) were analyzed at the item level and are presented in table 4. All items had excellent to almost perfect reliability.

Table 4.

Intraclass correlation coefficient for exercise prescription items (Part 3)

Item Intraclass correlation coefficient 95% CI
19. Do you prescribe exercise for people with mental illness 0.87 0.67 – 0.95
20. Do you undertake a formal assessment of the clients' suitability for exercise prior to prescribing a program? 0.85 0.63 – 0.94
21. When you prescribe exercise to people with a mental illness, what methods do you use? checkboxes
22. When you prescribe exercise to people with a mental illness, how often do you recommend they exercise? 0.92 0.81 – 0.97
23.When you prescribe exercise to people with a mental illness, how hard (what intensity) do you recommend they exercise? 0.70 0.26 – 0.88
24. When you prescribe exercise to people with a mental illness, how long do you suggest people try to exercise for at any one time? 0.88 0.70 – 0.95
25. When you prescribe exercise to people with a mental illness, what type of exercise do you suggest? checkboxes
Open in a new tab

Table 5.

Test-retest reliability of subsection A regarding barriers to prescribing exercise, and subsection B about barriers to exercise participation (Part 4)

Items Intraclass correlation coefficient 95% CI
26-36) Mean response to ten statements regarding the barriers to prescribing exercise to people with a mental illness 0.83 0.61 – 0.93
37-48) Mean response to twelve statements about the barriers to exercise participation for people with a mental illness 0.48 -0.19 – 0.77
Open in a new tab

Part 4 – Barriers to exercise

Part 4 is divided into two subsections: subsection A (items 26-36) asks respondents to rate their level of agreement with a series of statements regarding barriers to exercise prescription for people with mental illness; and subsection B (items 37-48) asks about barriers to exercise participation experienced by people with a mental illness. For both subsections, items were summed and analyzed using the mean response to the Likert scale response. The ICC for subsection A was 0.83 (0.61 – 0.93), representing excellent reliability. For subsection B, the ICC for the combined score was 0.48 (-0.19 to 0.77) representing moderate overall reliability.

Discussion

This paper describes the translation, and test-retest reliability of the EMIQ-HP-IT, a novel tool designed to assess health professionals’ knowledge, beliefs, behaviors and barriers regarding the use of exercise for the treatment of people with mental illness.

The EMIQ-HP-IT may play an important role in bridging the gap between research and practice. In fact, the questionnaire can inform the design of interventions aimed at promoting exercise as a treatment modality for people with mental illness. In pre-post study designs, the instrument can also be used to evaluate the effectiveness of training for health professionals on this specific topic. Similarly, the effectiveness of staff-based exercise interventions aiming to change health professionals’ attitudes can be evaluated through the EMIQ-HP-IT (Fibbins et al., 2018).

This study aimed to determine the test-retest reliability of the EMIQ-HP-IT. The questionnaire demonstrated acceptable test-retest reliability, with an overall acceptable ICC for each part of the questionnaire. The worst performing items were questions 14 “People with a mental illness do not exercise because they don’t think they can” and 17 “People with a mental illness who are prescribed exercise will not adhere to it”. The poor ICCs contrast with those of the original validation and development study and may be due to the negative phrasing of the questionnaire and potential translation issues. Caution should be used when interpreting these two items, and efforts should be made to ensure that health professionals completing the questionnaire are familiarized with these items. In any case and given the overall acceptable results of test-retest reliability, the EMIQ-HP-IT can be considered a reliable instrument for use within mental healthcare settings.

Some limitations of the present study should be acknowledged and further research with a larger and more diverse sample of mental health professionals working across different contexts are required. In fact, responses are likely to vary as a consequence of professional background (i.e., between psychiatrists, psychologists, nurses and exercise staff) and specific characteristics of different mental health services (e.g., inpatients or outpatient settings). Further research in these contexts would be of clinical relevance. The findings from such studies may inform whether discipline-specific or setting-oriented versions of the instrument are required. Future studies may also consider rewording items 14 and 17, for example modifying the double negative in the item 14.

Conclusions

The EMIQ-HP-IT appears to be a valid and reliable instrument to measure the knowledge, beliefs, attitudes, behaviors, and barriers of Italian speaking health professionals regarding the prescription of exercise for people with mental illness. The use of the EMIQ-HPIT may help inform future efforts aiming at increasing exercise participation for people with mental illness.

Acknowledgements

The authors would like to acknowledge the important contribution of the participants in this validation study.

Authors’ contributions

AC was the study coordinator, he led on the translation of the questionnaire, the recruitment of participants and data collection. EG assisted the study coordinator with the translation of the questionnaire and assisted SR in data analyses. GS assisted in revising the conceptual framework and revising the manuscript critically for intellectual content for publication. RS led on the drafting of the conceptual framework. SR was involved in the conception and design of the study, in data analyses and in the preparation of the manuscript.

References

  1. Craig, C. L., Marshall, A. L., Sjostrom, M., Bauman, A. E., Booth, M. L., Ainsworth, B. E., et al. (2003). International physical activity questionnaire: 12-country reliability and validity. Medicine and Science in Sports and Exercise, 35, 1381-1395. doi: 10.1249/01.MSS.0000078924.61453.FB [DOI] [PubMed] [Google Scholar]
  2. Czosnek, L., Lederman, O., Cormie, P., Zopf, E., Stubbs, B., & Rosenbaum, S. (2019). Health benefits, safety and cost of physical activity interventions for mental health conditions: A meta-review to inform translation efforts. Mental Health and Physical Activity, 16, 140-151. doi: 10.1016/j.mhpa.2018.11.001 [DOI] [Google Scholar]
  3. de Sousa Magalhães, S., Malloy-Diniz, L. F., & Hamdan, A. C. (2012). Validity convergent and reliability test-retest of the Rey Auditory Verbal Learning Test. Clinical Neuropsychiatry: Journal of Treatment Evaluation, 9(3), 129–137. [Google Scholar]
  4. Fibbins, H., Ward, P. B., Watkins, A., Curtis, J., & Rosenbaum, S. (2018). Improving the health of mental health staff through exercise interventions: A systematic review. Journal of Mental Health, 27(2), 184-191. doi: 10.1080/09638237.2018.1437614 [DOI] [PubMed] [Google Scholar]
  5. Furness, T., Hewavasam, J., Barnfield, J., McKenna, B., & Joseph, C. (2018). Adding an accredited exercise physiologist role to a new model of care at a secure extended care mental health service: A qualitative study. Journal of Mental Health, 27(2), 120-126. doi: 10.1080/09638237.2017.1294744 [DOI] [PubMed] [Google Scholar]
  6. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. 15. doi: 10.2307/2529310 [DOI] [PubMed] [Google Scholar]
  7. Lederman, O., Grainger, K., Stanton, R., Douglas, A., Gould, K., Perram, A., Baldeo, R., Fokas, T., Nauman, F., Semaan, A., Hewavasam, J., Pontin, L., & Rosenbaum, S. (2016). Consensus statement on the role of Accredited Exercise Physiologists within the treatment of mental disorders: A guide for mental health professionals. Australasian Psychiatry, 24(4), 347-351. doi: 10.1177/1039856216632400 [DOI] [PubMed] [Google Scholar]
  8. Lederman, O., Suetani, S., Stanton, R., Chapman, J., Korman, N., Rosenbaum, S., Ward, P. B., & Siskind, D. (2017). Embedding exercise interventions as routine mental health care: Implementation strategies in residential, inpatient and community settings. Australasian Psychiatry, 25(5), 451-455. doi: 10.1177/1039856217711054 [DOI] [PubMed] [Google Scholar]
  9. Mannocci, A., Di Thiene, D., Del Cimmuto, A., Masala, D., Boccia, A., De Vito, E., & La Torre, G. (2012). International Physical Activity Questionnaire: validation and assessment in an Italian sample. Italian Journal of Public Health, 7(4). doi: 10.2427/5694 [DOI]
  10. Negri, A., Andreoli, G., Mariani, R., De Bei, F., Rocco, D., Greco, A., & Bucci, W. (2020). First validation of the Referential Process Post-session Scale – Therapist version (RPPS-T). Clinical Neuropsychiatry, 17(6), 319-329. doi: 10.36131/cnfioritieditore20200601 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Petzold, M. B., Frank, G., Bendau, A., Plag, J., Betzler, F., & Ströhle, A. (2020). The German version of the Exercise in Mental Illness Questionnaire (EMIQ-G): Translation and testing of psychometric properties. Mental Health and Physical Activity, 19, 100353. doi: 10.1016/j.mhpa.2020.100353 [DOI] [Google Scholar]
  12. Probst, M., & Carraro, A. (2014). Physical activity and mental health: A practice-oriented approach. Edi. Ermes.
  13. Rankin, G., & Stokes, M. (1998). Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clinical rehabilitation, 12(3), 187-199. doi: 10.1191/026921598672178340 [DOI] [PubMed] [Google Scholar]
  14. Rosenbaum, S., Hobson-Powell, A., Davison, K., Stanton, R., Craft, L. L., Duncan, M., Elliot, C., & Ward, P. B. (2018). The role of sport, exercise, and physical activity in closing the life expectancy gap for people with mental illness: An international consensus statement by Exercise and Sports Science Australia, American College of Sports Medicine, British Association of Sport and Exercise Science, and Sport and Exercise Science New Zealand. Translational Journal of the American College of Sports Medicine, 3(10), 72-73. doi: 10.1249/TJX.0000000000000061 [DOI] [Google Scholar]
  15. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological bulletin, 86(2), 420-428. doi: 10.1037/0033-2909.86.2.420 [DOI] [PubMed] [Google Scholar]
  16. Stanton, R., Happell, B., & Reaburn, P. (2014). The development of a questionnaire to investigate the views of health professionals regarding exercise for the treatment of mental illness. Mental Health and Physical Activity, 7(3), 177-182. doi: 10.1016/j.mhpa.2014.06.001 [DOI] [Google Scholar]
  17. Stanton, R., Rosenbaum, S., Lederman, O., & Happell, B. (2018). Implementation in action: how Australian Exercise Physiologists approach exercise prescription for people with mental illness. Journal of Mental Health, 27(2), 150-156. doi: 10.1080/09638237.2017.1340627 [DOI] [PubMed] [Google Scholar]
  18. Stubbs, B., Vancampfort, D., Hallgren, M., Firth, J., Veronese, N., Solmi, M., Brand, S., Cordes, J., Malchow, B., Gerber, M., Schmitt, A., Correll, C. U., De Hert, M., Gaughran, F., Schneider, F., Kinnafick, F., Falkai, P., Möller, H., & Kahl, K. G. (2018). EPA guidance on physical activity as a treatment for severe mental illness: a meta-review of the evidence and Position Statement from the European Psychiatric Association (EPA), supported by the International Organization of Physical Therapists in Mental Health (IOPTMH). European Psychiatry: The Journal of the Association of European Psychiatrists, 54, 124-144. doi: 10.1016/j.eurpsy.2018.07.004 [DOI] [PubMed] [Google Scholar]
  19. Vancampfort, D., Stubbs, B., Ward, P. B., Teasdale, S., & Rosenbaum, S. (2015). Integrating physical activity as medicine in the care of people with severe mental illness. Australian & New Zealand Journal of Psychiatry, 49(8), 681-682. doi: 10.1177/0004867415590831 [DOI] [PubMed] [Google Scholar]
  20. World Health Organisation, WHO. (2007). Process of translation and adaptation of instruments. Published online 2007. Retrieved at: http://who.int/substance_abuse/research_tools/translation

Articles from Clinical Neuropsychiatry are provided here courtesy of Giovanni Fioriti Editore

RESOURCES