Skip to main content
PMC home page
International Journal of Methods in Psychiatric Research logoLink to International Journal of Methods in Psychiatric Research
. 2009 Feb 4;18(1):58–67. doi: 10.1002/mpr.277

Application of item response theory to achieve cross‐cultural comparability of occupational stress measurement

Akizumi Tsutsumi 1,✉, Noboru Iwata 2, Naotaka Watanabe 3, Jan de Jonge 4, Hynek Pikhart 5, Juan Antonio Fernández‐lópez 6, Liying Xu 7, Richard Peter 8, Anders Knutsson 9, Isabelle Niedhammer 10,11, Norito Kawakami 12, Johannes Siegrist 13
PMCID: PMC6878581  PMID: 19194857

Abstract

Our objective was to examine cross‐cultural comparability of standard scales of the Effort–Reward Imbalance occupational stress scales by item response theory (IRT) analyses. Data were from 20,256 Japanese employees, 1464 Dutch nurses and nurses' aides, 2128 representative employees from post‐communist countries, 963 Swedish representative employees, 421 Chinese female employees, 10,175 employees of the French national gas and electric company and 734 Spanish railroad employees, sanitary personnel and telephone operators. The IRT likelihood ratio model was used for differential item functioning (DIF) and differential test functioning (DTF) analyses. Despite the existence of DIF, most comparisons did not show discernible differences in the relations between Effort–Reward total score and level of the underlying trait across cultural groups. In the case that DTF was suspected, excluding an item with significant DIF improved the comparability. The full cross‐cultural comparability of Effort–Reward Imbalance scores can be achieved with the help of IRT analysis. Copyright © 2009 John Wiley & Sons, Ltd.

Keywords: cross‐cultural comparison, differential item functioning, differential test functioning, item response theory, stress

Full Text

The Full Text of this article is available as a PDF (249.8 KB).

REFERENCES

  1. Angoff W.H. (1993) Perspectives on differential item functioning methodology In: Differential Item Functioning (eds Holland P.W, Wainer H.), pp. 3–23, Lawrence Erlbaum. [Google Scholar]
  2. Antoniou A.‐S.G., Cooper C.L. (2005) Research Companion to Organizational Health Psychology (New Horizons in Management), Edward Elgar Publishing. [Google Scholar]
  3. Bjorner J., Kreiner S., Ware J., Damsgaard M., Bech P. (1998) Differential item functioning in the Danish translation of the SF‐36. Journal of Clinical Epidemiology, 51, 1189–1202, DOI: 10.1016/S0895‐4356(98)00111‐5 [DOI] [PubMed] [Google Scholar]
  4. Cooke D.J., Kosson D.S., Michie C. (2001) Psychopathy and ethnicity: structural, item, and test generalizability of the Psychopathy Checklist‐Revised (PCL‐R) in Caucasian and African American participants. Psychological Assessment, 13, 531–542. [DOI] [PubMed] [Google Scholar]
  5. de Jonge J., van der Linden S., Schaufeli W., Peter R., Siegrist J. (2003) Validity and reliability of the Dutch ERI scales: preliminary findings. ESF meeting. Düsseldorf.
  6. de Jonge J., van der Linden S., Schaufeli W., Peter R., Siegrist J. (2008) Factorial invariance and stability of the Effort–Reward Imbalance Scales: a longitudinal analysis of two samples with different time‐lags. International Journal of Behavioral Medicine, 15, 62–72. [DOI] [PubMed] [Google Scholar]
  7. de Smet P., Sans S., Dramaix M., Boulenguez C., De Backer G., Ferrario M., Cesana G., Houtman I., Isacsson S.O., Kittel F., Östergren P.O., Peres I., Pelfrene E., Romon M., Rosengren A., Wilhelmsen L., Kornitzer M. (2005) Gender and regional differences in perceived job stress across Europe. European Journal of Public Health, 15, 536–545, DOI: 10.1093/eurpub/cki028 [DOI] [PubMed] [Google Scholar]
  8. du Toit M. (2003) IRT from SSI: BILOG‐MG, MULTILOG, PARSCALE, TESTFACT, Scientific Software International.
  9. Embretson S.E., Reise S.P. (2000) Item Response Theory for Psychologists, Lawrence Erlbaum. [Google Scholar]
  10. Goldberg M., Chastang J.F., Leclerc A., Zins M., Bonenfat S., Bugel I., Kaniewski N., Schmaus A., Niedhammer I., Piciotti M., Chevalier A., Godard C., Imbernon E. (2001) Socioeconomic, demographic, occupational, and health factors associated with participation in a long‐term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population. American Journal of Epidemiology, 154, 373–384. [DOI] [PubMed] [Google Scholar]
  11. Hambleton R.K., Cook L.L. (1977) Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement, 24, 75–96. [Google Scholar]
  12. Hambleton R.K., Swaminathan H., Rogers H.J. (1991) Fundamentals of Item Response Theory (Measurement Methods for the Social Sciences), Sage Publications. [Google Scholar]
  13. Holland P.W., Wainer H. (1993) Differential Item Functioning, Lawrence Erlbaum. [Google Scholar]
  14. International Labour Organization (2004) Working Time and Workers' Preferences in Industrialized Countries: Finding the Balance, Routledge. [Google Scholar]
  15. Karasek R., Brisson C., Kawakami N., Houtman I., Bongers P., Amick B. (1998) The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. Journal of Occupational Health Psychology, 3, 322–355. [DOI] [PubMed] [Google Scholar]
  16. Karasek R., Theorell T. (1990) Healthy Work: Stress, Productivity, and the Reconstruction of Working Life, Basic Books. [Google Scholar]
  17. Kawakami N., Kobayashi F., Araki S., Haratani T., Furui H. (1995) Assessment of job stress dimensions based on the job demands‐control model of employees of telecommunication and electric power companies in Japan: reliability and validity of the Japanese version of the Job Content Questionnaire. International Journal of Behavioral Medicine, 2, 358–375. [DOI] [PubMed] [Google Scholar]
  18. Kawakami N., Haratani T., Kobayashi F., Ishizaki M., Hayashi T., Fujita O., Aizawa Y., Miyazaki S., Hiro H., Masumoto T., Hashimoto S., Araki S. (2004) Occupational class and exposure to job stressors among employed men and women in Japan. Journal of Epidemiology, 14, 204–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kristensen T.S., Bjorner J.B., Christensen K.B., Borg V. (2004) The distinction between work pace and working hours in the measurement of quantitative demands at work. Work & Stress, 18, 305–322. [Google Scholar]
  20. Li J., Yang W., Cheng Y., Siegrist J., Cho S.‐I. (2005) Effort–reward imbalance at work and job dissatisfaction in Chinese healthcare workers: a validation study. International Archives of Occupational and Environmental Health, 78, 198–204, DOI: 10.1007/s00420‐004‐0581‐7 [DOI] [PubMed] [Google Scholar]
  21. Lord F.M. (1980) Applications of Item Response Theory to Practical Testing Problems, Lawrence Erlbaum. [Google Scholar]
  22. Lord F.M., Novick M.R. (1968) Statistical Theories of Mental Test Scores, Addison‐Wesley. [Google Scholar]
  23. Macías Robles M.D., Fernández‐López J.A., Hernández‐Mejía R., Cueto‐Espinar A., Rancaño I., Siegrist J. (2003) [Measuring psychosocial stress at work in Spanish hospital's personnel. Psychometric properties of the Spanish version of Effort–Reward Imbalance model]. Medicina Clinica, 120, 652–657. [DOI] [PubMed] [Google Scholar]
  24. Marmot M., Siegrist J., Theorell T. (2006) Health and the psychosocial environment at work In: Social Determinants of Health (eds Marmot M, Wilkinson R.G.), pp. 97–130, Oxford University Press. [Google Scholar]
  25. Muthén L.K., Muthén B.O. (1998–2007) Mplus User's Guide, fifth edition, Muthén & Muthén. [Google Scholar]
  26. Niedhammer I., Tek M.‐L., Starke D., Siegrist J. (2004) Effort–reward imbalance model and self‐reported health: cross‐sectional and prospective findings from the GAZEL cohort. Social Science & Medicine, 58, 1531–1541, DOI: 10.1016/S0277‐9536(03)00346‐0 [DOI] [PubMed] [Google Scholar]
  27. Niedhammer I., Siegrist J., Landre M.F., Goldberg M., Leclerc A. (2000) Étude des qualités psychométriques de la version française du modèle du Déséquilibre Efforts [Psychometric properties of the French version of the Effort–Reward Imbalance model]. Revue d'Épidémiologie et de Santé Publique, 48, 419–437. [PubMed] [Google Scholar]
  28. Peng T.K., Peterson M.F., Shyi Y. (1991) Quantitative methods in cross‐national management research: trends and equivalence issues. Journal of Organizational Behavior, 12, 87–107. [Google Scholar]
  29. Peter R., Alfredsson L., Hammar N., Siegrist J., Theorell T., Westerholm P. (1998) High effort, low reward, and cardiovascular risk factors in employed Swedish men and women: baseline results from the WOLF study. Journal of Epidemiology and Community Health, 52, 540–547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Pikhart H., Bobak M., Siegrist J., Pajak A., Rywik S., Kyshegyi J., Gostautas A., Skodova Z., Marmot M. (2001) Psychosocial work characteristics and self rated health in four post‐communist countries. Journal of Epidemiology and Community Health, 55, 624–630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Raczek A., Ware J., Bjorner J., Gandek B., Haley S., Aaronson N., Apolone G., Bech P., Brazier J., Bullinger M., Sullivan M. (1998) Comparison of Rasch and summated rating scales constructed from SF‐36 physical functioning items in seven countries: results from the IQOLA Project. International Quality of Life Assessment. Journal of Clinical Epidemiology, 51, 1203–1214, DOI: 10.1016/S0895‐4356(98)00112‐7 [DOI] [PubMed] [Google Scholar]
  32. Reise S.P., Widaman K.F., Pugh R.H. (1993) Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552–566. [DOI] [PubMed] [Google Scholar]
  33. Siegrist J. (1996) Adverse health effects of high‐effort/low‐reward conditions. Journal of Occupational Health Psychology, 1, 27–41. [DOI] [PubMed] [Google Scholar]
  34. Siegrist J. (2005) Social reciprocity and health: new scientific evidence and policy implications. Psychoneuroendocrinology, 30, 1033–1038, DOI: 10.1016/j.psyneuen.2005.03.017 [DOI] [PubMed] [Google Scholar]
  35. Siegrist J., Peter R. (1997) Measuring Effort–Reward Imbalance at Work: Guidelines, Düsseldorf: Heinrich Heine Universität Düsseldorf. [Google Scholar]
  36. Siegrist J., Starke D., Chandola T., Godin I., Marmot M., Niedhammer I., Peter R. (2004) The measurement of Effort–Reward Imbalance at work: European comparisons. Social Science & Medicine, 58, 1483–1499, DOI: 10.1016/S0277‐9536(03)00351‐4 [DOI] [PubMed] [Google Scholar]
  37. Stansfeld S., Candy B. (2006) Psychosocial work environment and mental health – a meta‐analytic review. Scandinavian Journal of Work, Environment & Health, 32, 443–462. [DOI] [PubMed] [Google Scholar]
  38. Steenland K., Johnson J., Nowlin S. (1997) A follow‐up study of job strain and heart disease among males in the NHANES1 population. American Journal of Industrial Medicine, 31, 256–260, DOI: 10.1002/(SICI)1097‐0274(199702)31:2 [DOI] [PubMed] [Google Scholar]
  39. Thissen D., Steinberg L., Wainer H. (1988) Use of item response theory in the study of group differences in trace lines In: Test Validity (eds Wainer H, Braun I.), pp. 147–169, Lawrence Erlbaum. [Google Scholar]
  40. Thissen D., Steinberg L., Wainer H. (1993) Detection of differential item functioning using the parameters of item response models In: Differential Item Functioning (eds Holland P.W, Wainer H.), pp. 67–113, Lawrence Erlbaum. [Google Scholar]
  41. Tsutsumi A. (2004) Development and Application of the Japanese Version of Effort–Reward Imbalance Questionnaire. Report of Research Project, Grand in Aid for Scientific Research (C), 2001∼2004, Okayama University School of Medicine and Dentistry, Hygiene & Preventive Medicine.
  42. Tsutsumi A., Kawakami N. (2004) A review of empirical studies on the model of effort–reward imbalance at work: reducing occupational stress by implementing a new theory. Social Science & Medicine, 59, 2335–2359, DOI: 10.1016/j.socscimed.2004.03.030 [DOI] [PubMed] [Google Scholar]
  43. Tsutsumi A., Ishitake T., Peter R., Siegrist J., Matoba T. (2001) The Japanese version of the Effort–Reward Imbalance questionnaire: a study in dental technicians. Work & Stress, 15, 86–96. [Google Scholar]
  44. Van der Doef M., Maes S. (1999) The job demand‐control(‐support) model and psychological well‐being: a review of 20 years of empirical research. Work & Stress, 13, 87–114. [Google Scholar]
  45. van Vegchel N., de Jonge J., Bakker A.B., Schaufeli W.B. (2002) Testing global and specific indicators of rewards in the Effort–Reward Imbalance model: does it make any difference? European Journal of Work and Organizational Psychology, 11, 403–421. [Google Scholar]
  46. van Vegchel N., de Jonge J., Bosma H., Schaufeli W.B. (2005) Reviewing the effort–reward imbalance model: drawing up the balance of 45 empirical studies. Social Science & Medicine, 60: 1117–1131, DOI: 10.1016/j.socscimed.2004.06.043 [DOI] [PubMed] [Google Scholar]
  47. Xu L., Siegrist J., Cao W., Li L., Tomlinson B., Chan J. (2004) Measuring job stress and family stress in Chinese working women: a validation study focusing on blood pressure and psychosomatic symptoms. Women & Health, 39, 31–46. [DOI] [PubMed] [Google Scholar]

Articles from International Journal of Methods in Psychiatric Research are provided here courtesy of Wiley

RESOURCES