Validation of new psychosocial factors questionnaires: a Colombian national study

Abstract

Background:

The study of workers’ health problems possibly associated with stressful conditions requires valid and reliable tools for monitoring risk factors. The present study validates two questionnaires to assess psychosocial risk factors for stress-related illnesses within a sample of Colombian workers.

Methods:

The validation process was based on a representative sample survey of 2360 Colombian employees, aged 18–70 years. Worker response rate was 90%; 46% of the responders were women. Internal consistency was calculated, construct validity was tested with factor analysis and concurrent validity was tested with Spearman correlations.

Results:

The questionnaires demonstrated adequate reliability (0.88 to 0.95). Factor analysis confirmed the dimensions proposed in the measurement model. Concurrent validity resulted in significant correlations with stress and health symptoms.

Conclusions:

“Work and Non-work Psychosocial Factors Questionnaires” were found to be valid and reliable for the assessment of workers’ psychosocial factors, and they provide information for research and intervention.

INTRODUCTION

Occupational health experts, public health authorities and social partners have acknowledged that psychosocial conditions are a key occupational issue in different countries. Research evidence abounds on the relationship between stress and negative effects on an individual’s health and safety [Kasper et al., 1999; Martínez, 2001; Sauter et al., 2002; Kompier, 2005; Clarke, 2004].

A previous study designed to identify work-related stress factors conducted in 937 workplaces in Colombia [Ministerio de la Protección Social, 2007], identified four psychosocial risk factors most frequently reported: emotional demand from customer service, monotonous and repetitive work, ambiguous job tasks and lack of authorized breaks at work. In response to this study, the Colombian Ministry of Social Security identified intervention of psychosocial risks factors as a national priority, especially among critical economic activities, such as health care, education and banking.

The legal and scientific framework related to occupational health emphasizes exposure assessment as the first step in prevention and intervention [Starvorula et al., 2008]. Evaluation of psychosocial exposure among cohorts of workers would only become possible with a national policy, proper evaluation instruments and consistent information registries.

Although there has been some previous work in Colombian related to assessment methodologies of psychosocial factors [Villalobos, 1998; Bocanument et al., 1996; Ministerio de la Protección Social et al., 2004] the Colombian Ministry of Social Security provided the funding to develop a study aimed at designing and validating a set of new psychosocial assessment instruments for the Colombian working population. Such instruments would allow government and workplaces to prioritize psychosocial risks factors that are relevant for prevention of illnesses and promotion of health.

The purpose of this paper is to present the results of a validation study of a new set of psychosocial questionnaires developed by the authors according to the measurement model, to assess psychosocial risk factors at work and non-work settings: Work Psychosocial-Factor Questionnaire (WPFQ) [Form A and B], and “Non-work Psychosocial-Factor Questionnaire (N-WPFQ)”

Measurement model

The structure of the measurement model in which the questionnaires design was based, considered two main sources of psychosocial factors: work and non-work conditions (Figure 1). An additional set of questions was included to collect demographic and occupational information.

Figure 1.

Psychosocial factors measurement model.

Questionnaires for work psychosocial factors were based on theoretical contributions of the effort-reward imbalance model by Siegrist [Siegrist, 1996], and on Karasek [Karasek et al., 1990] model of demand–control and social support. Additionally, the measurement model considered some psychosocial dimensions based on practical experience and previous studies of the research group, as well as the country’s technical and legal requirements in relation to the assessment of psychosocial factors.

Theoretical Domains and Dimensions

The Work Psychosocial-Factor Questionnaires (WPFQ) Forms A and B are structured into four main domains (social relations and leadership at work, job demands, job control, and job rewards) [Ministerio de la Protección Social et al., 2010a]. Dimensions of each domain are explained in the following paragraphs.

Social Relations and Leadership at Work:

This domain considers three aspects: 1) the possibility of social relationship, 2) social support, and 3) the quality of the leadership interactions between coworkers and supervisors. This domain is structured in four dimensions: (i) Leadership characteristics refers to supervisory management attributes related to work planning and allocation o resources, results achievement, conflict resolution, participation, motivation, social support, interaction and communication with workers. (ii) Social relationships at work considers the characteristics of the interaction including contact with other individuals, the quality of peer interactions, the possibility of teamwork, and the group cohesion. (iii) Performance feedback describes whether or not the worker receives enough information on how his/her work has been done. This information allows the worker to identify his/her strengths and weaknesses and facilitate action to maintain or improve performance. (iv) Relationships with subordinates explores supervisor concerns about the worker’s relationship with subordinates as it relates to work performance, achieving results, conflict resolution, communication and participation.

Job Demands:

This domain refers to work requirements demanding a workeŕs effort. These demands have different dimensions according to their nature and source:

(i) Role consistency deals with the compatibility between various aspects of job tasks including technical efficiency, ethics and quality, and performance. (ii) The environmental and physical activity demands consider aspects of workstation design, exposure to physical, chemical, biological and safety stressors, which may require adaptation. (iii) Emotional demands concerns the affective content of the job that has the potential to interfere with the feelings and emotions of the worker and implies his/her ability to understand the situations and feelings of others, as well as to hide his or her own emotions. (iv) Quantitative demands consist of the relationship between the quantity of work and the time to do it, including the pace of work and work hours. (v) Work influence on the non-work setting refers to all job demands that produce negative effects in family relationships, such as absences from home and family activities. (vi) Position responsibility demands identify whether or not the worker must take specific responsibility for their own actions and the actions of others. (vii) Mental load demands refer to the cognitive processing requirements of the job for receiving information, processing it and making decisions. (viii) Working-time demands express requirements in terms of the duration and temporal organization of work which could lead to working at unusual or irregular shift work, as well as to shorter or no break periods.

Job Control

Job control domain refers to the workeŕs ability to influence and make decisions on various matters that affect his/her work. There are five control dimensions:

(i) The role clarity is the definition and communication of the role the worker is expected to play in the organization, in particular work objectives, functions, autonomy and the impact of his/her position. (ii) Training means the access to educative plans aligned with current job and career development that the organization provides to the worker in order to develop and strengthen his/her knowledge and skills. (iii) Participation and change management is defined as the set of organizational mechanisms aimed at increasing the adaptability of workers to the different transformations that occur in the employment context. (iv) Opportunities for knowledge and skills use and development refers to the opportunity the individual worker has learn and apply new skills and knowledge. (v) Control and work autonomy refers to the discretion an individual worker has to determine the methods, workload, rhythm, order of activities, working hours and the rest periods.

Job Rewards:

Job rewards domain refers to possible benefits the worker receives in exchange for his/her work. Work rewards considers two dimensions: (i) Rewards from work that correspond to feelings of pride and the sense of job security experienced by an individual as a result of belonging to an organization, as well as the sense of job satisfaction. (ii) Recognition and compensation (including remuneration, access to services and professional or personal development opportunities) the organization gives the employee in consideration of his/her effort at work.

Non-work Psychosocial-Factor Questionnaire (N-WPFQ) is comprised of seven dimensions related to use of non-work time, family relationships, communication and social relations, family economic situation, housing characteristics, influence of non-work conditions on work, and transportation between home and work. These dimensions are defined as: :(i) Use of non-work time relates to the time the individual engages in activities other than work, such as rest, sharing time with family and friends, personal care or domestic responsibilities, leisure activities and entertainment; (ii) Family relationships assesses the individual’s interactions with his/her family; (iii) Communication and social relations explores the qualities of the individual’s communication and interactions with his/her relatives and friends; (iv) Family economic situation explores the availability of funds for the worker and his/her family to meet their basic needs; (v) Housing characteristics refers to the physical structure and location of the worker’s home residence; (vi) Influence of non-work conditions corresponds to the demands family roles and personal well-being on work performance; (vii) Transportation between home and work looks at the conditions of the worker’s commute. This also includes the ease and convenience of transportation, and travel time.

METHODS

Participants

The validation study was carried out in a representative sample of 2360 workers from a population of 6,796,459 workers enrolled in the Workers’ Compensation System (WCS) in Colombia. The validation study was based on a multistage, non-probability sampling design by quotas.

Since the sample was meant to be proportionally representative of Colombian workers by region and by economic activity, the sampling process was dealt as follows:

1. All workplaces from a list provided by the Ministry of Social Security were classified into six regions and into four groups by economic activity within each region.

2. The expected sample of workers was proportionally distributed by quotas for each region and by economic activity within each region.

3. A group of workplaces by economic activity were randomly selected within each region in cities with high workplace concentration and geographic accessibility. Workplaces voluntarily participating were admitted in the study through snowball sampling.

4. Workers within each workplace were randomly selected by using the payroll or a numbered list, until the quota was met. Limits were set to avoid having sample quotas concentrated in few companies (a maximum of 20 workers from large, 7 or 8 from medium, and 1 or 2 from small workplaces).

Instruments

Three questionnaires were developed to assess the psychosocial risk factors at work and non-work settings. These questionnaires were tested and validated through the study described in this article. The study tested the psychometric characteristics of: the “Work Psychosocial-Factor Questionnaire (WPFQ)” Form A, designed for supervisors, professionals, and technicians; the “WPFQ” Form B, to be used with auxiliary and blue-collar workers; and the “Non-work Psychosocial-Factor Questionnaire (N-WPFQ),” designed for all worker populations. The WPFQ Form A has 123 items and Form B has 97; the N-WPFQ has 31 items. The respondent was asked about the frequency of a particular situation using a 5-point Lykert-type scale from never to always. According to the measurement model, work-related psychosocial factors are grouped in dimensions that constituted domains, in which items are distributed.

Three scales of the Ware and Gandeck [Ware et al., 1998; Ware et al., 2008] SF36v2 Health Survey: General Health, Mental Health and Vitality previously validated in Colombia [Lugo et al., 2006], and the “Stress symptoms questionnaire” (Villalobos, 2005, unpublished doctoral dissertation) were used to test concurrent validity. The research group decided to use the SF36v2 scales to estimate the concurrent validity, inspired in the structure of the Copenhagen Psychosocial Questionnaire (COPSOQ) [Kristensen et al., 2005] and its Spanish version (ISTAS21) [Moncada et al., 2002], in which these SF36v2 scales constitute part of these instruments.

The “Stress symptoms questionnaire” comprises 31 items to explore frequency of physiological, behavioral, intellectual and psycho-emotional symptoms (Villalobos G., 2005, unpublished doctoral dissertation) and was later published as part of a battery of psychosocial questionnaires developed for the Ministry of Social Security in Colombia [Ministerio de la Protección Social et al., 2010b]. This questionnaire was developed and validated in a sample of 2199 Colombian workers (0.89 Cronbach’s alpha coefficient).

The Job Content Questionnaire (JCQ) [Karasek et al., 1998], as well as the Effort Reward Imbalance Questionnaire (ERI) [Siegrist et al., 2004], and even the COPSOQ or the ISTAS21, were not used for the concurrent validity process because they are not validated in Colombian population, although a few studies have been recently developed to this end, but none of them have had a representative sample of the country workers.

The questionnaire design took into account the need to have culturally adapted instruments. To achieve this, the following procedures were carried out [Alexandre et al., 2002]:

(i) Cognitive interviews were conducted with 23 workers of different educational level and economic status. This process was particularly useful in ascertaining respondents’ understanding, in particular to questions that might be sensitive for certain groups [Drennan, 2003; Beatty et al., 2007]. Through this process of interviews we were able to test the “translation” of theoretical concepts to the daily realities of the Colombian worker. These included testing comprehension of individual items, information retrieval, decision process, and adequacy of response options. As a result of this process, the “WPFQ” resulted in two forms (A and B) due to differences we uncovered in the comprehension and relevance of dimensions between two main groups of workers: 1) managers, professionals and technicians, and 2) auxiliary and blue-collar workers.

(ii) Based on the results of the cognitive interviews, the individual questionnaire items were reviewed and revised as needed by a committee of seven experts in psychosocial factors and by trained members of the research group.

(iii) A pilot study with 132 workers from different economic activities, job positions and education levels was conducted and the results from the pilot were helpful to further adjust and refine the survey instruments.

(iv) The knowledge of the Colombian work reality and expertise in psychosocial factors of the research group acquired through several years of previous work in the field provided the skill base necessary to make final adjustments to the questionnaires that incorporated the results from the cognitive interviews and pilot testing that were culturally adapted to the Colombian situation.

In addition to the contributions of the research team, the workers’ participation during the questionnaire design process contributed greatly to the social and cultural interpretation of the psychosocial variables previously described [Brod et al., 2009].

Data collection

Fieldwork was undertaken in two stages: recruitment and data collection. Both activities were carried out following specific procedures. All first contacts with organizations and workers were made through letters informing them about the project’s objective, funding entity, benefits, and activities.

Shortlisted organizations were contacted by telephone, and we visited those interested to explain in detail the study activities, to collect organizational information, and to select a sample of workers from the payroll list. In total 790 organizations were contacted, though only 229 participated.

On the questionnaire administration day, the data collector handed each of the 2623 selected workers a study information letter. This letter included the informed consent, and those who chose to participate signed the form. Next, the questionnaires were given to each of the 2360 participants in a sealed envelope, marked with a code, and separated from the consent form. In most cases, questionnaires were self-administered; workers with limited literacy skills were interviewed. Once the questionnaires had been completed, respondents returned them to the data collector, who checked them for completeness.

Questionnaires and any other information related to individuals’ names were stored securely at the lead author’s institution and all data were processed together. The fieldwork team completed the Human Subjects Training Module [Health National Institute] before gathering data, and they received specific training for this study. Additionally, each member of the research group signed a confidentiality agreement as part of his/her contract. The Research Ethics Committee at Javeriana University School of Medicine approved the study. A research license for the use of SF36v2 was obtained.

Data analysis

Reliability of the questionnaires was evaluated by using Cronbach’s alpha coefficients. To analyze construct validity, an exploratory factor analysis using the principal component factor method and a promax rotation was performed. Additional analysis was conducted to confirm the construct validity of the questionnaires [Brown, 2000], Spearman coefficient correlations were calculated for each dimension within the domain it belongs to, and then compared with the correlations of the same dimension within other domains.

Concurrent validity was calculated with Spearman correlation coefficient among psychosocial questionnaires with the three scales of the SF36v2 Health Survey, and with the Stress Symptoms Questionnaire.

National reference values were constructed by using linear transformation of raw scores to scale scores ranging from 0 to 100 for each dimension and domain. Scale scores were divided into quintiles for the purpose of having homogeneous groups, which corresponded to defined exposure risk levels: no risk or insignificant risk (≤ 20 percentile), low risk (21–40 percentile), medium risk (41–50 percentile) high risk (51–60 percentile), and very high risk (≥ 80 percentile). In order to assess the magnitude and significance of psychosocial risk levels, the strength of association between levels of psychosocial risk factors and high levels of stress symptoms, Prevalence Rate Ratios (PRR) and their 95% confidence intervals (CI) were calculated.

To analyze the questionnaires discrimination power between groups the two-sample Wilcoxon rank-sum (Mann-Whitney) test was used. These data were processed by using SPSS (V16), Excel (2007) and Stata (V10).

RESULTS

Sample characteristics

The study involved 229 organizations across six regions. The sample was regionally and proportionally distributed by economic activity (Table I). Service sector was the major participant group, followed by industry, trade and farming/agricultural. Workers from farming/agricultural were mostly working in micro enterprises; trade and industry workers were mainly from small and medium enterprises. Private organizations had a higher participation than public organizations, which was coincident with the general distribution of workers in the country. Although the study involved the participation of companies of different sizes, more than half (58%) of them were small and medium-sized ones.

Table I.

Sample distribution of workers enrolled in the Workers’ Compensation System by region and economic sector.

Characteristics		Workers enrolled in the WCP		Study sample
		N	%	N	%
Region	Central Andean	2,854,513	42	935	40
	Amazonia	67,965	1	38	2
	Bogotá	2,174,867	32	767	33
	Caribbean	679,646	10	269	11
	Pacific	883,539	13	277	12
	Orinoquía	135,929	2	74	3
Economic sector	Farming/Agricultural	679,646	10	224	19
	Trade	1,291,327	19	426	18
	Industry	679,646	10	403	17
	Service	4,145,840	61	1,307	55

There were more male participants (54%), especially in the Amazonian and Pacific regions. The average age was 36 years (SD=10). Workers from all education levels participated in the study, some without any formal education (0.3%).

More than half (54%) of the study participants were married, and 66% had between one and three dependents. The median value for total years of work was 3 (percentile 25=1, percentile 75=8), which was higher to seniority at the current job (median value 2). Most reported having permanent contracts (61%) and fixed salary (88%).

Blue-collar workers had the highest proportion of participants (32%), followed by auxiliary, supervisors, professionals and technicians. Most (86%) worked between 8 and 10 hours daily (mean 9 and SD=1.5) (Table II).

Table II.

Demographic and occupational characteristics of sample

Demographic and occupational characteristics		Sample
		n	%
Sex	Male	1267	54
	Female	1093	46
Age	18–25 years old	389	16
	26–35	896	38
	36–45	609	26
	46–56	395	17
	57 or more	59	3
	No information	12	1
Marital Status	Single	881	37
	Married or cohabitation	1273	54
	Divorced	179	8
	Widow/widower	26	1
Level of education	No education	4	0
	Primary	118	5
	High school	692	29
	Technician	663	28
	College or higher	876	37
	No information	7	0
Number of economic dependants	None	391	17
	1 to 3	1560	66
	4 or more	361	15
	No information	48	2
Type of job	Manager/supervisor	500	21
	Professional/technician	469	20
	Clerk/assistant	623	26
	Blue collar	768	33
Type of contract	Temporary	730	31
	Permanent	1456	62
	Other	136	6
	No information	38	2
Number of daily working hours	<8 h/day	69	3
	8–10 h/day	2043	87
	11–15	243	10
	>16 h/day	3	0
	No information	5	0

Reliability

Internal consistency in 17 of the 19 dimensions within WPFQ Form A was above 0.70 (Table III). Working-time demands (0.53) and position responsibility demands (0.69) were the exceptions.

Table III.

WPFQ Forms A and B and domain reliability coefficients

WPFQ Domains	Form A (n=1576)		Form B (n=784)
	Number of items	Cronbach’s alpha	Number of items	Cronbach’s alpha
Characteristics of social relations and leadership at work	41	0.94	30	0.94
Job control	21	0.90	18	0.82
Job demands	50	0.89	39	0.82
Job rewards	11	0.82	10	0.79
WPFQ	123	0.95	97	0.93

In 10 of the 16 dimensions in WPFQ Form B, the internal consistency was equal or higher than 0.70 (Table III). Lower coefficients were found in environmental and physical effort demands (0.68); control and work autonomy (0.68); rewards from work and organization membership (0.67); work influence on non-work setting (0.62); working-time (0.60); and quantitative demands (0.57).

The dimensions and the entire N-WPFQ had internal reliability coefficients higher than 0.73 (Table IV).

Table IV.

N-WPFQ and dimension reliability coefficients

N-WPFQ Dimensions	No. items	Cronbach’s alpha (n=2360)
Use of non-work time	4	0.90
Family relationships	3	0.74
Communication and social relations	5	0.82
Family economic situation	3	0.73
Housing characteristics	9	0.75
Influence of non-work conditions on work	3	0.81
Transportation between home and work	4	0.75
N-WPFQ	31	0.88

Construct validity

Factor analysis confirmed the item-grouping described in the measurement model. As a result of the factor analysis, two dimensions in WPFQ Form B from the original measurement model were discarded. It was also found that three of the original dimensions considered in both WPFQ forms (teamwork, social support, and group cohesion) were assessing just one factor: social relationships at work.

WPFQ Form A resulted in 19 dimensions distributed across 24 factors; Form B resulted in 16 dimensions distributed across 22 factors (Table V). N-WPFQ resulted in 7 dimensions, (Table VI) but 2 of them were renamed after pre-testing to better reflect their content. Double role demands - work and home was changed to Influence of non-work conditions on work, and non-work social support was changed to communication and social relations. After factor analysis, 6 dimensions in Form A, 7 in Form B, and 2 in N-WPFQ remained unchanged.

Table V.

WPFQ- forms A and B: maximum and minimum items factor load

Domain	Dimension	Maximum and minimum items factor load
		Form A	Form B
Characteristics of social relations and leadership at work	Leadership characteristics	F1 (Max 0.84 Min 0.68)	F1 (Max 0.88 Min 0.54)
	Social relationships at work	F2 (Max 0.88 Min 0.61) F15 (Max 0.84 Min 0.62)	F2 (Max 0.86 Min 0.57) F21 (Max 0.83 Min 0.52)
	Performance feedback	F4 (Max 0.82 Min 0.67)	F4 (Max 0.84 Min 0.62)
	Relationship with subordinates	F10 (Max 0.93 Min 0.83)	Not applicable
Job control	Role clarity	F3 (Max 0.88 Min 0.63)	F3 (Max 0.94 Min 0.61)
	Training	F8 (Max 0.93 Min 0.89)	F8 (Max 0.93 Min 0.89)
	Participation and change management	F9 (Max 0.80 Min 0.41)	F20 (Max 0.80 Min 0.77)
	Opportunities for knowledge and skills use and development	F11 (Max 0.84 Min 0.49)	F9 (Max 0.89 Min 0.45)
	Control and work autonomy	F16 (Max 0.98 Min 0.76)	F22 (Max 0.83 Min 0.52)
Job demands	Role consistency	F5 (Max 0.75 Min 0.53)	Not applicable
	Environmental and physical effort demands	F7 (Max 0.83 Min 0.55) F19 (Max 0.89 Min 0.36) F27 (Max 0.84 Min 0.72)	F5 (Max 0.84 Min 0.31) F19 (Max 0.84 Min 0.30) F27 (Max 0.85 Min 0.70)
	Emotional demands	F13 (Max 0.88 Min 0.41)	F6 (Max 0.87 Min 0.48) F12 (Max 0.88 Min 0.88)
	Quantitative demands	F14 (Max 0.75 Min 0.43)	F16 (Max 0.77 Min 0.46)
	Work influence on the non-work setting	F17 (Max 0.85 Min 0.45)	F14 (Max 0.75 Min 0.32)
	Position responsibility demands	F20 (Max 0.71 Min 0.41) F22 (Max 0.76 Min 0.54)	Not applicable
	Mental load demands	F23 (Max 0.79 Min 0.50)	F18 (Max 0.81 Min 0.51)
	Working-time demands	F21 (Max 0.69 Min 0.58)	F10 (Max 0.90 Min 0.40) F25 (Max 0.78 Min 0.46)
Job rewards	Rewards from the work and the organization membership	F6 (Max 0.83 Min 0.33)	F15 (Max 0.73 Min 0.34)
	Recognition and compensation	F12 (Max 0.72 Min 0.36) F18 (Max 0.88 Min 0.45)	F7 (Max 0.79 Min 0.36) F17 (Max 0.84 Min 0.70)

Table VI.

N-WPFQ maximum and minimum items factor load

Dimension	Maximum and minimum items factor load
Use of non-work time	F1 (Max 0.92 Min 0.87)
Family relationships	F2 (Max 0.89 Min 0.68)
Communication and social relations	F3 (Max 0.91 Min 0.38)
Family economic situation	F4 (Max 0.81 Min 0.66)
Housing characteristics	F5 (Max 0.82 Min 0.61) F6 (Max 0.88 Min 0.55) F9 (Max 0.90 Min 0.75)
Influence of non-work conditions on the work	F7 (Max 0.85 Min 0.84)
Transportation between home and work	F8 (Max 0.86 Min 0.54)

Decisions for changing items between dimensions or eliminating some of them were made considering their low weight in the initial dimension (≤0.50), a higher weight in another dimension, their theoretical content appropriateness, their low discrimination power according to frequency of answers and their effect on the dimension reliability. Thus, 18 items were moved between dimensions, and 33 were discarded in all three questionnaires.

Correlation coefficients were used to test whether the questionnaires were measuring an underlying homogeneous construct. In both WPFQ Forms A and B, all correlations between domains with their respective dimensions were statistically significant (p=0.00), and higher than correlations of the same domain with other dimensions. Likewise, all correlations of N-WPFQ dimensions with the questionnaire’s total score were significant (p=0.00).

The following were the lowest and the highest correlations of each domain of the WPFQ form A with its own dimensions: Characteristics of social relations and leadership at work 0.28 to 0.85 (p=0.00); Job control 0.51 to 0.83 (p=0.00); Job demands 0.48 to 0.66 (p=0.00); and Job rewards 0.80 to 0.95 (p=0.00).

Concurrent validity

Results showed that SF-36v2 scales were inversely correlated with work (Forms A and B) and non-work psychosocial-factor questionnaires. To the contrary, it was found that psychosocial risks factors were directly correlated with stress symptoms. Comparing the correlations between the Psychosocial Factors Questionnaires with SF-36v2 and with the stress symptoms questionnaire, it was found that the latter coefficients were the highest. Likewise, WPFQ forms A and B domains showed higher association with the stress symptoms questionnaire than with the SF36v2 scales (Table VII).

Table VII.

Spearman correlation coefficients of WPFQ domains with SF-36v2 scales and with the stress symptoms questionnaire

	WPFQ Form A				WPFQ Form B
Correlated Variables	(1)*	(2)	(3)	(4)	(1)*	(2)	(3)	(4)
Stress symptoms questionnaire	1.00				1.00
SF-36v2 General health scale	−0.51	1.00			−0.48	1.00
SF-36v2 Vitality scale	−0.68	0.51	1.00		−0.61	0.55	1.00
SF-36v2 Mental health scale	−0.69	0.47	0.69	1.00	−0.64	0.48	0.66	1.00
Characteristics of social relations and leadership at work domain	0.32	−0.19	−0.30	−0.31	0.31	−0.14	−0.27	−0.27
Job demands domain	0.34	−0.20	−0.25	−0.26	0.18	−0.04	−0.14	−0.14
Job control domain	0.30	−0.18	−0.31	−0.29	0.20	−0.13	−0.21	−0.19
Job rewards domain	0.33	−0.19	−0.30	−0.29	0.29	−0.14	−0.29	−0.25
WPFQ total score	0.42	−0.24	−0.35	−0.36	0.31	−0.36	−0.45	−0.27

p=0.01 (one tailed).

^*(1) Stress Symptoms Questionnaire, (2) General health scale, (3) Vitality scale, (4) Mental health scale

The N-WPFQ resulted in significant correlations with stress symptoms (0.49, p=0.01), general health (−0.36, p=0.01), vitality (−0.45, p=0.01), and mental health (−0.47 p=0.01). All of these correlations were higher than those seen for the WPFQ.

Correlations between N-WPFQ dimensions and stress symptoms showed similar tendencies in the group of professionals, technicians and managers, compared with the group of auxiliary and blue-collar workers. The following dimensions showed stronger association with stress symptoms in both groups of workers: influence of non-work conditions on work (0.40 and 0.43, p=0.01, respectively), family economic situation (0.34, 0.41, p=0.01) and family relationships (0.33, 0.32, p=0.01). The lowest correlation with stress symptoms was seen with transportation between home and work dimension (0.20, 0.18, p=0.01). Main difference between the two workers groups was found in the correlation of stress with use of non-work time (0.37, 0.29, p=0.01 respectively).

National reference values

To test the convenience of the division by levels of risk defined in the reference values, we analyzed the strength of association between levels of psychosocial risk and high levels of stress symptoms by using prevalence rate ratios (PRR). For the above purposes all five levels of psychosocial risk factors were organized into three main risk groups: High (high, very high); medium; and low (insignificant, low).

The strength of association was significant for the WPFQ forms A and B and for their four domains when medium versus low psychosocial risk levels were compared with high stress symptom levels. The same results were obtained for high versus low psychosocial risk levels when compared with high stress symptom levels.

When comparing high versus low psychosocial risk level, job demands had the highest PRR in WPFQ Form A (PRR: 1.99, CI: 1.73–2.28), followed by job rewards (PRR: 1.87, CI: 1.63–2.14) and characteristics of social relations and leadership at work (PRR: 1.87, CI: 1.63–2.15). Likewise, for WPFQ Form B, the highest PRR was found for characteristics of social relations and leadership at work (PRR: 1.81, CI: 1.49–2.20), followed by job rewards (PRR: 1.55, CI: 1.28–1.87).

The same analysis was performed for the N-WPFQ. Results showed a significant strength of association between high levels of non-work psychosocial risk factors and high levels of stress symptoms when compared to low psychosocial risk level. The Influence of non-work conditions on the work dimension had the highest PRR (PRR: 3.38, CI: 2.62–4.36), followed by family economic situation (PRR: 3.31, CI: 2.56–4.28) and family relationships (PRR: 3.22, CI: 2.57–4.03).

Discrimination between groups:

When comparing the results obtained by different subgroups of workers using the two-sample Wilcoxon rank-sum (Mann-Whitney) test to evaluate the discriminatory power of the questionnaires, we found significant differences between workers in two occupational groups: (i) Supervisors, professionals and technicians and (ii) blue collar and auxiliary workers.

The WPFQ results showed significant differences in the following domains between the two groups of workers: Supervisors, professionals and technicians had higher levels of job demands (p <0.001) and lower levels of job rewards (p = 0.024) when compared to blue collar and auxiliary workers. By contrast, the results of the job control domain showed higher risk in the group of blue collar and auxiliary workers (p <0.001).

Finally, it was found that blue collar and auxiliary workers had higher risk scores for the total score of the N-WPFQ, when compared with supervisors, professionals and technicians (p <0.001).

DISCUSSION

The validation process of the three new questionnaires designed to evaluate work and non-work psychosocial risk factors, followed the recommended technical guidelines for scale development [DeVellis, 2003]. The psychometric test results showed that they have high levels of validity and internal consistency; similarly, factor analysis findings confirmed the measurement model proposed during the design phase of the instruments, thus preserving the theoretical principles on which their structure was based.

Regarding the understanding of work psychosocial factors, findings of the validation study allowed us to highlight the relevance and contribution of the different dimensions considered in the WPFQ Forms A and B, which are cited in the following section.

The role consistency dimension tried to identify conflicting demands on the task content, such as conflict between productivity and quality, performance requirements that involve ignoring safety rules, etc. [Karasek et al., 1998; Ivancevich et al., 1989], which were consolidated in one dimension of WPFQ Form A; however, this was not achieved in Form B, because the low factor load of its items led the authors to eliminate such dimension from this questionnaire. The previous finding, though consistent with other studies in which the relationship between role conflict and performance was stronger among managers compared to non-managers [Gilboa et al., 2008], sets up a need to find other valuation strategies for the auxiliary and blue collar group.

The underlying concept for introducing the responsibility demands dimension was that it could represent a psychosocial risk exposure [Cox et al., 2000]. The factorial analysis showed sufficient factor load of items in this dimension for Form A, but low for Form B, so that this dimension remained only in Form A, but with a moderate Cronbach’s alpha. The above results are consistent with other studies [Jackson et al., 1993] that have reported difficulties for measuring responsibility for production, and therefore suggest the need to supplement their evaluation with other tools such as job analysis, especially for auxiliary and blue collar workers.

Relationship with subordinates is a dimension included in WPFQ Form A, based on the authors’ experiences in psychosocial intervention processes and the findings of some studies reporting stress in managers, which derives from their relationship with subordinates [Deluga, 1991; Tepper et al., 1993]. Factor analysis showed high load for all nine items (0.80–0.90) of the dimension, suggesting that they consistently reflect the components of social support and assistance received by leaders from office staff. It is noteworthy that this dimension has not been explicitly registered in other psychosocial assessment instruments reviewed by the research team, thus highlighting their contribution to the understanding of psychosocial factors influencing people with leadership responsibilities.

Environmental and physical demands of work are an aspect considered by few psychosocial assessment tools [Guàrdia, 2008], but their impact on workers’ wellbeing needs to be considered [Evans et al., 1987]. The WPFQ forms A and B contains a dimension consisting of items with high factor loading (≥ 0.70), but the internal consistency of this dimension was higher in Form A than in B, which is an opportunity for improving the WPFQ Form B.

Dimensions evaluating social interactions with colleagues make up a domain consistent with findings from studies analyzing the effects of social support [Karasek et al., 1990; Johnson et al., 1996], and the results of internal consistency and factor analysis of WPFQ forms A and B confirm their suitability to account for such conditions.

Finally, as expected, the domain of recognition and compensation at work reflect an important source of organizational rewards, which corresponds to the core construct of the Effort-Reward Imbalance Theory by Siegrist [Siegrist, 1996; Siegrist et al., 2004]. In this regard, WPFQ Form A resulted in greater coefficients of internal consistency in the dimensions making up this domain as compared to Form B, although acceptable levels in the two dimensions making up the domain were reached in the latter (0.73 and 0.67).

The study of non-work psychosocial factors [Bianchi et al., 2005] has been gaining increased interest because of their interaction with people’s wellbeing and working conditions, and hence the contribution the N-WPFQ offers to understand the comprehensive nature of psychosocial factors, since it addresses factors other than those exclusively labor-related.

Factor analysis of the N-WPFQ resulted in seven clearly identified dimensions with strong factor loadings, therefore confirming the proper structure of the variables presented in the design phase; similarly, the high internal consistency shown by the questionnaire and its dimensions suggest that it was adequately developed.

Significant correlations between dimensions and their original domains also validated the soundness of the study model and the variables used in the measurement model. Moreover, correlations between dimensions in WPFQ belonging to different domains were lower than correlations between domains and their relevant dimensions, meaning as expected, that they evaluate different, but related theoretical constructs. Significantly positive correlations between dimensions and total score of the N-WPFQ, is a reflection of their relevance to assessing psychosocial conditions other than work.

Concurrent validity results showed significant, albeit moderated, correlations between work and non-work psychosocial risk-factor questionnaires and SF36v2 health scales. The highest correlations were found with vitality and mental health scales. Similar results were reported by Kristensen, Hannerz, Høgh and Borg [Kristensen et al., 2005], Moncada [Moncada et al., 2004], and Hanson, Schaufeli, Vrijkotte, Plomp and Godaert [Hanson et al., 2000], in validation studies of psychosocial factor questionnaires. On the other hand, correlation between work and non-work psychosocial risk factors and stress responses was higher than correlations with health scales, a result consistent with expectations under the questionnaires framework and the primary effects of psychosocial risk factors.

Concurrent validity results were also consistent with expected behavior. High job demands, lack of job control, scarce job rewards or poor social relations and leadership at work, were positively associated with stress symptoms and negatively with health. Non-work psychosocial risk factors equally showed the same tendency. Moreover, strength of association was greater for higher levels of work and non-work psychosocial risk factors and high levels of stress symptoms, a finding confirming the expectations of the conceptual framework on which designed and validated instruments were based.

While literature reports the use of other recognized questionnaires to access concurrent validity of new instruments, future studies well deserve to consider using the JCQ (Karasek) or the ERI (Siegrist), to estimate the WPFQ concurrent validity. However, since there are no similar developments in questionnaires to assess non-work psychosocial factors, the N-WPFQ validated in this study could be a contribution towards knowing the influence of psychosocial factors outside the workplace.

The analysis of differences in psychosocial factors between groups, suggests that the WPFQ forms A and B discriminate between two occupational groups of workers. Such differences behaved as expected in the light of the described theoretical models. Workers in positions of leadership and those in professional and technical categories tend to have more job control [Karasek et al., 1990], greater demands at work [Schreuder et al., 2008] and better rewards [Kalleberg et al., 1980], as compared to blue collar and auxiliary workers.

The lack of differences between the two groups concerning social relations and characteristics of leadership at work suggests that in addition to the specific aspects of work, personality traits and copying style [Kivimäki, 1996] may play a significant role in the interaction of these and other psychosocial factors. Continued research in integrated approaches to assessing psychosocial factors in and outside of work, is encouraged for a further understanding of the interactions between work, non-work and personal conditions.

The results of this study suggest that the sample size was representative of the working population affiliated to the national Workers’ Compensation System in Colombia, despite the fact that they represent 32% of the nation’s economically active population [DANE, 2010]. The sample used also reflects national distribution of target population by region and by economic activity, as well as their demographic characteristics.

To reduce selection bias, our sample was obtained from a large number of organizations nationwide, even larger than in other validation studies [Hanson et al., 2000; Holmstrom et al., 2008]. Sources of systematic errors caused by non-coverage bias were reduced by using two sample frames and by randomizing worker selection within each organization. Nevertheless, it should be noted that just 33% of the sample consisted of support staff and blue collar workers, which could imply limiting this group’s representativeness.

The expected organization response rate (75%) was lower (29%) than the rate of the First National Survey of Working Conditions in Colombia (79%). Two possible reasons might explain such low response rate: firstly, employers’ lack of knowledge on the psychosocial risks factors endangering workers’ health, and secondly, some organizations showed some type of reluctance to the study since it was state-funded, a fact construed by some employers as a way of monitoring compliance with their obligations regarding occupational health.

Contrary to the above results, the response rate of workers invited to participate in the study was 90%, higher than rates reported in other studies with large groups of workers [Kristensen et al., 2005; Wannstrom et al., 2009], suggesting an increased awareness of workers as regards their wellbeing, and the strength of the field work procedures followed for gathering the information.

The study’s greatest weakness was related to the reduced proportion of very small (micro) companies in the sample (11%), in relation to national distribution. This bias arose from practical considerations in the study, since nearly 80% of 459,224 workplaces recorded in the occupational risk official system database have less than 10 workers. Therefore, proportionally distributing the sample according to size of companies would have been unfeasible for the validation study. To handle such limitation, the sample was collected in companies having more than 25 workers, and 24 micro-firms were included, mainly from farming and agricultural sector. It should be noted that no sensitivity analysis was performed given the above limitation, which is an opportunity for improvement for future validation studies dealing with the same type of limitation.

Finally, since stress reactions can act as a precursor to injury or illness, early detection of factors associated with stress occurrence can contribute to preventing negative health effects on workers. Thus, WPFQ and N-WPFQ’s real value is their contribution as tools for disease prevention [Villalobos, 2004] and health promotion [Villalobos, 2007].

CONCLUSIONS AND IMPLICATIONS

According to the validation study results, the psychosocial-factor questionnaires developed and validated by the authors provide a validated methodology for assessing work and non-work psychosocial conditions that may be associated with illness and stress symptoms.

While the need to develop two forms of WPFQ is a fact, their reliability and validity results and factorial structure show that both forms (A and B) behave as similar and parallel questionnaires for two different groups of workers.

From a scientific point of view, all processes leading to the questionnaires’ validation study, such as developing the measurement model, designing the questionnaires, pretest and pilot study, and the active participation of workers for their construction and adaptation to the cultural reality of the country, made a significant contribution to achieving the results of this validation study. The questionnaires provide valid and reliable information to facilitate comparative studies with cohorts of workers and to increase awareness of psychosocial factors and related health conditions.

The social importance of the instruments lies in their applicability to identify psychosocial problems in the work setting, thus facilitating the definition of prevention and intervention policies in priority groups. The questionnaires also meet the need to evaluate workers exposures to risk factors, and they also provide useful information for further studies on causality of occupational illnesses.

Finally, given the fact that all three questionnaires are culturally adapted, they become a valuable contribution to the nation and a step forward for the Latin-American region.