Design of psychosocial factors questionnaires: a systematic measurement approach

Abstract

Background

Evaluation of psychosocial factors requires instruments that measure dynamic complexities. This study explains the design of a set of questionnaires to evaluate work and non-work psychosocial risk factors for stress-related illnesses.

Methods

The measurement model was based on a review of literature. Content validity was performed by experts and cognitive interviews. Pilot testing was carried out with a convenience sample of 132 workers. Cronbach’s alpha evaluated internal consistency and concurrent validity was estimated by Spearman correlation coefficients.

Results

Three questionnaires were constructed to evaluate exposure to work and non-work risk factors. Content validity improved the questionnaires coherence with the measurement model. Internal consistency was adequate (α=0.85–0.95). Concurrent validity resulted in moderate correlations of psychosocial factors with stress symptoms.

Conclusions

Questionnaires´ content reflected a wide spectrum of psychosocial factors sources. Cognitive interviews improved understanding of questions and dimensions. The structure of the measurement model was confirmed.

INTRODUCTION

Globalization and changes in socioeconomic conditions and organizational settings have prompted enormous transformations in job characteristics. More and more mental and emotional demands are placed on the worker to meet today’s client requirements. Workers are required to have different skills and continued learning, and increased competitiveness in the marketplace means higher job uncertainty, resulting in an economic situation in which everyone are vulnerable [Schnabel, 2010]. The effects of such demanding situations are diverse, specifically well-documented cases of injuries [Clarke et al., 2004], mental disorders [Stansfeld et al., 1999], cardiovascular diseases [Siegrist, 1996; Karasek et al., 1981], gastrointestinal [Drossman, 2004; Drossman et al., 1999] and musculoskeletal problems [Hurrell et al., 1996; Sauter et al., 1996], and negative job performance [Deluga, 1991], among others.

Surveys on health and work conditions have collected important information regarding risk exposure and have established technical recommendations [Houtman et al., 2007; Starvorula, 2008]. The European Risk Observatory (2007) conducted a survey to identify emerging occupational safety and health risks. The most important sources of psychosocial risks identified by the survey were precarious contracts in the context of unstable labor market; increased worker vulnerability in the context of globalization; new forms of employment contracts; feelings of job insecurity; aging workforce; long working hours; work intensification; lean production and outsourcing; high emotional demands at work and poor work-life balance [European Agency for Safety and Health at Work, 2007]. Colombia is not far from this reality. The First National Survey on Health and Working Conditions in Colombia [Ministerio de la Protección Social, 2007] carried out in 937 workplaces, identified a high prevalence of psychosocial risks factors, in particular, emotional demands from customer service (60.5%), and repetitive and monotonous tasks (49.5%). One fourth of respondents (26.3%) expressed having high levels of stress at work. Changes in working conditions, such as out-sourcing, downsizing and the consequently increase of workload, as well as different forms of violence in and outside the workplace, have increased work-related stress and negative health outcomes [International Labour Organization, 2010].

The scientific literature describes numerous assessment instruments based on different theoretical models [Peiró et al., 1994], but most are questionnaires aimed at evaluating stressors and stress symptoms, or specific aspects such as burnout, harassment, coping styles, and social support. Psychosocial questionnaires developed in Colombia have helped occupational health professionals and employers to understand psychosocial factors in the workplace, even though some of the questionnaires have not undergone national validation studies (⁽ⁱ⁾Psychosocial factors and ⁽ⁱⁱ⁾Stress Symptoms Questionnaires Ministerio de Trabajo and Universidad Javeriana -Villalobos G. 1996 [Villalobos, 1998]; ⁽ⁱⁱⁱ⁾Psychosocial factors Questionnaire Instituto de Seguros Sociales - Bocanument G, & Berján P [Bocanument et al., 1996]; and ^(iv)Questionnaires for the study of working violence, Ministerio de la Protección Social and Universidad de Antioquia [Ministerio de la Protección Social et al., 2004a].

Technical studies related to risk assessment have been useful for developing governmental policies. Some examples from the European Union are the directives that address working time (Directive 93/104/EC and Directive 2003/88/EC), discrimination (Directive 2000/43/EC), and equal treatment for men and women (Directive 2002/73/EC).

In Colombia, current legislation [Ministerio de Trabajo y Seguridad Social, 2008] has emphasized two salient aspects: evaluating job exposure to psychosocial factors to establish link between workers’ stress-related illnesses and intervention priorities in terms of the affected population, and the more harmful risks factors [Ministerio de Trabajo y Seguridad Social, 2009]. Additionally, employers have been assigned the responsibility of evaluating psychosocial factors and their effects, and to intervene to prevent negative health effects. Furthermore, technical orientation of employers has been delegated to worker compensation companies. Legal issues have increased employers’ and workers’ awareness on the effects of these psychosocial factors. Therefore, the need for useful information has become evident to identify and intervene in psychosocial harmful conditions, in order to prevent the associated effects on workers’ health, safety and production, as well as on workers’ compensation [Villalobos, 2007].

The need of having integrated psychosocial assessment tools validated through population studies, prompted the Colombian Ministry of Social Security to enlist the help of Javeriana University. The objective of this article is to present the methodological process used to construct the questionnaires and the results of the pilot test. Results of the national validation study are presented elsewhere. Other assessment instruments developed as part of the project are not within the scope of this article [Ministerio de la Protección Social et al., 2010].

METHODS

Literature review

A review of the literature began with a search of psychosocial factors specific to the workplace and outside it (work and non-work psychosocial factors). The search was restricted to books and published literature in the last 10 years and the search was conducted in both English and Spanish language databases and technical organizations. The definition of search terms was based on the search strategy used by Brewer et al. [Brewer et al., 2006], which included the three main categories of psychosocial factors, settings, and assessment; connected by "AND / OR / NOT Boolean operators."

A total of 82 documents were reviewed, which revealed nine explanatory models of stress and psychosocial factors, 19 questionnaires, 5 job observation instruments, and 3 mixed method studies that included self-report and observation. A total of 28 documents related to non-work psychosocial factors and 4 questionnaires were also reviewed.

The Copenhagen Psychosocial Questionnaire (COPSOQ) [Kristensen et al., 2005] was found to be one of the most complete, because it includes relevant elements (skill variety, autonomy and demands, social support, feedback, job future ambiguity and task identity) from seven major theories in occupational health psychology [Schbracq et al., 2003]: Job characteristics model (Hackman & Oldham's 1976), the Michigan organizational stress model (House, 1981), the demand–control and social support model (Johnson & Hall, 1988; Karasek & Theorell, 1990), the sociotechnical approach (Tavistock Institute), the action–theoretical approach (Freze & Zapf, 1994), the effort–reward–imbalance model (Siegrist J., 1996), and the Vitamin Model (Warr, 1987, 1994). The COPSOQ questionnaire and its Spanish version (ISTAS21) [Moncada et al., 2002] were valuable resources for the design of new Colombian questionnaires.

Two decisions were made as a result of this review: to develop new questionnaires culturally adapted and useful to obtain enough information required by the psychosocial factors national legislation in Colombia, and to develop a set of questionnaires and guidelines for psychosocial position analysis, semi-structured interviews, and focus groups [Ministerio de la Protección Social et al., 2010] as part of an integrated assessment methodology based on theoretical principles and practical requirements as a measurement triangulation strategy [Cox et al., 2000].

Measurement model

The structure of the measurement model considered the systemic character of the psychosocial factors; therefore work, non-work and individual conditions constituted the structural components of the system. These components maintain a dynamic relationship, so they are mutually influential. Some dimensions of these three components more clearly represent interfaces, such as the influence of work on family [Moncada et al., 2005], and family on work. Work and non-work components were defined as key measurement constructs. Individual conditions were excluded from the questionnaires’ objectives because of their complexity and to the existence of other instruments based on different personality theories, which have been validated among a Colombian population. However, due to their moderator roles, both individual demographic and occupational information were included as part of the questionnaires’ scope.

The work psychosocial factors questionnaires developed as part of the current project were based on the imbalance effort rewards model [Siegrist 1996] and the demand–control and social support model [Karasek et al., 1990; Johnson et al., 1996], both well-known and widely used. It was also considered the psychosocial factor structure used in the Copenhagen Psychosocial Questionnaire [Kristensen, 2002; Kristensen et al., 2005] and its Spanish version (ISTAS-21) [Moncada et al., 2002]. The content of the Non-work psychosocial questionnaire was based in the Ivancevich and Matteson stress theory [Ivancevich et al., 1989]. Additionally, the measurement model considered some psychosocial dimensions based on our own previous research and experience [Ministerio de la Protección Social et al., 2004b], as well as the country’s needs.

The Work Psychosocial Factors Questionnaire (WPFQ) required developing two forms for different occupational groups: Form A for supervisors, professionals, and technicians and form B to be used with auxiliary and blue-collar workers.

Both WPFQ forms were structured in domains with dimensions, where each domain is a scale (Table 1). The taxonomy of The Non-work Psychosocial Factors Questionnaire (N-WPFQ) was structured by dimensions related to family, economic, and housing conditions (Table 2). All domains, dimensions and risk factor indicators were operationally defined. Worker’s occupational and demographical information constituted a section in the questionnaires but they were not a risk measurement scale. All questionnaires used a summative frequency scale (five options from always to never), in which every response had a value, and values were summarized to obtain the total score for each dimension and domain, as well as for the entire questionnaire. Total scores of WPFQ and N-WPFQ can be summarized to obtain a general psychosocial factor result. Each item was used for scoring only one scale. In all cases, a higher score meant a higher psychosocial risk exposure. The four domains of WPFQ and the seven dimensions of N-WPFQ were expected to be related to stress symptoms and negative health status.

Table I.

Taxonomy of domains and dimensions underlying the WPFQ.

Domains	Dimensions
Characteristics of social relations and leadership at work	Leadership characteristics	WPFQ
	Social relationships at work
	Performance feedback
	Relationship with subordinates
Job control	Role clarity
	Training
	Participation and change management
	Opportunities for knowledge and skills use and development
	Control and work autonomy
Job demands	Role consistency
	Environmental and physical effort demands
	Emotional demands
	Quantitative demands
	Work influence on the non-work setting
	Position responsibility demands
	Mental load demands
	Working-time demands
Job rewards	Rewards from the work and the organization membership
	Recognition and compensation

Table II.

Taxonomy dimensions underlying the N-WPFQ.

Dimensions
Use of non-work time	N-WPFQ
Family relationships
Communication and social relations
Family economic situation
Housing characteristics
Influence of non-work conditions on work
Transportation between home and work

The general data section presented at the end of the questionnaires contains respondents’ demographic and occupational information. Questions are open-ended (e.g., year of birth, job position, city) and closed-ended (e.g., gender, type of job, highest educational level).

Questionnaires’ design

As part of the questionnaires' design, it became necessary to define what to measure, whether individual feelings related to the psychosocial factors or the characteristics of the psychosocial stimulus and demands themselves. We decided to: (a) develop questions targeted on the workers’ evaluation of psychosocial exposure frequency, and (b) construct parallel instruments that included guidelines for psychosocial position analysis, semi-structured interviews, and focus groups to identify the same exposures but considering also frequency, severity and duration of the exposure. Consequently, the questionnaires should provide information related to the individuals' experiences in relation to work and non-work conditions, minimizing whenever possible, the respondent's emotional processing.

Inspired by the content of the Copenhagen Psychosocial Questionnaire, we decided to estimate the concurrent validity of the new questionnaires by their correlation with two scales from the SF36v2 Health Survey (Mental Health and Vitality) [Ware et al., 1998; Ware et al., 2008] as well as with three scales from the Stress Symptoms Questionnaire validated in a sample of 2199 Colombian workers (Cronbach’s coefficient alpha 0.889. Villalobos, 2005, unpublished doctoral dissertation).

Based on the theoretical framework, on the defined measurement model, other questionnaires and the authors’ experience, a pool of 209 items related to work psychosocial factors and 50 to non-work were developed. Items reflected the variables and the psychosocial risk indicators defined in the measurement model. Additionally, questions selected mostly reflected observable facts, as a strategy to reduce the effect of excessive subjectivity, and to prevent threatening feelings when respondents were asked about sensitive topics.

Pretesting

Validation of the two questionnaires’ content was carried out by seven experts who analyzed and rated items using a qualitative-quantitative scale for clarity, relevance, relation to the dimension and sufficiency. Subject matter experts were academics with different professional backgrounds (Medical Doctors, Occupational Psychologists or Psychiatrists) each with more than 10 years of experience in their psychosocial applied research field.

Item quality was tested through cognitive interviews with 23 workers of different educational level and economic status. Workers helped us to clarify and to precise the items' meanings, and provided us with feedback to include or exclude other ones according to their relevance in everyday life. In cognitive interviews, all items were revised for comprehension (meaning and question objective), information retrieval (type of information and recall strategy), decision process (sensitivity and social desirability) and adequacy of response options. The research group discussed information gathered from sets of cognitive interviews, and items were improved and tested in subsequent rounds.

Pilot study

Participants

For the pilot study, a convenience sample of 132 employees from different economic activities, job positions and education levels was selected, to test instructions, questions and features. Questionnaires were generally self-administered, with the exception of workers with limited literacy skills, who were interviewed. Applications were tested on individuals and on groups. Questionnaire user manuals were also tested with occupational health professionals and employers. All workers’ questions and doubts as well as response time were recorded in a fieldwork log.

Pilot study participants were selected from 12 workplaces in the country’s capital city and in the nearby rural area. Potential participants had been contacted by letter. Then, each person volunteering to participate in the pilot study received a letter with the study information, and signed the written informed consent form after the purpose of the study was explained to them and they had an opportunity to ask any questions. The questionnaires were handed in sealed envelopes, marked with a code and separated from the informed consent forms. Completed questionnaires (response rate of 97%) were handed directly to the data collector, who checked each questionnaire for completeness (completeness rate 51.5%). Blank or missing items (no more than three in the uncompleted questionnaires) were reviewed and improved by the research group after the pilot test and before the validation study, by using additional cognitive interviews with the respondents.

Data collectors received specific training and were certified through a Human Subjects Training Module [Health National Institute] before being allowed to gather data. Information was processed as a group, and the research ethics committee at Javeriana University School of Medicine, approved the study.

Data analysis

Reliability of the questionnaires was evaluated by means of Cronbach’s alpha coefficients. Additionally, correlations between WPFQ and N-WPFQ were calculated to test if questionnaires’ reflected an integrated psychosocial measurement. Concurrent validity for the pilot sample was calculated by using Spearman´s correlation coefficient, between the psychosocial questionnaires and the mental health and vitality scales (Cronbach’s alpha coefficients 0.91 and 0.83 respectively) [Lugo et al., 2006] of the SF36 Health Survey [Ware et al., 1998; Ware et al., 2008], and between the psychosocial questionnaires and the physiological, social and cognitive symptoms from the Stress Symptoms Questionnaire (Cronbach’s alpha coefficient 0.73, 0.61 and 0.75 respectively. Villalobos, 2005, unpublished doctoral dissertation).

PILOT STUDY RESULTS

Three questionnaires were tested in the pilot study: WPFQ form A with 142 items, form B with 118 items and N-WPFQ with 34 items, and 18 demographical and occupational items.

The average answer time was longer for WPFQ form B (40 min) than for form A (25 min); the N-WPFQ average was shorter (6 min).

The pilot sample was made up by 132 workers, 103 of them answered WPFQ form A, and 29 form B. All subjects answered the N-WPF and the demographic and occupational questions.

Services was the economic sector with the largest representation in the pilot study sample (47%), followed by trade (18%) farming & agricultural (18%) and industry (17%).

Professionals, technicians and managers made up 58.3% of the total sample, followed by auxiliary and blue-collar workers (41.7%). The average age was 35 years (SD=9). The mean value for total years of work was 6 (SD=5.48), which was near to the seniority at the current job (mean=5.22, SD=5.14). The majority of the workers (86.6%) reported between 8 and 10 work hours daily (mean 8.3 and SD=1.54) (Table 3).

Table III.

Sample demographic and occupational characteristics

Demographic and occupational characteristics		Total sample
		n	%
Gender	Male	56	42.4
	Female	76	57.6
Age (years)	18–25	15	11.4
	26–35	59	44.7
	36–45	37	28.0
	46–56	16	12.1
	56 or older	2	1.5
	No information	3	2.3
Marital Status	Single	54	40.9
	Married or cohabiting	71	53.8
	Divorced	5	3.8
	Widow/widower	1	0.8
	No information	1	0.8
Level of education	Primary	8	6.1
	High school	38	28.8
	Technical	24	18.2
	University or higher	62	47.0
Number of economic dependants	None	38	28.8
	1 to 3	80	60.6
	4 or more	13	9.9
	No information	1	0.8
Type of job	Manager/supervision	52	39.4
	Professional/technician	25	18.9
	Clerk/assistant	25	18.9
	Blue collar	30	22.7
Type of contract	Temporary	37	28.0
	Permanent	82	62.1
	No information	12	9.1
	Self -employed	1	0.8
Number of daily working hours	< 8	9	6.8
	8–10	106	80.3
	11–15	10	7.6
	>16	0	0.0
	No information	7	5.3

Reliability

Reliability of the WPFQ and its domains was satisfactory and similar for both forms A and B (Table 4). Despite having the lowest Cronbach’s alpha among the three questionnaires, N-WPFQ’s reliability was adequate (0.85, n=132).

Table IV.

Cronbach’s alpha coefficients: WPFQ forms A and B

Questionnaire and domains	Cronbach’s alpha form A (n=103)	Number of items, form A	Cronbach’s alpha form B (n=29)	Number of items, form B
WPFQ	0.95	130	0.94	114
Demands	0.87	49	0.88	40
Job control	0.88	30	0.82	24
Leadership and social relationships at work	0.95	39	0.94	38
Job rewards	0.69	12	0.71	12

Correlations between WPFQ and N-WPFQ were also calculated to test whether the questionnaires reflected integrated psychosocial measurements. Significant correlations were obtained between WPFQ form A and N-WPFQ (0.50 p=0.01) and between WPFQ form B and N-WPFQ (0.60 p=0.01).

Content validity

As a result of the content validation findings, experts suggested adjustments, inclusion or elimination of some items of the following dimensions according to the measurement model: working time, cognitive, responsibility, emotional and environmental demands; thus, the WPFQ questionnaire was reduced from 209 to 154 items. Some questions related to housing conditions and family relationships were eliminated from or added to the N-WPFQ, which was reduced from 50 to 42 items.

After the cognitive interviews, some items were eliminated, reworded or reordered, and some others were added. All modified items were progressively tested through cognitive interviews, until they were sufficiently tested with workers of different educational levels. As a result of this process we decided that two forms of WPFQ (A and B) were necessary, because of differences in comprehension and relevance of dimensions between two main groups of workers: professionals, technicians and managers with 130 items, and auxiliary and blue-collar workers with 114 items. N-WPFQ items underwent the same kind of adjustments, and we determined that only one form (29 items) was needed.

As a result of the pilot study, items from WPFQ forms A and B were improved, leaving WPFQ form A with 142 items, WPFQ form B with 118 and N-WPFQ with 34. The user’s manual was also improved, and additional guidelines for questionnaire administration were included. Thereafter, questionnaires were prepared for the validation study.

Concurrent validity

Correlations between WPFQ domains and stress symptoms (physiological, social and cognitive), showed overall moderate but higher associations compared with vitality and mental health. Correlations with form A were stronger than those for form B (Table 5).

Table V.

Spearman correlations coefficients WPFQ and N-WPFQ with stress symptoms and health effects

Psychosocial factors	WPFQ	Job demands	Job control	Characteristics of social relations and leadership at work	Job rewards	N-WPFQ
Form A (n=103)
Physiological stress symptoms	0.34*	0.14	0.38*	0.32*	0.41*	0.45*
Social stress symptoms	0.47*	0.37*	0.38*	0.42*	0.38*	0.44*
Cognitive stress symptoms	0.54*	0.40*	0.50*	0.42*	0.53*	0.42*
Vitality	0.11	−0.09	−0.20**	0.14	0.15	0.11
Mental health	−0.05	0.04	−0.05	−0.01	−0.19**	−0.80
Form B (n=29)
Physiological stress symptoms	0.46**	0.33	0.31	0.34	0.52*	0.63*
Social stress symptoms	0.44**	0.36	0.24	0.30	0.37**	0.42**
Cognitive stress symptoms	0.53*	0.40**	0.38**	0.34	0.51*	0.53*
Vitality	−0.08	−0.21	−0.07	−0.11	−0.00	−0.02
Mental health	−0.18	−0.23	−0.14	−0.24	−0.13	−0.04

^*Correlation is significant at the 0.01 level (2-tailed).

^**Correlation is significant at the 0.05 level (2-tailed).

DISCUSSION

We designed three questionnaires to evaluate work and non-work psychosocial risk-factors for stress-related illnesses, by following a systematic process based on scientific literature, specific country requirements and workers participation. Pilot testing allowed us to verify the questionnaires’ appropriateness in terms of reliability and validity, and to get feedback related to practical issues for the validation process. The design of the two questionnaires and the pilot sample focused on workers enrolled in the national workers compensation system.

When evaluating psychosocial factors, a question related to usefulness arises. The answer partially depends on how well the factors reflect stressful situations in work and non-work settings and whether the measurement captures the main conditions under which these situations constitute a risk exposure, relevant enough to produce negative outcomes. The measurement objective, then, could be the risk factors (hazards) identification or even the risk assessment [Rick et al., 2001].

The WPFQ and N-WPFQ were designed using the criteria expressed above. Our scientific literature review, and also experts and workers, provided us with information that helped us to select relevant psychosocial conditions and to define situations indicating that they could become risk factors. Questions posed in the questionnaires represent these indicators so as to avoid, as far as possible, respondents’ negative emotions and patterns of erroneous answers. The individual’s decision process implies that he/she has considered whether the situation described in the item (indicator) occurs or not in his/her work or non-work setting. Then he/she would select the frequency option of the situation (from always to never). Frequency choices are a sensitive response format for the items; therefore, they contribute to measuring incidence of risk factors.

Measurement model structure

WPFQ forms A and B were structured in four domains: characteristics of social relations and leadership at work, job control, job demands, and job rewards. The first three correspond to the main postulates of the Demand, Control and Social Support model (Karasek et al.). The fourth one corresponds to the rewards component of the Effort-Reward-Imbalance theory (Siegrist), even though some other elements of that component are reflected in domains of job control and job demands of the WPFQ.

Domains and dimensions

The domain of social relations and leadership at work intended to reflect three facets of worker’s relationships in the work place: with the leader, with others, and with subordinates. This domain was expressed by four dimensions (Table 1): leadership characteristics, social relationships at work, and performance feedback, which had been previously considered in other questionnaires [Kristensen et al., 2005], and relationship with subordinates, not included in form B.

As a dimension, relationships with subordinates derive its importance from supervisors’ complaints commonly expressed during stress management programs. In WPFQ, questions related to this dimension refer to situations derived from difficult interactions with subordinates. This dimension is new in psychosocial questionnaires and therefore it merits being included as a research topic for studies of supervisors. The underlying hypothesis of the characteristics of the social relations and leadership domain, suggests that lack of social support and conflicting relationships among coworkers, supervisors and subordinates are associated with negative outcomes (stress and poorer health). This and similar hypotheses have been confirmed in numerous studies [Karasek et al., 1998] when using Karasek’s model.

The job control domain comprised five dimensions (Table 1), two of which (opportunities for knowledge and skills use and development, and control and work autonomy) reflect the skill discretion and decision authority elements from Karasek’s model [Theorell, 2003]. Three other dimensions (role clarity, training, and participation and change management) were included to broaden the spectrum of possibilities for workers’ job control. The importance of these dimensions has been previously reported [Häfner et al., 2010; Peiró, 1999; Decker et al., 1993].

Role clarity defines the communication that worker expects to receive from his/her role in the organization in terms of work objectives, results and autonomy level. The WPFQ included role clarity as a control dimension; role ambiguity is a psychological demand on Karasek’s scale. Role clarity was conceived as a source of control, and the questionnaire intends to identify whether the organization provides enough information, so that worker has greater clarity about his/her role.

Training refers to all coaching and learning activities the organization provides to individuals, to increase their work knowledge and abilities.

Participation and change management asks whether or not the organization provides the necessary mechanisms to increase worker’s adaptability to changes in the work context, and if the work changes negatively affect performance. The organizational mechanisms mentioned refer to clear, adequate and timely information and opportunities for worker participation.

The domain of job demands comprises eight dimensions (Table 1) mainly related to job content and work environment; one of them is focused on the interface between work and non-work settings. Some considerations were born in mind when structuring the measurement model of job demands.

The role consistency dimension mirrors the concept of role conflict expressed in the COPSOQ by Kristensen. This dimension refers to the compatibility between performance requirements and work conditions that worker faces daily. The design process for this dimension was difficult due to social desirability bias. Several items were tested with blue-collar workers, and their responses indicated threatened feelings and therefore this dimension was removed from WPFQ form B.

The emotional demands considered three main risk-exposure indicators: working with negative feelings or emotions or with harmful treatment from others; the requirement to face emotionally devastating situations; and the need to hide emotions at work. The last indicator had already been identified by Kristensen in the Copenhagen Psychosocial Questionnaire.

Quantitative demands, as suggested by Kristensen [Kristensen et al., 2004], consider the mismatch between task and time in terms of fast work pace and work overtime. However, this dimension also includes the reduction in the number and length of breaks during the workday due to the amount of work.

The dimension of work influence over the non-work setting, refers to job requirements (time and effort) negatively affecting the worker's personal and family life. This dimension can be justly criticized because its outcomes may mean both work demands and individual’s high need for control, or even for over commitment at work. If this work influence is considered as a work demand, then it could be positively correlated with high quantitative demands. However, this situation reveals a weakness of the questionnaire, which poses a challenge for future research to clarify the complexity of the individual and work interface, previously described by Siegrist and Peter [Siegrist et al., 1994], as a combined effect of workload and effort due to the high need for control coping style.

The dimension of responsibility demands related to the position required an extensive testing of items, in an attempt to avoid social desirability bias. As a result of the questionnaires’ validation study (presented elsewhere), this dimension was retained only in WPFQ form A, and its intent was to determine if the employee should bear direct responsibility for results and for maintaining control, given the impact of these responsibilities and their various determinants.

The content of mental load demands focuses on some cognitive task processing requirements [González et al. 2005; Nogareda, 2009] (attention, memory and analysis). This dimension underwent several changes during the design due to the difficulty for specifying overload as a different condition from the desirable mental load.

Finally, the dimension of working time demands is intended to find exposures to night shifts, work during days off, and work without breaks, which have been referred as being harmful conditions [Boivin et al., 2007]. Likewise, physiological effects with psychosocial impact have been described in other studies [Evans et al., 1987], and physical demands constitute a scale in the Job Content Questionnaire (Karasek).

The job reward domain contained in the WPFQ, adopted main postulates from Siegrist’s Effort-Reward-Imbalance model (status control, esteem and monetary recognition); however, both workers’ participation in cognitive interviews and the questionnaires’ field tests, contributed to a better fit of this domain’s dimensions. One of the job-reward dimensions addresses promotion, progress, and compensation that the organization gives to worker. Another dimension reflects job stability and the sense of pride that worker feels for his/her organization and for the work he/she performs. The feeling of pride explores the worker’s motivation and therefore is not limited to extrinsic work conditions. In this sense, questionnaire items describe a component of the person-task interface; thus, conclusions drawn from this dimension must be interpreted as clearly determined by the interface components.

Non-work psychosocial factors refer to all family, social and economic aspects of the worker’s life influencing his/her health and wellness. Stress is experienced when domestic demands and conflicts affect the work-family interface [Sauter et al., 1988]. Issues such as social support, family and housing demands, and the home-work interface, have been included in several studies [ten Brummelhuis et al., 2008; Ertel et al., 2008]. Despite the fact that the relationship between work and non-work factors has been study subject, little progress has been made towards a measurement methodology systematically approaching both constructs.

Both WPFQ and N-WPFQ have a strength related to their complementary characters when assessing work and non-work psychosocial constructs under the same measurement model and metrics. The questionnaires’ characteristics allow researchers, employees and employers to see the two main sources of risk exposure, and the questionnaires provide elements to intervene in the same systematic approach.

Pilot study

The overall format and items of the pilot questionnaires were well received by workers, employers and occupational health professionals from the volunteer convenience sample. Considering the response time, the questionnaires length was not a source of error. Most suggestions were related to instruction clarity, e.g. to changing job station by work place, and reordering items.

The reliability testing of the questionnaires revealed adequate to strong internal consistency, and the correlations between WPFQ and N-WPFQ suggest the existence of an integrated construct between the two questionnaires, which is consistent with the proposed measurement model.

The selection of two additional questionnaires to test concurrent validity was made with the aim of looking for early health and stress effects due to stress-related illness latency. Correlations between WPFQ and two selected SF-36v2 scales (mental health and vitality) were low and not significant, albeit in the right direction (higher risk psychosocial factors, less vitality and mental health). The reduced sample size, particularly in form B could explain these results. The vitality scale only showed a significant correlation with the job control domain, while the mental health scale correlated with the job rewards domain in form A. In spite of having a larger sample (n=859), similar low correlations have been reported in the validation study of COPSOQ in Spain [Moncada et al., 2002].

Stress symptoms (physiological, social and cognitive), however, showed a pattern of significant and higher correlations with WPFQ form A and with N-WPFQ, with the exception of some dimensions in form B, most probably a result of the reduced sample size (29 workers).

Concurrent validity findings resulted as expected according to stress theories. These results suggest that the measurement model and the questionnaire design were appropriate in relation to the initial objectives.

The use of a small convenience sample is a limitation of the pilot study; however, the results of the pilot were extremely useful to the design of the later validation of the questionnaires.

Since the length of the questionnaires was not burdensome for workers in the pilot study, we decided to include the SF36v2 General Health scale, and the entire Stress Symptoms Questionnaire (Cronbach’s alpha coefficient 0.89. Villalobos, 2005, unpublished doctoral dissertation) for the validation study of WPFQ and N-WPFQ.

CONCLUSIONS AND IMPLICATIONS

Instrument design followed a progressive process of defining each dimension in light of risk exposure indicators and based on literature review, workers participation and the authors’ previous experiences.

Expert suggestions and cognitive interviews with workers provided important feedback on item relevance and clarity, and this allowed us to make some adjustments which were tested in subsequent rounds of interviews. These findings provided some clues related to the questionnaires' cultural validity. Furthermore, quantitative data obtained through the pilot study suggested that the measurement model performed in the direction expected, and allowed us to base the national validation study on instruments previously tested in a smaller sample of workers.

Item design was mainly focused on identifying risk exposure through workers’ answers instead of workers’ feelings. This approach provides useful information for intervention planning and for change monitoring required by psychosocial and health surveillance programs.

The main challenge to improving the questionnaires is related to individual, work and non-work interfaces and to finding alternatives to assess their complexities and determinants.