Work design and management in the manufacturing sector: development and validation of the Work Organisation Assessment Questionnaire

Abstract

Objectives

To examine the factor structure, reliability, and validity of a new context‐specific questionnaire for the assessment of work and organisational factors. The Work Organisation Assessment Questionnaire (WOAQ) was developed as part of a risk assessment and risk reduction methodology for hazards inherent in the design and management of work in the manufacturing sector.

Method

Two studies were conducted. Data were collected from 524 white‐ and blue‐collar employees from a range of manufacturing companies. Exploratory factor analysis was carried out on 28 items that described the most commonly reported failures of work design and management in companies in the manufacturing sector. Concurrent validity data were also collected. A reliability study was conducted with a further 156 employees.

Results

Principal component analysis, with varimax rotation, revealed a strong 28‐item, five factor structure. The factors were named: quality of relationships with management, reward and recognition, workload, quality of relationships with colleagues, and quality of physical environment. Analyses also revealed a more general summative factor. Results indicated that the questionnaire has good internal consistency and test‐retest reliability and validity. Being associated with poor employee health and changes in health related behaviour, the WOAQ factors are possible hazards. It is argued that the strength of those associations offers some estimation of risk. Feedback from the organisations involved indicated that the WOAQ was easy to use and meaningful for them as part of their risk assessment procedures.

Conclusions

The studies reported here describe a model of the hazards to employee health and health related behaviour inherent in the design and management of work in the manufacturing sector. It offers an instrument for their assessment. The scales derived which form the WOAQ were shown to be reliable, valid, and meaningful to the user population.

Failures of work design and management are known to challenge both employee health and organisational behaviour and the healthiness and performance of the organisation.¹,² Such failures are often referred to as “work and organisational factors” in the occupational health literature and are the focus of much attention in relation to the assessment and reduction of work related risk at both the individual and organisational levels. This paper describes the development of a questionnaire based instrument for assessing such factors as risks to employee health and health related behaviour. This instrument is referred to as the Work Organisation Assessment Questionnaire (WOAQ).

The European Commission, and the governments of most of the Member States of the European Union, have enacted programmes to ensure that enterprises address the challenge posed by failures of work design and management and that they have the knowledge and tools to do so.³ The approach adopted by the European Commission, and in Great Britain by the Health and Safety Executive (HSE), is based on an adaptation of the traditional risk management paradigm for health and safety. In this context, the HSE has recently published its Stress Management Standards (SMS) approach.⁴,⁵,⁶

The SMS approach is based on the development of a standard questionnaire tool for the assessment of problems at work by employees in terms of perceived failures of work design and management. It can be argued that the tool is most useful as a means of identifying “hot spots” in organisations and informing subsequent discussions between managers and other employees over the exact nature of those problems and reasonable ways of dealing with them.

There are three challenges to the SMS approach: (1) the question of context specificity in the assessment of work and organisational factors, (2) the need to establish which factors are possible hazards in any particular context by exploring their association with health related outcomes, and (3) the use of sickness absence as the main or only source of outcome data has limitations and may lead to an underestimation of the real cost of psychosocial hazards at work.⁷

Standard instruments for use across all enterprises in all sectors, by necessity, have tended to address common issues and ignore those specific to any particular work group or workplace, enterprise, or sector. The “common issues” approach may not be capable of providing sufficiently detailed information to inform subsequent risk reduction for particular work groups or workplaces. Such instruments have often been used as vehicles for comparative research outside the risk management paradigm. There is a need for a balance between the requirements of context specificity for risk assessment and the resources required to complete that assessment exercise at the whole or cross‐enterprise level. Resolving the issue of context specificity is a difficult balance between the need for an instrument that is not expensive to use in terms of time or other resources (standardised approach) and one that is sufficiently informative to support risk reduction (tailored approach). Here a compromise is offered, tailored to context at the sector level: the manufacturing sector. An approach tailored at the organisational level is available elsewhere.¹,²,⁸,⁹,¹⁰

In addition to the SMS questionnaire, there are several other standardised instruments available for identifying problems in the design and management of work. Many of these, including the SMS questionnaire, used on their own, are not capable of establishing the “hazardous” nature of the problem identified because they do not establish any level of association between report of the problem (exposure) and health status (harm).

The methodology developed is based on the recognition and harvesting of employees' expert knowledge of their work and health, and their evaluations of the design and management of that work. It helps ensure employee and trade union acceptance.

Method

Sample and procedure

The development and testing of the WOAQ consisted of: (1) an initial questionnaire design phase, including a pilot study, (2) the main study which allowed configuration of the WOAQ, exploration of the underlying model of work and organisational factors and collection of validity data, and (3) a test‐retest reliability study.

Five enterprises in the manufacturing sector took part in this research. Their participation was facilitated by the EEF–The Manufacturers' Organisation. Data were collected from 11 different work sites and 710 white‐ and blue‐collar employees. Thirty employees took part in the pilot study, 524 in the main study, and 156 in the reliability study. The samples contributing to the latter two are described in table 1.

Table 1 Study participants.

	Main study (n = 524)	Test‐retest study (n = 156)
Age (years)	Mean 41.5 years (SD 11.3), range 17–64 years	Mean 39.3 years (SD 10.9)
Years of tenure	Mean 13.3 years (SD 10.8), range <1–42 years	Mean 10.2 years (SD 8.7)
Gender (males)	82.0%	75.0%
Married/cohabiting	99.0%	92.4%
Job categories:
Managerial/supervisory	14.0%	18.0%
Manual	58.4%	49.3%
Administrative/technical/customer related	27.4%	33.1%
Working full time	85%	92.1%
Response rate	28.4%–77.0%	15.3%–87.1%

Design phase

The WOAQ was developed as part of a practical instrument for the identification of hazards and the assessment of risk in relation to work and organisational factors, employee health, satisfaction, and health related behaviour. A list of potential hazards inherent in the design and management of work was developed from previous work at Nottingham¹,²,⁹,¹⁰ and reviews of the wider literature. These items were discussed with the EEF to test their current relevance to the manufacturing industry. Their nature and wording were modified to the needs and context of that sector.

The instrument was piloted with a small representative sample of 15 manual and 15 non‐manual employees randomly selected from employees at a large manufacturing organisation. Their demographic characteristics were similar to those of the main study sample (table 1). Each participant completed the pilot WOAQ and was then interviewed. The interviews were semistructured and explored the nature and wording of the items presenting the potential work and organisational hazards. They also considered the possibility of errors of omission. The WOAQ was modified on the basis of these data.

The WOAQ was combined with other questionnaires and scales (see below) to form a composite risk assessment instrument and was then employed in the main study.

Main study

The project was introduced to five small and medium sized enterprises through a series of meetings and discussions with employees. The risk assessment instrument was distributed to employees in each of these enterprises, based in 11 worksites by one of the researchers alongside a project champion.

The data collected from this main study allowed the exploration of the model underpinning the items on potential work and organisational hazards, the development of the associated measurement scales, and the initial testing of their (concurrent) validity.

Reliability study

The reliability study was conducted separately with a second group of participants from four of the 11 worksites. Based on the test‐retest paradigm, it examined the effects of three time intervals between test and retest from Time 1 to:

(i) Time 2: same day (n = 78)

(ii) Time 3: two weeks (n = 32)

(iii) Time 4: two months (n = 46)

The questionnaires were distributed and collected in the same way as for the main study.

Measures

A composite risk assessment instrument was used in the main study. It had three components: (1) introduction and demographic information, (2) the WOAQ, and (3) the General Well‐Being Questionnaire, subjective health and job satisfaction.

Introduction and demographic information

The first section of the questionnaire introduced the study and assured the participants that the data collected were voluntary, anonymous, and confidential. Demographic data were restricted to age, gender, job, and company details.

The WOAQ

The development of the WOAQ was described above. The final form for inclusion in the main study consisted of 28 items describing potential hazards inherent in the design and management of work in the manufacturing sector, each associated with a five‐point rating scale. Using their knowledge and experience, participants were asked to evaluate each aspect of their work in terms of how problematic (or good) it had been over the last six months using a five‐point Likert scale (5 = very good, 1 = major problem). The WOAQ uses situational rather than psychological reasoning: it asks “how good or poor do you and your colleagues think this aspect of work design (or management) is?” rather than “to what extent are you upset or distressed by this aspect of work design (or management)?” Because of the wording of the items, it also allows for some identification of the positive features of work design and management.

Employee wellbeing, health, and job satisfaction

Employee wellbeing was assessed by self‐report by use of the General Well‐Being Questionnaire (GWBQ).¹¹

The GWBQ was developed as a symptom based questionnaire to assess suboptimal health.¹² It consists of general, non‐specific symptoms of general malaise including reportable aspects of cognitive, emotional, behavioural, and physiological function, which are not clinically significant in themselves. Factor analytical studies have confirmed its two‐factor structure: (1) exhaustion and (2) tension, each defined by 12 symptoms. Feelings of exhaustion are defined by symptoms relating to fatigue, emotional lability, and cognitive confusion. Feelings of tension are defined by symptoms relating to worry, fear, and physical signs of anxiety.

Respondents were asked to indicate how often they had experienced the symptoms presented (within the last six months) on a scale from “never” (0) to “all the time” (4). Scores range from 0 to 48, a high score indicating lower wellbeing. A cut‐off of 24 is taken as indicative of an unacceptable level of exhaustion or tension. The scales have high internal reliability.¹² Cronbach's α for exhaustion was 0.9 (norms at 16.7 (SD 8.3), n = 2300) and for tension also 0.9 (norms at 10.7 (SD 7.4), n = 2313), controlling for age and gender.

Subjective health was measured by a single item,¹³ with responses from 1 (major problem) to 5 (very good). Participants' job satisfaction was also measured with one item¹⁴ (5 = very good to 1 = major problem).

Scales

The directionality of the scales used in the risk assessment instrument was varied to reduce the likelihood of response perseveration. The use of single item measures is defensible¹³,¹⁴,¹⁵ while that of direct questioning reduces problems of false accuracy (in the current context).

Results

The data of the main reliability studies are presented in three sections: (1) factor analytic modelling of the WOAQ items, (2) reliability, and (3) validity.

Structure of the WOAQ

Principal component analysis was performed on the 28 items that composed the WOAQ. Data were examined in terms of the unrotated solution and both orthogonal (varimax) and oblique rotations. The most meaningful model (statistically and theoretically) was orthogonal.

A series of pre‐analysis checks were completed before principal component analyses. Outliers were deleted where inspection of the questionnaires suggested improper completion. The inter‐item correlation matrix was examined to test whether r coefficients were appropriate for factor analyses, as determined by the Kaiser‐Meyer‐Olkin test of sampling adequacy (KMO = 0.9, >0.6) and the Bartlett test of sphericity (χ² = 5386.0, df = 378, p⩽0.01). Both were acceptable.

Examination of the unrotated factor structure revealed that a significant percentage of the variance (40.0%) was explained by a single summative factor which included 26 of the 28 items (not including those that referred to relationships with colleagues). All 26 items loaded positively and strongly on that factor. It was decided to retain this overall summative factor in subsequent analyses.

Varimax rotation was applied. Preliminary analyses yielded eight factors. After deletion of items with low communalities, examination of the scree plot, and expert judgement on the conceptual distinctions among the items, it was decided to retain five factors which represented the model of choice, explained 62.4% of the data variance, and incorporated all 28 items.

The variance explained by the individual factors ranged form 6.5% to 16.9%. All items had communalities above 0.5. A factor loading cut‐off of 0.5 was imposed on the model to aid interpretation. Although best practice suggests that factors with fewer than six items should not be accepted,¹⁶ in this study it made conceptual sense to accept the two factors with few items. They were judged strong enough to warrant inclusion in the model in terms of (1) the factor loadings of their items and (2) strong Cronbach's α.

The five factors that defined the model of hazards inherent in the design and management of work in the manufacturing sector were: quality of relationships with management (nine items), reward and recognition (seven items), workload (four items), quality of relationships with colleagues (two items), and quality of physical environment (six items). Table 2 presents the loadings of the 28 items on the factors and their communalities (h²).

Table 2 Factors, factor loadings, and communalities.

	Factor loadings	Summative factor loadings	h2
1. Quality of relationships with management
1 Communication with line manager/supervisor	0.8	0.7	0.8
2 Support from line manager/supervisor	0.7	0.7	0.6
3 Clear reporting line(s)	0.7	0.7	0.6
4 Senior management attitudes	0.7	0.7	0.7
5 Appreciation of efforts from line managers/supervisors	0.7	0.7	0.7
6 Clear roles and responsibilities	0.6	0.7	0.6
7 Clear company objectives, values, and procedures	0.6	0.7	0.6
8 Status/recognition in the company	0.6	0.7	0.7
9 Feedback on your performance	0.5	0.7	0.6
2. Reward and recognition
10 Opportunities to use your skills	0.8	0.7	0.7
11 Opportunities for learning new skills	0.7	0.7	0.7
12 Opportunities for promotion	0.7	0.6	0.5
13 Variety in different tasks	0.6	0.6	0.5
14 Sufficient training for your current job	0.6	0.6	0.5
15 Flexibility of working hours	0.5	0.6	0.5
16 Consultation about changes in your job	0.5	0.8	0.6
3. Workload issues
17 Impact of work on family/social life	0.8	0.6	0.7
18 Pace of work	0.8	0.6	0.7
19 How you rate your work load	0.7	0.6	0.7
20 Impact of family/social life on work	0.7	0.6	0.6
4. Quality of relationships with colleagues
21 How well you get on with your co‐workers (socially)	0.9	(0.3)	0.8
22 How well you work with your co‐workers (as a team)	0.9	(0.3)	0.8
5. Quality of physical environment
23 Safety at work	0.7	0.7	0.7
24 Exposure to physical danger	0.7	0.6	0.6
25 Facilities for taking breaks (rest rooms, lunch break areas, etc)	0.7	0.6	0.6
26 Work surroundings (noise, vibration, light, temperature, etc)	0.7	0.6	0.6
27 Work station/work space (or multisite location)	0.7	0.6	0.6
28 The equipment/IT that you use	0.5	0.6	0.5

The final WOAQ consists of the summative factor and the five orthogonal factors. The participants' scores on these six scales were calculated (by summation of the item scores involved) and the correlations among scales examined (table 3).

Table 3 Correlations between the five factors, demographics, and health outcome variables (zero order).

Factors and outcomes	M (SD)/%	n	1	2	3	4	5	6	7	8	9
1 Quality of relationships with management	3.0 (0.3)	452
2 Reward and recognition	3.0 (0.8)	476	0.8*
3 Workload	2.9 (0.8)	492	0.5*	0.5*
4 Quality of relationships with colleagues	3.8 (0.8)	505	0.3*	0.3*	0.3*
5 Quality of physical environment	3.0 (0.8)	437	0.6*	0.6*	0.6*	0.3*
6 Summative factor	83.7 (18.1)	524	0.9**	0.9**	0.7**	0.4*	0.8*
7 Tension	5.8 (6.3)	471	−0.2*	−0.3*	−0.3*	−0.1**	−0.3*	−0.3**
8 Exhaustion	15.1 (8.2)	479	−0.3*	−0.3*	−0.3*	−0.3*	−0.3*	−0.3**	0.8*
9 Subjective health	3.6 (1.1)	522	0.3*	0.3*	0.4*	0.3*	0.4*	0.4**	−0.4*	−0.4*
10 Job satisfaction	3.2 (1.1)	489	0.5**	0.5**	0.4**	0.3**	0.5**	0.6**	−0.2**	−0.3**	0.3**

*p⩽0.05; **p⩽0.01.

Correlations among the five factors ranged from 0.2 to 0.8. Quality of relationships with colleagues showed the lowest correlation with the other four scales (r = 0.2 to 0.3). Workload showed moderate to low correlations with all other scales (r = 0.3 to 0.6). Not surprisingly, quality of relationship with management and reward and recognition were strongly correlated (r = 0.8). The summative factor correlated strongly with the orthogonal factors (r = 0.7 to 0.9) except with quality of physical environment (r = 0.4).

Reliability of the WOAQ

Table 4 summarises the internal reliability data for the WOAQ factors. Using the data from the main study, all factors had acceptable internal reliability with Cronbach's α between 0.8 and 0.9, values above the recommended minimum of 0.7.¹⁷ All items contributed significantly to their scales. The split‐half reliability coefficients were acceptable for all five factors. Mean inter‐item correlations for both parts were above 0.4. Spearman‐Brown coefficients ranged from 0.8 to 0.9. Guttman split‐half coefficients were all above the recommended 0.7 lower bounds for true reliability.

Table 4 Reliability statistics.

	Quality of relationships with management	Reward and recognition	Workload	Quality of relationships with colleagues	Quality of physical environment	Summative factor
Items (n)	9	7	4	2	6	26
Factor loadings (range)	0.5–0.8	0.5–0.8	0.7–0.8	0.9–0.9	0.5–0.7	0.6–0.8
Cronbach's α Time 1	0.9 (n = 452)	0.9 (n = 476)	0.8 (n = 492)	0.8 (n = 505)	0.8 (n = 437)	0.9 (n = 346)
Test‐retest reliability:
Time 2 (n = 78)	0.9 (n = 74)	0.8 (n = 73)	0.8 (n = 77)	0.8 (n = 77)	0.8 (n = 71)	0.8 (n = 69)
Time 3 (n = 32)	0.9 (n = 26)	0.9 (n = 28)	0.9 (n = 30)	0.5 (n = 31)	0.9 (n = 29)	0.8 (n = 24)
Time 4 (n = 46)	0.9 (n = 43)	0.8 (n = 35)	0.7 (n = 44)	0.7 (n = 46)	0.7 (n = 44)	0.7 (n = 39)
Mean inter‐item correlations (range)	0.5 (0.4–0.7)	0.5 (0.3–0.6)	0.5 (0.4–0.7)	0.6 (2 items)	0.5 (0.4–0.7)	0.4 (0.1–0.7)
Spearman‐Brown coefficient	0.9*	0.9*	0.9†	0.8†	0.8†	0.9†
Guttman split‐half coefficient	0.9	0.8	0.9	0.8	0.8	0.9

*Unequal lengths.

†Equal length.

Data from the separate reliability study were used to calculate Cronbach's α for the subsequent time samples (Time 1, Time 2, and Time 3). These were all above 0.7 for four of the orthogonal factors: quality of relationships with management, reward and recognition, quality of physical environment, and workload. The time samples for the quality of relationships with colleagues showed a more variable pattern of internal reliability comparing across time samples.

The test‐retest data were reassuring. For short term measurement intervals (up to Time 2), r coefficients of dependability ranged from 0.8 to 0.9. Pearson's correlations between the different time measurements ranged from 0.6 to 0.9 (p⩽0.01); the lowest was for Time 1/Time 4 (r = 0.6). For any given measure, the test‐retest reliability should decrease with time. The α coefficients of stability for Time 1/Time 2 and Time 1/Time 3 ranged from 0.7 to 0.9. When the measurement time interval increases (two months), coefficients of stability are expected to drop to 0.6. In the current study, Time 1/Time 4 coefficients ranged from 0.6 to 0.9.

Validity of the WOAQ

There were small to moderate correlations between the WOAQ factors and wellbeing, subjective health and job satisfaction (r = 0.2 to 0.6, p⩽0.01; table 2). Feelings of tension and exhaustion were negatively correlated with the five orthogonal factors (mean r = −0.3, p⩽0.05), indicating that the greater the problems with the design and management of work, the worse participants' general wellbeing was reported to be. Wellbeing and subjective health were correlated with all five factors (mean r = 0.3, p⩽0.05). Self‐reports of health were positively related to apparent lack of problems with the design and management of work. The WOAQ factor with the weakest correlations with wellbeing and subjective health was quality of relationships with colleagues (mean r = 0.2, p⩽0.05). The factor with the strongest correlations with those variables was quality of physical environment (mean r = 0.4, p⩽0.05). Job satisfaction was strongly and positively correlated with all five factors and with the summative factor (mean r = 0.5, p⩽0.01).

Stepwise linear regression analyses examined the extent to which the summative and five orthogonal factors (separately for the former and the latter) could predict health and health related outcomes. This exercise provides some estimate of the risk associated with the five factors in terms of each outcome variable and is measurable by the beta weights for the appropriate predictive equation.

Missing cases were removed using the listwise method. There were no outliers. The ratio of cases to predictor variables (five WOAQ factors) was 87.3:1 which exceeds the minimum requirements of a 15:1 ratio.¹⁸ The recommended minimum number of cases (100) was also exceeded.¹⁹ Preliminary evaluation of the data against the normality requirements for linear regression led to the square root transformation of exhaustion and tension. All other variables were normally distributed. The majority of correlations between predictor and outcome variables were significant. Table 5 displays the predictors included in each of the five models, the variance contributed by each (R²), adjusted R² and additional R² explained by each extra variable (ΔR²), F values indicating the significance of the models, regression coefficients (unstandardised B and standardised β) and t values of the significance of the regression coefficients.

Table 5 Regression analyses of the summative and five factors on the outcome variables.

Outcomes and predictors	R2	Adj R2	ΔR2	F overall (df)	Beta	β	t
Job satisfaction
Summative factor	0.33	0.33		237.9** (1487)	0.03	0.6	15.4**
Quality of relationships with management	0.33	0.33	0.33	47.1** (4316)†	0.4	0.3	4.1**
Reward and recognition	0.35	0.35	0.02		0.3	0.2	3.1**
Workload	0.37	0.36	0.01		0.2	0.1	2.3*
Quality of relationships with colleagues	0.37	0.37	0.01		0.1	0.1	2.1*
Tension
Summative factor	0.05	0.05		25.6** (1469)	−0.02	−0.2	−5.1**
Workload issues	0.12	0.12	0.12	17.0** (3304)	−0.3	−0.2	−3.2**
Quality of physical environment	0.14	0.14	0.02		−0.3	−0.2	−2.8**
Exhaustion
Summative factor	0.06	0.06		31.9** (1477)	−0.1	−0.3	−5.7**
Quality of relationships with management	0.15	0.14	0.15	13.0** (5306)	−0.3	−0.2	−3.6**
Quality of relationships with colleagues	0.16	0.15	0.02		−0.2	−0.1	−2.2*
Workload issues	0.18	0.16	0.01		−0.2	−0.1	−2.0*
Subjective health
Summative factor	0.15	0.15		94.0** (1520)	0.02	0.4	9.7**
Workload issues	0.18	0.18	0.18	31.1** (3328)	0.4	0.3	5.2**
Quality of relationships with colleagues	0.21	0.20	0.02		0.2	0.2	2.8**
Quality of physical environment	0.22	0.22	0.01		0.2	0.1	2.4*

*p⩽0.05; **p⩽0.01.

†For the overall model including the five factors.

The summative factor proved a good predictor of the health and health related variables explaining between 5% and 33% of their variance. The strongest relationship was with job satisfaction (33% of variance; β = 0.6, p⩽0.01). The weakest relationships were with exhaustion and tension. Taken together, the five orthogonal factors were also predictive of wellbeing, subjective health, and job satisfaction. Interestingly, the pattern and strength of these predictions varied across outcome variables.

The main predictor for job satisfaction was quality of relationships with management (32.7% of variance; β = 0.3, t = 4.1, p⩽0.01). Reward and recognition, workload issues, and quality of relationships with colleagues made smaller contributions (explaining 0.9% to 2.4% of the variance). The less problematic work design and management was perceived to be, the better job satisfaction was. The final model explained a substantial 37.4% of the variance in job satisfaction (F(4316) = 47.1, p⩽0.01).

The main predictor of feelings of exhaustion was quality of relationships with management (10.3% of variance; β = −0.2; t = −3.6; p⩽0.01). Smaller contributions were made by quality of relationships with colleagues (1.6% variance), and workload (1.1% variance). Exhaustion was stronger, the more problematic the design and management of work was perceived to be. The final model explained 17.5% of the variance in exhaustion (F(4306) = 13.0, p⩽0.01).

The main predictor for feelings of tension was workload (12% of its variance; β = −0.2; t = −3.2; p⩽0.01). Quality of physical environment made a smaller contribution (2.2% variance). Tension was stronger, the more problematic the design and management of work was perceived to be. The final model explained 14.4% of the variance in feelings of tension (F(3304) = 17.0, p⩽0.01).

The main predictor for the rating of subjective health was workload issues (18.3% of variance; β = 0.3; t = −5.2; p⩽0.01). Smaller contributions were made by quality of relationships with colleagues (2.4% variance) and quality of physical environment (1.4% variance). The overall rating of health was better, the less problematic the design and management of work was perceived to be. The final model explained 22.2% of the variance in subjective health (F(3328) = 31.2, p⩽0.01).

Feedback to enterprises

A national WOAQ database has been established for this project using data collected from this study and others. It is being expanded through an online assessment and feedback system based at the EEF.²⁰ The developing database allows feedback to participating enterprises to be usefully augmented.

Feedback was given to enterprises on three levels: (1) scores on the summative factor, (2) scores on the five orthogonal factors, and (3) scores on individual items of importance.

Feedback on the summative factor is useful for benchmarking within the enterprise and across the sector and for prioritising further assessment or management initiatives. Feedback on the five orthogonal factors can be used for benchmarking and prioritisation. Feedback on individual items may provide sufficient detail to inform discussions on possible interventions and identify strong points (positive features).

Feedback from enterprises

Feedback from participating enterprises, many of which had used the tool in response to the HSE SMS initiative, was positive and encouraging. They found the risk assessment instrument meaningful and easy to complete. The particularly positive features identified from their comments were: the ability to compare the company‐specific findings with those for the rest of the manufacturing sector, the way that the data informed their discussions on risk reduction, and the prioritisation of such interventions.

Discussion

The studies reported here have allowed a new instrument to be developed for the identification and measurement of the hazards typically found in the design and management of work in the manufacturing sector.

A two level model of these hazards is supported by the factor analyses conducted on data from the main study. The higher level is described by a summative factor that involves 26 of the 28 items that make up the WOAQ. The lower level, including all items, is described by five orthogonal factors. This structure and the factors involved not only make conceptual sense, but have meaning and are useful for the user population. The five orthogonal factors can be viewed as the contextualised equivalent of the HSE SMS⁴,⁵,⁶ for the manufacturing sector, empirically derived and proven. At the same time, their origins in the taxonomy of psychosocial and organisational hazards are also apparent.² There is conceptual consistency in these different schemes.

The WOAQ was proved to have good concurrent validity and good reliability using a variety of techniques and analyses, particularly in terms of its association with overall job satisfaction and measures of exhaustion and tension. Interestingly, these relationships differ across outcome variables both in their pattern and strength. There is no simple halo effect obvious in these data.

Reports from the enterprises also suggest that the WOAQ has face validity (reported to be meaningful) and good utility (acknowledged to be short and reported to be easy to complete by both white‐ and blue‐collar workers).

The WOAQ factors were good predictors of the measures of wellbeing, subjective health, and job satisfaction used in the main study, explaining up to 37% of their variance. This percentage is high compared to reports in the general stress literature. For example, a review of studies reported that a combination of measures of 15 work stressors accounted for only 7% of the variance in health outcomes studied.²¹ Furthermore, the measures of work stressors used came from standardised questionnaires developed for use across occupations and organisations.

The use of linear regression techniques for the prediction of health and health related behaviour from the hazard measures (the WOAQ factors) makes available a range of statistics that might be taken as indicative measures of risk such as beta or the total variance accounted for and/or the reliability p of these measures. Such indicative coefficients of risk can be of use in prioritising areas of concern or subsequent interventions.

The WOAQ is not only a hazard identification tool; the wording of the items used and the response scales associated with those items allow the identification of positive work factors. This is an important subsidiary feature and begins to identify which aspects of work may be “good for you”.

The present studies demonstrate the feasibility of developing sector‐specific approaches to the assessment of risk associated with the design and management of work and the utility of doing so. This can be done within the framework of the HSE's SMS initiative while building on previous applied research. Essentially, the methodology used here¹,²,⁸,¹⁰ takes a situation or sector‐specific approach to measurement, is unconstrained by specific theories,²² is bottom‐up, and data driven.

Main messages

• Occupational context is important for the accurate and useful assessment of work stress.

• Feedback on assessment results is useful and practical if it is provided at three levels: generic, factor specific, and item specific.

• The method reflects Britain's HSE standards and related work on the development of the risk assessment methodology.

• The Work Organisation Assessment Questionnaire (WOAQ) is useful in identifying positive as well as negative aspects of work.

Policy implications

• Development of an assessment system for use by health and safety and occupational health management and line managers.

• Workable, bespoke system, capable of informing interventions to reduce risk.

• The risk assessment instrument relates the WOAQ with parameters of health and job satisfaction.

Conclusions

The studies reported here describe a two level model of the hazards to employee health and health related behaviour inherent in the design and management of work in the manufacturing sector. They support the use of a new questionnaire derived from that model, the WOAQ, by demonstrating its reliability, validity, and utility. The analyses revealed a strong 28‐item, five‐factor structure and a more general summative factor. The results also indicated that the questionnaire has good internal and test‐retest reliability and validity. It was shown that the problem areas identified by the WOAQ may be hazards in that they were associated with poor employee wellbeing, poor subjective health, and low job satisfaction. The strength of those associations offers some estimation of risk in this context. Feedback from the organisations involved indicated that the WOAQ was easy to use and meaningful to them as part of their risk assessment procedures.

Abbreviations

GWBQ - General Well‐Being Questionnaire

HSE - Health and Safety Executive

SMS - Stress Management Standards

WOAQ - Work Organisation Assessment Questionnaire