Skip to main content
BMJ Open logoLink to BMJ Open
. 2018 Mar 6;8(3):e018485. doi: 10.1136/bmjopen-2017-018485

Relationship between stress-related psychosocial work factors and suboptimal health among Chinese medical staff: a cross-sectional study

Ying-Zhi Liang 1,2,#, Xi Chu 3,#, Shi-Jiao Meng 4, Jie Zhang 1,2, Li-Juan Wu 1,2, Yu-Xiang Yan 1,2
PMCID: PMC5855386  PMID: 29511008

Abstract

Objectives

The study aimed to develop and validate a model to measure psychosocial factors at work among medical staff in China based on confirmatory factor analysis (CFA). The second aim of the current study was to clarify the association between stress-related psychosocial work factors and suboptimal health status.

Design

The cross-sectional study was conducted using clustered sampling method.

Setting

Xuanwu Hospital, a 3A grade hospital in Beijing.

Participants

Nine hundred and fourteen medical staff aged over 40 years were sampled. Seven hundred and ninety-seven valid questionnaires were collected and used for further analyses. The sample included 94% of the Han population.

Main outcome measures

The Copenhagen Psychosocial Questionnaire (COPSOQ) and the Suboptimal Health Status Questionnaires-25 were used to assess the psychosocial factors at work and suboptimal health status, respectively. CFA was conducted to establish the evaluating method of COPSOQ. A multivariate logistic regression model was used to estimate the relationship between suboptimal health status and stress-related psychosocial work factors among Chinese medical staff.

Results

There was a strong correlation among the five dimensions of COPSOQ based on the first-order factor model. Then, we established two second-order factors including negative and positive psychosocial work stress factors to evaluate psychosocial factors at work, and the second-order factor model fit well. The high score in negative (OR (95% CI)=1.47 (1.34 to 1.62), P<0.001) and positive (OR (95% CI)=0.96 (0.94 to 0.98), P<0.001) psychosocial work factors increased and decreased the risk of suboptimal health, respectively. This relationship remained statistically significant after adjusting for confounders and when using different cut-offs of suboptimal health status.

Conclusions

Among medical staff, the second-order factor model was a suitable method to evaluate the COPSOQ. The negative and positive psychosocial work stress factors might be the risk and protective factors of suboptimal health, respectively. Moreover, negative psychosocial work stress was the most associated factor to predict suboptimal health.

Keywords: psychosocial stress, suboptimal health, medical staff, confirmative factor analysis


Strengths and limitations of this study.

  • The study had high internal validity, with a good representation of medical staff.

  • To assess psychosocial factors at work among medical staff, a more parsimonious, modified second-factor model was finally built to replace the traditional method of calculating the average value of the Copenhagen Psychosocial Questionnaire, which ignored the effect of each item.

  • The study was conducted in Beijing (a dense city), adding evidence on these issues in a context different from the current literature.

  • Although the sample was representative of the diversity of medical staff in one geographical area of China, the data are not nationally representative and ethnic minority groups are particularly under-represented.

  • The study used a cross-sectional design, which is not well suited to assess the direction of causation.

Introduction

Work is viewed as an important aspect of psychosocial stress, and the impact of psychosocial work conditions on workers’ health has been well documented over the past decades. There is accumulating evidence indicating an association between a harsh working environment and a wide range of diseases, including mental disorders,1 2 diabetes3 and cardiovascular disease,4–6 among workers. So far, several theories have been established that predicted various consequences on health of workers when exposed to certain psychosocial risk factors at work.7 Seven influential theories are job characteristics model, the Michigan organisational stress model, the demand–control–(support) model, the sociotechnical approach, the action–theoretical approach, the effort–reward–imbalance model and the vitamin model.8 The Copenhagen Psychosocial Questionnaire (COPSOQ) is a comprehensive and generic instrument based on the integration of the common elements of seven kinds of pattern and development of some of the original entries (such as work content) at the same time to assess psychosocial factors at work. Exposure to workplace psychosocial risk factors varies according to the type of occupation and job role. Teachers, firefighters and hospital workers have been reported to experience higher than the average level of work-related stress.9 10

Due to demographic changes, the number of old people and the incidence of chronic diseases are rising in China. Meanwhile, dealing with chronic diseases, incurable or dying patients is emotionally demanding.11 In addition, there are rapid enhancements on treatment options and therapeutic strategies due to medical advances. These changes may lead to an increased workload and high quantitative demands for Chinese medical staff at hospitals. Recent studies have demonstrated that the prevalence of burnout and stress is relatively high among medical staff.12 13 Stress fatigue and burnout further have a detrimental influence on physicians’ quality of life and may result in early retirement or reduced quality of patient care, and negatively affects healthcare systems.14 15 What is more, studies have shown that medical staff are at increased risk for ill-health, including musculoskeletal disorders16 and mental health problems,17 caused by adverse workplace factors. Consequently, we need to pay attention to the psychosocial work characteristics of medical staff.

Since the ancient time, traditional Chinese medicine has been identifying a physical status between health and disease, which we coined as suboptimal health status (SHS).18 SHS is characterised by functional somatic syndromes or symptoms that are medically undiagnosed. Nowadays, much attention has been paid on perceived poor health ‘somatization’ and ‘medically unexplained symptoms’ in community and primary care systems located in developed countries.19 20 Undoubtedly, SHS is becoming a global issue. Recent studies reported that 60% of students21 and 50%–60% of occupational population22 23 suffered from suboptimal health in China. Unfortunately, impaired quality of life, frequent hospital visits and incurrence of significant medical expenses were often accompanied with SHS.24 Our previous studies have shown that SHS may contribute to the progression or development of chronic diseases, such as cardiovascular disease.25 Although the aforementioned study has demonstrated the prevalence of SHS and its consequences, few studies have addressed the issue of stress-related psychosocial work factors and suboptimal health among medical staff in China. This study aimed to evaluate the impact of stress-related psychosocial work factors on SHS and their associations.

Participants and methods

Ethics statement

All study participants provided written informed consent prior to enrolment in the study.

Participants

This cross-sectional study was conducted using clustered sampling method. The current analysis included 914 medical staff from Xuanwu Hospital who participated in the 2014 annual health medical examination (including physicians, nurses, medical technicians, management staff and others). All participants of this study were older than 40 years of age. The data were collected through the Suboptimal Health Status Questionnaires-25 (SHSQ-25) and the COPSOQ. The subjects were divided into ‘SHS’ and ‘non-SHS’ groups depending on their scores on SHSQ-25.

Instruments

Copenhagen Psychosocial Questionnaire

The COPSOQ is a comprehensive and generic instrument used to assess psychosocial factors at work. The Chinese translation and adaptation of COPSOQ had been tested in a population with different professions, and had shown good reliability and validity, with a Cronbach’s alpha coefficient of 0.7 for most scales.26 27 This instrument includes three versions: a long version for research use, a medium-length version to be used by work environment professionals and a short version for workplaces. Our study was based on the short Chinese version of COPSOQ, which consists of 44 questions forming 8 scales. We selected 34 questions including 5 dimensions from a short version of COPSOQ, namely ‘Demands at work’, ‘Influence and development’, ‘Interpersonal relations and leadership’, ‘Insecurity at work’ and ‘Job satisfaction’, to assess psychosocial factors at work for stress.8 In this survey, the remaining three health-related dimensions, namely ‘general health’, ‘mental health’ and ‘vitality’, in the original short version of COPSOQ were not used. For most of the questions, we used either intensity (from ‘to a very small extent’ to ‘to a very large extent’) or frequency (from ‘never/hardly ever’ to ‘always’). All items of COPSOQ were transformed on a value that ranges from 0 to 100 points, with 0 representing the lowest degree of the measured psychosocial factor ‘never/hardly ever’ or ‘to a very small extent’, and 100 representing the highest ‘always’ or ‘to a very large extent’ (online supplementary table S1). In most scales, a high score was considered desirable. On the contrary, a low score was considered desirable for ‘Demands at work’ and ‘Insecurity at work’.

Supplementary file 1

bmjopen-2017-018485supp001.pdf (433.9KB, pdf)

As a default generic method, the average scores for each dimension of COPSOQ were calculated and compared. But this method ignored the relationship between each item and corresponding dimension. To explore the association among each of the dimensions of COPSOQ, we conducted confirmatory factor analysis (CFA),28 which could estimate the relationship between each latent variable (ie, each dimension of COPSOQ) and between observed variables (ie, items of dimensions), as well as corresponding latent variables.

Suboptimal Health Status Questionnaires-25

Prior to survey, participants had attended a hospital annual health examination, comprising a medical history, physical examination, blood biochemical examination, routine urinalysis, rest ECG, chest radiography and so on. According to medical history and physical examination results, participants diagnosed with clinical diseases by an associate chief physician or more professional clinical doctors were excluded. Then, the SHS of the other participants was measured by SHSQ-25,18 including 25 items and encompassing five subscales: fatigue, the cardiovascular system, the digestive tract, the immune system and mental status. The SHSQ-25 is short and easy to complete, and therefore suitable for use in general population and primary care service.23 Each individual was asked to rate a specific statement on a 5-point Likert-type scale based on how often they suffered various specific complaints in the preceding 3 months: never/hardly ever, occasionally, often, very often and always. The scores on the questionnaire were coded as 0–4. SHS scores ranged from 0 to 100 and were calculated for each respondent by summing the ratings for the 25 items. A high score represents a high level of SHS (poor health).

There are no cut-off scores. The sample did not have high levels of suboptimal health (online supplementary table S2); therefore, for an easier interpretation, participants with a SHSQ-25 score higher than 31 (median of the total sample) were classified as ‘SHS’, and those with a score equal to or lower than 31 were classified as ‘non-SHS’. The sensitivity of our results to this choice was examined further in sensitivity analyses by classifying the respondents with SHSQ-25 scores in the 75th percentile (P75) and above (a score higher than 43) and in the 90th percentile (P90) and above (a score of 53 and above) as SHS, and all others as non-SHS.

Statistical analysis

Descriptive statistics were used to describe the overall population. Univariate analyses were used to compare variations in demographic characteristics among medical staff with different SHS; for binary and categorical variables, χ2 test was used, while ordinal variables were analysed by Kolmogororv-Smirnov Z test. For non-parametric data, Mann-Whitney U test was used to assess stress-related working factors among medical staff with different health status. Demographic missing data were coded as missing and excluded from relevant analysis. A Cronbach’s alpha of >0.70 is considered to be an acceptable reliability coefficient for determining the internal consistency of the scale.29 Model testing was conducted by CFA and structural equation modelling (SEM) analyses. To assess global fit of the model by total sample, we calculated five goodness-of-fit indices. They were χ2 and its subsequent ratio with df (χ2/df), adjusted goodness-of-fit index, comparative fit index, standard root mean square residual and root mean square error of approximation. Evaluation standards were described in a previous literature.30–32 The first-order factor model was used to analyse the correlation among the five dimensions of COPSOQ, and the second-order factor model was used to establish the evaluating method of COPSOQ for comparing psychosocial work characteristics among medical staff. A multivariate logistic regression model was used to estimate the relationship between suboptimal health status and psychosocial factors at work. Potential confounders including age, gender, education level, occupation, physical exercise, drinking behaviour and smoking status were adjusted. Two-tailed P<0.05 was considered statistically significant. The statistical packages SPSS V.22.0 and AMOS V.22.0 (Chicago, Illinois) were used for statistical analysis.

Results

Baseline characteristics

Among 914 of the medical staff who participated in the 2014 annual health medical examination, 797 eligible questionnaires were retrieved, with a retrieval rate of 87.20%. The mean age was approximately 50. More than half of the participants were female (n=554, 69.5%). Table 1 shows the descriptive analyses of participants according to SHS. The differences in age, gender, education level, occupation, and status of physical exercise, smoking and drinking between individuals with and without SHS were statistically significant (all P<0.05; table 1). Sensitivity analyses of the participants according to SHS (P75 and P90) reported the same results. Moreover, compared with non-SHS individuals, SHS individuals were statistically significantly (P<0.05) more likely to have longer weekly working hours when using P75 as an SHS cut-off (online supplementary table S3). There were 396 (49.7%) individuals considered as SHS based on the SHSQ-25 score (median). Among 396 individuals with suboptimal health, 80.6% were female, nearly half (48.2%) with the highest record of formal schooling were junior college, 31.8% careered in nursing, 59.8% were without the habit of physical exercise, and most (>80%) were not smoking and drinking (table 1). This advantage in the proportion of corresponding variables above still existed and became more obvious on sensitivity analyses (online supplementary table S3).

Table 1.

Descriptive analyses of participants according to suboptimal health status and the stress-related psychosocial work factors as a total sample

 Demographics Total (n=797) SHSQ-25 (P50) Second-order factor of COPSOQ
n (%) Non-SHS SHS P value D1 negative psychosocial work stress factor P value D2 positive psychosocial work stress factor P value
Age group (years) 0.005* 0.003 0.007
 40~ 270 (33.9) 118 (29.4) 152 (38.4) 0.21±1.65 −0.80±7.00
0.15 (−1.14, 1.32) 0.15 (−5.52, 4.17)
 45~ 245 (30.7) 126 (31.4) 119 (30.1) 0.07±1.72 −0.29±6.84
−0.1 (−1.28, 1.24) 0.46 (−4.39, 4.76)
 55~68 282 (35.4) 157 (39.2) 125 (31.6) 0.26±1.70 1.01±7.35
−0.47 (−1.55, 0.88) 1.08 (−2.93, 6.78)
Gender <0.001 0.292 <0.001
 Male 243 (30.5) 166 (41.4) 77 (19.4) −0.05±1.88 1.24±7.29
−0.17 (−1.67, 1.20) 1.54 (−3.33, 6.82)
 Female 554 (69.5) 235 (58.6) 319 (80.6) 0.02±1.62 −0.54±6.96
−0.11 (−1.22, 1.14) 0.24 (−5.02, 4.31)
Education level 0.003* <0.001 0.111
 High school and below 122 (15.3) 65 (16.20) 57 (14.4) −0.41±1.81 0.55±7.79
−0.62 (−1.90, 0.64) 1.19 (−3.74, 6.23)
 Junior college 321 (40.3) 130 (32.4) 191 (48.2) −0.07±1.65 −0.78±7.44
−0.21 (−1.39, 1.04) 0.17 (−5.81, 4.35)
 University 182 (22.8) 102 (25.4) 80 (20.2) −0.15±1.71 0.62±6.68
−0.37 (−1.52, 1.01) 1.22 (−3.60, 5.47)
 Graduate students and above 172 (21.6) 104 (25.9) 68 (17.2) 0.57±1.56 0.42±6.30
0.70 (−0.65, 1.63) 0.66 (−4.28, 5.17)
Occupation <0.001 <0.001 0.001
 Nurses 188 (23.6) 62 (15.5) 126 (31.8) 0.22±1.57 −1.15±7.35
0.13 (−0.99, 1.16) −0.10 (−7.17, 4.10)
 Medical technicians 187 (23.5) 83 (20.7) 104 (26.3) −0.30±1.59 −0.85±6.75
−0.43 (−1.64, 1.02) −0.20 (−5.48, 3.46)
 Doctors 208 (26.1) 125 (31.2) 83 (21.0) 0.37±1.72 0.70±6.97
0.31 (−0.93, 1.53) 0.94 (−4.19, 6.67)
 Others 214 (26.9) 131 (32.7) 83 (21.0) −0.29±1.78 1.06±7.13
−0.60 (−1.77, 0.89) 1.44 (−2.78, 5.51)
Physical exercise 0.003 0.001 0.022
 Yes 363 (45.5) 204 (50.9) 159 (40.2) −0.19±1.76 0.51±7.30
−0.48 (−1.69, 1.14) 1.12 (−4.27, 5.77)
 No 434 (54.5) 197 (49.1) 237 (59.8) 0.16±1.63 −0.43±6.92
0.12 (−1.02, 1.18) 0.33 (−5.03, 4.32)
Smoking <0.001 0.082 0.013
 Yes 93 (11.7) 63 (15.7) 30 (7.6) −0.19±2.04 1.58±7.80
−0.90 (−1.77, 1.16) 2.16 (−3.16, 7.23)
 No/Quit 704 (88.3) 338 (84.3) 366 (92.4) 0.02±1.65 −0.21±6.99
−0.09 (−1.28, 1.16) 0.47 (−4.72, 4.78)
Drinking <0.001 0.081 0.012
 Yes 166 (20.8) 105 (26.2) 61 (15.4) −0.18±1.78 1.18±7.13
−0.66 (−1.56, 1.04) 1.39 (−2.83, 6.72)
 No/Abstained 631 (79.2) 296 (73.8) 335 (84.6) 0.05±1.68 −0.31±7.07
−0.84 (−1.30, 1.16) 0.37 (−4.93, 4.75)
Night shift 0.774 <0.001 0.445
 Yes 331 (41.5) 169 (42.1) 162 (40.9) 0.27±1.71 −0.21±7.04
0.22 (−1.08, 1.36) 0.46 (−4.42, 4.78)
 No 466 (58.5) 232 (57.9) 234 (59.1) −0.19±1.67 0.15±7.16
−0.39 (−1.53, 0.96) 0.67 (−4.68, 5.22)
Weekly working hours 0.455 <0.001 0.827
 ≤40 hours 270 (33.9) 141 (35.2) 129 (32.6) −0.49±1.60 0.13±6.92
−0.71 (−1.78, 0.56) 0.57 (−4.45, 5.07)
 >40 hours 527 (66.1) 260 (64.8) 267 (67.4) 0.25±1.70 −0.06±7.21
0.17 (−1.10, 1.43) 0.70 (− 4.62, 5.11)

*Analysed by Kolmogororv-Smirnov Z test.

COPSOQ, Copenhagen Psychosocial Questionnaire; P50, 50th percentile; SHS, suboptimal health status; SHSQ-25, Suboptimal Health Status Questionnaires-25.

Reliability

COPSOQ showed a very high overall internal consistency, with a Cronbach’s alpha of 0.849 for the total scale (items 1–34). The internal consistency characteristics of COPSOQ showed good reliability. The Cronbach’s alpha of five dimensions ranged from 0.791 to 0.891 (online supplementary table S4).

Confirmatory factor analysis

We performed CFA based on the five theoretical dimensions of COPSOQ. Parameters were estimated for the CFA model based on the maximum likelihood procedure involving fitting the variances and covariances among observed scores. AMOS therefore created a covariance matrix, including the variances and covariances among observed scores. The next step was to illustrate the observed (items) and unobserved (factors) in the hypothesised model (see online supplementary figure S1). The goodness-of-fit index was unacceptable in M1a (table 2). After modification according to modification index,33 the modified first-order factor model (M1b, see online supplementary figure S2) for COPSOQ had adequate fit of the model to the data (table 2). However, Pearson correlations between first-order factors in the M1b model showed that most of the first-order factors correlated with each other (online supplementary table S4). These results supported the notion that COPSOQ comprised five factors subsumed under one or two higher order factors. Based on the theoretical model of COPSOQ, high scores in F1 (demands at work) and F4 (insecurity at work) mean being susceptible to work strain. Conversely, high scores in F2 (influence and development), F3 (interpersonal relations and leadership) and F5 (job satisfaction) may protect people from work strain.26 According to the theory, the second-order factor model of COPSOQ might be existed. We next conducted CFA to formally test the fit of our hypothesised, second-order factor model (M2a) of COPSOQ. This model, depicted in online supplementary figure S3, did not have good overall model fit (table 2). This suggested M2a needs further modification. M2a was modified (figure 1) and the fit of the modified second-order model (M2b) was acceptable (table 2).

Table 2.

Goodness-of-fit indices for the different models

Model χ2 (df) χ2/df CFI AGFI SRMR RMSEA
M1a 4216.39 (517) 8.16 0.737 0.708 0.0802 0.095
M1b 1699.40 (503) 3.40 0.915 0.864 0.0696 0.055
M2a 4276.30 (521) 8.21 0.733 0.707 0.0852 0.095
M2b 1664.67 (508) 3.28 0.918 0.866 0.0659 0.053

Model fitting criteria were as followed: a CFI value of greater than 0.90 showed a psychometrically acceptable fit to the data; the value of AGFI ranged between 0 and 1, a value of 1 indicated a perfect fit; for the SRMR, values of 0.08 or lower represented good fit; the value of RMSEA should be below 0.06 to show good fit.

M1b: the modified first-order factor model; M2a: the second-order factor model; M2b: the modified second-order factor model; M1a: the first-order factor model.

AGFI, adjusted goodness-of-fit index; CFI, comparative fit index; RMSEA, root mean square error of approximation; SRMR, standard root mean square residual.

Figure 1.

Figure 1

Standardised coefficients for modified second-order confirmatory factor analysis (CFA) of Copenhagen Psychosocial Questionnaire (M2b). The structural model consisted of seven interrelated constructs. F1 refers to demands at work; F2, influence and development; F3, interpersonal relations and leadership; F4, insecurity at work; F5, job satisfaction; D1, negative psychosocial work stress factor; and D2, positive psychosocial work stress factor. The observed variables, unobserved variables and measurement error were represented as rectangles, ellipses and circles, respectively. The arrow between the unobserved variable and the observed variable represented a regression path and its number represented the standardised regression weight. The arrow between a small circle and the observed variable represented a measurement error term. The double-headed arrows represented the correlation between two unobserved variables (factor covariances) of the model. CFA results indicated the goodness-of-fit index is fairly good: χ2=1664.67; df=508; χ2/df=3.28; comparative fit index=0.918; root mean square error of approximation=0.053; standardised root mean square residual=0.066. M2b, the modified second-order factor model.

Supplementary file 2

bmjopen-2017-018485supp002.pdf (567.7KB, pdf)

The overall fit of modified factor first-order model (M1b) and modified second-order factor model (M2b) was similar. Thus, we further compared these two models. As a result, a χ2 difference test revealed that modified second-order factor model was significantly better than modified factor first-order model (Δχ2=34.73, P<0.05), which suggested that the more parsimonious, modified second-factor model (M2b) would be favoured for COPSOQ. In M2b, D1 which referred to negative psychosocial work factor included two first-order factors (F1 demands at work and F4 insecurity at work), and D2 positive psychosocial work factor was composed of the rest of the three first-order factors (influence and development, interpersonal relations and leadership, and job satisfaction). All standardised factor coefficients of this model were significant (P<0.05; figure 1), but the relationship between insecurity at work and D1 negative psychosocial work factor was not significant (r=0.18, P>0.355; figure 1). Thus, demand at work was the largest contributor to the negative psychosocial work stress in the current study.

Assessment of stress-related psychosocial work factors among medical staff with different individual and work characteristics

We used the two second-order factors (D1 negative psychosocial work stress factor and D2 positive psychosocial work stress factor) to assess the psychosocial work factors among medical staff. The scores of the factors were calculated by standardised regression coefficients. In SEM, the standardised regression coefficients, also called standardised factor loadings, actually are the correlation coefficients between indicators and their latent variables. The form of standardised factor scores of the ith factor in the first-order model is:

ZFi=jbij(ZCjZCj)

where bij is the standardised regression weight, ZCj is the standardised score of the jth questionnaire item, and ZCj is the average standardised score. i=1,2,3,4,5, j=1,2,3,4,5,6,7,8,9,10.

The form of standardised factor scores of the ith factor in the first-order model is:

S=Wi(ZFiZFi)

where Wi is the standardised regression weight, ZFi is the standardised score of the ith latent variable, and ZFi is the average standardised score of five latent variables. i=1,2,3,4,5.

Based on the above two formulas, we can get the score of D1 (negative psychosocial work factor) and D2 (positive psychosocial work factor) among the medical staff. Scores on two factors did not meet the normal distribution assumptions, and were conducted using the Mann-Whitney U non-parametric test by ranks. Table 1 shows the score of stress-related psychosocial work factors based on different individual and work characteristics. The score of negative psychosocial work stress factor was significantly different among medical staff with different age, education level, occupation, physical exercise, night shift and weekly working hours (P<0.05), while the difference between men and women was not significant (P=0.292). On the other hand, the score of positive psychosocial work stress factor was significantly different among medical staff with different age, gender, occupation, and status of physical exercise, smoking and drinking (P<0.05). Then, we explored the score of psychosocial work stress factors between individuals with and without SHS, with the results shown in table 3. The scores of negative and positive psychosocial factor were significantly different between SHS and non-SHS groups (P<0.05). Briefly, the individuals with SHS were likely to get higher score in negative psychosocial work factor and lower score in positive psychosocial work factor, respectively. This difference stayed statistically significant when using SHS cut-offs of either P75 or P90 (table 3).

Table 3.

Assessment of stress-related psychosocial work factors between individuals with and without SHS

Groups Second-order factor of COPSOQ
D1 negative psychosocial work stress factor P value D2 positive psychosocial work stress factor P value
Suboptimal health status (P50) <0.001 <0.001
 Non-SHS −0.53±1.60 1.15±6.97
−0.77 (−1.83, 0.54) 1.61 (−2.79, 6.34)
 SHS 0.53±1.63 −1.16±7.06
0.46 (−0.67, 1.60) −0.57 (− 5.75, 3.94)
Suboptimal health status (P75) <0.001 <0.001
 Non-SHS −0.24±1.67 0.71±7.08
−0.39 (−1.62, 0.91) 1.24 (−3.85, 5.69)
 SHS 0.77±1.57 −2.29±6.73
0.70 (−0.44, 1.86) −1.93 (−7.11, 2.60)
Suboptimal health status (P90) <0.001 0.005
 Non-SHS −0.09±1.68 0.29±7.13
−0.20 (−1.47, 1.08) 0.78 (−4.39, 5.22)
 SHS 0.90±1.68 −2.14±6.57
0.57 (−0.44, 2.19) −1.93 (−7.23, 2.18)

COPSOQ, Copenhagen Psychosocial Questionnaire; P50, 50th percentile; P75, 75th percentile; P90, 90th percentile; SHS, suboptimal health status.

Relationship between stress-related psychosocial work factors and suboptimal health (P50, P75 and P90)

Multivariate stepwise logistic regression models showed a statistically significant inverse relationship between positive psychosocial work stress factor and suboptimal health, and a positive relationship between negative psychosocial work stress factor and suboptimal health. Regarding the negative psychosocial factor in the total sample, those who got higher scores in negative psychosocial work stress factor had higher risk of being suboptimal than individuals with low scores (model 1: OR (95% CI)=1.47 (1.34 to 1.62), P<0.001). This relationship remained statistically significant in the adjusted models (model 2: OR (95% CI)=1.50 (1.36 to 1.66), P<0.001; model 3: OR (95% CI)=1.57 (1.42 to 1.75), P<0.01) (table 4) and when using SHS cut-offs of either P75 or P90. Considering the total sample, individuals with higher scores in positive psychosocial work stress factor had a lower risk of having suboptimal health compared with those who got lower scores (model 1: OR (95% CI)=0.96 (0.94 to 0.98), P<0.001). This relationship remained statistically significant in the adjusted models (model 2: OR (95% CI)=0.97 (0.95 to 0.99), P=0.003; model 3: OR (95% CI)=0.97 (0.95 to 0.99), P=0.012) and in the majority of SHS sensitivity analyses (using cut-offs of P75 and P90), with the exception of the first-step adjusted and fully adjusted models using P90 as the SHS cut-off (model 2: OR (95% CI)=0.97 (0.94 to 1.01), P=0.155; model 2: OR (95% CI)=0.98 (0.95 to 1.02), P=0.325).

Table 4.

Sensitivity analyses with multivariate models assessing the relationship between stress-related psychosocial work factors and suboptimal health (P50, P75 and P90)

Model/variables Suboptimal health status (P50, P75, P90)
OR (95%  CI) P value
Suboptimal health status (P50)
 Model 1
  D1 negative psychosocial work stress factor graphic file with name bmjopen-2017-018485ileq01.jpg 1.47 (1.34 to 1.62) <0.001
  D2 positive psychosocial work stress factor 0.96 (0.94 to 0.98) <0.001
 Model 2
  D1 negative psychosocial work stress factor 1.50 (1.36 to 1.66) <0.001
  D2 positive psychosocial work stress factor 0.97 (0.95 to 0.99) 0.003
 Model 3
  D1 negative psychosocial work stress factor 1.57 (1.42 to 1.75) <0.001
  D2 positive psychosocial work stress factor 0.97 (0.95 to 0.99) 0.012
Suboptimal health status (P75)
 Model 1
  D1 negative psychosocial work stress factor 1.39 (1.26 to 1.54) <0.001
  D2 positive psychosocial work stress factor 0.95 (0.93 to 0.97) <0.001
 Model 2
  D1 negative psychosocial work stress factor 1.42 (1.28 to 1.58) <0.001
  D2 positive psychosocial work stress factor 0.95 (0.93 to 0.98) <0.001
 Model 3
  D1 negative psychosocial work stress factor 1.44 (1.29 to 1.61) <0.001
  D2 positive psychosocial work stress factor 0.96 (0.93 to 0.98) 0.001
Suboptimal health status (P90)
 Model 1
  D1 negative psychosocial work stress factor 1.36 (1.18 to 1.57) <0.001
  D2 positive psychosocial work stress factor 0.97 (0.93 to 1.00) 0.037
 Model 2
  D1 negative psychosocial work stress factor 1.43 (1.23 to 1.67) <0.001
  D2 positive psychosocial work stress factor 0.97 (0.94 to 1.01) 0.155
 Model 3
  D1 negative psychosocial work stress factor 1.46 (1.25 to 1.70) <0.001
  D2 positive psychosocial work stress factor 0.98 (0.95 to 1.02) 0.325

Model 1: unadjusted. Model 2: adjusted by age and gender. Model 3: adjusted by age, gender, education level, occupation, physical exercise, drinking behaviour and smoking status.

P50, 50th percentile; P75, 75th percentile; P90, 90th percentile.

Discussion

With social economic development and rapid pace of life, the public have paid more and more attention to the importance of suboptimal health. SHS is regarded as a subclinical, reversible stage of chronic disease characterised by a decline in vitality, in physiological function and in the capacity for adaptation within a period of 3 months.18 To measure SHS, we developed SHSQ-25 and adopted it as an instrument in this study. SHSQ-25 has good internal consistency, with item–subscale correlations ranging from 0.51 to 0.72, and with a Cronbach’s alpha of 0.70 or higher for all subscales.23 The good internal consistency (Cronbach’s alpha of 0.943) was also verified in our study (data not shown). However, there are other SHS questionnaires in China, such as the Sub-Health Measurement Scale V.1.0 (SHMS V1.0) and the Multidimensional Subhealth Questionnaire of Adolescents (MSQA). MSQA is aimed at adolescents, while SHMS V.1.0 is a 39-item questionnaire that includes physiological, psychological and social dimensions. Previous research has shown SHMS V.1.0 has good internal consistency in a population of southern Chinese medical staff.34 However, the SHSQ-25 was reliable and valid in a large-sample health status survey in Beijing.23 On the other hand, the content and function of the social symptoms dimension of SHMS V.1.0 were repeated with COPSOQ, which is used to assess the social-psychological factors at work in our study. In comparison, SHSQ-25 is shorter and easier to complete, and therefore suitable for use in studies of the medical staff in our study. Multiple factors that were influential to SHS, including gender, age, physical activities, dietary habits, emotional problems, social adaptation and others, have been found in recent studies.22 25 Correspondingly, age, gender, education level, job, physical exercise, smoking and drinking were significant factors that may influence the status of health among medical staff in the current study. There is no internationally accepted cut-off value to diagnose SHS. Thus, we further conducted a sensitivity analysis, the results of which were also valid. Overall, the female nurses without ways to relieve stress, such as habit of physical exercise, smoking and drinking, had higher scores in SHSQ-25 (poorer health).

Over the last 20 years, rare longitudinal and many cross-sectional studies have highlighted that work organisation conditions, including repetitive work,35 decision authority,36 physical and emotional demands,37 irregular schedules and long hours,38 39 and job insecurity,40 were the stress-related work factors that explain the emergence or aggravation of mental illness. In addition, different studies have also found low job satisfaction to be an important contributor to occupational stress in healthcare settings.41 42 The relation of mental workload and health status based on documented measuring instruments that covered all important aspects was undisputedly increased. For enterprises and organisations, the COPSOQ questionnaire is a qualified screening instrument for psychosocial factors at work.43 It has good internal consistency, with a Cronbach’s alpha of 0.79 or even higher for all subscales in our study. But scale scores were computed as the average of the values of the single aspects, and this method ignored the relationship among each of the dimensions. Previous studies28 also showed that the factor loadings calculated by traditional factor analysis were less accurate and precise than that calculated by SEM, because the traditional method could not control the effects of other variables and caused message loss when extracting common factors. By contrast, SEM could get factor loadings both of indicators to first-order factors and first-order factors to second-order factors. The standardised regression coefficients estimated the relational degree between indicators and first-order factors, first-order factors and second-order factors by controlling other variables. Another difference with traditional method is that SEM allows measurement error of indicators.

Based on the above comparison and consideration, we conducted first-order and second-order factor models to explore the association among the dimensions of COPSOQ. In this study, a modified second-factor model with best fit indexes was considered to be favoured for COPSOQ. Therefore, the stress-related psychosocial work factors of medical staff were assessed by modified second-order factor model (M2b). In M2b, the relationship between insecurity at work and negative psychosocial work factor was not significant. This result reflected that the risk of subjects facing unemployment was very low in our study. It was accorded with an actual investigation in which the subjects were on-the-job medical staff (older than 40 years of age) whose careers have ‘reached a stable position’. The prevalence rate of SHS was 49.7% when using P50 as the cut-off value in our study. Although they had low insecurity at work, they were in high risk of SHS because of the high pressure during the inservice.

Clinicians as a kind of special population need to possess highly concentrated attention, sensible thinking, exquisite techniques and experiences. Moreover, lasting work and intensive labour intensity make them suffer more stress than other medical specialties. Previous research suggested that psychosocial stress may result from gendered processes,44 such as uneven family responsibilities, gender-specific harassment or discrimination, and unequal levels of poverty, which mainly limited the professional influence and development in women. In the current study, the difference in scores in negative psychosocial work stress factor between men and women was not significant. But women had lower scores in positive psychosocial work stress factor than men (P<0.05). In other words, women were more likely to suffer from stress-related psychosocial work factors than men. The gender gap in SHS in our study may be explained by the discriminatory impact of gender on susceptibility to stress-related psychosocial work factors, and the individuals with high level of psychosocial work stress were a high-risk group of SHS. Additionally, age was also a significant factor affecting stress levels.45 46 Meanwhile, individuals with higher levels of education report greater psychological demands.47 Similarly, we found that older male non-clinical medical staff with habits of physical exercise, smoking and drinking reported higher score of positive psychosocial work factors (less susceptible to work strain), while younger clinical doctors with a graduate degree or above who lack exercise, are on night shift and had longer man-hour (longer than 40 hours per week) reported higher scores in negative psychosocial work factor (more susceptible to work strain). In our study, psychosocial work stress factors, especially the negative side, was the factor influencing the risk of suboptimal health among medical staff. This relationship was also found in a population of executive employees.48

The results of this study provided some important insights for supervisors and managers in hospitals. Positive effects of work in the medical services should be maximised, and the consequences of work-related risk factors, such as demands and insecurity at work, in this important profession should be prevented. Moreover, Yan et al25 indicated that SHS is associated with cardiovascular risk factors and contributes to the development of cardiovascular disease. Therefore, it is less likely to be a question that the above measures are effective in preventing SHS and further reduce the risks of cardiovascular disease.

Conclusion

The modified second-order factor model was a suitable method to evaluate COPSOQ among medical staff. In this population, the negative and positive psychosocial work stress factors might be the risk and protective factors of suboptimal health, respectively. Negative psychosocial work stress was the most associated factor to predict suboptimal health.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

The authors would like to thank the staff at Xuanwu Hospital for their support. They also would like to thank all participants in this study for their voluntary participation.

Footnotes

Y-ZL and XC contributed equally.

Contributors: YY, lead and corresponding author, had full access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. YL wrote the majority of the manuscript, provided critical revisions to the manuscript and aided substantially in the preparation of the revised submission. XC participated in study design, performed data analysis and aided with interpretation. SM, LW and JZ collected and inputted questionnaires. All authors read and approved the final manuscript.

Funding: This work was supported by the National Science Foundation (81573214), the Beijing Municipal Nature Science Foundation (7162020), the Scientific Research Project of Beijing Municipal Educational Committee (KM201510025006), and the National Key Research and Development Plan (2016YFC0900603).

Competing interests: None declared.

Patient consent: Obtained.

Ethics approval: This study and associated protocols were conducted after approval by the Research Ethics Committee of Capital Medical University. All research participants consented to having their anonymous data included in the analyses reported herein.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: The data sets used and/or analysed during this study are available from the corresponding author on reasonable request.

References

  • 1.Chen SW, Wang PC, Hsin PL, et al. Job stress models, depressive disorders and work performance of engineers in microelectronics industry. Int Arch Occup Environ Health 2011;84:91–103. 10.1007/s00420-010-0538-y [DOI] [PubMed] [Google Scholar]
  • 2.Grynderup MB, Mors O, Hansen ÅM, et al. Work-unit measures of organisational justice and risk of depression--a 2-year cohort study. Occup Environ Med 2013;70:380–5. 10.1136/oemed-2012-101000 [DOI] [PubMed] [Google Scholar]
  • 3.Pranita A, Balsubramaniyan B, Phadke AV, et al. Association of occupational & prediabetes statuses with obesity in middle aged women. J Clin Diagn Res 2013;7:1311–3. 10.7860/JCDR/2013/5466.3134 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Du CL, Du CL, Cheng Y, et al. Workplace justice and psychosocial work hazards in association with return to work in male workers with coronary heart diseases: a prospective study. Int J Cardiol 2013;166:745–7. 10.1016/j.ijcard.2012.09.176 [DOI] [PubMed] [Google Scholar]
  • 5.Park J, Kim Y, Cheng Y, et al. A comparison of the recognition of overwork-related cardiovascular disease in Japan, Korea, and Taiwan. Ind Health 2012;50:17–23. 10.2486/indhealth.MS1317 [DOI] [PubMed] [Google Scholar]
  • 6.Belkic KL, Landsbergis PA, Schnall PL, et al. Is job strain a major source of cardiovascular disease risk? Scand J Work Environ Health 2004;30:85–128. 10.5271/sjweh.769 [DOI] [PubMed] [Google Scholar]
  • 7.Fishta A, Backé EM. Psychosocial stress at work and cardiovascular diseases: an overview of systematic reviews. Int Arch Occup Environ Health 2015;88:997–1014. 10.1007/s00420-015-1019-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kristensen TS, Hannerz H, Høgh A, et al. The copenhagen psychosocial questionnaire--a tool for the assessment and improvement of the psychosocial work environment. Scand J Work Environ Health 2005;31:438–49. 10.5271/sjweh.948 [DOI] [PubMed] [Google Scholar]
  • 9.Tsutsumi A, Kayaba K, Nagami M, et al. The effort-reward imbalance model: experience in japanese working population. J Occup Health 2002;44:398–407. 10.1539/joh.44.398 [DOI] [Google Scholar]
  • 10.Johnson S, Cooper C, Cartwright S, et al. The experience of work‐related stress across occupations. Journal of Managerial Psychology 2005;20:178–87. 10.1108/02683940510579803 [DOI] [Google Scholar]
  • 11.Wallace JE, Lemaire JB, Ghali WA. Physician wellness: a missing quality indicator. Lancet 2009;374:1714–21. 10.1016/S0140-6736(09)61424-0 [DOI] [PubMed] [Google Scholar]
  • 12.Alacacioglu A, Yavuzsen T, Dirioz M, et al. Burnout in nurses and physicians working at an oncology department. Psychooncology 2009;18:543–8. 10.1002/pon.1432 [DOI] [PubMed] [Google Scholar]
  • 13.Sehlen S, Vordermark D, Schäfer C, et al. Job stress and job satisfaction of physicians, radiographers, nurses and physicists working in radiotherapy: a multicenter analysis by the DEGRO Quality of Life Work Group. Radiat Oncol 2009;4:6–15. 10.1186/1748-717X-4-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Firth-Cozens J, Greenhalgh J. Doctors' perceptions of the links between stress and lowered clinical care. Soc Sci Med 1997;44:1017–22. 10.1016/S0277-9536(96)00227-4 [DOI] [PubMed] [Google Scholar]
  • 15.Fahrenkopf AM, Sectish TC, Barger LK, et al. Rates of medication errors among depressed and burnt out residents: prospective cohort study. BMJ 2008;336:488–91. 10.1136/bmj.39469.763218.BE [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Alexopoulos EC, Burdorf A, Kalokerinou A. Risk factors for musculoskeletal disorders among nursing personnel in Greek hospitals. Int Arch Occup Environ Health 2003;76:289–94. 10.1007/s00420-003-0442-9 [DOI] [PubMed] [Google Scholar]
  • 17.Rafnsdottir GL, Gunnarsdottir HK, Tomasson K. Work organization, well-being and health in geriatric care. Work 2004;22:49–55. [PubMed] [Google Scholar]
  • 18.Wang W, Russell A, Yan Y. Global Health Epidemiology Reference Group (GHERG). Traditional Chinese medicine and new concepts of predictive, preventive and personalized medicine in diagnosis and treatment of suboptimal health. Epma J 2014;5:4 10.1186/1878-5085-5-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Woolfolk RL, Allen LA. Treating somatization: a cognitive-behavioural approach. New York, NY: Guilford, 2007. [Google Scholar]
  • 20.Brown RJ. Introduction to the special issue on medically unexplained symptoms: background and future directions. Clin Psychol Rev 2007;27:769–80. 10.1016/j.cpr.2007.07.003 [DOI] [PubMed] [Google Scholar]
  • 21.Bi J, Huang Y, Xiao Y, et al. Association of lifestyle factors and suboptimal health status: a cross-sectional study of Chinese students. BMJ Open 2014;4:e005156 10.1136/bmjopen-2014-005156 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Wu S, Xuan Z, Li F, et al. Work-Recreation balance, health-promoting lifestyles and suboptimal health status in southern china: a cross-sectional study. Int J Environ Res Public Health 2016;13:339–54. 10.3390/ijerph13030339 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Yan YX, Liu YQ, Li M, et al. Development and evaluation of a questionnaire for measuring suboptimal health status in urban Chinese. J Epidemiol 2009;19:333–41. 10.2188/jea.JE20080086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Joustra ML, Janssens KA, Bültmann U, et al. Functional limitations in functional somatic syndromes and well-defined medical diseases. Results from the general population cohort LifeLines. J Psychosom Res 2015;79:94–9. 10.1016/j.jpsychores.2015.05.004 [DOI] [PubMed] [Google Scholar]
  • 25.Yan YX, Dong J, Liu YQ, et al. Association of suboptimal health status and cardiovascular risk factors in urban Chinese workers. J Urban Health 2012;89:329–38. 10.1007/s11524-011-9636-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Shang L, Liu P, Fan LB, et al. Psychometric properties of the chinese version of copenhagen psychosocial questionnaire [article in china]. J Environ Occup Med 2008;25:572–6. [Google Scholar]
  • 27.Meng SJ, Yan YX, Liu YQ, et al. Reliability and validity of copenhagen psychosocial questionnaire of chinese [article in China]. Chin Prev Med 2013;14:12–15. [Google Scholar]
  • 28.Babyak MA, Green SB. Confirmatory factor analysis: an introduction for psychosomatic medicine researchers. Psychosom Med 2010;72:587–97. 10.1097/PSY.0b013e3181de3f8a [DOI] [PubMed] [Google Scholar]
  • 29.Nunnally JC, Bernstein IH. Psychometric theory. 3rd Ed New York, London: McGraw-Hill, 1994. [Google Scholar]
  • 30.Kline RB. Measurement models and confirmatory factor analysis Principles and practice of structural equation modeling. 3rd Edn New York, London: The Guilford Press, 2005:230–51. [Google Scholar]
  • 31.Lt H, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling 1999;6:1–55. 10.1080/10705519909540118 [DOI] [Google Scholar]
  • 32.Tavakol S, Dennick R, Tavakol M. Psychometric properties and confirmatory factor analysis of the Jefferson Scale of Physician Empathy. BMC Med Educ 2011;11:54–61. 10.1186/1472-6920-11-54 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Barbara MB. Structural equation modeling with AMOS: basic concepts, applications, and programming. 2nd Ed New York: Taylor & Francis Group; London, 2001:108–11. [Google Scholar]
  • 34.Xu J, Feng LY, Luo R, et al. Assessment of the reliability and validity of the Sub-health Measurement Scale Version1.0. Nan Fang Yi Ke Da Xue Xue Bao 2011;31:33–8. [PubMed] [Google Scholar]
  • 35.Shirom A, Westman M, Melamed S. The effects of pay systems on blue-collar employees' emotional distress: the mediating effects of objective and subjective work monotony. Human Relations 1999;52:1077–97. 10.1177/001872679905200805 [DOI] [Google Scholar]
  • 36.Stansfeld SA, Fuhrer R, Shipley MJ, et al. Work characteristics predict psychiatric disorder: prospective results from the Whitehall II Study. Occup Environ Med 1999;56:302–7. 10.1136/oem.56.5.302 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Bültmann U, Kant IJ, Van den Brandt PA, et al. Psychosocial work characteristics as risk factors for the onset of fatigue and psychological distress: prospective results from the Maastricht Cohort Study. Psychol Med 2002;32:333–45. 10.1017/S0033291701005098 [DOI] [PubMed] [Google Scholar]
  • 38.Spurgeon A, Harrington JM, Cooper CL. Health and safety problems associated with long working hours: a review of the current position. Occup Environ Med 1997;54:367–75. 10.1136/oem.54.6.367 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Bohle P, Tilley AJ. The impact of night work on psychological well-being. Ergonomics 1989;32:1089–99. 10.1080/00140138908966876 [DOI] [PubMed] [Google Scholar]
  • 40.Virtanen P, Janlert U, Hammarström A. Exposure to temporary employment and job insecurity: a longitudinal study of the health effects. Occup Environ Med 2011;68:570–4. 10.1136/oem.2010.054890 [DOI] [PubMed] [Google Scholar]
  • 41.Fiabane E, Giorgi I, Musian D, et al. Occupational stress and job satisfaction of healthcare staff in rehabilitation units. Med Lav 2012;103:482–92. [PubMed] [Google Scholar]
  • 42.Weinberg A, Creed F. Stress and psychiatric disorder in healthcare professionals and hospital staff. Lancet 2000;355:533–7. 10.1016/S0140-6736(99)07366-3 [DOI] [PubMed] [Google Scholar]
  • 43.Nübling M, Stößel U, Hasselhorn HM, et al. Measuring psychological stress and strain at work - evaluation of the COPSOQ Questionnaire in Germany. Psychosoc Med 2006;3:1–14. [PMC free article] [PubMed] [Google Scholar]
  • 44.Springer KW, Mager Stellman J, Jordan-Young RM. Beyond a catalogue of differences: a theoretical frame and good practice guidelines for researching sex/gender in human health. Soc Sci Med 2012;74:1817–24. 10.1016/j.socscimed.2011.05.033 [DOI] [PubMed] [Google Scholar]
  • 45.Choi ES, Ha Y. Work-related stress and risk factors among Korean employees. J Korean Acad Nurs 2009;39:549–61. 10.4040/jkan.2009.39.4.549 [DOI] [PubMed] [Google Scholar]
  • 46.Purcell SR, Kutash M, Cobb S. The relationship between nurses' stress and nurse staffing factors in a hospital setting. J Nurs Manag 2011;19:714–20. 10.1111/j.1365-2834.2011.01262.x [DOI] [PubMed] [Google Scholar]
  • 47.Landsbergis PA, Grzywacz JG, LaMontagne AD, et al. Work organization, job insecurity, and occupational health disparities. Am J Ind Med 2014;57:495–515. 10.1002/ajim.22126 [DOI] [PubMed] [Google Scholar]
  • 48.Hsu SH, Chen DR, Cheng Y, et al. Association of psychosocial work hazards with depression and suboptimal health in executive employees. J Occup Environ Med 2016;58:728–36. 10.1097/JOM.0000000000000760 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary file 1

bmjopen-2017-018485supp001.pdf (433.9KB, pdf)

Supplementary file 2

bmjopen-2017-018485supp002.pdf (567.7KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES