The characteristics of non-respondents and respondents of a mental health survey among evacuees in a disaster: The Fukushima Health Management Survey

Abstract

The Fukushima Medical University conducted a mental health care program for evacuees after the Fukushima Daiichi nuclear power plant accident. However, the mental health status of non-respondents has not been considered for surveys using questionnaires. Therefore, the aim of this study was to clarify the characteristics of non-respondents and respondents. The target population of the survey (FY2011-2013) is people living in the nationally designated evacuation zone of Fukushima prefecture. Among these, the participants were 967 people (20 years or older). We examined factors that affected the difference between the groups of participants (i.e., non-respondents and respondents) using multivariate logistic regression analysis. Employment was higher in non-respondents (p=0.022) and they were also more socially isolated (p=0.047) when compared to respondents; non-respondents had a higher proportional risk of psychological distress compared to respondents (p<0.033). The results of the multivariate logistic regression analysis showed that, within the participants there was a significant association between employment status (OR=1.99, 95% confidence interval [CI]:1.12-3.51) and psychological distress (OR=2.17, 95% CI: 1.01-4.66). We found that non-respondents had a significantly higher proportion of psychological distress compared to the respondents. Although the non-respondents were the high-risk group, it is not possible to grasp the complexity of the situation by simply using questionnaire surveys. Therefore, in the future it is necessary to direct our efforts towards the mental health of non-respondents and respondents alike.

Introduction

The accident of the Fukushima Daiichi Nuclear Power Plant occurred after the Great East Japan Earthquake, which took place on March 11, 2011. Fukushima Medical University has, commissioned by Fukushima Prefecture, conducted “the Fukushima Health Management Survey”¹⁾ based on the health effects of radiation due to the accident. The Mental Health and Lifestyle Survey²⁾ (MHLS) is one of the detailed surveys included in the project. Participants are about 2.1 million people in 13 municipalities within the evacuation area. Based on the questionnaire survey respondents, we have been conducting telephone support for evacuees who need mental health support and lifestyle-related diseases support since FY2011. However, the response rate of the survey participants is low and the characteristics of the non-respondents are unclear. Previous studies have found that many evacuees face the problem of psychological distress after disaster^3,4). The same result was also obtained after this disaster⁵⁾. In addition, previous studies reported that non-respondents who did not return the questionnaire were worse off with regard to their subjective health, had more negative health behaviors, and had higher age-adjusted mortality rates than respondents^6-8). Therefore, the aim of this study was to clarify the characteristics of non-respondents of MHLS (FY2011-FY2013).

Materials and Methods

Participants

The target population of the MHLS (20 years or older) for each year was 173,537 (FY2011), 174,337 (FY2012), and 177,146 (FY2013) in the nationally designated evacuation zone including the 13 municipalities of Fukushima prefecture. In addition, the survey response rate was 41.0%, 30.3%, and 25.4%, respectively. Among these, 967 participants who had lived in two municipalities in Fukushima prefecture and evacuated inside or outside Fukushima prefecture, were randomly selected. We performed 30 to 60-minute face-to-face interviews with questionnaires on basic attributes, socioeconomic status, and mental health of 387 people (40.0% participation) from October 2014 to January 2015. Consequently, excluding the 74 participants whose response history of MHLS was unknown in this study, a total of 313 people (32.4%) were included in the study analyses.

In this study, we examined the characteristics of non-respondents in the MHLS conducted in FY2013. Regarding the representativeness, we found that the K6, PCL-S, and LSNS-6 scores among the participants of this study were similar to those of the survey population in FY2013 (K6: 5.8 vs. 5.2, PCL-S: 30.7 vs. 30.4, LSNS-6: 12.3 vs. 13.1)⁹⁾. However, some results were not similar; the participants in this study were older (58.0 vs. 54.7 years old, p=0.002), had a higher proportion of women (women: 58.1% vs. 51.2%, p=0.014).

Data collection

We divided the remaining 313 people into two groups: non-respondents and respondents. Respondents refer to the people who answered the survey one or more times over three years. We collected the following participant characteristics data for the analysis of both groups: age, gender, living place, family structure, educational attainment, employment status, financial circumstances, and the Lubben Social Network Scale-6 (LSNS-6)^10,11). If a participant scored <12 on the LSNS-6, he/she was defined as socially isolated^10,11).

The psychological distress of evacuees was measured using the Japanese version¹²⁾ of the 6-item Kessler Psychological Distress Scale (K6)¹³⁾. The K6 is a self-administered questionnaire consisting of six questions that evaluate depressive mood and anxiety over the preceding 4 weeks on a 5-point response scale from 0=None of the time to 4=All of the time (range of the scale score: 0-24). In this study, we used a cut-off score of 13 to identify cases with psychological distress, as was used in previous studies of community populations in Japan^14,15).

Posttraumatic stress disorder (PTSD) of evacuees was measured using the PTSD Checklist Stressor-Specific Version (PCL-S)^16-18). The PCL-S is a self-administered questionnaire consisting of 17 questions that evaluate PTSD based on the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV)¹⁹⁾. Participants indicated whether they were bothered by symptoms due to a traumatic event in the past month on a 1-5 Likert scale (1=Not at all to 5=Extremely), with the sum of all responses generating a total score ranging from 17-85. The score providing the optimal screening cut-off point has been shown to be ≥44^17,20,21), including in this study.

Statistical analysis

Statistical comparisons between groups (non-respondents and respondents) were conducted using a chi-square test, t-test, and Mann-Whitney U test. Moreover, a multivariate logistic regression analysis was performed to determine differences between groups. The multivariate logistic regression analysis was used only for K6 scores, because a strong correlation was found between the K6 and PCL-S scores¹⁸⁾ in a previous study.

The scores were expressed as means and standard deviations (SD), and p<0.05 was regarded to indicate statistical significance. All statistical procedures were performed using SPSS for Windows (version 21; Armonk, NY:IBM).

Ethical considerations

This study was approved by the ethics review committee of Fukushima Medical University (No. 2086). The study objective was explained to all participants and all were advised that 1) participation in the study was completely voluntary, 2) they could withdraw from the study at any time, and 3) if they chose to withdraw, they would not be disadvantaged in any way.

Results

The characteristics of the participants in the two groups (non-respondents and respondents) are shown in Table 1. Non-respondents were more likely to be employed (p=0.022) and socially isolated (p=0.047) compared to respondents.

Fig. 1.

Selection of participants

Table 1.

Characteristics of participants

	total n=313		non-respondents n=144		respondents n=169		p
Age
Mean (SD)	58.0±18.7		57.7±20.5		58.9±16.9		0.588
Gender
Men	131	(41.9)	66	(45.8)	65	(38.5)	0.207
Women	182	(58.1)	78	(54.2)	104	(61.5)
Living place
Inside Fukushima prefecture	145	(46.3)	61	(42.4)	84	(49.7)	0.212
Outside Fukushima prefecture	168	(53.7)	83	(57.6)	85	(50.3)
Family structure
Single and Spouses	117	(40.1)	47	(35.9)	70	(43.5)	0.230
Other	175	(59.9)	84	(64.1)	91	(56.5)
Educational attainment
Elementary or junior high school	60	(20.5)	32	(24.2)	28	(17.5)	0.190
High school or more	232	(79.5)	100	(75.8)	132	(82.5)
Employment status
Unemployed	179	(61.1)	71	(53.8)	108	(67.1)	0.022
Employed	114	(38.9)	61	(46.2)	53	(32.9)
Financial circumstances
Good	200	(67.8)	90	(67.2)	110	(68.3)	0.901
Bad	95	(32.2)	44	(32.8)	51	(31.7)
LSNS-6
Median (25%-75%)	12.0	(7.0-17.8)	10.0	(6.0-17.3)	13.0	(8.0-18.0)	0.125
<12	144	(48.6)	74	(55.2)	70	(43.2)	0.047
≥12	152	(51.4)	60	(44.8)	92	(56.8)

n (%) Chi-square test, t-test, and Mann-Whitney U test were used.

Note: Socially isolated was measured using LSNS-6 scale; a participant with a LSNS-6 score of <12 was defined as socially isolated.

Table 2 shows the results of the chi-square tests and Mann-Whitney U test for associations between non-response and mental health (psychological distress and PTSD). The K6 and PCL-S scores of non-respondents had a lower median compared to those of respondents. On the other hand, non-respondents had a higher proportional risk of psychological distress and PTSD compared to respondents.

Table 2.

The number (rate) of non-respondents corresponding to psychological distress and posttraumatic stress disorder

	total n=313		non-respondents n=144		respondents n=169		p
K6
Median (25%-75%)	4.0	(1.0-9.0)	4.0	(0-10.0)	5.0	(1.0-10.0)	0.941
<13	254	(87.3)	107	(82.3)	147	(91.3)	0.033
≥13	37	(12.7)	23	(17.7)	14	(8.7)
PCL
Median (25%-75%)	27.0	(20.0-36.5)	25.0	(19.0-36.8)	29.0	(20.0-37.0)	0.256
<44	243	(84.1)	101	(78.9)	142	(88.2)	0.036
≥44	46	(15.9)	27	(21.1)	19	(11.8)

n (%) Chi-square test and Mann-Whitney U test were used.

Note: Psychological distress was measured using K6 scale, a participant with a K6 score of ≥13 was defined as having psychological distress. Posttraumatic stress disorder was measured using PCL scale; a participant with a PCL score of ≥44 was defined as having posttraumatic stress disorder.

Table 3 shows the results of the multivariate logistic regression analysis of factors that affected the difference of the participants of the self-administered questionnaire. Non-respondents in the survey had a significant association with employment (OR=1.99, 95% confidence interval [CI]:1.12-3.51) and psychological distress (OR=2.17, 95% CI: 1.01-4.66).

Table 3.

Factors associated with non-response to the self-administered questionnaire in the multivariate analysis

Variable		Total
		OR	95% CI
Age		1.00	0.99-1.02
Gender	(Ref. Men)	0.68	0.40-1.14
Living place	(Ref. Inside Fukushima prefecture)	1.69	0.99-2.86
Family structure	(Ref. Single and Spouses)	1.63	0.97-2.73
Educational attainment	(Ref. Elementary or junior high school)	0.58	0.30-1.10
Employment status	(Ref. Unemployed)	1.99	1.12-3.51	*
Financial circumstances	(Ref. Good)	1.26	0.72-2.23
LSNS-6 <12	(Ref. ≥12)	0.80	0.47-1.35
K6 ≥13	(Ref. <13)	2.17	1.01-4.66	*

Multivariate logistic regression was used to calculate ORs and 95% confidence intervals (95% CI) after controlling simultaneously for independent variables. N=418

*p<0.05

Note: K6 score ≥13 was defined as psychological distress.

Table 4 shows the results of why respondents took part in the study and the suggested improvement plan regarding the submission of the questionnaire. “For contribution (48.5%)” was the most popular reason as to why respondents took part in the study, followed by “to tell my thoughts (45.6%)” and “to take part in a study at the Fukushima Medical University (40.2%).” On the other hand, “having no time to spare (46.7%)” was the highest-ranking reason for non-respondents, followed by “no need to do this for myself (26.8%),” “asking too many questions (16.0%),” and “stress brought about by the answers (16.0%).” Furthermore, “to reduce the number of questions (45.9%)” was highest in the suggestions for improving the response rate. Followed by “the study should be repeated every few years (18.7%)” and “to be able to participate in the study at the same time as the health checkup (16.0%)” were also of high importance to the participants.

Table 4.

Reasons for answering the questionnaire and proposal for improvement in the Mental Health and Lifestyle Survey

	total		non-respondents		respondents
					n=169
The reason for response
For contribution	-		-		82	(48.5)
To tell my thoughts.	-		-		77	(45.6)
To take part in a study at the Fukushima Medical University	-		-		68	(40.2)
To always participate	-		-		33	(19.5)
To have time to spare.	-		-		23	(13.6)
Because a result report arrived	-		-		7	(4.1)
Because it was recommended to be submitted to the family	-		-		4	(2.4)
Other	-		-		13	(7.7)
	n=257		n=144		n=113*
The reason for non-response
To have no time to spare	120	(46.7)	59	(41.0)	61	(54.0)
No need for myself	69	(26.8)	38	(26.4)	31	(27.4)
For asking too many questions.	41	(16.0)	20	(13.9)	21	(18.6)
Stress of answers	41	(16.0)	31	(21.5)	10	(8.8)
Do not remember	25	(9.7)	19	(13.2)	6	(5.3)
Do not understand the purpose of the study.	23	(8.9)	17	(11.8)	6	(5.3)
Because it was a study at the Fukushima Medical University	9	(3.5)	9	(6.3)	0	(0.0)
Because a result report did not arrive	3	(1.2)	1	(0.7)	2	(1.8)
Other	59	(23.0)	25	(17.4)	9	(8.0)
	n=257		n=144		n=113*
Suggestions for improving the response rate
To reduce the number of questions.	118	(45.9)	44	(30.6)	74	(65.5)
The study should be repeated every few years	48	(18.7)	22	(15.3)	26	(23.0)
To be able to participate in the study at the same time as the health checkup	41	(16.0)	29	(20.1)	12	(10.6)
To make a result report arrive	27	(10.5)	17	(11.8)	10	(8.8)
Do not submit so troublesome	9	(3.5)	8	(5.6)	1	(0.9)
Interview study	7	(2.7)	7	(4.9)	0	(0.0)
Improvement of the question items	5	(1.9)	4	(2.8)	1	(0.9)
To receive benefits.	3	(1.2)	3	(2.1)	0	(0.0)
Other	36	(14.0)	31	(21.5)	5	(4.4)

n (%)

*We excluded those who participated all three times.

Discussion

In this study, we examined the characteristics of non-respondents in a mental health survey after a disaster. We found that participants (non-respondents and respondents) were significantly associated with employment and the prevalence of high-risk psychological distress. Furthermore, non-respondents were significantly more employed and socially isolated than respondents. Further, non-respondents were more often male, younger, and with a lower educational history. Although these findings were not statistically significant, similar trends have been observed in other studies^7,22). In addition, regarding representativeness of the MHLS in this research, we believe that the target of this survey is a representative group.

Non-respondents had a significantly higher proportion of psychological distress compared to respondents. As there was no difference in the median between these groups, we considered that people with psychological distress were more likely to become non-respondents. Une et al.²³⁾ found differences in non-respondents and respondents in a population-based mail survey: non-respondents had higher age-adjusted mortality rates, cancer, heart disease, and stroke compared to respondents. In addition, non-respondents have been characterized as having poorer mental health than respondents²⁴⁾. From the above, it is necessary to consider efforts that ensure the mental and physical health of evacuees.

In addition, non-respondents had higher ratios of social isolation. Kawachi et al.²⁵⁾ stated that social support may act on the pathway between the stressful event and mental illness. Recently, Goodwin et al.²⁶⁾ reported psychological distress and social support among refugees in the Great East Japan Earthquake and Tsunami. As people who are socially isolated have poorer mental health²⁷⁾, socially isolated evacuees, in particular, need to be considered and require support.

Furthermore, non-respondents had a higher proportion of employment than respondents. This may have reflected the fact that many employees do not have time to spare when it comes to responding to questionnaires²⁸⁾, because the proportion of non-respondents was high among men and young people⁷⁾. However, because non-respondents are potentially at higher risk for psychological distress, we need to encourage them to take part in surveys with an aim to improve the situation of non-respondents.

One of the reasons non-respondents did not submit the questionnaire was that they “do not have time to spare.” As the proportion of employment was higher in non-respondents, not enough time is considered to be one of the factors that had an effect on response rate. The proportion of non-respondents was significantly higher than that of previous research²⁹⁾. As there may be people at high risk for mental health problems among non-respondents, it would be useful to employ measures that encourage response. We consider it important to connect to these non-respondents over the telephone to support these evacuees. In addition, Edwards et al.³⁰⁾ stated that response was more likely when short questionnaires were used, when a monetary incentive was used, and when follow-up contact was conducted. Similarly, in terms of suggestions for improving the response rate in our study, “to reduce the number of questions” obtained the highest score. For this reason, we decided to change the study to use a short questionnaire that was easy to answer in FY2014 in the MHLS. Further, to increase the response rate, we took the approach that if participants did not return the results of the questionnaire, we sent a reminder, and called participants to complete the survey through cooperation with municipalities. In a post-disaster survey, researchers recognize that participants want “to contribute” or “to share their thoughts.” However, we also need to consider research methods that do not create stress among evacuees.

There are several limitations to this study. First, the study participants were evacuees after a disaster rather than community residents. In addition, the evacuation area was quite broad. This limits the generalizability of our findings. Second, the use of the K6 scale is inferior to a clinician administered diagnostic tool. However, measures of psychological distress such as the K6 have been used for evacuees who experience continuous living stress. Third, the response rate of 40.0% was not high. In addition, there is also a possibility of a lack of representativeness of the participants in the FY2013 survey; it is possible that selection bias occurred. Despite these limitations, the present study is aimed at clarifying the characteristics of non-respondents in a post-disaster study. We hope that our findings will contribute to the study of post-disaster.

In this study, we examined the characteristics of non-respondents and respondents by age, gender, socioeconomic status, and psychological distress. We found that non-respondents were significantly more likely to suffer from psychological distress compared to respondents. Although the non-respondents were a high-risk group, it is not possible to grasp the full situation simply by using questionnaire surveys. In addition, we found non-respondents felt stress about answering the questionnaire compared to respondents. Therefore, efforts are necessary to provide mental health care after a disaster while considering the characteristics of non-respondents.

Acknowledgements

This study was supported by the Grant-in-Aid for Scientific Research (C) (No. 26510011), and was conducted as part of Fukushima prefecture’s post-disaster recovery plan.

Conflict of interest

The authors declare no conflicts of interest.