＜Editors’ Choice＞ : Developing online lectures using text mining reduces health workers’ anxiety in non-epicenter areas of COVID-19

ABSTRACT

COVID-19 is indirectly associated with various mental disorders such as anxiety, insomnia, and depression, and healthcare professionals who treat COVID-19 patients are particularly prone to severe anxiety. However, neither the anxiety of healthcare workers in non-epicenter areas nor the effects of knowledge support have been examined thus far. Participants were 458 staff working at the Toyota Regional Medical Center who completed a preliminary questionnaire of their knowledge and anxiety regarding COVID-19. Based on text mining of the questionnaire responses, participants were offered an online lecture. The effect of the lecture was analyzed using a pre- and post-lecture rating of anxiety and knowledge confidence, and quantitative text mining. The response rates were 45.6% pre- and 62.9% post-lecture. Open-ended responses regarding anxiety and knowledge were classified into seven clusters using a co-occurrence network. Before the lecture, 28.2%, 27.2%, and 20.3% of participants were interested in and anxious about “infection prevention and our hospital’s response,” “infection and impact on myself, family, and neighbors,” and “general knowledge of COVID-19,” respectively. As a result of the lecture, Likert-scale ratings for anxiety of COVID-19 decreased significantly and knowledge confidence increased significantly. These changes were confirmed by analyses of open-ended responses about anxiety, lifestyle changes, and knowledge. Positive changes were strongly linked to the topics focused on in the lecture, especially infection prevention. The anxieties about COVID-19 of healthcare workers in non-epicenter areas can be effectively reduced through questionnaire surveys and online lectures using text mining.

INTRODUCTION

In Japan, mortality from COVID-19 has been low since the beginning of the COVID-19 epidemic compared to the corresponding rates in other developed countries.¹ Toyota Regional Medical Center to which the authors belong is a small-scale hospital with 150 beds located in Toyota City, Aichi Prefecture, Japan, that accepts suspected COVID-19 patients as outpatients and performs PCR (Polymerase Chain Reaction) tests. The hospital had experienced no confirmed cases or hospitalizations related to COVID-19 throughout this study. Despite this situation, even staff who do not directly treat COVID-19 patients often report anxiety due to concerns about the potential impact on their own and their families’ health as a result of being in close proximity with COVID-19 patients.

COVID-19 is indirectly associated with various disorders such as anxiety, insomnia, depression, and post traumatic stress disorder.^2-6 Moreover, doctors and nurses treating COVID-19 are at a high risk of infection and are thus vulnerable to severe anxiety , and even physical symptoms can appear as a result.^7-10 However, it has been reported that other nurses and the general public have a higher level of anxiety than nurses on the frontline,^11,12 which can be attributed to the difference in their knowledge of COVID-19. Countermeasures for COVID-19-related anxiety among healthcare professionals include being in an environment that provides knowledge support, consultation with an expert, training in concrete measures to deal with COVID-19, and improving accessibility to formal psychological support.^12,13

Currently under consideration for healthcare professionals in COVID-19 epidemic areas are methods for providing mental healthcare such as cognitive behavior therapy,^14-16 as well as targeted and specific social support to improve their anxiety, self-efficacy, and sleep quality.¹⁷ However, no evaluation has been conducted on the extent to which anxiety can be reduced by providing appropriate knowledge to medical staff in non-epicenter areas. Therefore, we delivered the lecture on COVID-19 countermeasures to reduce anxiety by providing correct knowledge. We delivered the lecture using an online conference system because the lecture should be delivered without contact directly for prevent infection, and since previous research suggests that online support for mental healthcare is useful in eliminating anxiety.^16,18

The content of the lecture was decided using text mining. The subjects of the lecture were medical professionals, but staff with poor medical knowledge such as clerks were also targeted, so it was assumed that there are differences in the areas of interest and the level of knowledge that staff wants to know. Therefore, we conducted a pre-questionnaire to all staff and selected the content of the lecture. We developed the lecture content using quantitative text mining. Quantitative text mining can be used to analyze a large amount of data¹⁹ and perform multivariate analysis that cannot be done using ordinary qualitative research methods.²⁰ Moreover, quantitative text mining is highly suitable for the analysis of questionnaire surveys that are difficult to quantify.

The main outcome of this research is to evaluate whether online lectures developed by text mining reduces health workers’ anxiety in non-epicenter areas of COVID-19. We analyzed the results using not only a quantitative but also a qualitative evaluation of pre- and post-lecture questionnaires because we wanted to evaluate the content that cannot be expressed quantitatively, such as what caused the anxiety to be resolved, as an educational effect. Text mining is also a method that can evaluate the educational effect qualitatively and objectively,^20,21 and we decided to use it for post evaluation.

METHODS

Study Design

This study is an unpaired before-after study. A qualitative evaluation is added to the quantitative evaluation to ascertain whether the before and after changes are due to the intervention. The qualitative evaluation was converted into quantitative data by text mining to ensure reproducibility.

Participants and Procedure

This study was conducted with 458 staff members who were working at Toyota Regional Medical Center as of April 27, 2020. A pre-lecture survey was conducted on April 27 and 28 that included open-ended interview questions concerning participants’ knowledge and anxiety regarding COVID-19 (Supporting Information 1). We analyzed the responses using quantitative text mining to determine the instructional content for the lecture. We distributed to staff information regarding the lecture content and the times that the lecture would be given. A synchronous virtual lecture of one hour length was given on April 30 and May 1. The staff who could not watch the lecture on the same day watched the recorded lecture between May 2 and 15. The post-lecture questionnaire survey was conducted from May 15 to 22 (Supporting Information 2). The pre- and post-lecture questionnaire surveys were conducted anonymously. The purpose of this study was explained in the questionnaire explanation form, and the submission of the completed questionnaire was considered to indicate consent to participate. Because we conducted analyses for each item using all the valid responses, we included all the participants with response to at least one item. This study was approved by the Ethics Review Committee of Toyota Regional Medical Center and was carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki).

Measurements

Participants could respond to the questionnaire survey either through a hardcopy form or using Google Forms. As descriptive statistical data, participants’ type of profession, working department, and age groups were collected. In the post-lecture questionnaire, we additionally collected the lecture viewing method (online conference or watching video later), whether participants worked with COVID-19 suspected cases, and gender. Regarding anxiety, we asked, “How anxious do you feel about the novel coronavirus?” using an 11-point Likert scale on both the pre- and post-lecture questionnaires. Moreover, we asked the open-ended questions, “What are you most worried about regarding the novel coronavirus today?”, “How did your anxiety about the novel coronavirus change after this lecture?”, and “How will your behavior and life change or how have they changed after hearing this lecture?” in the post-lecture questionnaire. Our knowledge questions included, “How much do you know about the novel coronavirus?” (knowledge confidence) and an 11-point Likert scale was used for the responses. We asked the open-ended questions, “What do you want to know about the novel coronavirus?” in the pre-lecture questionnaire. We asked, “What did you gain/not gain about the novel coronavirus?” and “Please ask any questions that this lecture has not covered, and state what further information you require” in the post-lecture questionnaire.

Statistical Analyses

We performed a χ² test to examine differences between pre and post questionnaire groups by type of profession, department, and age groups. The median (MD) and interquartile ranges (IQR) were calculated for the pre- and post-lecture ratings of anxiety and knowledge confidence and a Mann-Whitney test was used to examine changes in the levels of anxiety and knowledge confidence. The questionnaire was completed anonymously because the questionnaire may have contained sensitive information that they did not want to identify individuals. This resulted in a pre- and post-evaluation with no response, and we couldn’t use the Wilcoxon test. A sub-analysis was performed for ward nurses because the number of ward nurses increased significantly in the post-lecture questionnaire. We set cut off p value of 0.05 for statistical significance for all analyses. The statistical analysis was conducted using R Commander ver. 2.6-2 software.

A qualitative evaluation of open-ended answers regarding anxiety and knowledge was performed using the free software KH Coder ver. 3. Beta.01a.^22-24 In this study, it was necessary to perform an exploratory analysis of a large number of open-ended answers in a short time, search for the co-occurrence of relationships through the simultaneous analysis of open-ended answers and clusters, and perform cross-tabulation. Thus, we selected KH Coder since it can perform text mining analysis and handle qualitative data. KH Coder calculates word frequencies and visualizes the co-occurrence network between each word using R.²⁵ Chasen,²⁶ a Japanese morphological analysis tool owned by the Nara Institute of Science and Technology, Matsumoto Lab, was used as a program to support R.

Developing Online Lectures Using Text Mining

We break down free descriptions that appeared frequently in the open-ended responses staff provided in the pre-lecture questionnaire regarding what they want to know and their anxiety regarding COVID-19 into words using KH Coder. Based on the results, we created a co-occurrence network (Figure 1). The minimum number of occurrences of a word to be used for drawing a co-occurrence network was set to at least 5 times, which generally contains words characteristic of this theme. Co-occurrence relations were drawn using the Jaccard coefficient of the top 60. The Jaccard coefficient is a coefficient that measures the similarity of a set. We used the Jaccard coefficient to determine the strength of the co-occurrence relationship because the frequency of the same word appearing in a document was small. To determine how many of the Jaccard coefficients to use in the analysis, we tried several conditions and looked at the actual drawn diagrams to determine the best rank for clustering. To determine how many of the top Jaccard coefficients to include in the analysis, we actually drew the co-occurrence network under several conditions and decided on the one that seemed best for clustering. Clustering was done automatically by KH Coder based on the Jaccard coefficients and their relationships (ie, color coding in the figure), but for co-occurrence relationships and clustering where the author had questions, the clustering was adjusted by directly checking the usage of each word in the original sentence. There are seven named clusters.

Fig. 1.

Co-occurrence network created with the words from the questionnaire regarding knowledge and anxiety

The minimum number of occurrences is five, the total number of words is 89, and the co-occurrence relationships are created using the top 60 Jaccard coefficients. The size of the circle in the figure indicates the frequency of appearance of the word, and strong co-occurrence relationships between words are connected by lines (the dotted line shows that KH Coder judged it to be another cluster, but the co-occurrence relationship is as strong as the solid line.) The boundary of each cluster is shown by a thick blue line. The subgroups are the result of automatic KH coder clustering.

To analyze by cross-tabulation the knowledge and anxiety of the participants, coding rules were created from the words unique to each cluster (Table 1). Coding rules were first created from the words forming each cluster of the co-occurrence network, excluding universal and featureless words (eg when, know). Frequent words not appearing in the co-occurrence network that were considered to be characteristic of the cluster were included in the coding rules after checking their usage in the original text (eg nosocomial infection → Infection prevention and our hospital’s response).

Table 1.

Correspondence between cluster names and coding rules

Cluster name	Coding rule
General knowledge of COVID-19	corona or virus or information or correct or knowledge or disease or severe illness or pneumonia or risk or symptom
Infection prevention and our hospital’s response	patient or gown or mask or N95 or splash or contact or fever or outpatient or protection or countermeasure or response or material or shortage or hospital or occurrence or medical care or center or staff or nosocomial infection or prevention or method or disinfection or pathway or concentrated or nosocomial
Examination	examination or PCR or positive or negative or antibody
Treatments and vaccines	treatment or vaccine or medicine
Perspective	convergence or finished or time of year or spread
Infection and impact on myself, family, and neighbors	myself or family or child or neighbors or surrounding or influence or living
Impact on the workplace	workplace or bother or work

The cross-tabulation of each cluster according to the coding rules is in Table 2. Among worried staff members, “infection and impact on myself, family, and neighbours” was the most frequently mentioned concern (46.9%, 98/209), followed by “infection prevention and our hospital’s response” (28.7%, 60/209) and “general knowledge of COVID-19” (25.0%, 52/209). Moreover, the topic that staff most wished to know about was “infection prevention and our hospital’s response” (28.7%, 60/209), followed by “treatment and vaccines” (22.0%, 46/209), “general knowledge of COVID-19” (15.8%, 33/209), and “perspective” (14.4%, 30/209), indicating their interest in receiving more specific content. Based on the results, the lecture focused on “general knowledge of COVID-19” and “infection prevention and our hospital’s response”. The topic of “ infection and impact on myself, family, and neighbours “ was of concern mainly because of anxiety, so the lecture covered not only itself but also how to deal with anxiety in general. “Examinations,” “treatments and vaccines,” “perspective,” and “impact on the workplace” covered a relatively small percentage of the lecture minutes. To determine whether the lecture was in line with the proportions of each cluster, the number of seconds devoted to the content of each cluster was recorded from the video of the lecture taken at a later, and the proportions were calculated. As a result, the percentage of responses in each cluster was well proportional to the number of seconds in the lecture.

Table 2.

Comparison of pre-lecture responses by cluster with the content of the lecture

Cluster Name	Anxiety (%)	Wanted Knowledge (%)	Total (%)1	Lecture contents (%)2
General knowledge of COVID-19	25.0	15.8	20.3	22.4
Infection prevention and our hospital’s response	28.7	27.8	28.2	24.7
Examination	3.8	9.1	6.5	6.9
Treatments and vaccines	3.4	22.0	12.7	3.0
Perspective	9.1	14.4	11.7	3.2
Infection and impact on myself, family, and neighbors	46.9	7.6	27.2	10.5
Impact on the workplace	8.1	1.0	4.6	4.4
Anxiety in general				10.4
Others			22.3	14.6

¹The total percentage is not 100% because all cluster names are assigned to the target if KH Coder determines that the target belongs to multiple clusters.

²The percentage of lecture contents is obtained by calculating the total time spent talking about each content as a percentage of the total time in minutes.

RESULTS

Sample Characteristics

The total number of participants was 458, of which 435 (95%) were included in the study. The remaining 23 people were not eligible for the survey because they were unable to watch the video within the specified deadline. The pre-lecture questionnaire was collected from 209 people, while the post-lecture questionnaire was collected from 288 people, and the collection rates were 45.6% and 62.9%, respectively. The most frequently reported type of profession in both surveys was that of a nurse, followed by that of a therapist (Table 3). The number of nurses among respondents was 44 (21.1%) for the pre-lecture questionnaire and 144 (50.0%) for the post-lecture questionnaire. The collection rate of nurses increased significantly from the pre- to post-lecture questionnaire. The highest collection rate increase was inpatient nurses, followed by outpatient nurses and home-visit nurses. The number of ward nurses among respondents increased from 56 (26.8%) for the pre-survey questionnaire to 134 (46.5%) for the post-survey questionnaire. Additionally, the ratio of ward nurses to total nurses increased significantly from 13 (29.5%) to 76 (52.8%). The ratio of outpatient and home nurses slightly decreased from 17 (38.6%) to 46 (31.9%) and from 6 (13.6%) to 15 (10.4%), respectively. The ratio of the age groups (ie, 20s to 50s) did not change significantly before and after the online lecture. Responses to three items (gender, viewing method, and contact with suspected COVID-19 cases) were only received from the post-lecture questionnaire. The percentage of female participants was 207 (71.9%). The lecture was attended by 104 people (36.1%) using an online conference system in real-time, while 184 people (63.9%) watched the recorded video. 53 participants (18.4%) had responded to at least one suspected COVID-19 case.

Table 3.

Comparison of participants’ general characteristics between pre- and post-lecture questionnaires

	Pre-questionnaire (n=209)	Post-questionnaire (n=288)	P value 1
Type of profession, %			<.001
Doctor	20 (9.6)	12 (4.2)
Nurse	44 (21.1)	144 (50.0)
Therapist	33 (15.8)	36 (12.5)
Medical technician	12 (5.7)	4 (1.4)
Caregiver	10 (4.8)	10 (3.5)
Clerk	16 (7.7)	30 (10.4)
Others2	13 (6.2)	7 (2.4)
No answer	61 (29.2)	45 (15.6)
Department, %			<.001
Ward	56 (26.8)	134 (46.5)
Outpatient	35 (16.7)	55 (19.1)
Home healthcare	35 (16.7)	34 (11.8)
Others3	9 (4.3)	8 (2.8)
No answer	74 (35.4)	57 (19.8)
Age groups, %			0.23
20s	34 (16.3)	55 (19.1)
30s	32 (15.3)	39 (13.5)
40s	48 (23.0)	78 (27.1)
50s	39 (18.7)	60 (20.8)
60s	8 (3.8)	14 (4.9)
70s	0 (0.0)	1 (0.3)
No answer	48 (23.0)	41 (14.2)
Gender, %			-
Male	-	43 (15.0)
Female	-	207 (71.9)
No answer	-	38 (13.2)
Video viewing, %			-
Online conference	-	104 (36.1)
Watch later	-	184 (63.9)
Contact with suspected COVID-19			-
Yes	-	53 (18.4)
No	-	223 (77.4)
No answer	-	12 (4.2)

¹Chi-square test

²Others include pharmacist, public health nurse, nutritionist, and details unknown.

³Others mainly include clerks working across the ward, outpatient, and home healthcare sectors.

Quantitative Analysis of Anxiety and Knowledge confidence

The quantitative analysis of change in anxiety and knowledge confidence before and after the lecture were performed. The anxiety before the lecture (MD, IQR: 7, 5–8) was significantly reduced after the lecture (MD, IQR: 6, 5–8) (p <.001). The knowledge confidence was (MD, IQR: 4, 3–5) before the lecture, which increased significantly to (MD, IQR: 5, 5–6) after the lecture (p <.001). Anxiety among ward nurses remained the same at (MD, IQR: 7, 5–8) before and after the lecture. Ward nurses’ knowledge confidence was (MD, IQR: 4, 3–5) before the lecture, which increased significantly to (MD, IQR: 5, 3–6) after the lecture. Anxiety in the post-questionnaire was higher among ward nurses than in the entire sample, while knowledge confidence was the same.

Qualitative Analysis of Post-Lecture Anxiety and Lifestyle Changes

A co-occurrence network was created using participants’ open-ended responses regarding changes in anxiety level and lifestyle changes in the post-lecture questionnaire using the same method as was previously described (Figures 2 and 3). The co-occurrence network of anxiety changes had a minimum appearance frequency of 10, and the co-occurrence relationship included the top 30 Jaccard coefficients in the drawing range. The minimum frequency of lifestyle changes was five, and the co-occurrence relationship included the top 40 Jaccard coefficients in the drawing range. In the co-occurrence network, change in anxiety was divided into “no change” (21.7%, 84/388) and “reduced anxiety” (31.4%, 122/388) (Figure 2). “Reduced anxiety” is linked to the three clusters of “self-behavior and infection,” “infection of others,” and “acquiring knowledge and countermeasures.” A strong correlation was observed with the contents that were dealt with intensively in the lecture, especially “infection prevention and our hospital’s response.” In terms of changes in lifestyle, changes in specific actions (eg “enhanced infection prevention” and “changes in family life”), changes in motivation for future actions (eg “changes of mind”), and changes in psychology that reduced stress (eg “reducing anxiety through appropriate countermeasure”) formed a cluster (Figure 3). Although the “reduction of anxiety by an appropriate response” was low at 5.7% (22/388), this reduction was directly life-changing. Conversely, there was “no change” in lifestyles for some clusters. Within the “no change” result, there were two clusters: one for no change, and a positive cluster, composed of those who had already taken measures against COVID-19 and who stated that it would not change any further.

Fig. 2.

Co-occurrence network created with the words from the questionnaire regarding post-lecture anxiety

The minimum number of occurrences is 10, the total number of words is 23, and co-occurrence relationships are created with the top 30 Jaccard coefficients.

*The parts of speech are different in Japanese.

Fig. 3.

Co-occurrence network created with the words from the questionnaire regarding post-lecture lifestyle change

The minimum number of occurrences is five, the total number of words is 41, and co-occurrence relationships are created with the top 40 Jaccard coefficients.

Cross-tabulation of Knowledge Pre/Post-Lecture

Open-ended answers about knowledge of COVID-19 were examined through a comparison of pre- and post-lecture questionnaires. We created a co-occurrence network by gathering open-ended answers in advance (ie, what I wanted to know) and afterward (ie, what I gained or didn’t gain), and created clusters and coding rules using the same method as was previously described. Based on the result, the derived cluster included “risk of infection,” “characteristics of COVID-19,” “infection prevention,” “perspective,” “treatments and vaccines,” “the correct information,” “life and family,” “hospital countermeasures,” “television and news,” and “examination.” As shown in Figure 4, the frequency of “not gained knowledge” was lower than for the category of “wanted knowledge.” In particular, the “characteristics of COVID-19,” “the correct information,” “television and news” related to “general knowledge of COVID-19”; “infection prevention,” and “examination” were the most frequently cited contents of gained knowledge, while there was a large decrease in the overall frequency of “not gained knowledge.” Conversely, the proportion of lecture time spent on “perspective” and “treatments and vaccines,” for which we did not yet possess the relevant knowledge, was less than the proportion of interest in these topics based on the preliminary questionnaire (Table 2). For these topics, the frequency of “gained knowledge” is low, while the frequency of “not gained knowledge” remained relatively high.

Fig. 4.

Cross-tabulation of wanted knowledge, gained knowledge, and not gained knowledge for each cluster

This figure shows the frequency and Pearson relative standard deviation (RSD) of wanted knowledge, gained knowledge, and not gained knowledge within each cluster. The frequency of occurrence of words is the size of a square, and the Pearson RSD is higher for red and lower for blue.

DISCUSSION

We analyzed anxiety and knowledge of COVID-19 of the staff by text mining in advance and used the results to develop and conduct an online lecture. The results show that many staff anxious about COVID-19, especially “infection and impact on myself, family, and neighbours”, and it was suggested that reducing anxiety and increasing confidence in knowledge may be linked to online lectures. Moreover, using qualitative analysis, it could be used to support the possibility that changes were based on the intervention of this lecture.

These changes were anchored by a) the comparison between open-ended responses regarding anxiety and knowledge in the pre-lecture questionnaire and the contents of the lecture (Table 2), b) the open-ended response analysis of anxiety and changes in lifestyle, and c) a cross-tabulation (Figure 4) of the open-ended questions regarding knowledge. The results show that the lecture is well designed based on the anxiety and knowledge-interest expressed in the preliminary questionnaire. Moreover, even if the contents of the lecture provide multiple viewpoints, the audience nevertheless learns the material intended by the developers of the lecture. To the extent possible, we were able to match what the audience wanted to know with the contents provided in the lecture; thus, we felt that the audience learned what we wanted them to know and participants’ ratings of the confidence level of their knowledge may have increased. Moreover, “reduced anxiety” was found to correlate with the content focused on in the lecture, which is attributable to the knowledge gained in these content areas. Thus, even if the pre- and post-questionnaires have no correspondence, we believe that by linguistically visualizing the content of the pre- and post-questionnaire changes through text mining analysis, we can evaluate whether the changes are based on the intervention or not.

This research is also valuable because it shows how the results of quantitative text mining may be reflected not only in the evaluation of online lectures but also in the content of lectures. Quantitative text mining can be used to analyze a large amount of data¹⁹ and perform multivariate analysis that cannot be done using ordinary qualitative research methods.²⁰ Moreover, quantitative text mining is highly suitable for the analysis of questionnaire surveys that are difficult to quantify. Therefore, we evaluated the lecture content using metric text mining such as the KH coder.^20,27 In creating the lectures, we did not directly calculate the percentage of each category, but simply referred to the co-occurrence networks that was formed to determine the categories and their portions to be covered in the lecture. However, as shown in Table 2, the results of the questionnaire and the content of the lecture were very close. This shows that we can accurately interpret the results of the questionnaire just by referring to the co-occurrence network, which is very easy to create. Regarding using data mining for education, Penteado et al stated that it is important to understand the needs of learners and reflect their needs in education is the most improvement in improving the outcome of learners.²¹ Thus, the needs of the hospital staff must match the contents of an online lecture to reduce anxiety. For these reasons, we believe that this methodology is extremely useful for visually exploring the learners’ needs to categorize them into components and prioritizing them quantitatively in a short time.

This study has three limitations. The first is that the use of the Likert scale and quantitative text mining to measure anxiety and knowledge confidence is not standard for each evaluation scale. The State-Trait Anxiety Inventory and Beck Anxiety Inventory are valid and accepted measures for anxiety.²⁸ However, since we needed to reflect as many open-ended answers as possible, it was necessary to simplify the questionnaire. In the process of creating co-occurrence networks and clustering in quantitative text mining, the author arbitrarily sets the minimum frequency of occurrence of words, the number of Jaccard coefficients to be used, and the point from which the Jaccard coefficients should be considered as a strong relationship. Therefore, there is a risk that the results may change depending on the analyst. However, by showing the values of each setting and the results of clustering automatically performed by KH Coder, I have taken care to allow the reader to judge the validity of the results.

The second limitation is that we did not ensure that the before and after responses to the questionnaires were by the same respondents. The questionnaires answered in hardcopy were anonymized to ensure privacy. Therefore, we did not perform a multivariate analysis for these responses because the pre- and post-responses were not linked by the respondent. However, by combining quantitative analysis with qualitative analysis, it was possible to show that the change in anxiety and confidence in knowledge was a true change, allowing us to explore the relationships between them. Since it was possible to analyze more open-ended changes than the standard analysis of changes in the prescribed items we had prepared, this analytic method may be more realistic in capturing small changes.

The third limitation is that there was a change in the prevalence of COVID-19 during the month in which the study was conducted. In Japan, there were 284 diagnosed cases of COVID-19 on April 28, compared to 31 on May 22. However, there was no change in the sense of urgency among health care workers in non-epicenter areas not to bring even a single case of COVID-19 into the hospital. In addition, the co-occurrence network showed that the change in anxiety about COVID-19 was related to the content of the lecture, as shown in the results.

Medical workers in non-epicenter areas have various anxieties that can be effectively reduced through online lectures using text mining. When a large amount of data is analyzed in a short time and applied to a lecture such as ours, there may be a bias in the analysis process, such as the tendency to direct focus based on one’s interests or easiness of answer. More rigorous methods are impractical in such a time-pressured environment. Thus, we conducted the analysis so that objectivity and a conclusion could be obtained quickly. To efficiently intervene and reduce the anxiety about COVID-19 of staff members, it is important to use an approach that is appropriate to the needs of the target population, not generalized interventions. Thus, we conclude that online lectures using text mining can be useful in improving the mental health of staff members.

ACKNOWLEDGMENTS

We would like to thank Daiki Kobayashi, St. Luke’s International University and Edanz Group (www.edanzediting.com/ac) for editing a draft of this manuscript.

CONFLICT OF INTEREST

The authors declare they have no conflict of interest with respect to this research study and paper.

SUPPLEMENTS

Sup. 1

A pre-lecture survey in hardcopy form included an 11-point Likert scale and open-ended interview questions concerning participants’ knowledge and anxiety regarding COVID-19

Sup. 2

A post-lecture survey in hardcopy form included an 11-point Likert scale and open-ended interview questions concerning participants’ knowledge and anxiety regarding COVID-19

Abbreviations

PCR

polymerase chain reaction

MD

median

IQR

interquartile ranges