Evaluating the Training of Chinese-Speaking Community Health Workers to Implement a Small-Group Intervention Promoting Mammography

Abstract

This study evaluated the training of Chinese American Community Health Workers (CHWs) to implement a small-group mammography video and discussion program as part of a randomized controlled trial that had the goal to increase adherence to mammography screening guidelines among Chinese American women. A total of 26 Chinese American CHWs in the metropolitan Washington DC area, Southern California, and New York City participated in a 4-hour training workshop and completed surveys before and after the workshop to assess their knowledge regarding mammography screening guidelines and human subjects protection rules. The results showed significantly increased knowledge of mammography screening guidelines and human subjects protection rules (both p<0.01) after the training. CHWs were also trained to lead a discussion of the video, including screening benefits and misconceptions. Forty-three audio-recordings of discussions led by 13 active CHWs were transcribed and qualitatively analyzed to assess implementation fidelity. Ten out of 13 active CHWs fully addressed about 3 of the 5 benefit items, and 11 out of 13 CHWs fully addressed more than 5 of the 9 misconception items. Chinese CHWs can be trained to implement research-based intervention programs. However, a one-time training resulted in moderate adherence to the discussion protocol. Ongoing or repeat trainings throughout the intervention period may be needed to enhance implementation fidelity.

INTRODUCTION

The community health worker (CHW) model in the United States was formally proposed in the 1960s to meet health and social challenges of marginalized migrant communities [1]. Since CHWs are highly involved in local communities, employing them as part of intervention teams has been an increasingly popular strategy for reaching underserved and hard-to-reach ethnic minority groups in community-based health programs over the past five decades [2, 3]. The role of CHWs has been constantly evolving and has become more important in public health promotion. Research indicates that CHWs can liaise between communities and health professionals, increase recruitment, participation and retention of target populations, provide social support, and ensure that the interventions are culturally and linguistically appropriate [3–5].

Although CHW-led interventions have shown positive effects in promoting health behaviors in various ethnic groups [6, 7], scarce research has evaluated the trainings that CHWs received and the implementation fidelity of CHW-led programs. Prior published literature on CHWs’ trainings primarily evaluated training efforts by using pre- and post-test approaches to assess their knowledge and attitudinal changes [8–11], but little is known about how well lay CHWs deliver science-based intervention messages to educate their peers and whether they are consistent in implementing intervention protocols.

In this paper, we evaluated the training of Chinese American CHWs who implemented a small-group mammography video and discussion (SMVD) program for a multi-site randomized controlled trial (RCT) that had the goal to increase adherence to mammography screening guidelines among Chinese American women. Specifically, we quantitatively assessed CHWs’ changes in knowledge regarding mammography screening guidelines and human subjects protection rules and qualitatively analyzed audio-recordings of CHW-led discussions to evaluate implementation fidelity. To our knowledge, this is the first study that evaluates Chinese American CHWs’ training and intervention implementation. Results from this process evaluation will be considered when we will analyze the data from the RCT to estimate the effectiveness of the intervention.

METHODS

The entire RCT including CHW training and SMVD intervention (mammography video and CHW-led small group discussion) has been approved by the institutional review boards of Georgetown University, University of California Los Angeles, and Temple University.

CHW-led SMVD intervention

Chinese American women who had not had a mammogram for two years viewed a culturally appropriate, Chinese-language mammography video and discussed the content of the video and their mammography concerns and viewpoints. This discussion (all in Chinese language) led by trained Chinese-speaking CHWs was aimed to increase peer interaction to foster mutual learning of mammography screening benefits and to correct mammography misconceptions. Each small group consisted of 5 Chinese American women on average (range of participants: 2–11). The video guided by the Health Belief Model has shown significant effects in improving non-adherent Chinese American women’s knowledge and attitudes about mammography screening and increasing their mammography use [12–14]. The SMVD attempted to increase Chinese American women’s adherence to the American Cancer Society (ACS) mammography screening guidelines [15]. Annual mammography screening was still recommended at the time of this study.

CHWs implemented the SMVD program in two steps: 1) viewing the 20-minute mammography video and 2) discussing breast cancer and mammography screening information from the video for about 30 minutes on average (see details in the training section below). At the end of the group discussion, participants received a copy of the video in DVD format and an information sheet indicating local free or low-cost mammography screening programs. Additionally, a trained research staff member attended each small group discussion and corrected information if they noticed that a CHW misinterpreted or provided incorrect information about breast cancer and mammography screening.

Recruitment of Chinese-speaking CHWs

Chinese American CHWs were recruited through 8 Chinese American community-based organizations (CBOs) in the metropolitan Washington DC area, Southern California, and New York City. The eligibility criteria for CHWs were: 1) Chinese American women over age 40 or Chinese American breast cancer survivors over age 21, 2) high-school diploma or higher education, 3) fluency in Mandarin or Cantonese, and 4) interest in promoting mammography screening. Twenty-six eligible CHWs were enrolled by CBOs to participate in our training workshops. After a 4-hour one-time training workshop, 17 of the 26 (65.4%) CHWs agreed to lead small group discussions, while the remaining 9 withdrew from the program due to time concerns or lack of interest. Finally, 13 out of the 17 CHWs actively implemented the SMVD throughout the intervention period.

Training Chinese-speaking CHWs

Training workshops were held at each participating CBO site. The 4-hour training consisted of four steps: 1) learning about human subjects research and protection, 2) viewing the mammography video, 3) reviewing a printed, structured small group discussion guide, and 4) rehearsing through a mock small group discussion session. The human subjects research and protection training was required because CHWs were engaged in research and had access to participants’ personally identifiable private information if they shared their concerns and experiences about breast cancer and screening as part of the small group discussion. The structured discussion guide delineated Chinese women’s perception of screening benefits and misconceptions. Specific timestamps in the video illustrating those benefits and misconceptions were also provided in the guide for CHWs to rehearse at home. All CHWs received one in-person training.

Additionally, CHWs received two auxiliary tools to help them lead the group discussion. One tool was a flip chart with scientific information from the video such as a graph indicating that breast cancer risk increases with age and data regarding breast cancer prevalence and mammography screening. CHWs could use this flip chart to explain screening benefits during the discussion. The second tool was a palm-sized summary of the discussion guide used as a handy reminder during the group discussion. Each CHW was asked to review all intervention materials at home before leading a small group discussion. CHWs were paid $250 per small group session to compensate for time and effort in preparing the discussion, contacting participants to convene a small group, commuting, and leading the SMVD.

Measurement

Training outcomes

CHWs were invited to complete a paper-and-pencil test in Chinese immediately before and after the training. The test assessed two training outcomes: 1) CHWs’ knowledge of mammography screening guidelines (2 items) including age when screening should start and time interval between screenings, and 2) CHWs’ knowledge of human subjects protection rules (7 items; see Appendix 1) adapted from the Human Subjects Protection Training for Community Workers questionnaire [16]. Correct answers were scored as “1”, while incorrect answers were scored as “0”. Scores for each outcome were added.

Fidelity of intervention implementation

To assess the fidelity of intervention implementation, each CHW’s first two small group discussions were required to be audio-recorded. We continued to record additional sessions if CHWs skipped key items in the structured guide and/or provided incorrect or ambiguous information. Trained staff and the last author listened to the recordings and provided prompt feedback to each CHW. We recorded 70 of the 95 small group discussions, which were led by active CHWs. Due to the relatively large volume of the recordings, we sampled and transcribed 43 of the 70 recordings for analysis. For CHWs who conducted fewer than 4 discussions, we included all of their recordings in the analysis. We determined CHWs’ implementation fidelity by evaluating how well they addressed the following two categories: 1) screening benefits and 2) misconceptions regarding breast cancer and mammography. There were 5 mammography benefit items including: 1) saving lives, 2) detecting small lumps that cannot be felt by hand, 3) detecting breast cancer early, 4) increasing chance of survival, and 5) being the most cost-effective screening test. The 9 misconception items included participants’ beliefs regarding the cause of breast cancer: 1) poor diet, 2) lack of exercise, 3) emotional stress, 4) personal destiny; and beliefs that 5) breast cancer only occurs among women with family history of breast cancer, 6) older women do not develop breast cancer, 7) radiation from mammography can cause breast cancer, 8) women with small breasts are less likely to get breast cancer, and 9) breast self-examination is effective. Coders evaluated whether a CHW successfully utilized facts and data provided in the video and the flip chart to explain screening benefits, and to address participants’ concerns without asserting her personal opinions or presenting conflicting messages. Coders only coded messages delivered by CHWs and did not code any explanations by research staff. Situations where CHWs did not or only partially utilized the provided information were coded as “noncompliant” (score=0). Situations where CHWs successfully and correctly addressed an item were coded as “compliant” (score=1). CHW’s scores on each category were summed to a total compliance score which was divided by the total number of items in each category to obtain a mean score.

Two Chinese-speaking research assistants transcribed the recordings and two different Chinese-speaking research assistants independently coded each of the verbatim transcripts following a coding scheme developed by the research team. The inter-coder reliability rate was 82.5%. Coders also listened to randomly selected recordings to check the consistency between audio files and Chinese transcripts. The two coders discussed differences in coding results to reach consensus. Examples of compliant and non-compliant statements were translated from Chinese to English and provided in the results below.

Self-evaluation of small group discussions

After the last small group discussion, each CHW was invited to complete a debriefing survey that assessed how comfortable they felt when conducting the SMVD program, their intention to lead future small groups, and how much time they spent reviewing the intervention materials prior to each group discussion.

Data Analyses

Analysis of training outcomes was based on all 26 CHWs who completed the training sessions as well as the pre- and post-tests. Analysis of implementation fidelity was based on the 13 active CHWs. Sample characteristics of the CHWs were summarized by frequency and percentage. Paired t-tests were conducted to examine mean changes in knowledge of the screening guidelines and human subjects protection rules between pre- and post-test. All t-tests were two-sided and a p-value less than or equal to 0.05 was considered statistically significant. CHWs’ implementation fidelity and self-evaluation data were qualitatively analyzed and presented by frequency or mean values. Statistical analyses were conducted using SPSS 22.

RESULTS

Sample characteristics

Table 1 presents baseline sample characteristics. The mean age of all CHWs (N=26) was 56 years and most of them (73.1%) were over age of 50. Approximately 52% of the CHWs had lived in the U.S. for more than 25 years. Most CHWs had a college education or higher (73%) and a full- or part-time job (84.6%). There were no significant differences in demographic characteristics between all CHWs and the 13 active CHWs.

Table 1.

Chinese American Community Health Worker (CHW) Characteristics

	All CHWs		Active CHWs
	N=26	%	N=13	%
Age
< 40	2	7.7	0	0
⩾ 40 and < 50	5	19.2	3	23.1
⩾ 50 and < 75	19	73.1	10	76.9
Years in the US1
< 25	12	48.0	5	41.7
⩾ 25	13	52.0	7	58.3
Study site
Washington DC Area	6	23.1	4	30.8
Southern California	12	46.1	6	46.1
New York City	8	30.8	3	23.1
Education
High School, some college or technical training school	7	26.9	3	23.1
College and above	19	73.1	10	76.9
Medical background2
Yes	15	57.7	7	53.8
No	11	42.3	6	46.2
Employment
Full time	13	50.0	7	53.8
Part time	9	34.6	4	30.8
Not employed	4	15.4	2	15.4
Language spoken fluently (Check all apply)
Mandarin	26	100.0	13	100.0
Cantonese	9	34.6	5	38.5
English	19	73.1	10	76.9
Taiwanese	6	23.1	4	30.8
Other	1	3.9	1	7.7
Had breast cancer
Yes	3	11.5	1	7.7
No	23	88.5	12	92.3

¹Years in the US: 1 was missing, total number of respondents to this question were 25 and 12, respectively.

²Medical background was defined as having expertise in nursing, nutrition, medical technology, medicine, and dentistry.

Changes in knowledge scores

The 26 CHWs significantly increased their screening knowledge mean scores from 1.3 to 1.9 on a 2-point scale (Δ=0.6, p<0.01, see Table 2) after the training. Similarly, their knowledge of human subjects protection rules mean scores increased from 5.1 to 6.6 (Δ=1.5, 7-point scale, p<0.01) after the training.

Table 2.

Mean Scores on Community Health Workers’ Knowledge of Screening Guidelines and Human Subjects Protection Rules before and after the Training (N=26)

	Pre-training score Mean (SD)	Post-training score Mean (SD)	P value
Mammography screening guidelines (2 items)	1.3 (0.7)	1.9 (0.3)	<0.01
Human subjects protection rules (7 items)	5.1 (1.5)	6.6 (1.6)	<0.01

Note. SD= standard deviation

Fidelity of intervention implementation

The qualitative analysis of audio recordings indicated that 10 out of 13 CHWs (77%) successfully and correctly addressed about 3 out of the 5 mammography screening benefit items (mean score= 2.8 out of 5 points). The following example was considered as fully addressing a screening benefit and coded as compliant (score = 1). “If cancer is found at the earliest stage, five-year survival rate is 93%. However, if it is found at the last stage, only 15% can survive five years after diagnosis, which means most patients passed away within five years after diagnosis. Five-year survival rate decreases as the cancer stage increases” [EDU3].

Situations where CHWs attempted to explain the survival rate but exaggerated the statistical data were considered as partially compliant (score = 0). For example, “The figure shows that if you find breast cancer at stage 0 or stage 1, you don’t even have to receive chemotherapy. Just remove the tumor and the survival rate can be 99–100%” [EDU 12].

Similarly, 11 CHWs (85%) fully addressed at least 5 out of 9 misconceptions (mean =5.5 out of 9). For example, when addressing the misconception regarding the effectiveness of breast self-examination, CHWs received a score of 1 if they provided a correct, in-depth explanation: “In lots of cases, cancers were detected by mammography not by breast self-exam… Here’s the comparison of the tumor size detected by breast self-exam and by mammography. If you get an annual mammogram, cancer can be found at this very small size. It can’t be felt by hand, unless the tumor has grown big… Don’t wait until it grows this big and spreads to see the doctors. It is too late.” [EDU8]. This CHW did not assert any personal opinions and clearly explained the difference between mammography and self-examination.

Although we asked CHWs not to promote self-examination because research has not proven its effectiveness [17], some CHWs would make statements like: “Self-exam is a way to detect breast cancer. Do a self-exam while you take a bath or lie on bed. Sometimes, the lump can be felt. In the video, there are eight lumps in the breast model; four big ones can be felt by hand. It is always better to do a self-exam than not. However, the small lumps can’t be felt by hand, you still need to get a mammogram. That is why we remind everyone to get an annual mammogram, early detection and early treatment makes you suffer far less” [EDU 12]. Such statements including the CHW’s personal opinions were coded as partially correct (score = 0).

CHWs’ self-evaluation results

Nine out of 13 active CHWs felt very comfortable in leading the groups and were very interested in continuing their role in future group discussions. The remaining four were somewhat interested. Only 8 of the 13 CHWs reported reviewing all intervention materials before every group discussion. Reviewing materials took 30 minutes on average. Reasons for not being able to review materials before every group discussion were attributed to urgent tasks at work or home before small group events.

DISCUSSION

This study indicates that Chinese-speaking lay women can be recruited and trained to implement a mammography screening education program in Chinese American communities. A one-time training is effective in increasing CHWs’ knowledge of breast cancer screening guidelines and human subjects protection rules. The training showed a moderate effect on CHWs’ implementation fidelity. These results can inform future CHW trainings and evaluations of CHW-led intervention programs.

Our finding that a one-time training increased CHWs’ knowledge of mammography screening is consistent with prior research findings [8–10]. The study’s novel finding is that after discussing realistic scenarios that CHWs might encounter, almost all CHWs earned perfect scores and were fully aware of the need to protect participants’ privacy and the importance of informed consent and voluntary participation (Appendix 1) [7, 18]. This result demonstrates that it is feasible to train CHWs to comply with human subjects protection rules [4]. We recommend this methodology for future CHW trainings.

CHWs’ adherence to the research protocol is important but may be difficult to measure in community-based programs [19]. Evaluation approaches have to be feasible and largely depend on the study design. We utilized a relatively time-consuming but rigorous evaluation approach – scoring audio recordings of actual small group discussions - to assess how well CHWs delivered intervention components. We found that although CHWs followed the training instructions to cover required topics in small group discussions, it was challenging for them to fully clarify the benefits and misconceptions of mammography screening using the scientific data and information in the video. This may be due to CHWs’ personal beliefs (e.g., in the usefulness of self-exams), preferences (e.g., stating precisely 93% vs. generally 99–100 % survival rates of early diagnosis), and insufficient rehearsing of the study materials due to home- and work-related issues, as reflected in CHWs’ self-evaluation.

The community agencies that partnered with us preferred a one-time training rather than several training sessions for a more efficient use of time and resources and since some CHWs were working full- or part-time. This approach may be convenient for CHWs but requires CHWs’ self-practice. Our results suggest that ongoing or repeat trainings and rehearsals throughout the intervention period and corrective feedback may be needed to enhance implementation fidelity. This will allow CHWs to repeatedly practice implementation details and to learn from each other. It may also lead to a more standardized intervention delivery, which is especially important if several CHWs are involved. Prior research has shown that continuous training throughout the study period (in addition to initial training) would increase CHWs’ compliance and performance [20, 21].

Our results regarding implementation fidelity were based on 13 active CHWs. This small sample precluded our ability to examine factors associated with intervention fidelity such as CHW demographic background, the length of each discussion session, or the number of discussion group participants. Future studies should explore this issue.

In conclusion, this study contributes to the body of research regarding the evaluation of CHW trainings and the implementation fidelity of CHW-led intervention programs in community settings. Future research should evaluate whether repeat training throughout the intervention period improves CHWs’ delivery of scientific information in an accurate and standardized manner and enhances the fidelity of intervention implementation.

Appendix 1. Items to Assess Knowledge of Human Subjects Protection Rules (adapted from Hatcher and Schoenberg [16])

1. A participant can end her participation in the study at any time, without any negative consequences or explanations. (true/false)

2. A casual conversation about a study participant with someone other than the investigator is a breach of confidentiality. (true/false)

3. A person may withdraw from the study without written documentation. (true/false)

4. You may tell your family who is in the study as long as you don’t tell what the participant said. (true/false)

5. You have just completed an interview with Ms. Ko. Her neighbor spots you leaving her home. The neighbor has heard about the “Breast Health in Chinese Women” at her community center and is curious about who is in it and how it works. Later you run into that neighbor in the grocery store and she asks you if Ms. Ko is part of the “Breast Health in Chinese Women” study. What would be the most appropriate action to take? (choose one of four response options)

6. You are entering information on your laptop at home regarding a research participant. Your young son asks to use the computer to play games. What action would be appropriate to protect the confidential information you have entered? (choose one of four response options)

7. When community educators contact participants for their most convenient date and time for small group discussions, one participant asks, “Could you tell me who else is participating in the small group discussion? I would only participate if other people I know are attending.” What is the most appropriate response in this situation? (choose one of four response options)