Skip to main content
NCBI home page
Health Education Research logoLink to Health Education Research
. 2013 Oct 11;29(1):158–165. doi: 10.1093/her/cyt100

Assessing the fidelity of the Kin KeeperSM prevention intervention in African American, Latina and Arab women

Sabrina Ford 1,*, Cristian Meghea 1,2, Tamika Estes 1, Hiam Hamade 3, Murlisa Lockett 1, Karen Patricia Williams 1
PMCID: PMC3894668  PMID: 24122323

Abstract

Background: We evaluated a randomized controlled treatment, utilizing Community Health Workers (CHW) to deliver breast and cervical cancer education intervention to African American, Latina, and Arab women in Detroit and Dearborn, Michigan. The main objectives of the study are to: (1) examine fidelity and consistency of treatment delivery and (2) assess qualitative elements of the intervention. Methods: We surveyed 305 women who received the intervention and 16 CHWs Survey included questions regarding the treatment integrity, treatment received, and training provided. Surveys included both quantitative and questions. Results: The intervention group (n = 305) was made up of 48% Black, 11% Latina, and 41% Arab women. Almost all (≥ 90%) women agreed that they received the treatment in the way that it was intended. Sixteen CHWs responded affirmatively as well. Conclusions: Both participants and CHWs indicated that the program was mutually rewarding, indicating that there was “cross fertilization and cross benefit” of working with each other. These benefits served to endorse and confirm that CHWs are a very important mechanism in increasing health literacy in the community and referring underserved individuals to health providers. Second, with strong treatment fidelity the Kin KeeperSM program and has the potential to be replicated for a number of diseases in a variety of venues especially for those facing health disparities.

Introduction

As community health workers (CHWs) are a cost-effective way to reduce health disparities and reach underserved individuals, they have received increased attention as potential healthcare system navigators [1, 2]. CHWs can educate and refer clients to healthcare providers for specific health problems such as diabetes and heart disease [3, 4] and can also be utilized to encourage screening and surveillance for devastating diseases such as cancer [5, 6]. CHW programs have been described and examined in the literature, but most of these examinations are process evaluations and few programs have been studied for intervention fidelity and model consistency [7]. Intervention fidelity refers to participants receiving the intervention in the same way every time, ‘faithful’ to the proposed model [8, 9] and with the trained CHW delivering the intervention consistently to each participant with the same quality as other CHWs.

Assessing intervention fidelity is important, especially in community-based participatory research (CBPR), to validate the accuracy and quality of the intervention and to ensure CHW adherence to intervention delivery protocols [8, 9]. Here, we evaluated an intervention program known as the Kin KeeperSM Cancer Prevention Intervention that utilizes CHWs to deliver breast and cervical cancer education to African American (herein referred to as Black), Latina and Arab women [10]. The main objectives of this evaluation were to: (i) examine intervention fidelity and consistency to different ethnic groups and between CHWs and (ii) assess the qualitative elements of the intervention pertaining to training and intervention delivery. Identifying the Kin KeeperSM model’s strengths and limitations, if any, will allow us to build capacity for the model and replicate it in various venues and with other diseases.

Methods

Kin KeeperSM is a CBPR project with community partners whose clientele reflect the urban population of Southeast Michigan: Black, Latino and Arab. It is an established university–community partnership that has been in existence for over 7 years with two NIH funded projects. Over that time, the intervention has proven to be efficacious [11, 12] and is presently being continued on a larger scale. This intervention program utilizes: (i) an evidence-based and standardized curriculum that is culturally and linguistically sensitive [13]; (ii) culturally and linguistically validated outcome measures and (iii) CHWs with racial and ethnic concordance to research participants [14]. The Latina CHWs are bilingual in English and Spanish and the Arab CHWs are bilingual in English and Arabic, respectively. CHWs that participated in this study were obtained from our community health clinic partners from the Detroit Department of Health and Wellness Promotion and the Arab Community Center for Economic and Social Services.

The Kin KeeperSM CHWs were first given 24 h of training in breast and cervical cancer education prevention intervention and early screening detection [10]. Each CHW was given an educational kit that included a scripted flip chart and demonstration models (e.g. breast models). CHWs also received training in the protection of human subjects in research. Then, in 2011 and 2012, the CHWs recruited participants from their roster of clients and this was condition of client eligibility for the study. Using a randomized controlled trial design, participants were randomized into the Kin KeeperSM group or the control group. Participants then recruited at least two directly related female family members (mother, daughter, sister, aunt and grandmother), yielding a total of 514 participants in the study (305 in the Kin KeeperSM group, 209 in the control group). More details, including recruitment, participation rate and characteristics of the CHWs and study participants are provided elsewhere [10].

CHWs applied the Kin KeeperSM Intervention model based on three main principles [10, 11]: (i) women will understand that family history of cancer plays a role in the early detection of cancer; (ii) women will make a clear connection between family history of cancer and the need for screening and (iii) family ties will strengthen the possibility of change in screening behavior. The Kin KeeperSM Intervention group received two home visits wherein the CHWs delivered two, 2-h education sessions—one session pertaining to breast cancer and the other pertaining to cervical cancer using the materials described above. The education was administered in the language of the participants’ choice accurately translated across all languages. CHWs had the flexibility to provide additional explanations as necessary.

Intervention fidelity was evaluated only for the Kin KeeperSM intervention group because they received the comprehensive, 4-h behavioral intervention, a more robust intervention. The control group was given only reading materials in a single session and not considered an intervention, but standard of care such as literature given in a doctor’s office. For this intervention fidelity evaluation, we utilized a tool created by the Treatment Fidelity Workgroup of the National Institutes of Health (NIH) Behavior Change Consortium [15] to assess the treatment fidelity (TF) of interventions [15, 16]. TF includes five elements: (i) study design (SD); (ii) training provided (TP); (iii) treatment integrity (TI); (iv) receipt of treatment (TR) and (v) enactment of treatment skills (SE). The NIH TF model was incorporated into our original research design of this NIH grant and was used to design the TF survey items used here (Table II). Two separate surveys (one for the breast cancer education and one for the cervical cancer education interventions) were administered as a self-report after the interventions were given to capture the information about TI and TR from the participants.

Table II.

Community health worker treatment evaluation questions and results

Item Mean (SD)
1. How long have you worked as a community health worker? (mean years, range in years: 1–20) 9.09 (5.13)
2. How many different jobs that could be described as a community health worker or similarly titled job have you had? (mean jobs, range: 2–4) 3.00 (0.64)
3. On average, how many total hours do you work each week as a community health worker? (mean hours, range: 5–40) 21.33 (11.64)
4. On average, how many total clients would you say you serve in any given month? (mean hours, range: 5–55) 23.30 (15.30)
5. How many hours a week do you meet with your supervisor about your clients? (mean hours, range: 0–5) 1.60 (0.69)
6. How long have you been working with the Kin KeeperSM Cancer Research Project? (mean years, range: 1–2) 2.09 (1.04)
Following questions are reported in percentages
7. Do you feel you have all the tools you need for the Kin KeeperSM home visits? (yes) 100
8. Do you feel that you deliver the breast and cervical education the same way as the other community health workers do? (yes) (TI) 90.09
9. Do you feel like you make an impact or a difference with educating the Kin KeeperSM families? (yes) 90.09
10. What kind of training did you have before you started your current job? (TP) College CHW Both None
18.2 36.4 27.3 9.1
11. How difficult is it to schedule Kin KeeperSM home visits? Easy Neither Difficult
45.50 54.50 0.0
12. How satisfied are you with the Kin KeeperSM Cancer Prevention Intervention training that you received? (TP) Satisfied Neutral Dissatisfied
90.90 9.01 0
13. How helpful do you feel your supervisor is in giving you Kin KeeperSM Education support? (TP) Helpful Neither Not Helpful
90.09 9.01 0
14. How effective do you feel the models and other visuals are in teaching the women about breast and cervical cancer prevention? (TI) Effective Neither Not Effective
90.09 9.01 0
Sample of narrative responses
15. What do you like most about working with the Kin KeeperSM families? ‘I like educating the families about cancer prevention. Feeling like I am saving lives’.
… opportunity to serve the community and educate people and myself …’
16. What do you like least about working with the Kin KeeperSM research? ‘Sometimes hard to contact families’.
‘Paperwork’.
17. What is the biggest barrier you face when doing your work with the Kin KeeperSM research? ‘Getting families to make time for the surveys’. ‘Getting families to complete all phases of the paper work’.
18. Please express any other opinions about Research Project. ‘… wish it could be part of a regular presentation done everywhere-churches, companies, fairs, etc.’
‘It’s been educational and empowering’.
Open in a new tab

TI, treatment integrity; TP, training provided

As with many CBPR programs, CHWs are considered research participants to inform policy [1, 2] and are part of this intervention fidelity evaluation. CHW survey questions were based on the Michigan Department of Community Health Survey [17] that included a total of 19 questions: 15 quantitative questions and 4 qualitative questions to examine TI and TP (Table II). All CHW surveys were given anonymously and were blind to the CHW supervisors and the research team. The tables also indicate which questions addressed TI, TR and TP. (SD and SE will be evaluated during follow-up interviews to be completed over the next 3 years of the project.) Analyses were mainly descriptive including means, standard deviations (SDs), percentages and comparisons of the items across racial/ethnic groups using χ2 tests.

Results

Table I presents participant demographics: of the 305 participants, 147 (48%) were Black, 33 (11%) were Latina and 125 (41%) were Arab. The women ranged in age from 18–88 years with a mean age of 43.79 years (SD = 12.63) years and the majority are married (72%). Approximately, 60% of the women have a high school education or less, 44% are unemployed and 50% have a household income <$20 000. Shown in Table II, for the majority of the TF items, most (>91%) women agreed that they received the treatment in the way that it was intended. Items 2–7 addressed TI and items 8 and 9 addressed TR. We found no differences by race/ethnicity (all P-values were >0.10). For the cervical cancer fidelity survey, Latinas answered with slightly lower frequency on item 1, (I completed a form that asked me questions about how my family and I talk to each other) than Black or Arab women with 85% of Latina women stating they completed the family form compared with >90% for the Black and Arab women. After we examined the database, we found this was due to underreporting because we confirmed Latina participants had completed the form. We also noted that only 59% of Latinas reported receiving a gift bag (item 11 on the cervical survey) versus over 90% of Black and Arab women. We have explored possible reasons by asking CHWs and their supervisors what may account for the difference with no resolution and note it as a spurious finding.

Table I.

Participant breast and cervical cancer intervention fidelity items and results

Race African American Latina Arab Total
N 147 33 125 305
(%) (48.20) (10.82) (40.98) (100)
Mean age in years 42.89 41.06 44.80 43.79
(SD) (14.06) (10.58) (12.24) (13.02)
Married (%) 91.03 82.54 48.58 71.89
Education less than high school (%) 43.66 89.06 66.67 59.88
First survey—breast cancer items percent Yes (%) (%) (%) (%)
1. I completed a form that asked me questions about my education, my health and other similar things. 99.17 100 95.77 97.64
2. I heard a talk about breast cancer. (TI) 100 100 99.31 99.67
3. Before the talk, the community health worker asked me questions about what I knew about breast cancer. (TI) 99.19 100 98.60 99.00
4. After the talk, the community health worker asked me questions about what I knew about breast cancer. (TI) 99.19 100 99.30 99.33
5. The community health worker used flip charts during the talk. (TI) 100 100 98.61 99.33
6. During the talk, the community health worker showed some models of women’s breast. (TI) 99.19 100 99.31 99.33
7. The community health worker answered any questions about breast cancer. (TI) 100 100 98.59 99.33
8. I enjoyed the talk about breast cancer. 99.19 100 99.31 99.33
9. I understand what the CHW said about breast cancer. (TR) 100 96.97 99.30 99.33
10. I learnt a great deal from what the CHW said about breast cancer. (TR) 99.18 96.97 97.92 98.33
11. The community health worker gave out gift bags. 91.23 100 97.67 93.04
12. We rescheduled our next home visit. 100 100 98.40 99.15
Second survey—cervical cancer items percent Yes
1. I completed a form that asked me questions about how my family and I talk to one another. 98.39 85.85 97.14 96.30
2. I heard a talk about cervical cancer. (TI) 100 100 99.31 99.67
3. Before the talk, the community health worker asked me questions about what I knew about cervical cancer. (TI) 97.58 100 95.86 97.01
4. After the talk, the community health worker asked me questions about what I knew about cervical cancer. (TI) 100 96.97 98.61 99.00
5. The community health worker used a flip chart during the talk. (TI) 99.19 100 99.31 99.33
6. During the talk, the community health worker showed what cervical cancer looks like. (TI) 100 100 97.87 98.99
7. The community health worker answered any questions about cervical cancer. (TI) 100 100 99.31 99.67
8. I enjoyed the talk about cervical cancer. 99.19 100 99.29 99.32
9. I understood what the community health worker said about cervical cancer. (TR) 100 100 97.93 99.00
10. I learnt a great deal from what the CHW said about cervical cancer. (TR) 100 100 99.31 99.67
11. The community health worker gave out gift bags, 99.16 58.62 92.08 91.57
12. I was told that I would get a post card in the mail the next time the community health worker contacts me. 100 96.88 99.27 99.31
Open in a new tab

TI, treatment integrity; TR, treatment received. Values for χ2 tests not reported. all P-values were >0.10

Of the total 16 CHWs, 8 (50%) are Black, 2 (13%) are Latina and 6 (38%) are Arab. CHWs reported they have worked in their field for an average of 8.62 years (SD = 5.13), have been with the Kin KeeperSM study for 2.25 years (SD = 1.03) and have held an average of three CHW positions. They work an average of 20 h a week, seeing slightly more than 23 clients per month and receive ∼1 h of supervision per month. Approximately 18% have attended college, 36% have a CHW certificate and 27% have both a certificate and attended college (yielding 63% with a CHW certificate). The majority (>90%), felt that they delivered the treatment how it was intended and delivered it in the same way that other Kin KeeperSM CHWs did. Qualitatively, 91% of CHWs felt they made an impact on participants’ cancer literacy. Many CHWs indicated that ‘serving and educating their community’ was important to them. They also appreciated being involved with a large research project with one participant stating ‘it is educational and empowering’. Many CHWs conveyed that it was challenging to negotiate scheduling with families. These results are shown in Table II which includes a sample of the responses to the narrative questions. Although scheduling was challenging, a 100% follow-up rate was achieved between the 2-week period between the breast and cervical cancer education. Currently, this longitudinal study is in Year 3 with twice yearly follow-up, thus all rates and effect size of increased breast and cervical cancer screening will be reported at the end of the study.

Discussion

Results of our study suggest that the Kin KeeperSM Model has strong intervention fidelity, consistency and quality as indicated by both participants and CHWs. This has positive implications on a number of levels. Both participants and CHWs expressed that the program is mutually rewarding, indicating ‘cross fertilization and cross benefit’ of working with each other as validated by previous CHW studies [1, 2, 18]. These benefits served to endorse and confirm that properly trained CHWs are a very important means in delivering health information accurately and conducting consistent, quality CBPR. The intent underlying the use of visiting health workers is to engage and educate the community on the ground-level so that individuals who have little or no access to adequate healthcare can learn how to segue into a formal health system, ask for what they need, receive preventative care and in turn, reduce health disparities [1, 2, 19]. It is important that underserved individuals receive correct and specialized information that can actually be used to reduce these disparities. In addition, given the mutual benefit for the CHW and the participant, programs like the Kin KeeperSM have the potential to inform policy and deliver information in ‘healthcare deserts’ where there are no adequate healthcare providers (e.g. available providers do not take public insurance). In these scenarios, CHWs reported they are making an impact in their communities in an economic and equitable manner and empowering individuals to seek out healthcare.

With its robust intervention fidelity, this program has the potential to be replicated in a number of ways including diseases such as other cancers, diabetes, cardiovascular disease that can take hold in disenfranchised communities where diagnoses are often made much later than for those who are able to participate in routine healthcare. Not only can the model be used for education and prevention, but to increase surveillance and screening of chronic diseases [4, 20–22].

Limitations

Although the participants and CHWs reported strong fidelity and high quality of the model, they also reported challenges in scheduling appointments with participants. This occurred for several reasons: (i) the model is intended to be delivered to the family members together and it was sometimes difficult to schedule the family group all at the same time; (ii) for the Arab women and CHWs, some Muslim holidays occurred during the follow-up period and lay activities are curtailed during these holidays; (iii) often, Black and Latina participants had ‘competing priorities’, such as lack of childcare or a family crisis that presented difficulties in scheduling an appointment with the CHW, which is considered a lower priority. This is often the case when providing services for underserved populations and scheduling flexibility is helpful in addressing this challenge and (iv) there was greater attrition for Latinas due to immigration and deportation issues faced by the Detroit metro area at the beginning of the study as seen by the small number enrolled in the study. Another limitation is the small sample of CHWs. Even though the findings were robust, it would be helpful to evaluate the treatment fidelity with a larger set of CHWs from a statewide or national program to examine if these findings can be replicated. Nevertheless, even though the CHW sample size was small, very similar answers were observed on the open-ended questions.

Conclusion

We are encouraged by these findings of solid intervention delivery and fidelity for this complex intervention model that using family participants and CHWs of different races in a community setting. Over the next year, further evaluations will be conducted to identify methods for reducing scheduling difficulties. In addition, upon completion of the study, a full intervention fidelity evaluation will be completed by examining fidelity of the SD and the SE by the participants. Assessing treatment fidelity was an important step in validating of the Kin KeeperSM model and its instruments for building capacity in preparation for broader dissemination.

Acknowledgements

The authors would like to acknowledge Lihong Yang and Justice Wilson for their diligent work in managing the study data and preliminary analyses and Kachen Kimmell for her editorial assistance. S.F. designed the study with the collaboration of K.P.W. and C.M. S.F. provided the main writing for the manuscript. C.M. advised the analytical approach, performed statistical analyses and manuscript editing. T.M., H.H. and M.L. provided intellectual content, managed data for the study and provided editing from the field perspective. K.P.W. is the principal investigate for the parent study and senior author who provided advice and expertise throughout this study.

Funding

NIH National Institute of Nursing Research (Grant number 1R01NR011323).

Conflict of interest statement

None declared.

References

  • 1.U. S. Department of Health and Human Services, Health Resources and Services Administration, Bureau of Health Professions. Community Health Worker National Workforce Study. March 2007. Available at: http://bhpr.hrsa.gov/healthworkforce/reports/chwstudy2007.pdf. Accessed: 26 September 2012. [Google Scholar]
  • 2.Viswanathan M, Kraschnewski JL, Nishikawa B, et al. Outcomes and costs of community health worker interventions: A systematic review. Med Care. 2001;48:792–808. doi: 10.1097/MLR.0b013e3181e35b51. [DOI] [PubMed] [Google Scholar]
  • 3.Spencer MS, Rosland A, Kieffer EC, et al. Effectiveness of a community health worker intervention among African American and Latino adults with type 2 diabetes: A randomized controlled trial. Am J Public Health. 2011;101:2253–60. doi: 10.2105/AJPH.2010.300106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Cornell CE, Littleton MA, Greene PG, et al. A community health advisor program to reduce cardiovascular risk among rural African American women. Health Ed Res. 2009;24:622–33. doi: 10.1093/her/cyn063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Heart Disease and Strong Prevention Addressing chronic disease through community health workers. A Policy and Systems-Level Approach. Available at: http://www.cdc.gov/dhdsp/docs/chw_brief.pdf. Accessed: 26 September 2012. [Google Scholar]
  • 6.Fernandez ME, Gonzales A, Tortolero-Luna G, et al. Effectiveness of Cultivando La Salud: A breast and cervical cancer screening promotion program for low-income Hispanic women. Am J of Pub Health. 2009;99:936–43. doi: 10.2105/AJPH.2008.136713. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gearing RE, El-Bassel N, Chesquiere A, et al. Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011;31:78–88. doi: 10.1016/j.cpr.2010.09.007. [DOI] [PubMed] [Google Scholar]
  • 8.Dumas JE, Lynch AM, Laughlin JE, et al. Promoting intervention fidelity: conceptual issues, methods, and preliminary results from the EARLY ALLIANCE prevention trial. Am J Prev Med. 2001;20:38–47. doi: 10.1016/s0749-3797(00)00272-5. [DOI] [PubMed] [Google Scholar]
  • 9.Cordray DS, Hulleman C. Implementation Fidelity and Achieved Relative Strength in Randomized Controlled Trials. Invited Presentation for the IES 4th Annual Research Conference. Washington, DC: June 4, 2009. Available at: http://www.ies.ed.gov/director/conferences/09ies…/ppt/cordray_hulleman.ppt. Accessed April 15, 2013. Accessed: 25 April 2013. [Google Scholar]
  • 10.Williams KP, Roman L, Meghea C. Kin KeeperSM: Design and baseline characteristics of community-based randomized controlled trial promoting cancer screen in Black, Latina, and Arab women. Contemp Clin Trials. 2013;34:312–9. doi: 10.1016/j.cct.2012.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Williams KP, Mullan PB, Todem D. Moving from theory to practice: Implementing the Kin KeeperSM Cancer Prevention Model. Health Ed Res. 2009;29:1338–1342. doi: 10.1093/her/cyn026. [DOI] [PubMed] [Google Scholar]
  • 12.Williams KP, Mabiso A, Todem D, et al. Differences in knowledge of breast cancer screening modalities among African-American, Latina and Arab-American Women. Prev Chronic Dis. 2011;8:1–8. [PMC free article] [PubMed] [Google Scholar]
  • 13.Williams KP, Mabiso A, Jackson TL, et al. Breast cancer and cervical cancer control program enrollees inform the Kin KeeperSM curriculum. J of Cancer Ed. 2009;24:257–60. doi: 10.1080/08858190902972939. [DOI] [PubMed] [Google Scholar]
  • 14.Rivera-Vasquez O, Mabiso A, Hammad A, Williams KP. A community-based approach to translating and testing cancer literacy assessment tools. J Cancer Educ. 2009;24:319–25. doi: 10.1080/08858190902997373. [DOI] [PubMed] [Google Scholar]
  • 15.Bellg AJ, Borrelli B, Resnick B, et al. Treatment fidelity workgroup of the NIH behavior change consortium. Health Psychol. 2004;23:443–51. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
  • 16.Borrelli B, Sepinwall D, Ernst D, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73:852–60. doi: 10.1037/0022-006X.73.5.852. [DOI] [PubMed] [Google Scholar]
  • 17.Michigan Department of Community Health. Community Health Worker Survey. Chicago, IL: Glengariff Group Inc.; 2011. [Google Scholar]
  • 18.Mousa SM, Brooks E, Dietrich M, et al. Community health workers speak out about the Kin KeeperSM model. J of Cancer Ed. 2010;25:236–41. doi: 10.1007/s13187-010-0049-2. [DOI] [PubMed] [Google Scholar]
  • 19.Cornell CE, Littleton MA, Greene PG, et al. A community health advisor program to reduce to reduce cardiovascular risk among rural African American women. Health Educ Res. 2009;24:622–33. doi: 10.1093/her/cyn063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Kreuter MW, Kegler MC, Joseph KT, et al. The impact of implementing selected CBPR strategies to address disparities in urban Atlanta: A retrospective case study. Health Educ Res. 2012;27:729–41. doi: 10.1093/her/cys053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Smith SA, Blumenthal DS. Community health workers support community-based participatory research ethics: Lessons learned along the research-to-practice-to-community continuum. J Health Care Poor Underserved. 2012;23:76–87. doi: 10.1353/hpu.2012.0156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Balcazar H, Rosenthal EL, Brownstein JN, et al. Community health workers can be a public health force for change in the United States: Three actions for a new paradigm. Am J Public Health. 2011;101:2199–203. doi: 10.2105/AJPH.2011.300386. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Health Education Research are provided here courtesy of Oxford University Press

RESOURCES