Abstract
Process evaluation is essential to understanding and interpreting the results of randomized trials testing the effects of behavioral interventions. A process evaluation was conducted as part of a comparative effectiveness trial testing a mailed, tailored interactive digital video disc (DVD) with and without telephone-based patient navigation (PN) to promote breast, cervical and colorectal cancer screening among rural women who were not up-to-date (UTD) for at least one screening test. Data on receipt, uptake, and satisfaction with the interventions were collected via telephone interviews from 542 participants who received the tailored interactive DVD (n = 266) or the DVD plus telephone-based PN (n = 276). All participants reported receiving the DVD and 93.0% viewed it. The most viewed sections of the DVD were about colorectal, followed by breast, then cervical cancer screening. Most participants agreed the DVD was easy to understand, helpful, provided trustworthy information, and gave information needed to make a decision about screening. Most women in the DVD+PN group, 98.2% (n = 268), reported talking with the navigator. The most frequently discussed cancer screenings were colorectal (86.8%) and breast (71.3%); 57.5% discussed cervical cancer screening. The average combined length of PN encounters was 22.2 minutes with 21.7 additional minutes spent on coordinating activities. Barriers were similar across screening tests with the common ones related to the provider/health care system, lack of knowledge, forgetfulness/too much bother, and personal issues. This evaluation provided information about the implementation and delivery of behavioral interventions as well as challenges encountered that may impact trial results.
Keywords: cancer screening, rural women, tailored intervention, patient navigation, process evaluation
Process evaluation of behavioral interventions is essential to understanding and interpreting the results of intervention trials; such evaluation provides information about the implementation and delivery of interventions as well as challenges encountered that may impact results.
Graphical Abstract
Implications.
Practice: Interventions that can be successfully delivered to rural women in their homes to increase completion of needed cancer screening tests should be integrated into clinical practice settings.
Policy: Policymakers and health systems who want to increase participation in early detection and prevention of cancer through screening should support cost-effective, accessible, and acceptable interventions that can be widely implemented to reach rural populations.
Research: Future research should include comprehensive process evaluation to assess the receipt, uptake, and satisfaction of effective interventions to increase cancer screening participation.
Introduction
Residents in rural areas of the United States (U.S.) experience higher cancer incidence and mortality rates, as well as lower screening rates compared to those in metropolitan areas [1–5]. Cancer disparities among rural populations have been linked to social determinants of health including socioeconomic status (SES), lifestyle factors, access to health care, health insurance, or a combination of these factors [3–6]. Many rural women do not complete cancer screening at recommended intervals [5]. In an observational study conducted among rural women in Appalachian Ohio, medical record-validated completion of guideline-based screening were low: 32% for breast cancer (BC), 36% for cervical cancer (CC), and 30% for colorectal cancer (CRC) [7]. Only 8.6% of the women were up-to-date (UTD) for all three cancer screenings [7].
Prior studies showed sociodemographic factors limit UTD screening for these cancers among rural populations. Such factors include lower educational attainment, less knowledge about screening, lower income, poor access to health care, and greater social deprivation [1]. Given the substantial contribution guideline-based cancer screening provides for lowering cancer mortality [2–4] effective interventions to increase breast, cervical, and colorectal cancer screening could increase rates of being UTD with screening guidelines and decrease the disparate cancer mortality experienced by rural women [5].
These data suggest that cost-effective interventions are needed to support adherence to cancer screening guidelines for rural women. To address this need, we conducted a comparative effectiveness trial to test the hypothesis that the addition of telephone-based Patient Navigation to a tailored interactive Digital Video Disc (DVD+PN) will be more effective than either a mailed DVD alone or usual care (UC) [6]. The main efficacy results of this trial are reported elsewhere [8].
In addition to examining the outcomes of a health behavior change intervention, careful evaluation of the processes of delivering such interventions is essential. Understanding the degree to which interventions are delivered, received, taken up, and satisfactory to participants is necessary to interpret the success or failure of interventions as well as any limitations [9, 10]. Few studies have documented results of the process evaluation of screening interventions tested in randomized trials. While there are no standard methods for process evaluation of behavioral interventions, the approach used in this study was modeled after previous evaluations of similar interventions conducted by this team [11, 12].
Process measures are quantifiable metrics that provide information to describe the implementation process, including intermediate steps that lead to intervention outcomes. Process evaluation is used to identify how an intervention is delivered as well as implementation failures and/or problems that reduce the likelihood of achieving the desired outcomes [9, 13]. This paper reports the results of process evaluations of the two interventions, DVD and the DVD+PN, designed to promote BC, CC, and CRC screening among rural women who were not UTD with one or more screenings. The larger trial, including the process evaluation, was approved by the Institutional Review Boards at two midwestern universities.
Methods
Interventions to promote cancer screening
Detailed descriptions of the interventions have been published [6] and are briefly described here.
Tailored DVD intervention
The theory-based tailored, interactive DVD included an introduction and three separate sections focused on BC, CC, and CRC screening. The introduction focused on how cancer develops and the importance of being UTD for cancer screening. Each cancer site section started with a discussion of the importance of the specific cancer screening, an overview of screening test recommendations, and used narratives of women who addressed screening barriers and highlighted the importance of completing screening. Women received tailored messages based on their responses to questions they answered using the DVD remote control about their cancer screening history, risk factors, cancer knowledge, intention to get screened, confidence to complete cancer screening, and barriers to complete screening. Examples of barriers addressed in the DVD included: concerns about costs, pain, embarrassment, lack of transportation, finding a location to complete screening, and lack of doctor recommendation.
Patient navigation intervention
Patient navigators were experienced social workers who were trained by study investigators. Navigators had a comprehensive understanding of cancer screening tests, common barriers associated with those tests, and local rural culture and communities. They developed and maintained county-level resource lists that included healthcare facilities and screening services provided at each, health departments, and services such as transportation and childcare to better assist participants in their communities. Navigators had strong communication skills and used plain language when talking to community members. Navigators attempted to contact women by phone, mail, or email approximately 1 week after the DVD was mailed and made up to ten attempts to reach each participant to complete navigation. Navigators tracked the number of contact attempts, information/content discussed during encounters, actions taken by the navigator to address screening barriers, and coordinating time spent on navigation activities in a computer-based navigator documentation system.
Navigators first contacted participants by telephone to ascertain whether they received and viewed the DVD. If a participant had not yet viewed the DVD, they were encouraged to do so, and a follow-up call was scheduled to begin navigation. Navigators assessed each woman’s knowledge of needed screenings and then assessed barriers to getting screened. They provided additional educational information, mailed cancer screening brochures to participants who requested additional materials, addressed screening barriers, provided support and encouragement, and assisted with follow-up after any abnormal screening test. At exit, navigators also contacted any participant found to have been diagnosed with BC, CC, or CRC cancer during the intervention period to assist them if needed, regardless of a participant’s randomization condition. To assess intervention fidelity, all navigator calls were recorded, and a random sample of recordings were reviewed by the project manager to confirm concordance between recordings of barriers, navigation provided, and data recorded by the navigators.
Recruitment, eligibility and randomization
Detailed information on recruitment methods and eligibility criteria are described in detail elsewhere [6]. Briefly, mailing lists of female residents aged 50–74 living in 98 rural counties in Indiana and Ohio were obtained from Marketing Systems Group (white pages, commercial and U.S. postal service lists). Promotional materials (flyers/brochures/postcards) were displayed in public spaces including libraries, senior centers, and health departments. Finally, targeted Facebook advertising was employed, focusing on age-eligible women who lived in target counties of both states. Eligible women were: (i) aged 50–74 years; (ii) not UTD (overdue) for one or more recommended screenings for BC, CC, or CRC; (iii) had no personal history of cancer (other than non-melanoma skin cancer); (iv) resided in one of the target counties in Indiana or Ohio; (v) able to speak/read English; (vi) able to provide informed consent; and (vii) able to access a device to play a DVD. The U.S Preventive Services Task Force 2016 guidelines that were published at the time of the start of the study were used to determine whether a woman was adherent to screening guidelines [2–4].
Potentially eligible women were mailed a letter and information sheet introducing the study, followed by a telephone call by trained interviewers. In addition, potentially eligible women were able to contact the study team via a toll-free number or through the study’s email address. Women who clicked targeted Facebook ads were directed to a REDCap page that included a description of the study, eligibility criteria, and a brief set of questions to partially assess eligibility and determine the best time(s) for an interviewer to contact the individual to complete the consent process and baseline survey.
During this initial phone call, interviewers explained the study, verified eligibility, obtained verbal informed consent and conducted the baseline telephone survey. Of 1852 eligible participants who verbally consented to complete the baseline survey, 209 refused to provide consent to medical record review (MRR) to verify screening status for each of the three cancers. Participants were then mailed an additional short survey and a medical record release form with a stamped, pre-addressed envelope. Those who did not return the release form within 2 weeks were re-contacted by the interviewer. Documentation of screening tests in the medical record were verified by each woman’s health care provider. Among the remaining 1643 women, 658 were found to be ineligible after MRR, leaving 985 eligible women randomized to one of three study arms: (i) 382 to the tailored interactive DVD alone (DVD); (ii) 388 to DVD+PN, and (iii) 194 to UC. Randomization was stratified by age (50-64 vs. 65–74 years old) and by seven categories reflecting each woman’s needed cancer screening(s): (BC only, CC only, CRC only, BC+CC, BC+CRC, CC+CRC, or BC+CC+CRC).
Only women who were randomized to the two intervention study arms—those who received the DVD alone (n = 382) or the DVD+PN (n = 388) were asked to complete the process interview.
Data collection and measures for process evaluation
Data were collected through structured telephone interviews to evaluate receipt, uptake, and satisfaction with the DVD and DVD+PN interventions. Process evaluation interview calls commenced 2 months after mailing the DVD intervention. Ten attempts were made to reach participants by phone interviewers. Interview questions focused on the DVD receipt, viewing, and if participants had discussions with others about the DVD content. Four closed-ended items asked whether women had received the DVD (yes/no), viewed it (yes/no), which sections were watched (introduction, breast, cervical, and/or colorectal), and whether they discussed the DVD with others (yes/no). Women who did not watch the DVD were asked an open-ended question about why they did not watch it, and women who responded that they discussed the DVD with others were asked to indicate with whom they discussed it (spouse/partner, child, other family member, friend/neighbor, doctor/healthcare provider, or others).
Participants who viewed the DVD were then asked about their perceptions of the relevance, helpfulness, and satisfaction with the DVD’s content. Women were asked to indicate their level of agreement with 12 statements on a 5-point Likert scale where 1 = strongly disagree to 5 = strongly agree. Statements included that the DVD: was easy to understand, was helpful, gave information that was new, gave information that was important, was embarrassing, took too long, was convenient, gave information that could be trusted, included women like me, included scenes I could relate to, included people I could trust, and should be sent to other women in my community.
To assess receipt of the PN intervention, women randomized to the DVD+PN group were asked if they received a call from the navigator (yes/no), had talked to the navigator (yes/no) or had intention to speak with the navigator (yes/no). If no contact occurred, reasons for not talking with the navigator were documented. If participants reported talking with the navigator, data were collected about the content of their discussion (BC, CC, and/or CRC screening) and whether they discussed screening with anyone after the navigator call (yes/no). Those who reported a discussion with others were asked to indicate whom they discussed screening with (spouse/partner, child, other family member, friend/neighbor, doctor/healthcare provider, or others).
For participants who talked with the navigator, satisfaction, relevance, and helpfulness of the navigator was then assessed. Women were asked to indicate their level of agreement with 11 statements on a 5-point Likert scale where 1 = strongly disagree to 5 = strongly agree. Statements included that the navigator: was easy to understand, was helpful, gave information that was new, gave information that was important, discussed things that were embarrassing, covered just the right amount of information, took too much time, was convenient, was someone I could trust, sounded like she knew people in my community, and should talk to other women in my community about screening. Women were also asked to indicate their level of agreement on the same 5-point Likert scale with two additional statements: (i) the navigator gave me the information I needed to get: BC, CC, and/or CRC screening; and (ii) the navigator helped me decide to get BC, CC, and/or CRC screening.
Data on the number and types of contacts made by the patient navigator as well as the types of actions taken by navigators, including barriers addressed, were extracted from the computer-based navigator documentation database created for this study.
Analysis
Descriptive statistics are reported for the process evaluation. Specifically, categorical, and continuous variables were described with frequencies (%) and means (standard deviation), respectively. We also compared the demographic characteristics of women who completed the process interview (n = 542) with those who did not complete the process interview (n = 228) using Pearson’s chi-squared test and two-sided Wilcoxon rank sum test for categorical and continuous demographic variables, respectively. The item response metrics of variables are described in the section above. Perceptions of the relevance, helpfulness, and satisfaction (12 items pertaining to DVD content and 11 items about the PN) were analyzed at the item level (i.e. as categorical variables) to report a rich and nuanced descriptive assessment, and because the percentage who agreed or strongly agreed provided an intuitive descriptive metric. Which variables were analyzed categorically or continuously is clearly discerned in all tables where we report frequency and percentage or mean (SD) and median (IQR) and range, respectively.
Results
Participant characteristics
Of 770 participants assigned to receive the interventions, 542 (70.4%) were reached to complete the process interview and provided data for this analysis. No significant differences were observed in the baseline characteristics of women who completed the process interview compared with those who did not (Table 1); no differences between completers and non-completers were observed when tested separately within intervention arm (not shown). Participants who completed the process interviews had a mean age of 58.8 years, 97.8% were White, 79.3% were married or living as married, and 86.8% had at least some college. Two-thirds were currently employed, 77.2% had annual household incomes above $40 000 with 41.0% reporting annual incomes above $80 000, and 95.7% had health insurance.
Table 1.
Baseline characteristics by process evaluation completion status
| Characteristic | Overall n = 7701 |
Completed process interview n = 542 |
Did not complete process interview n = 228 |
P-value2 |
|---|---|---|---|---|
| Age at consent | .053 | |||
| Mean (SD) | 58.5 (6.3) | 58.8 (6.5) | 57.7 (5.8) | |
| Median (IQR) | 58.0 (53.0, 63.0) | 58.0 (53.0, 64.0) | 57.0 (53.0, 62.0) | |
| Range | 50.0, 74.0 | 50.0, 74.0 | 50.0, 74.0 | |
| Race | .484 | |||
| Non-White | 19 (2.5) | 12 (2.2) | 7 (3.1) | |
| White | 751 (97.5) | 530 (97.8) | 221 (96.9) | |
| Marital status | .061 | |||
| Married/living as married | 592 (77.0) | 429 (79.3) | 163 (71.5) | |
| Divorced/widowed/separated | 146 (19.0) | 93 (17.2) | 53 (23.2) | |
| Never married | 31 (4.0) | 19 (3.5) | 12 (5.3) | |
| Unknown | 1 | 1 | 0 | |
| Education | .129 | |||
| HS/GED or less | 114 (14.8) | 71 (13.1) | 43 (18.9) | |
| Some college or associate degree | 297 (38.6) | 210 (38.7) | 87 (38.2) | |
| Bachelor’s degree | 192 (24.9) | 135 (24.9) | 57 (25.0) | |
| Graduate degree | 167 (21.7) | 126 (23.2) | 41 (18.0) | |
| Currently employed | .538 | |||
| No | 255 (33.1) | 181 (33.4) | 74 (32.5) | |
| Yes—part time | 134 (17.4) | 89 (16.4) | 45 (19.7) | |
| Yes—full time | 381 (49.5) | 272 (50.2) | 109 (47.8) | |
| Household Income | .825 | |||
| <$40 000 | 144 (18.7) | 103 (19.0) | 41 (18.0) | |
| $40 000–$79 999 | 281 (36.5) | 196 (36.2) | 85 (37.3) | |
| $80 000+ | 318 (41.3) | 222 (41.0) | 96 (42.1) | |
| Unknown | 27 (3.5) | 21 (3.9) | 6 (2.6) | |
| Health insurance status | .132 | |||
| None | 37 (4.8) | 23 (4.3) | 14 (6.1) | |
| Public only | 74 (9.6) | 50 (9.2) | 24 (10.5) | |
| Private only | 529 (68.8) | 367 (67.8) | 162 (71.1) | |
| Public and private | 129 (16.8) | 101 (18.7) | 28 (12.3) | |
| Unknown | 1 | 1 | 0 | |
| Smoking status | .162 | |||
| Never | 505 (65.6) | 348 (64.2) | 157 (68.9) | |
| Former | 198 (25.7) | 151 (27.9) | 47 (20.6) | |
| Current | 46 (6.0) | 29 (5.4) | 17 (7.5) | |
| Unknown | 21 (2.7) | 14 (2.6) | 7 (3.1) |
Table includes participants randomized to the DVD and DVD+PN groups at baseline.
HS/GED, High School/General Educational Development.
1“Mean (SD = standard deviation),” “Median (IQR = interquartile range).
2Pearson’s Chi-squared test; Wilcoxon rank sum test.
DVD intervention
Descriptive statistics for the DVD intervention included receipt, adherence with viewing, and occurrence of discussions after DVD viewing (Table 2). All participants reported receiving the DVD, and 93.0% viewed it. Among the 39 women who did not view the DVD, 38 provided reasons (i.e. barriers) for not viewing it which included not having time (28.9%), failure to get around to it (21.1%), a malfunctioning DVD player (44.7%), or the DVD being difficult to use (7.9%). A high percentage of participants watched DVD sections for each of the three cancer types (63%–77%). The sections with the highest viewing were CRC (76.9%) and BC (75.9%). Almost half (45.9%) of participants reported discussing the DVD with someone else after viewing it, with the most frequent discussion being with the participant’s health care provider (37.7%).
Table 2.
DVD receipt, adherence with viewing, and discussions (n = 542)
| n | % | |
|---|---|---|
| DVD receipt and viewing | ||
| Received DVD: Yes | 542 | 100 |
| Viewed DVD: Yes | 503 | 93.0 |
| Barriers to viewing DVD (n = 38)a | ||
| No time | 11 | 28.9 |
| Did not get around to it | 8 | 21.1 |
| No DVD player/not working | 17 | 44.7 |
| DVD difficult to use | 3 | 7.9 |
| Other | 3 | 7.9 |
| DVD sections watched:a | ||
| Introduction only | 46 | 9.1 |
| Breast | 382 | 75.9 |
| Cervical | 317 | 63.0 |
| Colorectal | 387 | 76.9 |
| DVD discussions | ||
| Discussed DVD with others: Yes | 231 | 45.9 |
| Discussed DVD with:a | ||
| Spouse/partner | 44 | 19.0 |
| Child | 18 | 7.8 |
| Other family member | 45 | 19.5 |
| Friend/neighbour | 64 | 27.7 |
| Doctor/Health care provider | 87 | 37.7 |
| Other (co-workers, patients) | 18 | 7.8 |
DVD, Digital Video Disc.
aPercentages exceed 100% because multiple responses were allowed. For example, 38 women reported 42 barriers to viewing the DVD.
Relevance, helpfulness and satisfaction with the DVD
Results for participants’ perceptions of the relevance, helpfulness of, and satisfaction with the DVD are shown in Table 3. Overall, the percentage of participants who rated “strongly agree or agree” was high for satisfaction, relevance, and helpfulness. Items that more than 90% of participants strongly agreed or agreed with included the DVD:"was easy to understand” (99.4%), “included people I could trust” (93.6%), “included scenes I could relate to” (92.0%), “gave important information” (97.8%), “gave trusted information” (96.6%), “was helpful” (95.0%), “was convenient” (91.8%), and “should be sent to other women in my community” (90.6%). Most women strongly agreed or agreed that the DVD gave them information they needed to get BC (86.6%), CC (87.0%), and CRC screening (91.0%). About half of women agreed that the DVD gave them information that helped them decide to get BC (49.9%) or CC screening (47.3%) while a larger percentage reported it helped them decide to get CRC screening (62.5%). Very few women reported that the DVD was embarrassing (1.4%) or took too long to watch (5.4%).
Table 3.
DVD: satisfaction, relevance and helpfulness (n = 503 patients who viewed DVD)
| Strongly agree/agree | Neutral/not sure | Strongly disagree/disagree | |
|---|---|---|---|
| n (%) | n (%) | n (%) | |
| Participants’ perception of DVD | |||
| Easy to understand | 500 (99.4) | 3 (0.6) | – |
| Helpful | 478 (95.0) | 13 (2.6) | 12 (2.4) |
| Gave information that was new to me | 275 (54.7) | 56 (11.1) | 172 (34.2) |
| Gave information that was important | 492 (97.8) | 6 (1.2) | 5 (1.0) |
| Embarrassing | 7 (1.4) | 3 (0.6) | 493 (98.0) |
| Took too long to watch | 27 (5.4) | 15 (3.0) | 461 (91.7) |
| Gave information I could trust | 485 (96.6) | 12 (2.4) | 5 (1.0) |
| Convenient | 462 (91.8) | 12 (2.4) | 29 (5.8) |
| Included women like me | 441 (88.9) | 45 (9.1) | 10 (2.0) |
| Included scenes I could relate to | 460 (92.0) | 34 (6.8) | 6 (1.2) |
| Included people I could trust | 467 (93.6) | 30 (6.0) | 2 (0.4) |
| Should be sent to other women in my community | 453 (90.6) | 36 (7.2) | 11 (2.2) |
| Gave information I needed to get | |||
| Breast cancer screening (n = 382) | 331 (86.6) | 21 (5.5) | 30 (7.9) |
| Cervical cancer screening (n = 315) | 274 (87.0) | 18 (5.7) | 23 (7.3) |
| Colorectal cancer screening (n = 387) | 352 (91.0) | 14 (3.6) | 21 (5.4) |
| Gave information that helped me decide to get | |||
| Breast cancer screening (n = 381) | 190 (49.9) | 42 (11.0) | 149 (39.1) |
| Cervical cancer screening (n = 317) | 150 (47.3) | 35 (11.0) | 132 (41.6) |
| Colorectal cancer screening (n = 387) | 242 (62.5) | 30 (7.8) | 115 (29.7) |
DVD, Digital Video Disc.
Patient navigation intervention
For the PN intervention, descriptive statistics for the receipt and for whether participants had screening discussions with the navigator during the call and/or with others after the call are shown in Table 4. Almost all women recalled receiving a call from the navigator (98.9%, n = 273), and 98.2% (n = 268) reported talking with the navigator. Among five women who did not speak with the navigator, reasons given were having no time and not getting around to watching the DVD (n = 2 each). The screenings that the largest percentage of women discussed with the navigator were CRC (86.8%) and BC (71.3%); only 57.5% of participants discussed CC screening with the navigator. About half of participants discussed screening with someone after talking with the navigator, and the most frequent discussion was with the participant’s health care provider (45.6%).
Table 4.
Patient navigation (PN): receipt and discussions (n = 276 patients in DVD+PN group who completed the process evaluation interview)
| n | % | |
|---|---|---|
| PN receipt | ||
| Received call: Yes | 273 | 98.9 |
| Talked with navigator: Yes | 268 | 98.2 |
| Reason for not talking with navigator (n = 5) | ||
| No time | 2 | 40.0 |
| Didn’t get around to it (hadn’t watched the DVD) | 2 | 40.0 |
| Other (rescheduled call, but no connection) | 1 | 20.0 |
| PN screening discussions | ||
| Breast cancer screening | 189 | 71.3 |
| Cervical cancer screening | 153 | 57.5 |
| Colorectal cancer screening | 230 | 86.8 |
| Discussed screening with anyone after talking with navigator: Yes | 136 | 51.3 |
| Discussed with:a | ||
| Spouse/partner | 30 | 22.1 |
| Child | 6 | 4.4 |
| Other family member | 28 | 20.6 |
| Friend/neighbour | 31 | 22.8 |
| Doctor/health care provider | 62 | 45.6 |
| Other (co-workers, insurance co., patients) | 8 | 5.9 |
aCounts exceed 100% because multiple responses allowed.
Relevance, helpfulness, and satisfaction with the patient navigator
Data on participants’ perceptions of relevance, helpfulness, and satisfaction with the PN intervention are shown in Table 5. Items that 90% or more of participants strongly agreed or agreed included that the navigator: “was easy to understand” (99.6%), “was helpful” (98.5%), “gave information that was important” (97.4%), “covered just the right amount of information” (96.2%), “was convenient” (95.1%), and “was someone I could trust” (94.0%). A few women strongly agreed or agreed that the navigator “discussed things that were embarrassing” (6.0%) or that the navigator “took too much time” (4.1%). Only 20.1% thought “the navigator sounded like she knew people in my community,” although 89.4% strongly agreed or agreed that “the navigator should talk to other women in my community.” More than three-quarters of women strongly agreed or agreed that the navigator gave them the information they needed to get BC (77.0%), CC (77.6%), and CRC screening (83.9%). More than half of participants strongly agreed or agreed that the navigator helped them decide to get screened for BC (54.8%) and CC (51.0%) while almost two-thirds (64.8%) agreed that the navigator helped them decide to get screened for CRC.
Table 5.
Satisfaction, relevance and helpfulness of patient navigator (n = 268 patients in DVD+PN group who talked with navigator)
| Strongly agree/agree | Neutral/not sure | Strongly disagree/disagree | |
|---|---|---|---|
| n (%) | n (%) | n (%) | |
| The Navigator… | |||
| Was easy to understand | 266 (99.6) | – | 1 (0.4) |
| Was helpful | 262 (98.5) | 2 (0.8) | 2 (0.8) |
| Gave me information that was new | 154 (57.7) | 34 (12.7) | 79 (29.6) |
| Gave me information that was important | 259 (97.4) | 2 (0.8) | 5 (1.9) |
| Discussed things that were embarrassing | 16 (6.0) | 5 (1.9) | 246 (92.1) |
| Covered just the right amount of information | 256 (96.2) | 4 (1.5) | 6 (2.3) |
| Took too much time | 11 (4.1) | 3 (1.1) | 252 (94.7) |
| Was convenient | 254 (95.1) | 8 (3.0) | 5 (1.9) |
| Sounded like she knew people in my community | 52 (20.1) | 116 (44.8) | 91 (35.1) |
| Was someone I could trust | 251 (94.0) | 13 (4.9) | 3 (1.1) |
| Should talk to other women in my community about screening | 235 (89.4) | 23 (8.7) | 5 (1.9) |
| Gave me the information I needed to get | |||
| Breast cancer screening (n = 187) | 144 (77.0) | 10 (5.3) | 33 (17.6) |
| Cervical cancer screening (n = 152) | 118 (77.6) | 8 (5.3) | 26 (17.1) |
| Colorectal cancer screening (n = 230) | 193 (83.9) | 9 (3.9) | 28 (12.2) |
| Helped me decide to get | |||
| Breast cancer screening (n = 188) | 103 (54.8) | 25 (13.3) | 60 (31.9) |
| Cervical cancer screening (n = 151) | 77 (51.0) | 23 (15.2) | 51 (33.8) |
| Colorectal cancer screening (n = 230) | 149 (64.8) | 24 (10.4) | 57 (24.8) |
Time spent and actions taken by patient navigators
Data on the amount of time spent with, and on behalf of, participants as well as the types of actions taken by navigators are listed in Table 6. Of the 276 participants assigned to receive navigation, 95.3% had at least one encounter (phone conversation) with a navigator. Participants had an average of four encounters with navigators with a range of 1–14. The total length of encounters as documented by navigators was 22.2 minutes on average with a range of 1–136 minutes. Navigators also documented the amount of time they spent on “coordinating” activities which included the amount of time they spent preparing for and following-up on the encounter. Such coordinating activities included finding resources for the participant, contacting clinics/providers, mailing additional materials, or making appointments. Navigators spent an average of 21.7 minutes, with a range of 1–102 minutes on coordinating activities for each participant. The total number of contacts made by navigators to reach participants was 22.8 (range = 2–73), with most being telephone contacts (mean = 15.2, range = 1–48). In addition, navigators’ actions included sending participants additional educational materials by U.S. mail an average of 5.3 times (range = 1–17), by email an average of five times (range 1–35), and by text only one time.
Table 6.
Navigator encounter and time data (n = 276 patients in DVD+PN group who completed the process interview)
| Characteristic | n = 2761 |
|---|---|
| Total number of contacts per participant by navigator | |
| Mean (SD) | 22.8 (14.3) |
| Median (IQR) | 20.0 (11.0, 32.0) |
| Range | 2.0, 73.0 |
| Total number of telephone contacts per participant by navigator | |
| Mean (SD) | 15.2 (10.3) |
| Median (IQR) | 12.0 (8.0, 21.0) |
| Range | 1.0, 48.0 |
| Total number of U.S. mail contacts per participant by navigator | |
| Mean (SD) | 5.3 (3.2) |
| Median (IQR) | 5.0 (3.0, 7.0) |
| Range | 1.0, 17.0 |
| Total number of email contacts per participant by navigator | |
| Mean (SD) | 5.0 (6.3) |
| Median (IQR) | 2.0 (1.0, 6.0) |
| Range | 1.0, 35.0 |
| Total number of text contacts per participant by navigator | |
| Mean (SD) | 1.0 (NA) |
| Median (IQR) | 1.0 (1.0, 1.0) |
| Range | 1.0, 1.0 |
| Number of encounters per participant | |
| 0 Navigator encounters | 13 (4.7%) |
| 1 or more navigator encounters | 263 (95.3%) |
| Number of encounters per participant | |
| Mean (SD) | 4.0 (2.4) |
| Median (IQR) | 4.0 (2.0, 5.5) |
| Range | 1.0, 14.0 |
| Median encounter length per participant (minutes) | |
| <5 | 51 (19.4%) |
| 5–14 | 177 (67.3%) |
| 15–29 | 34 (12.9%) |
| 30–44 | 1 (0.4%) |
| 45–60+ | 0 (0.0%) |
| Total encounter time per participant (minutes) | |
| Mean (SD) | 22.2 (21.1) |
| Median (IQR) | 16.0 (8.0, 26.5) |
| Range | 1.0, 136.0 |
| Median coordinating time per participant (minutes) | |
| <5 | 84 (31.9%) |
| 5–14 | 147 (55.9%) |
| 15–29 | 30 (11.4%) |
| 30–44 | 2 (0.8%) |
| 45–60+ | 0 (0.0%) |
| Total coordinating time (minutes) | |
| Mean (SD) | 21.7 (14.0) |
| Median (IQR) | 18.0 (15.0, 25.0) |
| Range | 1.0, 102.0 |
1 n (%).
Barriers addressed by patient navigators
Navigators devoted a significant amount of time during their encounters discussing reported barriers to screening and offering support to overcome them. Figure 1 illustrates the barriers addressed by navigators in descending order of frequency. The most frequently discussed barriers were related to screening not being a priority because the participant had no symptoms/problems and concerns about the health care provider or health system (no screening recommendation, no provider, lack of trust, poor communication or discrimination, lack of knowledge and/or time).
Figure 1.
Number of barriers addressed by navigators by type of screening test. *Type of barrier on y-axis is sorted according to total number of times each type was checked for all four tests (Breast + Cervical + Colonoscopy + FOBT/FIT) with greatest number at the top of the graph and smallest number at the bottom
Most barriers were similar across screening tests but slight differences were noted (Table 7). While the top five barriers that were most frequently discussed were fairly consistent across screenings (not a priority/no symptoms, provider/health care system barriers, lack of knowledge, too much bother/forgetfulness, and other health problems/disability), the number of navigator encounters where barriers to CRC screening were discussed was higher than other screening tests. Of the 3171 total barriers to screening discussed with navigators, 1904 (60.0%) were related to CRC screening with 1009 (31.8%) barriers to stool testing and 895 (28.2%) barriers to colonoscopy. Lack of knowledge and screening not being a priority was the most frequently cited barrier for stool blood testing.
Table 7.
Frequency of barriers addressed by navigators by screening tests (among 1281 patient-navigator encounters for patients in DVD+PN group who talked with a navigator)
| Barrier category | BC Screening |
CC Screening |
CRC Screening: colonoscopy |
CRC Screening: stool test |
Total |
|---|---|---|---|---|---|
| Not a priority/no symptoms | 100 | 86 | 117 | 164 | 467 |
| Provider/healthcare system (lack of provider, no recommendation, mistrust, discrimination) | 66 | 133 | 90 | 125 | 414 |
| Lack of knowledge | 51 | 73 | 71 | 167 | 362 |
| Too much bother/forgetfulness | 75 | 53 | 56 | 106 | 290 |
| Other health problems/disability | 50 | 45 | 78 | 77 | 250 |
| Insurance: none or inadequate | 41 | 41 | 72 | 76 | 230 |
| Lack of time | 36 | 32 | 61 | 64 | 193 |
| Family responsibilities | 36 | 24 | 56 | 59 | 175 |
| No family history/not at risk | 22 | 64 | 27 | 31 | 144 |
| Cost of the test | 23 | 18 | 23 | 35 | 99 |
| Fear of the procedure | 32 | 5 | 41 | 21 | 99 |
| Facility (distance, unfamiliar, long wait for appointment) | 33 | 11 | 29 | 24 | 97 |
| Employment/no time off | 13 | 8 | 30 | 22 | 73 |
| Prep concerns | 0 | 0 | 64 | 1 | 65 |
| Discouragement from others | 9 | 4 | 18 | 6 | 37 |
| Lack of transportation | 3 | 2 | 24 | 3 | 32 |
| Fear of results | 7 | 6 | 8 | 9 | 30 |
| Pain/discomfort | 18 | 2 | 5 | 1 | 26 |
| Embarrassment | 4 | 10 | 9 | 2 | 25 |
| Other | 6 | 6 | 6 | 5 | 23 |
| Religious/cultural beliefs | 6 | 4 | 7 | 5 | 22 |
| Procedure is messy/gross | 0 | 0 | 3 | 5 | 8 |
| Exposure to radiation | 7 | 0 | 0 | 0 | 7 |
| Worry about results | 2 | 0 | 0 | 1 | 3 |
| Totals | 640 | 627 | 895 | 1009 | 3171 |
BC, breast cancer; CC, cervical cancer; CRC, colorectal cancer.
Discussion
This paper reports the results of process evaluations for two interventions, DVD compared to DVD+PN, designed to promote BC, CC, and CRC screening among rural women who were not UTD with one or more screenings. In this study, all participants who completed the process interview reported receiving the DVD in the mail and 93% viewed it. Participants overwhelmingly agreed that the DVD was easy to understand, helpful, provided trustworthy information, and gave them information needed to make a decision about screening. Development of the DVD intervention required substantial investment of time and resources, but once developed, dissemination and delivery of the DVD through the U.S. mail were relatively easy and inexpensive.
Receipt, uptake, and satisfaction with the mailed DVD used in this study was very high and superior to several studies testing similar interventions [12, 14–16]. In another study conducted by this team, results of a process evaluation of a mailed interactive DVD designed to promote CRC screening among low-income and minority patients in a safety net hospital who did not attend their screening colonoscopy appointment were similar [12]. Katz et al. reported that of 243 participants randomized to receive the interactive DVD, 194 (80%) viewed it, 99% reported the DVD was helpful and 99% would recommend it to others. In the current study of rural women, a higher percentage (93%) of participants viewed the mailed DVD. Since this DVD included information on BC, CC, and CRC screening, the content may have been more salient to this study population.
In another study testing the effects of a complex intervention delivered to patients who had an expired order for colonoscopy, the intervention consisted of a personalized reminder letter from their physician, an educational brochure and a mailed DVD about CRC and screening [14]. Of 109 patients contacted for the process evaluation, 95% reported receiving the mailed materials, 98% reported reading the letter and brochure, but only 30% watched the DVD. Reasons for not watching the DVD were that participants: were too busy/had no time (32%); already had sufficient information about screening (21%); were not interested in screening (13%); or did not have or were unable to operate a DVD player (10%).
Levy and colleagues [15] conducted a process evaluation of a multilevel intervention to increase CRC screening. The intervention consisted of a mailed DVD plus written materials from the American Cancer Society and the Centers for Disease Control. They found that 67% of participants reported they viewed at least 75% of the DVD, 94% reported they understood at least 75% of the material, and 96% reported that the amount of material was just right. Zapka and colleagues [16] tested the effects of a mailed video to promote CRC screening to patients prior to a healthcare provider visit. Among 450 patients who received the mailed video, 193 (43%) viewed at least 50% of the video and 174 (39%) watched all of it. Of 243 patients who watched less than 50% of the video, 43% watched none of it. Screening outcomes were directly tied to viewing the video.
In our study, higher viewership of the DVD may be due to a variety of reasons. Paramount to the success of these interventions was our use of community-engaged strategies in the planning, development, implementation, and evaluation of the interventions. We placed considerable value on feedback from community members and the input and recommendations from the study’s community advisory board members. Additionally, this was a gender-specific study and females tend to be more focused on their health and willing to participate in, and follow through, with health-related interventions. Furthermore, the interactive nature of the DVD may have resulted in higher levels of engagement and viewership. Specifically, DVD messages were tailored to address the specific cancer screening needs of each user and included messages delivered by community members to address the unique barriers identified by each participant. The tailoring process required women to answer several questions using the remote control requiring active engagement of the user while viewing the entire DVD. Finally, women assigned to the DVD+PN group were reminded to view the DVD by navigators, given time to review the DVD if needed, and called back at a later time to complete navigation.
Patient navigation has been widely adopted in recent years and found to be highly effective in promoting cancer screening uptake [17, 18]. A comprehensive review of the efficacy and cost-effectiveness of PN programs showed positive outcomes, specifically increased uptake and adherence to cancer screening [19]. Recent reviews of studies testing interventions to increase BC and CC screening among rural women showed that PN is effective [20, 21]. Details about the reach, uptake, and satisfaction with navigation, however, were limited in these reviews. Falk made a strong case for the use of social workers as navigators which were employed in our study [20].
In the current study, patient navigators encouraged women to view the DVD (if they had not already done so), assessed their knowledge about the screening(s) for which they were not UTD, and addressed barriers to getting screened. Almost all (98.9%) women assigned to receive the PN intervention remembered receiving a call from the navigator and 98% reported talking with the navigator. The ability of navigators to reach participants by phone in this study exceeded those in other studies [22, 23]. Reasons for the success of navigators in this study included their experience navigating participants in previous cancer screening intervention studies, their understanding of the culture of rural women since they are residents of rural counties themselves, and their persistence because of their belief that they are helping other women. Navigators made 10 attempts to reach participants and varied those attempts across calls, texts, and emails as well as days, evenings, and weekends. If navigators did not reach the participant after 10 attempts, a letter with the navigator’s contact information was mailed to the participant stating the navigator was trying to reach them and asked for a call back. Finally, most women had received and viewed the DVD prior to being contacted by the navigator which may have contributed to their acceptance of navigation.
In a trial showing navigators were effective at promoting BC, CC, and CRC screening among 792 patients in a primary care practice network who were overdue for screening, 151 (19%) patients assigned to the intervention could not be reached despite an average of 3.3 attempts [22]. Myers and colleagues tested a navigation intervention to increase CRC screening and reported that the navigator was unable to reach 31 (16%) patients assigned to receive navigation [23].
Consistent with most studies, the majority of participants reported being highly satisfied with navigation and the services navigators provided [24–26]. Navigators made an average number of 22.8 contacts with participants including communication by phone, email, U.S. mail, and text. Navigators had an average of four encounters per participant, spending an average of 22 minutes in total encounter time with participants. Navigators also spent an average of 22 minutes on coordinating activities on behalf of participants.
In this study, barriers discussed by navigators with participants were fairly consistent across types of screening and included screening not being a priority/no symptoms, provider/health care system barriers, lack of knowledge, too much bother/forgetfulness, and other health problems/disability. Common barriers to CC screening were slightly different and included perceptions of not being at risk for CC—a belief that may be attributed to several factors. Older women enrolled in the study who had a hysterectomy were no longer at risk for CC. In addition, like most cancers, it is a common belief that one is not at risk for that cancer if they have no family history of the disease. Barriers addressed in this study were similar to those reported in a national study of patient barriers identified by 437 navigators employed in two large federally-funded cancer screening programs. Barriers to BC, CC, and CRC screening were similar and included: (i) lack of knowledge about cancer, cancer screening procedures, and the benefits of screening; (ii) low motivation to get screened; (iii) transportation; and (iv) health insurance [27].
Limitations
Several limitations should be considered when interpreting results. First, participants in this study reflected the population of rural women in Ohio and Indiana; they were primarily White, married, well-educated, and had health insurance. Rural women of other races who are less educated and under- or uninsured may have responded differently to the DVD or PN interventions. Second, while process data were collected from 93% of participants who viewed the DVD and 98% who spoke with the navigator, we were unable to reach all who were randomized to receive these interventions to assess receipt, uptake, and satisfaction. Of 770 participants assigned to receive the interventions, 542 (70.4%) were reached to complete the process interview and provided data for this analysis. Although satisfaction with both DVD and PN was high, the ability to implement either into clinical practice has yet to be tested. Additionally, while the DVD was relatively inexpensive to deliver, the cost of development was substantial and changes in guidelines for screening now necessitate changes to the DVD content. In a similar manner, satisfaction with PN was high but the feasibility of a busy clinic supporting at least four or more navigation calls for each woman who need cancer screening needs to be explored.
Conclusion/Implications
The current process evaluation provided important data regarding the reach/receipt, uptake, and satisfaction with both behavioral interventions to promote BC, CC, and CRC screening among rural women who were not UTD with at least one screening test. Process evaluation provides important information to determine the reach and extent a behavioral intervention was implemented as intended, the quality, relevance, and acceptability of the intervention’s content among participants, and identifies areas for improvement. These issues are critical to understand the results of interventions tested in randomized controlled trials and for dissemination of effective trials.
Process evaluation is often overlooked when planning and implementing health promotion interventions because it can be tedious, time-consuming, incurs additional cost, and requires additional record-keeping. However, it often provides clues about why an intervention did or did not produce the desired behavioral change. Future investigators who are planning clinical trials need to include a comprehensive process evaluation so that implementation and fidelity of the intervention are monitored since these factors influence the outcome of every intervention trial. Process evaluation will support development of future interventions that are acceptable to participants which may increase intervention effects.
Contributor Information
Susan M Rawl, Simon Comprehensive Cancer Center, School of Nursing, Indiana University, Indianapolis, IN, USA.
Ryan Baltic, Comprehensive Cancer Center, The Ohio State University, Columbus, OH, USA.
Patrick O Monahan, Department of Biostatistics and Health Data Science, School of Medicine, Indiana University, Simon Comprehensive Cancer Center, Indianapolis, IN, USA.
Timothy E Stump, Department of Biostatistics and Health Data Science, School of Medicine, Indiana University, Indianapolis, IN, USA.
Madison Hyer, Center for Biostatistics, College of Medicine, The Ohio State University, Columbus, OH, USA.
Alysha C Ennis, College of Public Health, The Ohio State University, Columbus, OH, USA.
Jean Walunis, Comprehensive Cancer Center, The Ohio State University, Columbus, OH, USA.
Katherine Renick, The Ohio State University, Columbus, OH, USA.
Karen Hinshaw, Indiana University School of Medicine, Indianapolis, IN, USA.
Electra D Paskett, College of Medicine, Comprehensive Cancer Center, The Ohio State University, Columbus, OH, USA.
Victoria L Champion, School of Nursing, Indiana University, Simon Comprehensive Cancer Center, Indianapolis, IN, USA.
Mira L Katz, College of Public Health, Comprehensive Cancer Center, The Ohio State University, Columbus, OH, USA.
Conflict of Interest
EP reports being the principal investigator on grants to The Ohio State University from Pfizer, Merck Foundation, Guardant Health, and Genentech, not related to this research. She is also an Advisory Board Member for Glaxo Smith Kline. The remaining authors have no relevant financial or non-financial interests to disclose.
Funding
National Cancer Institute (R01 CA196243), Recruitment, Intervention and Survey Shared Resource at The Ohio State University Comprehensive Cancer Center (P30 CA016058), and the Ohio State University CTSA (UL1TR002733) from the National Center for Advancing Translational Sciences. Funding sponsors did not have any role in the study design; collection, analysis, and interpretation of data; writing the report; and the decision to submit the report for publication.
Human rights
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was approved by The Ohio State University and Indiana University Institutional Review Boards. This article does not contain any studies with animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Study registration
This study is registered at www.clinicaltrials.gov NCT02795104 as of October 2016.
Analytic plan pre-registration
The analysis plan was registered prior to beginning data collection at www.clinicaltrials.gov NCT02795104 as of October 2016.
Data Availability
De-identified data from this study are not available in a public archive. De-identified data will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.
Analytic code availability
Analytic code used to conduct the analyses presented in this study are not available in a public archive. They may be available by emailing the corresponding author.
Materials availability
Materials used to conduct the study are not publicly available
References
- 1. Sabatino SA, Thompson TD, White MCet al. Cancer screening test receipt—United States, 2018. Morb Mortal Wkly Rep 2021;70:29–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Curry SJ, Krist AH, Owens DKet al.; US Preventive Services Task Force. Screening for cervical cancer: US preventive services task force recommendation statement. JAMA 2018;320:674–86. [DOI] [PubMed] [Google Scholar]
- 3. Siu AL; U.S. Preventive Services Task Force. Screening for breast cancer: U.S. preventive services task force recommendation statement. Ann Intern Med 2016;164:279–96. [DOI] [PubMed] [Google Scholar]
- 4. Bibbins-Domingo K, Grossman DC, Curry SJet al.; US Preventive Services Task Force. Screening for colorectal cancer: US preventive services task force recommendation statement. JAMA 2016;315:2564–75. [DOI] [PubMed] [Google Scholar]
- 5. Sabatino SA, Lawrence B, Elder Ret al.; Community Preventive Services Task Force. Effectiveness of interventions to increase screening for breast, cervical, and colorectal cancers: nine updated systematic reviews for the guide to community preventive services. Am J Prev Med 2012;43:97–118. [DOI] [PubMed] [Google Scholar]
- 6. Biederman E, Baltic R, Katz MLet al. Increasing breast, cervical, and colorectal cancer screening among rural women: baseline characteristics of a randomized control trial. Contemp Clin Trials 2022;123:106986. [DOI] [PubMed] [Google Scholar]
- 7. Katz ML, Reiter PL, Young GS, et al. Adherence to multiple cancer screening tests among women living in Appalachia Ohio. Cancer Epi Bio & Prev 2015;24;1489–1494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Champion VL, Monahan PO, Stump TEet al. The effect of two interventions to increase breast cancer screening in rural women. Cancers (Basel) 2022;14:4354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Moore GF, Audrey S, Barker Met al. Process evaluation of complex interventions: Medical Research Council guidance. Bmj 2015;350:h1258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Luckmann R, White MJ, Costanza MEet al. Implementation and process evaluation of three interventions to promote screening mammograms delivered for 4 years in a large primary care population. Transl Behav Med 2017;7:547–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Skinner CS, Buchanan A, Champion Vet al. Process outcomes from a randomized controlled trial comparing tailored mammography interventions delivered via telephone vs. DVD. Patient Educ Couns 2011;85:308–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Katz ML, Emerson B, Champion VLet al. Process evaluation of a mailed interactive educational DVD in a comparative effectiveness trial to promote colorectal cancer screening. Health Promot Pract 2022;23:874–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Subramanian S, Hoover S, Tangka FKLet al. A conceptual framework and metrics for evaluating multicomponent interventions to increase colorectal cancer screening within an organized screening program. Cancer 2018;124:4154–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Cameron KA, Persell SD, Brown Tet al. Patient outreach to promote colorectal cancer screening among patients with an expired order for colonoscopy: a randomized controlled trial. Arch Intern Med 2011;171:642–6. [DOI] [PubMed] [Google Scholar]
- 15. Levy BT, Daly JM, Xu Yet al. Mailed fecal immunochemical tests plus educational materials to improve colon cancer screening rates in Iowa Research Network (IRENE) practices. J Am Board Fam Med 2012;25:73–82. [DOI] [PubMed] [Google Scholar]
- 16. Zapka JG, Lemon SC, Puleo Eet al. Patient education for colon cancer screening: a randomized trial of a video mailed before a physical examination. Ann Intern Med 2004;141:683–92. [DOI] [PubMed] [Google Scholar]
- 17. Wells KJ, Battaglia TA, Dudley DJet al.; Patient Navigation Research Program. Patient navigation: state of the art or is it science? Cancer 2008;113:1999–2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Paskett ED, Harrop J, Wells KJ.. Patient navigation: an update on the state of the science. CA Cancer J Clin 2011;61:237–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Bernardo BM, Zhang X, Beverly Hery CMet al. The efficacy and cost-effectiveness of patient navigation programs across the cancer continuum: a systematic review. Cancer 2019;125:2747–61. [DOI] [PubMed] [Google Scholar]
- 20. Falk D. A mixed methods review of education and patient navigation interventions to increase breast and cervical cancer screening for rural women. Soc Work Public Health 2018;33:173–86. [DOI] [PubMed] [Google Scholar]
- 21. Atere-Roberts J, Smith JL, Hall IJ.. Interventions to increase breast and cervical cancer screening uptake among rural women: a scoping review. Cancer Causes Control 2020;31:965–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Percac-Lima S, Ashburner JM, Zai AHet al. Patient navigation for comprehensive cancer screening in high-risk patients using a population-based health information technology system: a randomized clinical trial. JAMA Intern Med 2016;176:930–7. [DOI] [PubMed] [Google Scholar]
- 23. Myers RE, Stello B, Daskalakis Cet al. Decision support and navigation to increase colorectal cancer screening among Hispanic patients. Cancer Epidemiol Biomark Prev 2019;28:384–91. [DOI] [PubMed] [Google Scholar]
- 24. Post DM, McAlearney AS, Young GSet al. Effects of patient navigation on patient satisfaction outcomes. J Cancer Educ 2015;30:728–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Dillon EC, Kim P, Li Met al. Breast cancer navigation: using physician and patient surveys to explore nurse navigator program experiences. Clin J Oncol Nurs 2021;25:579–86. [DOI] [PubMed] [Google Scholar]
- 26. Dwyer AJ, Weltzien ES, Harty NMet al. What makes for successful patient navigation implementation in cancer prevention and screening programs using an evaluation and sustainability framework. Cancer 2022;128:2636–48. [DOI] [PubMed] [Google Scholar]
- 27. Barrington WE, DeGroff A, Melillo Set al. Patient navigator reported patient barriers and delivered activities in two large federally-funded cancer screening programs. Prev Med 2019;129S:105858. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
De-identified data from this study are not available in a public archive. De-identified data will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.