Evaluating Outcomes of Community-Based Cancer Education Interventions: A Ten-Year Review of Studies Published in the Journal of Cancer Education

Abstract

The public is encouraged to participate in cancer education programs because it is believed that acquiring health-promoting knowledge will motivate participants to make the recommended, evidence-based behavioral modifications that should lead to reductions in cancer morbidity and mortality. Because of the extended time that elapses between conducting a health education program and the amassing of the scientific evidence needed to establish that an education program has ultimately resulted in a reduction in morbidity and mortality, researchers have sought more proximal and intermediate outcome measures as substitutes for the more distal desired outcomes. This paper presents an analysis of research published in the Journal of Cancer Education from 2000 through 2010, in which the impact of cancer education interventions was evaluated. The focus was to identify the proximal, intermediate, and distal outcome measures used to evaluate the impact of cancer education interventions. The results show that researchers primarily focus on measuring the varied proximal outcomes (e.g., knowledge and attitude changes) of cancer education interventions. Intermediate outcome measures (the desired behavior change itself) received less attention, while distal outcomes (changes in morbidity and mortality) were never measured. This review gives cancer education researchers a review of the proximal and intermediate outcome measures and strategies that behavioral scientists recently used to overcome the challenges of measuring distal outcomes. Future reviews could expand this analysis to studies published in other journals and health disciplines.

Introduction

Decreasing the impact of cancer is vital. Cancer education interventions can help to ease the impact of cancer in a variety of ways. They can increase general knowledge about cancer, including modifiable and non-modifiable risk factors. These include early warning signs, screening and diagnostic options, prevention strategies, treatment options and clinical trials. Cancer education interventions have the potential to benefit the public by increasing the frequency of constructive health behaviors being practiced (e.g., using sunscreen, not smoking, obtaining cancer screening). The overall desired outcome of education interventions is that cancer morbidity and mortality rates will decrease because individuals are taking measures that have been associated with reducing cancer risk.

The manner in which the impact of these interventions is measured is key. Potential outcomes for cancer education interventions can be categorized in three levels [1, 2]. The first is the proximal outcome, which can be thought of as the change in knowledge, attitudes, beliefs, or intentions that results from exposure to an education intervention. The second is the intermediate outcome or the behavioral change that results from exposure to the intervention, and is hypothesized to result from the proximal changes. For example, the intermediate outcome of an intervention teaching the importance of sunscreen could be the individual’s regular application of sunscreen following the educational intervention. The third is the distal outcome. True distal outcomes of education interventions measure whether rates of diagnoses, severity or diagnostic stage, or mortality in the population decrease due to that intervention.

All three of these outcomes are important to understanding the impact of cancer education interventions. An additional, essential consideration is the assessment method used to measure outcomes at each level. At the proximal level, self-report is typically used to capture changes in knowledge, attitudes, beliefs, and intentions, all of which are generally conceptualized as intrapersonal variables not easily observed by others. Intermediate outcomes, in contrast, are by nature behavioral, and can be assessed either through self-monitoring (e.g., of daily food intake), through observation by others (e.g., of use of sun protective clothing), or through quantified adherence to medical proscriptions (e.g., medical records showing that an annual mammogram was done). It is optimal if measures at all levels can be standardized and have been shown to be reliable and valid indicators of the target outcome(s) in the populations studied. However, sometimes standardized and validated measures are not available for the variable in question or for the community of concern. It can also be appropriate to develop and use non-standardized measures to capture outcomes related to measuring a specific aspect of an education program. For example, knowledge questionnaires are often most useful when they are specifically designed to assess the content taught in the cancer education program being evaluated.

The purpose of the current study was to examine which outcomes are being measured by researchers studying cancer education interventions, how frequently each type of outcome is assessed, and whether standardized instruments are used when they are an appropriate option. The researchers investigated four questions: 1) Would the majority of studies use proximal outcome measures?; 2) Would intermediate outcome measures be used in at least half the interventions?; 3) Would distal outcome measures be used in at least ten percent of the studies?; and 4) Would the majority of studies employ standardized and validated instruments in the assessment of outcomes?

Method

Selection Criteria

Research studies evaluating the impact of community-based cancer education interventions that were published in Volumes 15 (2000) through 25 (2010) of the Journal of Cancer Education were used for this descriptive literature review. One criterion for eligibility was that a study’s evaluation sample consisted of adult participants, aged 18 and over. Because pediatric cancers are rare and there are no recommended prevention or early detection strategies deemed appropriate for inclusion in a public health program, studies related to children were not included in this study. The second was that the recruitment of participants was community-based (e.g., not hospitalized patients). The third was that the participants were being recruited as members of the lay community (e.g., not healthcare professionals or medical students). Finally, the review included only those studies that measured change in targeted outcomes both pre- and post-intervention. Studies that described the development of new programs or pilot studies to explore feasibility were not included in the review, as their dependent variables typically did not align with the outcomes in question. The studies’ findings were not taken into account; the focus of this study was only to identify the measurement strategies used to assess outcomes.

Operational Definitions

Proximal outcomes were defined as changes in knowledge, attitudes, beliefs, or intentions. Intermediate outcomes were defined as changes in engagement in tangible behaviors or practices. Distal outcomes were defined as changes in overall rates and stages of diagnosis and mortality in the population.

Results

After reviewing the titles and abstracts of the articles published in the Journal of Cancer Education from 2000 through 2010, 138 studies were deemed to be potentially eligible for inclusion. Subsequently, 108 of those studies were excluded from the analysis because they failed to meet one or more inclusionary criteria; the remaining 30 studies fit all selection criteria [3–32]. These 30 studies described evaluations of interventions that used a variety of educational media (e.g., web-based, telephone, video) and covered a variety of cancers. The intervention studies are presented alphabetically by year of publication (earliest to latest) in Table 1. The defined proximal, intermediate, and distal outcomes are listed for each study as applicable, along with the amount of time that elapsed between the intervention and the point at which the measurement outcomes were assessed.

Table 1.

Descriptions of each intervention (including types of outcomes measured, time of assessment and method of measurement), displayed alphabetically by year of publication.

Title of Article	Proximal Measure	Intermediate Outcomes
		Measure	Time
Tailored, interactive soap operas for breast cancer education of high-risk Hispanic women [3]	Breast cancer knowledge and beliefs questionnaire	N/A	N/A
Impact of a cancer education multimedia device on public knowledge, attitudes, and behaviors: A controlled intervention study in southern Sweden [4]	Knowledge and attitudes about malignant melanoma questionnaire	Sun-protective behavior questionnaire	2 years
Screening to the converted: An educational intervention in African American churches [5]	Cancer health knowledge and screening history questionnaire	Telephone interview asking about obtaining cancer screening	7 months
Black cosmetologists promoting health program: Pilot study outcomes [6]	Breast cancer knowledge and attitudes questionnaire	N/A	N/A
Breast cancer education program based in Asian grocery stores [7]	Breast cancer-related knowledge and attitudes survey	Telephone interview or questionnaire asking about scheduled mammography	Two weeks
Two community outreach strategies to increase breast cancer screening among low-income women [8]	N/A	Changes in screening rates in the county mammography registry	Not reported
Compliance with fecal-occult blood test screening among low-income medical outpatients: A randomized controlled trial using a videotaped intervention [9]	Knowledge about colorectal cancer, perceived risk, self-efficacy, and intent to use fecal occult blood testing questionnaire	Compliance with FOBT determined by medical records	Three months
Bringing breast cancer education to Deaf women [10]	Breast cancer-related knowledge survey and focus groups	N/A	N/A
Impact of a multimedia breast cancer education intervention on use of mammography by low-income Latinas [11]	Breast cancer-related knowledge, recall of four themes in intervention, and intention to screen, assessed by survey and oral interview	Telephone interview asking about mammography screening status	Four months
Evaluation of a preparatory community-based prostate health education program [12]	Prostate cancer knowledge, program evaluation, and intention to maintain prostate health survey	N/A	N/A
Impact of culturally, linguistically, and literacy relevant cancer information among Hispanic farmworker women [13]	Knowledge, intervention satisfaction, and comfort level discussing cancer information surveys	Mammography screening status determined by medical records, telephone calls and/or home visits	Six weeks
A breast cancer education and on-site screening intervention for unscreened African American women [14]	Cancer history, screening, structural barriers to care, and behavioral barriers to care questionnaire	On-site test records and telephone interview asking about completion of breast cancer screening	Three months
Promoting informed decision making: Evaluating a community-based prostate health awareness program [15]	Prostate cancer knowledge survey	Telephone interview asking about visiting a physician to discuss prostate health	Three months
Video breast health kits: Testing a cancer education innovation in older high-risk populations [16]	Breast-Self Examination Proficiency Rating Instrument*	Telephone interview asking about mammography screening status	Eight months
Increasing mammography screening through inpatient education [17]	N/A	Questionnaire or telephone interview asking about mammography screening status	Three months
A woman in transition: Can drama deliver a cancer awareness message? [18]	Knowledge, attitudes, and beliefs about breast cancer and intention to obtain mammography questionnaire	N/A	N/A
Choice of screening modality in a colorectal cancer education and screening program for the uninsured [19]	N/A	Screening status and type of test determined by hospital registration form	Not reported
The effectiveness of training for breast cancer and breast self-examination in women aged 40 and over [20]	Breast cancer-related knowledge questionnaire or interviews, competency of completing a breast self-exam using a breast simulator	N/A	N/A
Prostate cancer knowledge and screening attitudes of inner-city men [21]	Prostate cancer knowledge and attitudes about screening questionnaire	N/A	N/A
Promoting breast cancer screening among Chamorro women in southern California [22]	Breast cancer knowledge, attitudes and beliefs survey	Breast cancer screening status questionnaire	One year
The impact of cervical cancer education for Deaf women using a video educational tool employing American sign language, open captioning, and graphics [23]	Cervical cancer knowledge survey	N/A	N/A
Predictors of new screening for African American men participating in a prostate cancer educational program [24]	Prostate cancer knowledge, attitudes, and beliefs questionnaire	Prostate cancer screening behavior questionnaire	Three and six months
The effects of peer education in university students’ knowledge of breast self-examination and health beliefs [25]	Breast self-examination questionnaire and Champion Health Belief Scale*	N/A	N/A
Efficacy of breast cancer appeals for promoting physical activity [26]	Intention to increase physical activity level, message believability and message relevance questionnaire	N/A	N/A
Theater as a tool to educate African Americans about breast cancer [27]	Breast Cancer Health-Related Behaviors and Attitudes Scale	N/A	N/A
Communication message strategies for increasing knowledge about prostate cancer screening [28]	Prostate cancer knowledge survey	Questionnaire or telephone interview asking about prostate-specific antigen screening status	Six to 11 months; 12 to 18 months
Improving cancer knowledge and screening awareness: Test of a telephone interviewer intervention [29]	Telephone interview asking about cancer knowledge and intention to obtain cancer screenings	N/A	N/A
Patient education cards for skin cancer detection and treatment [30]	N/A	Having skin lesions identified, diagnosed or treated determined by billing data	Not reported
Colorectal cancer video for the Deaf community: A randomized control trial [31]	General and colorectal cancer knowledge survey	N/A	N/A
Effective colorectal cancer education for Asian Americans: A Michigan program [32]	Colorectal cancer knowledge and intention to obtain screening questionnaire	Screening status and type of test questionnaire	6 to 12 months

Note. Instruments with an asterisk have been standardized and validated.

Question one was answered affirmatively, with the majority of the studies (86.7%, n = 26) measuring proximal outcomes. Proximal outcomes included recall of intervention messages or themes, comfort level when talking about cancer, willingness to miss work in order to obtain screening, and intention to increase physical activity. Although this was not a requirement for an outcome to be considered proximal, in all 26 studies, proximal outcomes were measured immediately following the education intervention. Thus, there is no report for time elapsed between the intervention and assessment of proximal outcomes in Table 1.

Question two was answered affirmatively; slightly more than half of the studies (56.7%, n = 17) measured intermediate outcomes. Intermediate outcomes included obtaining screening, fruit and vegetable intake, amount of fat consumed, and having skin lesions diagnosed or treated.

Question three was answered negatively; none of the studies measured distal outcomes. The final question was also answered negatively, in that the majority of the studies used original data collection measures rather than standardized and validated instruments. As shown in Table 2, there was a wide range of assessment tools used to measure proximal and intermediate outcomes and of the points in time when assessments of intermediate outcomes were initiated.

Table 2.

Frequency of measurement tool utilization, original versus validated measure utilization, and length of time after the study for initiation of intermediate outcome assessments.

Outcome Factor	%	n
Proximal outcome assessment methods
Questionnaires or surveys	84.6	22
Telephone or in-person interview	3.9	1
Combination of methods	11.5	3
Utilized original measurement tool	77.0	20
Utilized validated measurement tool	23.0	10
Intermediate outcome assessment time
Immediately to < 3 months	11.8	2
3 months to < 6 months	29.4	5
6 months to < 9 months	11.8	2
1 year to < 2 years	5.8	1
2+ years	5.8	1
Multiple time points	17.7	3
Declined to state	17.7	3
Intermediate outcome assessment methods
Questionnaires or surveys	23.5	4
Telephone or in-person interview	23.5	4
Medical records	5.9	1
County screening rates	5.9	1
Billing data	5.9	1
Combination of methods	29.4	5
Utilized original measurement tool	100.0	17

Discussion

This review of the literature was done to identify and describe the proximal, intermediate, and distal outcomes that were recently measured to evaluate the impact of cancer education interventions and the frequency with which the various types of outcomes were measured.

Proximal outcomes were the most consistently measured, probably because they are the easiest to assess. Many of the proximal outcomes are cognitive variables (e.g., knowledge, attitudes, beliefs) that are typically assessed via self-report and questionnaires. This information can be quickly and easily collected at baseline and again after the intervention is delivered, when most, if not all, of the participants are still present to respond and the intervention messages are still new.

Intermediate outcomes are usually more difficult to measure because they require monitoring of participants and follow-up assessment. However, they are highly desirable assessments to make because intermediate outcomes are generally viewed as the intended effects of the interventions and contribute to the achievement of the distal outcomes. Further, since they usually occur at a chronologic distance from the conclusion of the study’s intervention, they are a reflection of the sustained impact of the intervention. However, this follow-up can be problematic because of measurement challenges, including expense, participant retention, and protection of participant privacy, particularly when the topic being addressed could be viewed as stigmatizing. In other cases, intermediate outcomes are behaviors that may be difficult to assess. For example, a person’s appropriate and consistent use of sunscreen, as the intermediate outcome following exposure to a skin cancer education program, is difficult to capture effectively and efficiently, and may necessarily involve observation that is intrusive. Other behaviors cannot be captured by an intermediate outcome assessment at the end of the intervention or even shortly after the intervention; some are not practiced on a day-to-day basis and others take time to establish (e.g., smoking cessation, colonoscopy exam, increased daily intake of vegetables and fruits).

While researchers have considered behavioral intention as a proxy when actual behaviors are not realistic to measure [33], intention is more appropriately thought of as a proximal outcome, as it was classified here, because it represents motivation, not actual behavior. Another challenge is attrition, with the possible consequence that the subset of participants who provide post-intervention data may constitute a biased sample. Additionally, the release of new health guidelines and the media’s scientific reporting of guideline changes can create confusion about proper health behaviors and disrupt the educational interventions’ messages. For instance, the United States Preventive Services Task Force reformed mammography guidelines in 2009, which caused confusion regarding women’s optimal health behaviors [34]. Changes of this nature can also make intermediate outcomes of cancer education interventions more difficult to measure, because it can be difficult to establish which policy people are following. However, these actual behavioral outcomes are crucial to measure because it is the behavioral engagement that is anticipated to help reduce the impact of cancer. The impact of proximal outcome changes alone does not guarantee that the intermediate and distal changes will be achieved; this is of even greater concern when an extended period of time will elapse before those changes might be seen [33].

Distal outcomes are the most difficult to measure for a number of reasons. The time between the intervention and collection of statistical evidence reporting diagnostic and mortality rates is often very extended, and the funding available for most cancer education studies is generally insufficient to allow for the considerable expense associated with the assessment of long-term outcome measures. Further, direct attribution to a specific intervention or style of educational intervention is virtually impossible since multiple factors can be interacting. Finally, the high rates of attrition at this stage may limit the generalizability of the findings. Perhaps the more important question for the community of health education researchers to address is whether distal measures are essential. Aside from the fact that distal outcome measures are expensive and difficult to accomplish, education interventions most often evolve from the application of findings derived from astute clinical observations or large population-based epidemiological studies. Such studies suggest that people who engaged (or failed to engage) in a particular behavior had lower morbidity and/or mortality rates than their counterparts. For example, large-scale epidemiological studies have reported that as the early detection rate for of melanoma increases and the morbidity rate decreases. Evolving from this finding were studies to determine if a melanoma education intervention could increase people’s knowledge of how to find melanomas early and secure appropriate treatment (proximal), increase monthly melanoma self-exams for discovering and responding to early changes in moles (intermediate), and decrease the morbidity and mortality rates for melanomas (distal). The distal finding has already been established: morbidity rates increase as stage of melanoma increases. Consequently, the most important outcome measures are the proximal findings related to changes in knowledge, attitudes, and intentions (e.g., observing changes in moles) and the intermediate findings related to increased and continued engagement in periodic self-exams for changes in moles, whether an abnormal mole triggers a visit to the doctor, and whether the intervention has enabled the individual to discern which moles are truly at increased likelihood of being melanomas. In this instance, the measurement of distal outcomes may be unnecessary, as the changes in melanoma knowledge, self-monitoring behaviors, detection, and subsequent treatment are assumed to have an effect on morbidity and mortality rates for melanomas.

Using standardized measures can be important for assessing proximal and intermediate outcomes. The evaluation of changes in the construct(s) of interest may be better accomplished by using reliable and valid standardized instruments. Such measures can enable researchers to better understand the impact of educational interventions and allow comparisons to other studies using the same measure(s). Standardized measures may also enrich the discovery process by allowing researchers to gain different perspectives through the examination of differential program impact across diverse sociodemographic and psychosocial characteristics. For example, in another study published in the Journal of Cancer Education during the time period covered of this review but deemed ineligible for inclusion, investigators examined whether internal health locus of control, or the belief that one has personal control over his/her health, influenced the knowledge gained and retained by Deaf women receiving a cervical cancer prevention education program [35]. To examine internal health locus of control as a moderator of program impact, the researchers used the Multidimensional Health Locus of Control scales, which have been widely used and have established reliability and validity for measuring dimensions of health locus of control. Such data can help health educators gain nuanced understandings of why a program works or does not work and whether those observations are operating differentially among the sample’s subgroups. However, there are some instances in which psychometrically validated instruments are not crucial. For example, researchers may develop knowledge questionnaires that are specifically tied to the content of the particular education program being studied.

Limitations and Future Directions

Published studies from only one journal were considered for this analysis. This precluded consideration of measured outcomes in cancer education intervention studies published elsewhere. However, given that the Journal of Cancer Education is the official journal of both the American and European Associations for Cancer Education, it was viewed as the most appropriate and representative publication. The current review was also limited to cancer education interventions, thereby excluding education intervention studies related to other chronic illnesses.

Future studies could analyze comparable research related to cancer education interventions published in other journals, as well as education interventions for other chronic and life-threatening diseases. This could help to highlight the fuller array of outcome measures available for use, as well as to identify gaps where new measures are needed. The development of new measures to fill such measurement gaps offers an important collaborative opportunity for the American and European Associations for Cancer Education’s members to address.