Progress in the Pipeline: Replication of the Minority Training Program in Cancer Control Research

Abstract

Background

This paper evaluates the replicability of an NCI-funded didactic/experiential program to increase the diversity of doctorally-prepared cancer disparities investigators.

Methods

The program was developed and operated successfully for three years in Northern California when a replication was established at UCLA. Feasibility, process, impact, and outcome measures on UCSF and UCLA summer-institute participants were compared.

Results

Average participant rankings of the influence of the program on intention to apply for a doctorate were 9.1/10 (UCSF) and 8.6/10 (UCLA). A total of 22.5% of UCSF and 10% of UCLA participants have enrolled in, been accepted by, or completed doctoral programs. Among these alumni, 68% (21/31) of UCSF and 60% (3/5) of UCLA participants plan to conduct their doctoral research in cancer control.

Conclusions

This program has been successfully replicated and has met its objective to increase the pipeline of ethnically diverse doctoral-level public health researchers. Expansion of the program to other regions of the US is feasible and indicated.

There is extensive documentation of the disproportionate burden borne by minority and underserved populations in morbidity and mortality due to cancer.^1–7 The body of research aimed at redressing these disparities is large and cuts across disciplines including epidemiology, genetics, health education/behavioral science, health psychology, clinical medicine, health economics, nutrition, communications, and many more. It is indisputable that there is much more to be learned from these fields so that all populations may benefit equally from prevention, early detection, cancer treatment modalities and survivorship interventions.

While increased resources for research are needed,⁸ an infusion of targeted research funds cannot itself eliminate disparities. The research should be developed and conducted by investigators who can best understand and address the needs of culturally diverse communities. The term for this in anthropology is “insider researcher,” an investigator who possesses an innate knowledge of the culture under investigation because they have lived it. While individuals cannot presume to speak for an entire cultural group, there is no substitute for the combination of innate knowledge of a culture and strong research skills.⁹ This applies to development of appropriate research goals, concepts, study designs, measurement tools, intervention strategies, analytic approaches and interpretation, and the acceptance of study methods and their findings in the communities for which they are intended. Indeed, researchers of color should be over-represented in proportion with the demographics of cancer disparities.

In addition to the importance of diversity among those trained at the doctoral level, there are key roles in cancer control research and practice for minority graduates at the master’s level. These individuals are needed for mid-level positions such as project coordinators/managers on studies. While cultural concordance between investigators and the intended audience greatly enhances the appropriateness of the design and methods of the research, representative staff is the link to the community. For effective community relations involving surveys or interventions, it is important to consider the fact that people are drawn to others like themselves and are more likely to trust those to whom they relate to based on shared culture.^10,11

THE MINORITY TRAINING PROGRAM IN CANCER CONTROL RESEARCH

The Minority Training Program in Cancer Control Research (MTPCCR) was conceived in the early 1990’s, when the program founder (RP) served as Project Director on a large National Cancer Institute (NCI) program project grant that addressed cancer screening disparities in four ethnic groups. The research team included leadership at the PhD and MD levels from the ethnic groups targeted by the study. Her appreciation for the unique perspective of the “cultural insider researcher” raised concern that few such researchers were active in cancer control research or “present in the pipeline.”

In collaboration with faculty from San Francisco State University, San José State University, University of California at Berkeley, University of California at San Francisco, and the Northern California Cancer Center, the Northern California (NCal) research team was awarded a National Cancer Institute (NCI) training grant establishing the MTPCCR to encourage minority master’s level students in health programs to pursue a doctorate and a career in cancer control research.

The core of the MTPCCR is a 5-day Summer Institute, “Careers in Cancer Control Research,” designed to showcase the needs, opportunities, and resources for doctoral studies for ethnically underrepresented students. The program also includes internships, i.e., mentored research experiences, each year for 4–8 participants, and Doctoral Application Support Awards (DASA) of up to $2,000. DASA are awarded competitively to 2–4 program alumni annually to offset costs associated with applying to doctoral programs (e.g., GRE preparatory classes, application fees, travel). The program is described in greater detail elsewhere.⁹

Participant recruitment is conducted primarily through collaborating faculty at partner institutions within the California State University (CSU) system and University of California (UC) systems. A faculty advisor at each institution is paired with a student advisor. To be eligible, applicants must: a) have completed at least one year of a master’s level program in health sciences; b) have graduate school grades averaging at least “B”; c) be a member of an underrepresented minority group in the cancer control research field and from a community that suffers a disproportionate or unknown burden of cancer.

One particular distinguishing element of this program is the effort made, throughout the Summer Institute, to honor and celebrate as assets the cultural heritage and diversity represented by the participants. Because students of color often share a different path to graduate school built upon experience with cultural insensitivity and outright racism both from students, faculty, and administrators, many students have viewed their ethnicity as a disadvantage. The introductory Summer Institute session creates a safe space to honor ethnic heritage and cultural differences, integrating affective and cultural elements in a manner that provides, for many students, their first opportunity to express the emotional pain and joy attendant to their “difference.” The entire Institute goes on to underscore the value of participants’ cultures and the importance of their place in the fight against cancer. Our initial hope for the MTPCCR was that one or two students in each class might have the potential and commitment to pursue and complete a doctoral degree. In reality, there have only been few participants who did not evince this capability. Because of initial uncertainty due to lack of any precedent for this program, our first grant proposal to the National Cancer Institute requested modest support for a three-year program. However, when the results far exceeded these expectations, a second proposal was submitted for a five-year continuation plus replication.

MTPCCR REPLICATION

The replication program is based at UCLA, under the leadership of two senior UCLA Public Health faculty members who were participants in the NCal program. Following extensive quantitative and qualitative evaluation of the NCal curriculum,⁹ programmatic content and delivery were maintained with maximal fidelity to test the portability and replicability of the original model. Both UCLA faculty members maintained their participation in the NCal program (which during this time relocated to the University of California, San Francisco Comprehensive Cancer Center) as they teamed up to develop the Southern California (SCal) program. Similarly, directors of the NCal program, joined with the core faculty for the UCLA Summer Institute and the program adhered to the principles and procedures of the original. The fidelity was deliberately pursued by the SCal team to determine whether the cohesiveness and emotional impact of each cohort in NCal could be duplicated. This intangible “magic” of the combination of didactic with affective elements seems to be a key to the program’s success. This report describes the outcome of the replication as indicated by our assessment of feasibility, and comparability of process, impact, and outcome results from both locations.

The primary replication question was whether a second program, like the original, could be created, that not only fulfilled the aims of the program as stated, but also replicated the strengths of the program that extend beyond the initial aims. The replication program would be in a different location characterized by different demographics and a unique cultural climate, operated by different staff with different Summer Institute faculty and internship mentors, and students recruited from a different region. If this goal could be achieved, then similar programs could be created in any other region of the country where there is diversity at the Master’s level.

METHODS

We identified six criteria to assess the success of the replication: feasibility, impact on participants of the Institute curriculum, change in intention to apply to a doctoral program, number of students who matriculated to doctoral programs, those who did not matriculate to doctoral programs but changed their field of work to cancer control, and the extent to which these participants attributed their decisions to the MTPCCR. We developed measures for each criterion that included overall program evaluation at three levels: process, impact, and outcome. Replication of program strengths that could not be measured quantitatively were gauged by the qualitative comments participants wrote in their evaluation forms and verbally expressed to each other and to the faculty and staff.

Program Evaluation

Overall evaluation of the program is conducted at three levels, process, impact, and outcome. The process evaluation provides immediate feedback on intermediate objectives such as number and diversity of program participants. Process measures include number of applications, Summer Institute attendance, and participant satisfaction as indicated by the daily surveys, and a final survey following the Institute. Impact evaluation measures participant intentions to apply for a doctoral program in surveys administered before and after the Summer Institute, and on an ongoing basis through an annual alumni survey until participants either submit a doctoral program application or indicate that they likely never will do so. Outcome evaluation measures include tracking actual applications to, and enrollment in, doctoral programs through regular informal contact between alumni and program staff, and the annual mailed alumni survey.

Outcome evaluation

Data are presented here for the annual survey mailed to all 6 classes in 2004 class who did not complete the annual survey until late 2005. The alumni survey contains 14 items covering professional and academic plans for the next 5 years, current enrollment in a doctoral program, and intentions to apply in the next 1–2 years. For those who have applied, we asked for their three greatest challenges in the application process, the three most helpful factors, the extent to which their plans were influenced by the MTPCCR, and the field of study they are pursuing. All contact with students for evaluation/research purposes was approved by the Institutional Review Boards of UCLA and UCSF.

Results

Comparisons between NCal and SCal outcome data must be interpreted cautiously because many of our alumni who matriculate do so several years after participating in our program. Since there are six years of followup data for NCal and only two years of data for SCal, outcome conclusions for the newer program must be regarded as preliminary.

Feasibility

Feasibility of the replication is indicated by the extent to which the original model could be implemented in Southern California. A multi-ethnic, multi-disciplinary team comprises the core staff in the two locations, as well as student and faculty advisors from the partnering academic institutions. The disciplines represented on the core team of both NCal and SCal are health education, anthropology, medicine/public health, health psychology, social welfare, and public administration, and the ethnicities of the core team include white European-American, Japanese American, African American, Latina and Filipina. The SCal program was successful in recruiting partnering institutions that serve the large public university system in that region including the four California State University (CSU) campuses around UCLA, CSUs of Dominguez Hills; Fullerton; Los Angeles; and Northridge, and the three major Universities: University of California, Irvine, University of California, San Diego, and University of Southern California. Half of the faculty advisors and all of the student advisors are from underrepresented ethnic groups. All members are actively involved in all phases of planning, revision, and implementation. The majority of Summer Institute faculty presenters in each location also represent ethnically underserved groups. One current staff member for the program in each site (NCal and SCal) is a former program participant.

Process Evaluation

Recruitment

The number of applicants has increased yearly, from 1.2 per participant slot in 1999 to 2.1 (NCal) and 2.0 (SCal) per slot in 2004. Overall, 188 participants completed the Summer Institute through 2004, as a part of 6 NCal classes (n=138) and 2 SCal classes (n=50). Twenty to 25 participants attended the Institute each year in NCal, with 25 in each of the 2 years in SCal. The minimum class size to date was 20 (one class), and the average size has been 23.5 (Table 1). The overall ethnic distribution of the participants was 26% African-American, 20% Latino, 45% Far East and Southeast Asian Americans/Pacific Islanders, and 9% others—Native Americans, Africans, gays/lesbians and those of mixed ethnic heritage (Table 2).

Table 1.

Change in Intention to Apply to Doctoral Program^*

	Year	n	Mean Change in Intention	P Value
NCal	1	25	3.6	.001
	2†	25	0.4	.042
	3	20	1.4	.006
	4	21	1.0	.01
	5	22	1.2	.01
	6	25	1.3	.005
SCal	5	25	0.2	.75‡
	6	25	1.13	.001

^*Ncal indicates Northern California; Scal, Southern California.

^†The scale used in all years was 1 through 10 except for Year 2 when the scale was 1 to 5.

^‡In this class with 22 completed pre- and post-surveys, 2 students’ post-institute rating was lower than the pre-institute ranking due to the time frame of the question. One of these noted: “I will apply in 3 years”.

Table 2.

Race/Ethnicity of MTPCCR Participants by Site^*

	Northern California 1999–2004 (n=138)		Southern California 2003–2004 (n=50)		Total (n=188)
Race/Ethnicity	%	No.	%	No.	%	No.
African American	27.0	37	24	12	26.1	49
American Indian	0.7	1	2	1	1.0	2
Asian Amer/Pacific Islander	48.5	67	34	17	44.7	84
Latino/Hispanic	14.5	20	34	17	19.7	37
Other/mixed ethnicity†	9.4	13	6	3	8.5	16

^*MTPCCR indicates Minority Training Program in Cancer Control Research.

^†Gay/lesbian, Persian, Paraguayan/Ukranian, Taiwanese/Irish, African French/Black, Palestinian, Chinese/Mexican, Persian, African American/Latino/Hispanic.

To assess the comparability of each program’s ratings of the qualifications of applications, we compared the average score (range: 0–30) and the range for the two years when both programs were accepting applications. Inter-program scoring reliability was assessed by exchanging 3 applications from each program, which were then scored by both groups. The scores for the six applications were indistinguishable.

Summer Institute Program

The sessions offered in the original NCal program are comparable to those offered at the SCal program. Days 1–3 provide an introduction and overview of the importance of cancer control research covering topics such as epidemiology, behavioral research, surveillance, intervention, and cancer survivorship. Days 5 and 6 focus on providing information, resources, tools, and motivation needed to apply to a doctoral program, in addition to exploring funding opportunities. Overall ratings of daily program sessions averaged 4.55 (NCal) and 4.53 (SCal) on a scale from 1=poor to 5=excellent (see Table 3), indicating no significant differences between sites.

Table 3.

Average Rating for each summer institute program day^*

	Northern California						Southern California†
Day	Year 1	Year 2	Year 3	Year 4	Year 5	Year 6	Year 5	Year 6
1	4.5	4.6	4.4	4.1	4.4	4.2	4.5	4.5
2	4.1	4.4	4.5	4.2	4.4	4.5	4.6	4.4
3	4.8	4.4	4.5	4.5	4.7	4.7	4.5	4.5
4	4.5	4.6	4.7	4.4	4.7	4.5	4.2	4.5
5	4.9	4.8	4.9	4.9	5.0	4.8	4.9	4.7
Mean	4.56	4.56	4.60	4.42	4.64	4.54	4.54	4.52

^*Scale was 1–5: 1, poor, to 5, excellent.

^†Southern California’s first summer institute was Year 5.

Comments made by participants about the “safe space” created in the Institute for self-reflection speak to the sense of wholeness that is a goal of the Institute. One participant wrote: “I found the day (1) to be emotionally, spiritually and intellectually inspiring”. Another wrote about one of the group exercises: “To hear others speak of their lives and issues helps you realize you’re not alone and you can make it and balance (the ‘balls in your life’). The culture exercise helped to initiate a bond between the group - and that was just in the first three hours!” The awakening of what cultural differences mean is also a major transition for many of the participants.

Internships

In NCal, 28 internships were granted, and 24 were completed (over six Institutes). In SCal, 15 internships were granted, and 14 were completed (over two Institutes). Incomplete internships at both sites were mainly due to extenuating personal circumstances.

Doctoral Application Support Awards (DASAs)

A total of 16 DASAs have been awarded, 14 in NCal, and 2 in SCal. Many recipients have indicated that the availability of the DASAs eliminates the financial barrier posed by the application process and reduces the time to application.

Impact Evaluation

Alumni survey

We present data from the 2004 annual survey that was sent to the 138 alumni since 2003 (5 classes in N. Cal and 1 in S. Cal). In addition, informal updates were provided by alumni from the 2004 class who did not complete an annual survey until late 2005. A total of 113 responses were received, 105 from the annual survey plus 8 additional responses from alumni in the 2004 class. On a scale of 1–10 (with 10=very certain will apply), 20% (24) of NCal participants and 30% (8) of those in SCal rated their plans to apply to a doctoral program in the next 1–5 years at 7 or above; of those, 63% (15) in NCal and 75% (6) in SCal are currently working or plan to work in the field of cancer control. In addition, there were 22% (13) of NCal participants and 26% (7) in SCal who do not have plans to apply to a doctoral program but are currently working or plan to work in the field of cancer control. Among the 36 respondents currently enrolled or who have graduated from doctoral programs, 67% chose 7 or above regarding their intention to work in cancer control (see Table 4).

Table 4.

MTPCCR Alumni Accepted/Enrolled in Doctoral Programs and PhD Students Studying Cancer Control^*

	Accepted/Enrolled In/ Completed PhD Programs	PhD Students /Graduates in Cancer Control
Program Year	n	n
1	5	4
2	5	3
3	3	3
4	8	6
5†	7	4
6†	8	4
Total	36 (19%)	24 (67%)

^*Data from 2004 Alumni Survey (1999–2003 classes) plus informal reports of current acceptances and field of study. MTPCCR indicates Minority Training Program in Cancer Control Research.

^†Northern and Southern California programs combined.

Quantitative Results

Applicants’ pre-test/post-Summer Institute level of certainty that they would ultimately apply to a doctoral program (defined as a score of 7 or higher on a Likert scale) increased significantly in all years (p < .05) in NCal, (Table 1). In SCal, a non-significant increase was observed in one year, but we believe this was due to limitations in the question format. Across all classes and locations, positive qualitative responses indicated that the participants gained increased confidence, heightened focus, more motivation and determination to pursue a doctorate. Respondents also cited continuing barriers to further education such as family commitments and financial barriers (13% of respondents), and 19% stated they were still undecided because they needed more work experience or higher GRE scores before applying.

Overall, participant rankings of the influence of the MTPCCR on their intention to apply to a doctoral program averaged 8.6 and 9.1 on a scale of 1–10 for the NCal and SCal programs. Fifty-five percent of NCal participants, and 68% of SCal, indicated that the MTPCCR had a positive influence on their plans to pursue doctoral training compared to 27 percent in the pre-institute survey.

Qualitative Results

Open-ended survey responses included numerous comments about the program’s value in addressing the isolation, low expectations from faculty, fears of failure, and other challenges respondents face as students of color. Despite their desire for higher education, they had been made to feel that academia was not a place for them. Many also spoke of how the program inspired them through a sense of shared passion for addressing inequities and commitment to “community.” Importantly, one of the major, unanticipated benefits to students is recognition that their cultural background and identity is an asset for their research “toolbox.” Another outcome of value identified by the participants was the connection the students felt with the MTPCCR faculty.

Outcome Evaluation

Of the 188 alumni through the time of the 2004 survey, 34 are enrolled in doctoral programs - 21% (29) in NCal and 10% (5) in SCal. An additional two alumni, one each from the 1999 and 2000 NCal classes, have graduated with their PhDs (one is currently in a post-doctoral fellowship at NCI). Sixty-seven percent of these individuals report that they are conducting or planning to conduct their doctoral research in some aspect of cancer control (Table 4). Note, however, that the data only show a one year follow up for the SCal cohort compared to five years for NCal.

One of the major themes, in responses to questions on the impact of the program, is that the Institute provided participants with social and emotional support, networking opportunities among the other students who think like they do and face similar challenges, and also importantly, exposure to minority leaders in public health and cancer control. Statements included: “It reassured me that I can go for my doctorate,” or “[I now have] a stronger self-confidence in pursuing a doctorate,” and “I was exposed to resources within my reach,” “this has increased my sense of self-efficacy,” and “now I know all I need to do is ask for assistance.”

Completion of an internship was significantly associated with current enrollment or immediate plans to enroll in a doctoral program, with more than twice the proportion of interns moving on to doctoral programs, compared with those not interning in the program (chi square=0.018). In NCal, 28 internships were granted and 24 were completed and in SCal, 15 internships were granted and 14 were completed. Among alumni who completed internships, 67% (16) of those in NCal and 21% (3) in SCal, are applying or enrolled in a doctoral program. Two additional alumni who completed internships in NCal applied to a doctoral program, one enrolled but later dropped out, and the other was denied acceptance. Of the alumni who did not participate in the internships, 21% (23) in NCal are either applying or enrolled in a doctoral program (21), or have completed a PhD (2), and 4% (2) in SCal are applying or enrolled. Similarly, completion of an internship was associated with intent to pursue a career in cancer control, as 54% (13) former interns in NCal and 36% (5) in SCal expressed certainty about this field, compared with 44% (36) of non-interns in NCal and 31% (11) in SCal (chi square=0.031).

DISCUSSION

The evaluation suggests that the MTPCCR has been effective in expanding the pipeline of doctoral-level, “cultural insider” public health researchers, particularly those in cancer control, and that MTPCCR is a replicable model. From a research standpoint, construction of matched control groups for future cohorts would be optimal, but is not feasible for this evaluation. In the absence of a controlled experiment, we cannot, with certainty, attribute doctoral applications to participation in the program. Changes in intention and alumni testimonials help, but cannot completely address this issue.

It should be noted that the lower number of doctoral applicants/students from SCal is, in all likelihood, a result of the newness of the program there. In fact, many of the current doctoral students from MTPCCR stated that it took 3–5 years or more before pursuing their doctoral studies, but still ultimately attribute their doing so to the program.

The hallmark Summer Institute seems to function as a stand-alone program component, as it does not appear that internships were an incentive for Summer Institute participation. Among the 64 individuals who were accepted into the program but whose application for an internship was denied (due to lack of a mentor match or over-subscription to the internships), only one elected not to attend the Summer Institute. Thus, applicant interest in the program is not likely based on the prospect of a paid internship. It is probable, however, that the internships are sought by students more focused on doctoral study and provide opportunities for development of needed skills and intangible attributes and assets needed for matriculation.

Replication of the program was deemed successful because of all our criteria for feasibility, impact, outcomes and qualitative intangibles were met. From a service perspective, these results support further replication of this program in other regions of the US, particularly those with racial/ethnic diversity at the master’s level. The essence of the program appears to be the blending of psycho-social support strategies with the academic or didactic aspects of the program.

Future replication efforts would need to include this essential aspect of the program. Whether or not this program would be replicable in academic environments with less diversity among faculty and senior investigators is unclear, since the qualitative data suggest that interaction with cultural insider/role model researchers is a necessary and powerful program element. This lack of faculty diversity nationally, is, of course, also a major barrier to recruitment and retention of minority doctoral students (the focus of this program). Additionally, feedback from students who are currently in doctoral programs indicate that ongoing academic and psychosocial support of these students is warranted. The barriers and isolation that they report was expected, but they lack the skills to comfortably cope with them without moral support, at a minimum, and strategic support, more specifically.

This program remains singular in the nation in addressing the need to prepare cancer control researchers in disparities. As one student poignantly stated: “The goal no longer seems unattainable, the path less of a mystery. I’m excited to see where the road takes me but I think I have a better idea of the direction to which I’m heading.”