Abstract
Objective To develop and implement Project LEAD (leadership, education, and advocacy development), a science course for breast cancer activists.
Population Students were breast cancer activists and other consumers, mainly affiliated with advocacy organizations in the United States of America.
Setting Project LEAD is offered by the National Breast Cancer Coalition; the course takes place over 5 days and is offered 4 times a year, in various cities in the United States of America.
Results The Project LEAD curriculum has developed over 5 years to include lectures, problem‐based study groups, case studies, interactive critical appraisal sessions, a seminar by an ‘expert’ scientist, role play, and homework components. A core faculty has been valuable for evaluating and revising the course and has proved necessary to provide consistent high quality teaching. Course evaluations indicated that students gained critical appraisal skills, enhanced their knowledge and developed confidence in selected areas of basic science and epidemiology.
Conclusions Project LEAD comprises a unique curriculum for training breast cancer activists in science and critical appraisal. Course evaluations indicate that students gain confidence and skills from the course.
Keywords: breast cancer, education, patient advocacy, peer review
Introduction
The National Breast Cancer Coalition (NBCC), a breast cancer advocacy organization in the USA, 1 , 2 , 3 , 4 , 5 was founded in 1991 and comprises over 500 member groups and 60 000 individual members. The NBCC was founded to: promote research into the cause, optimal treatments and cure for breast cancer; improve access to high quality breast cancer screening, diagnosis, treatment and care for all women; and increase the involvement and influence of those living with breast cancer. An overarching objective common to each of these goals is to ensure that consumers have a seat at the decision‐making table in the research, access and influence arenas.
Project LEAD (leadership, education, and advocacy development), is a science course for breast cancer activists. 6 It was developed in response to a need for breast cancer activists to be sufficiently grounded in science concepts so that they might participate more fully in decision‐making related to biomedical research. The goal of the course is to help develop in students an understanding of scientific concepts and critical appraisal skills in general, not to increase knowledge about breast cancer, breast cancer treatment or the best health care.
Methods
Development and implementation
Project LEAD is organized centrally by staff at the NBCC offices in Washington DC. Faculty include a ‘core’ group of seven scientist‐teachers (two of whom are also breast cancer survivor‐advocates), faculty who teach once or twice a year in Project LEAD and a faculty consultant specializing in adult education. The course is offered 4 times a year at geographically dispersed cities in the USA. The sessions, course materials and meals are provided by the NBCC without charge, and travel scholarships are available for students who otherwise would not be able to attend.
Each course includes about 30 activists who are mainly from the USA and breast cancer survivors. Students are selected by NBCC staff on the basis of: experience with breast cancer advocacy in the community; interest in learning the language and concepts of science; a clear personal connection to breast cancer; a commitment to become part of the breast cancer research/policy process; and a commitment to prepare for and actively participate in the entire course.
Evaluation
To guide instruction and inform modifications, we developed two types of evaluations. The first involved open‐ended questionnaires and verbal feedback sessions on each of the lectures and other sessions, administered at the end of each day and the end of the course. The second evaluation was primarily intended to guide instruction by evaluating student attitudes, knowledge and confidence at the beginning and end of the course. It was structured and administered both before the course began (‘pre‐course’) and at the conclusion of the course (‘post‐course’). The pre‐ and post‐course test forms are identical and they were modified over time as a result of piloting and curriculum changes.
The evaluation contained three sections. The first contained a scenario constructed to represent an individual treatment decision based on evidence from a randomized clinical trial the students have read as homework. Use of critical appraisal skills was assessed by asking the student to choose three questions she would ask regarding the proposed intervention. The students had a list of questions to choose from that included five that we classified as ‘not using critical appraisal skills’ and four classified as ‘using critical appraisal skills’.
The evaluation then moved to questions related to two journal articles, one describing basic concepts in genetic testing and breast cancer (‘basic science article’) 7 and one, noted earlier, reporting on the results of a clinical trial (‘epidemiology article’). 8 Students were assigned to read both articles before the course, and the articles were subsequently critically appraised in an interactive session. Thus, after the course students presumably had additional tools for understanding the articles.
The third area of questioning involved asking the student whether she would be able to explain to a friend approximately 20 concepts covered by the course (e.g. how RNA is involved in making proteins, relative risk). Finally, students were asked to draw conclusions about causality from a graph showing a linear relationship between different levels of fat consumption and breast cancer mortality, in various countries. Because of modifications to the course and the evaluation form in the first 2 years of Project LEAD, not all questions were asked of participants in each session and the total numbers of responses varies by section of the form.
We did not perform any statistical tests as our analyses of the evaluation responses are for descriptive purposes only and do not address research questions.
Results
Development and implementation: course description
As of December 1998, 15 Project LEAD sessions had been held over 4 years, attended by 460 participants (see Table 1). Initial costs were considerable and included: funding a meeting of scientists to develop the initial curriculum; designing, drafting and revising of course materials; and legal fees. Ongoing costs include: NBCC staffing and other operational costs (20% of total ongoing costs); travel costs for faculty and staff (20%); scholarships (13%); mailings and other outreach costs (5%); course materials (12%); faculty honoraria, some faculty decline (15%); and facility, including meals (12%).
Table 1.
Characteristics of students
The key components of the course have remained fairly constant over time: introduction to principles of basic science and of epidemiology; training on critical appraisal of the scientific literature, grant proposals and research seminars; presentations on clinical aspects of breast cancer and the process of research; advocacy training for participating in research decisions; and development of an individual action plan for involvement in research.
The core faculty finalized a set of lecture outlines and materials, and a schedule of sessions, and has continued to revise and refine the course on an ongoing basis. A variety of pedagogical approaches are used to convey the material.
One‐hour lectures are the primary means of presenting new material. Students are exposed through college‐level lectures to the elements of genetics, DNA replication and transcription, protein synthesis, mutation, the cell cycle, carcinogenesis, basic laboratory techniques, and more. They are introduced to epidemiology, including observational and experimental study design, screening, and systematic reviews and meta‐analysis. Statistical concepts are woven into each lecture. With very few exceptions, lecture topics focus on educating participants about science generally, while the examples and documents critiqued are breast cancer‐related. Other lecture topics include behaviour strategies for activists who want to serve as members of research and related groups. We also present students with a full spectrum of advocacy opportunities that include science and encourage students to seek opportunities that best fit their strengths, interests and resources.
At the course outset, students are randomly assigned to study groups of about six people; these meet between lectures and discuss the material presented using a list of questions or exercises as a guide. They also work together to present role play activities and a critical appraisal of a research seminar, described later.
Interactive discussion of ‘case studies’, which use a patient scenario and a basic science journal article (e.g. focusing on BRCA1), places the basic science concepts students have learned into a clinical and social context. Interactive sessions are also used to teach critical appraisal of basic science and clinical research articles. A key component of the course is a research seminar presented by an eminent scientist. Study groups prepare questions and critiques, and select a panel representative to present discussion points to the scientist for his or her response.
At the start of each course, students receive notebooks containing standardized material relating to each session, including lecture outlines, homework, questions for study groups, and reference materials. In the evening, students complete homework assignments, either alone or in groups.
Role play is used to prepare students for participation on an IRB or grant review study section. Each study group reviews a successful grant proposal and prepares discussion points for a role play. The role play is critiqued by the faculty and other students both in terms of the scientific approach and the group’s interactions and problem‐solving skills.
Results of the knowledge and confidence evaluation
Four hundred and forty‐two students completed both a pre‐course and post‐course evaluation that examined their critical appraisal skills, knowledge and confidence levels; 392 completed pre‐ and post‐course questions on the clinical scenario, 392 on the epidemiology article, 255 on basic science, and 372 on their level of confidence with science concepts.
Post‐course, students were more likely to respond to the clinical scenario by asking questions that used critical appraisal skills; 13% of participants pre‐course and about 23% of participants post‐course selected only ‘critical appraisal‐type’ questions as those they wished to ask (see Table 2). In addition, students were more likely to respond post‐course than pre‐course that they would use MEDLINE, other Internet resources, or the public library, to answer questions they had; pre‐course, they were more likely to rely on asking the advice of their doctors or friends.
Table 2.
Questions asked in response to clinical scenario
Overall, students seemed to have understood the article reporting on the results of the clinical trial quite well pre‐course (see Table 3). The students exhibited greater confidence post‐course that all patients randomized were accounted for in the analysis (74% vs. 55%), and more believed post‐course (83%) than pre‐course (71%) that randomized controlled trials are extremely important.
Table 3.
Responses to questions testing understanding of epidemiology and basic science articles
Pre‐course, students appeared to have read and understood points made in the basic science article (Table 3). Substantial improvement was observed post‐course in students’ understanding of substantive basic science concepts.
Overall, students showed a gain in confidence between pre‐ and post‐course tests, when they were asked whether they would feel able to explain basic science and epidemiology concepts to a friend (Table 4). Pre‐course, students felt most confident about explaining why researchers study samples rather than entire populations and least confident about explaining the role of RNA in making proteins, and the concepts of P‐value and odds ratio. Post‐course, students were most confident about their ability to explain mutations and how mutations lead to cancer, and least confident in their ability to explain P‐value, odds ratio and relative risk. Percentage change from pre‐ to post‐course ranged from 31% (why researchers study samples) to 71% (the role of RNA).
Table 4.
Concepts able to explain to a friend or fellow advocate
Although the students improved considerably post‐course in their interpretation of the ecological association between dietary fat consumption and breast cancer mortality in selected countries, 22% still believed incorrectly post‐course that the association was sufficient to infer a causal relationship.
Comment
To our knowledge, Project LEAD is the first national programme to educate health activists about science. It is significant that this effort is organized and guided by activists themselves in cooperation with core faculty. The importance of the general public’s perspective in guiding research at the advisory level has been recognized for many years (for example, public participation is required on NIH advisory panels 9 ), but only recently have consumers been admitted to the traditionally scientist‐only spheres of peer review 10 , 11 and research project planning in the USA. 12 , 13
Implementation of the curriculum was ultimately most successful after a core teaching team was assembled, starting in the second year of the programme. A consistent core teaching team allowed the faculty to modify the curriculum, integrate the various sessions, and develop teaching techniques that were best suited to the learning needs of the majority of the participants.
The course is meant to be the beginning of a life‐long educational process. Thus, teaching materials and approaches stress critical thinking. Students learn the basics of critical appraisal and do not concentrate on results of individual research studies; they are taught science concepts and not about breast cancer specifically; and finally, they are provided with information about how to access available resources. In addition, the notebook provided to all students with course and reference materials serves as a resource for the future, particularly for those serving on study sections or other scientific committees.
While maintaining its overall structure, Project LEAD has been modified in content to respond to new scientific knowledge, and in approach to reflect the unique teaching environment and opportunities as well as the student evaluations. For example, in response to students’ expressed lack of confidence post‐course about their understanding of P‐value, odds ratio and relative risk, these areas have been strengthened in the epidemiology section. The fact that the curriculum changed in response to the evaluations represents a limitation in the interpretation of the grouped evaluation data: pre‐ and post‐course differences may have been smaller before the curriculum changes (presumably improvements) were implemented.
The results of the pre‐ and post‐course evaluations have provided information useful for building and revising the course, and to a limited extent also provide information about how well the students assimilated the material presented. The faculty, who are also invested in the course’s success, believe that over the course of Project LEAD students acquire a deeper and more complex understanding of science concepts as well as increased confidence in explaining or discussing them. It is the faculty’s impression that this level of understanding was facilitated by the small group sessions, which were designed to encourage active engagement with the scientific concepts presented in lectures.
There are some potential qualifications relating to our findings. Since we have not tested the students beyond the immediate post‐course period, we cannot comment on students’ long‐term retention of course material or confidence. However, many students do participate in Project LEAD journal clubs and yearly ‘refresher courses’; these activities plus active participation on scientific committees is likely to be useful in maintaining the skills learned. Project LEAD students are experienced activists and could not be considered typical of the lay public. We expect that a subset of activists in other areas of health are similarly motivated and active in learning about the science aspects of their fields.
Although Project LEAD includes some advocacy training aimed at helping activists fulfil their role in a science setting, this is not the objective of the course. NBCC’s Annual Advocacy Conference, which specifically focuses on training activists, is useful in this regard. Many of the workshop offerings at that meeting focus on ways to bring advocacy skills to the scientific process. Although informal surveys have been conducted to learn about LEAD graduate activities, the proportion serving in a decision‐making capacity should not serve as a measure of the success of the programme. Indeed, one might view the course as serving its purpose when it results in an activist deciding she needs more training before serving on a research committee, or deciding she does not want to serve in this capacity at all.
We believe that Project LEAD is a model programme for other areas. The Department of Defense, The National Institutes of Health, and other funding agencies, are now including patient advocates on study sections, 14 as well as advisory councils. At the Department of Defense, many or most of the study sections deal with basic science proposals. There is therefore an immediate need to ensure that consumers (as well as all other members of these committees) participate effectively. To be effective, consumer education programmes need to be conducted in partnership with community action groups. Otherwise, there will be no guarantee that informed consumers representing a defined constituency will be the ones contributing to decision‐making processes. In addition, partnerships with community groups ensure a mechanism for information flow in both directions. Agencies promoting and involving consumers in the research process and policy‐making related to science and technology should consider supporting science education projects for advocates.
Acknowledgements
Funding of Project LEAD: Nathan Cummings Foundation (1995, 1996), Bristol Myers Squibb (1995–98), Glaxo‐Wellcome (1995–99), Avon Breast Cancer Awareness Crusade (1996–99).
We would like to thank Sara Collina JD, Margo Michaels MPH and Jane Reese‐Coulbourne MBA as National Breast Cancer Coalition (NBCC) staff directing Project LEAD, and Millie Domenech, Becky Levin and Aimee Maceda, as NBCC staff assistants, for their contributions to development and implementation of Project LEAD. We would also like to thank Barbara Crawley MS, Angelique Griffen MS and Qi Zhu MS for performing the evaluation analyses.
Some of the material in this manuscript was presented at the BMJ‐sponsored conference, 20–30 October 1998, ‘50 years of Clinical Trials: Past Present, and Future’ (Kay Dickersin presented), and the 126th Annual meeting of the American Public Health Association, Washington, DC, 15–19 November 1998 (Braun L, Dickersin K, McCarthy M, Mead M, Michaels M, Millikan R, Pietenpol J, Troyan S, Wu A, Visco F. Development of Project LEAD, a science education course for breast cancer activists).
References
- 1. Dickersin K & Schnaper L. Reinventing medical research. In: Moss K. (ed.) Looking Forward: Defining a Health Agenda for Women in the 21st Century. Durham: Duke University Press, 1996: 57–76.
- 2. Marshall E. The politics of breast cancer. Science, 1993; 259 : 616–617. [DOI] [PubMed] [Google Scholar]
- 3. Erickson J. Breast cancer activists seek voice in research decisions. Science, 1995; 269 : 1508–1509. [DOI] [PubMed] [Google Scholar]
- 4. Altman R. The Politics of Breast Cancer: Waking Up, Fighting Back Boston: Little, Brown, 1996.
- 5. Love S & Lindsey K. Dr Susan Love’s Breast Book Reading, Massachusetts: Addison‐Wesley, 1995.
- 6. Erickson J. NBCC’s new goal: make patient advocates part of all breast cancer research decision‐making. Oncology Times, 1995; 17 : 1‐1 16‐18. [Google Scholar]
- 7. Weber BL. Genetic testing for breast cancer. Sci Am Sci Med, 1996; 3 : 12–21. [Google Scholar]
- 8. Spiegel D, Bloom JR, Kraemer HC, Gottheil E. Effect of psychosocial treatment on survival of patients with metastatic breast cancer. Lancet, 1989; 2 : 888–891. [DOI] [PubMed] [Google Scholar]
- 9. 42 U.S.C. 284a; Section 406 of the Public Health Service Act, as amended, and Pub. L. 92–463, as amended, 1998.
- 10. Andejeski Y, Crawford I, Dickersin K et al. and the US, Army Medical Research, Materiel Command Fiscal Year 1995 Breast Cancer Research Program Integration Panel. Impact of Including Consumers in the Scientific Review of Research Proposals. Submitted.
- 11. Mervis JUS. Army to use peer review in brest cancer programme. Nature, 1993; 363 : 195–195. [DOI] [PubMed] [Google Scholar]
- 12. Vanchieri C. Patient advocates help researchers avoid ‘bumps in the road’. Journal of the National Cancer Institute, 1998. ;90: 1193–1195.DOI: 10.1093/jnci/90.16.1193 [DOI] [PubMed] [Google Scholar]
- 13. Agnew B, NIH invites activists into the inner sanctum. Science, 1999; 283 : 1999–2000. [DOI] [PubMed] [Google Scholar]
- 14. NCI peer review: advocates named to panels reviewing center grants, cooperative group renewal. The Cancer Letter, 1998; 24 : 5–6. [Google Scholar]