What Good Is a Scientist in the Classroom? Participant Outcomes and Program Design Features for a Short-Duration Science Outreach Intervention in K–12 Classrooms

Abstract

Many short-duration science outreach interventions have important societal goals of raising science literacy and increasing the size and diversity of the science workforce. Yet, these long-term outcomes are inherently challenging to evaluate. We present findings from a qualitative research study of an inquiry-based, life science outreach program to K–12 classrooms that is typical in design and excellent in execution. By considering this program as a best case of a common outreach model, the “scientist in the classroom,” the study examines what benefits may be realized for each participant group and how they are achieved. We find that K–12 students are engaged in authentic, hands-on activities that generate interest in science and new views of science and scientists. Teachers learn new science content and new ways to teach it, and value collegial support of their professional work. Graduate student scientists, who are the program presenters, gain teaching and other skills, greater understanding of education and diversity issues, confidence and intrinsic satisfaction, and career benefits. A few negative outcomes also are described. Program elements that lead to these benefits are identified both from the research findings and from insights of the program developer on program design and implementation choices.

INTRODUCTION

National concern is high over the quality and equity of science education (National Research Council [NRC], 2005; Bush, 2006), and efforts in both formal and informal K–12 education are needed to foster improved public science literacy and to maintain a strong and diverse science workforce. A variety of organizations, including professional societies, universities, government and industrial laboratories, and informal science institutions such as museums and planetaria, seek to aid schools and to engage the public in science, and scientists are increasingly asked to participate in this work by their funders and taxpayers (Dolan et al., 2004; Andrews et al., 2005). To judge the effectiveness of such investments, it is important to understand what outreach programs can achieve in the “best-case” scenario in which they are well run and well implemented.

The “scientist in the classroom” is a common outreach model that seeks to bring to schools the content expertise and enthusiasm of practicing professional scientists to stimulate student learning, interest in science, and consideration of science careers. We use this term to refer to programs that offer short-duration visits to a classroom where the scientist may give a presentation, lead a hands-on activity, or discuss scientific careers with students, and we distinguish these programs from longer-term partnerships (Alberts, 1993). Programs may be sponsored by universities, professional groups, or community organizations with external or internal funding, and they may stand alone or contribute to a portfolio of outreach efforts within an organization. The model of the visiting content professional is also widely applied in art, music, literature, and other fields.

Despite the popularity of this model, the positive effects of scientist in the classroom programs are largely a matter of faith: little research literature documents their effectiveness. Although both audiences and presenters are reported to be enthusiastic about their participation (Hood, 1994; Koehler et al., 1999; Swim, 1999), most of the published literature consists of outreach program descriptions and advice from experienced program developers (Hood, 1994; Koehler et al., 1999; Munn et al., 1999; Swim, 1999; DeLooper et al., 2000; Evans et al., 2001; Pelaez and Gonzalez, 2002; Dolan et al., 2004; Halford, 2005). These descriptions provide valuable information from experienced practitioners, especially in helping to improve program delivery, but many of the claims made are not supported by evidence gathered using methodologically sound research and evaluation approaches.

The reasons for this lacuna in the research base are not hard to understand. Like many other informal science education programs (Crane, 1994), the change goals of science classroom outreach programs are generally long-term and societal—for example, to increase the number of students pursuing education and careers in science, technology, engineering, and mathematics (STEM) fields or to increase representation of women and minorities in science. When the intervention strategy chosen is short in duration, like a scientist's visit to a classroom, the immediate outcomes of such events are primarily affective, compared with other strategies such as curriculum change or teacher professional development where deeper learning may take place. Short-duration intervention strategies are based on a change model (Seymour, 2002) with the premise that developing interest and enthusiasm around science, having positive experiences with science, meeting science role models, and learning about science careers will translate down the road to more students pursuing advanced science education and careers in high school, college, and beyond. As Bruce et al. (1997) point out, positive attitudes and early positive experiences with science have been shown to correlate positively with outcomes such as choice of high school science courses, college major, and careers, but this link has been shown largely for longer-duration, school-based experiences. Evaluations of informal science programs of any type are rare, and to date these evaluations focus on longer-duration, more intensive opportunities such as summer (Knox et al., 2003; Markowitz, 2004) or after-school programs (Fancsali, 2002).

Given the choice of short-duration change strategies, program outcomes are inherently difficult to evaluate. Although they take place in schools, scientist in the classroom programs share difficulties with other types of informal science learning (Crane, 1994): Because they are typically offered to a wide range of classrooms and schools, the student audience is thus inhomogeneous in grade, age, gender, ethnicity, and socioeconomic status. The course, teacher, and school hosting the presentation also differ. When there are multiple presenters and topics, the content, style, and quality of the presentations may vary as well; they are not part of a regular, annually repeated curriculum. In these circumstances, no common success criteria for students, such as a common assessment of student learning, can be established, nor is content learning necessarily the primary objective.

Methodological limitations also apply. Affordable methods, such as surveys, largely measure attitudes, or, if the instrument is sufficiently sensitive and administered both before and after the event, attitudinal changes. Yet, whether these changes will be permanent or lead to real, long-term behavior changes is unknown. We also do not understand fully what leads to attitude change. Responses to surveys “given at the end of the event when participants are revved up … are a better indication of a participant's enjoyment rather than whether the objectives of the activity were met. … While it is certainly important for the success of a program to have the participants enjoy it, from the point of view of larger goals and objectives, it is inadequate” and should be verified by other types of data (Bogue, 2005). Even survey data that indicate attitudinal changes may not be sufficient to establish what caused those changes (Lott, 2003).

In contrast, a very large, long-term tracking study with multiple control groups would be required to demonstrate statistically significant differences between participating and nonparticipating school-aged students in desirable outcomes such as high school completion rates, high school courses chosen, college attendance and major, or career choices. Such a “gold standard” study would be prohibitively expensive, and the likelihood would be small that all confounding variables, such as students' home and school environments, could be anticipated and controlled in the design to enable unambiguous attribution of the outcomes to any particular influence. Thus, the desired ultimate outcomes for short-duration outreach programs are not meaningfully measured by the evaluation tools most readily available. Yet, the very premise of many outreach programs is that these short-duration events can and do have beneficial results on long-term outcomes.

We attempted to address this problem in our study of a well-established and long-lived scientist in the classroom program. The Science Squad, developed by the Biological Sciences Initiative (BSI) at the University of Colorado at Boulder is an outreach program that supports university science and engineering graduate students to give interactive, hands-on science presentations in area K–12 schools. The BSI staff's long experience with the program and iterative use of formative evaluation data to refine it, the data gathered from internal evaluations, and the high demand for the program by local teachers indicated that the program was well executed and well received—successful by all apparent measures. The scientist presenters were extensively trained; their work was observed and refined to ensure that the programs were inquiry based and well delivered; and the program had encountered, but had already solved, some of the problems met by other scientist in the classroom programs (Bruce et al., 1997; Community Resources for Science [CRS], 2006). Together, these lines of evidence were persuasive that the program was functioning well, but we wanted to know more about the specific ways in which the program might be making a difference: What impacts of the program could be documented for K–12 students and teachers and for the Science Squad presenters, and how did these impacts come about? Thus, the program staff (J.G. and her colleagues) worked together with an independent research group (S.L., C.L., and H.T.) to design and conduct a summative study to examine the longer-term outcomes for participants and presenters and to elucidate the processes by which these outcomes arose (Laursen et al., 2004, 2005). We chose qualitative methods for their strength in detecting outcomes that are likely to be primarily affective and attitudinal rather than cognitive, and in understanding processes and interaction (Denzin, 1989).

Beyond its evaluative utility to the program itself, the study findings have broader applicability. The evidence indicates that this program represents a best-case scenario for its type, the short-duration scientist in the classroom outreach program. Thus, a study of this program offers the chance to determine whether such outreach programs, when well implemented, have any lasting benefit. The findings are of use beyond this program because the model is widely used, although little studied, and because we can link program outcomes to elements of the program design that can be incorporated or adapted by others.

In this report, we present study findings about the outcomes for students, teachers, and presenters. Student outcomes are most difficult to measure, yet the findings are consistent as seen by two groups of adult observers in indicating the types of benefits that may be anticipated, by using different types of measures than the typical postevent survey. Moreover, when added to the student benefits, the additional benefits to teachers and especially to the presenters present a powerful argument for the merits of this type of program as one variation within the portfolio of efforts across the outreach community.

Our aim here is to present the research findings in a manner useful for practitioners—the designers, organizers, and sponsors of outreach programs. To shed further light on how the program outcomes are achieved, we also include practical insight about the design and support of the program that may aid others. Thus, this article includes both findings from a qualitative study by the independent research team and advice distilled from years of experience by the program staff. These types of information are based on different kinds of evidence, and we are careful to distinguish them in our narrative. More information about the program design and logistics is available at the BSI website (http://www.colorado.edu/Outreach/BSI/k12/sciencesquad.html) and from J.G.

BRIEF PROGRAM DESCRIPTION

The Science Squad is one of several outreach programs sponsored by the BSI under a series of multiyear grants from the Howard Hughes Medical Institute. The Science Squad comprises four to six graduate students from biomedically related science and engineering departments who each visit K–12 school classrooms to lead inquiry-based, hands-on science presentations.

Science Squad members are selected in a rigorous application and interview process. Once chosen, they receive compensation comparable to teaching assistantships in their graduate departments; most participate in the Science Squad instead of working as a teaching assistant (TA) for a given year, while continuing their graduate research. Together with BSI staff, each member creates a set of four presentations that includes hands-on activities and that are related to their area of scientific expertise. They typically offer these presentations 2 days a week to several classes at one school each day. The program thus provides both an intensive teaching experience to Science Squad members and a science enrichment experience for many K–12 students and teachers.

According to its mission statement, the BSI seeks to “increase the number of students interested in careers in the biological or medical sciences, to strengthen their biology education, and especially to encourage minority and women students entering the sciences” (http://www.colorado.edu/Outreach/BSI/k12/sciencesquad.html). To achieve the latter goal within the Science Squad program (one of several run by the BSI), schools are prioritized for Science Squad presentations that have low-income and high-minority student populations, although other schools may and do participate. Science Squad members are deliberately selected for their potential to be role models for all students.

The program is advertised by mailed brochures and on the Internet. Teachers request program dates by phone or on a Web-based calendar system, and they may schedule up to four presentations during a school year. In a typical year, the Science Squad reaches an average of 14,980 students (46% minority, 56% female) and 273 teachers (data from program averages 1996–2004).

BSI staff members monitor the program through teacher feedback forms and reflective end-of-year surveys completed by Science Squad members. One-on-one coaching, classroom observation, and monthly team meetings provide feedback to members to help them improve their presentations and brainstorm solutions to problems. Additional program features are discussed below, where they bear on and give insight into the study results. Implementation details are the focus of the section titled Developer's Advice: Program Design Choices.

STUDY METHODS

In this report, we share the direct perspectives of some important constituent groups: the Science Squad presenters and the teachers, two groups who were interviewed in the study, and the program director, whose experience is shared directly in Developer's Advice: Program Design Choices. The choice not to include students' perspectives directly in the study design was made after discussion of the attendant challenges, including the general difficulties discussed in the Introduction and the limited funding available for this study.

This choice does not discount the importance of student outcomes, but rather it acknowledges the difficulty of measuring the outcomes that matter. We chose to solicit observations from two groups of adults about children's responses—the behaviors and changes in behavior that these adults noticed in participating students. The two groups observed students under different circumstances: Science Squad members encountered many students unfamiliar to them and for a short time only, whereas teachers could observe their own students in the days and weeks after the Science Squad presentation. That these observations are consistent across the two groups supports the validity of this approach. It is also clear that further research is required to understand the longer-term implications of these medium-term student outcomes.

The interview samples are intentional, reflecting patterns in the nature of the scientist presenters and the teachers and schools seeking their assistance. The interviews included two groups: Science Squad members and K–12 classroom teachers. A nonrandom, purposive sample (Berg, 1989) of 16 “frequent user” teachers was chosen from among teachers who had used the Science Squad for several years, often multiple times a year. Given the dearth of published evidence about the outcomes of short-duration outreach interventions, these teachers were chosen as those most likely to be able to report any effects on students, due to their multiple opportunities to observe students' responses to the program in their classrooms. This choice probably biases the sample toward teachers with a positive view of the program, but a positive view is also typical: Internal evaluation data reflect that few teachers report negative experiences with the program. Indeed, the only consistent negative report from teachers is of difficulties in scheduling, because there are too few Science Squad presenters to meet the demand. The study questions were summative; for formative purposes, it would have been critical to include teachers who discontinued use of the program.

The teachers in the study were all white and mostly female, typical of the teacher population in the schools served. The Science Squad primarily serves secondary schools: of the teacher sample, eight (half) taught high school (grades 9–12), six taught middle school (grades 6–8), and two taught elementary grades (K–5). Five taught in urban schools, and the remainder were in suburban settings of varied socioeconomic status. Three taught in high-minority (≥75% nonwhite) schools. In addition to using the Science Squad, most of the teachers interviewed participated in other BSI outreach programs, such as teacher workshops. Overall, the teacher sample has a high self-reported use of inquiry teaching strategies for science and high interest in providing role models and authentic science experiences for their students, more than typical in other teacher populations we have studied, and the findings reflect their perspective.

Interviews with 24 Science Squad members were conducted with former Science Squad members whom we could locate, out of a total pool of 34 past members who had participated during the 10-yr period from 1992–1993 to 2001–2002. The sample of 20 women and four men reflects the historical gender makeup of the program. Most interviewees and past Science Squad members were white. Members came primarily from biology departments but also anthropology, engineering, and geography departments. Their Science Squad experience ranged from one to six semesters; multiyear participation accounts for the low total head count. Most participated as graduate students, with a few postbaccalaureate or postdoctoral scientists. Many had extensive teaching experience with youth in a wide array of informal and experiential education settings, and all expressed high interest in teaching.

Interview protocols addressed the benefits to students, teachers, and Science Squad members of participating in the program and how these benefits were achieved. Benefits questions in particular asked for respondents' evidence for their claims: What behaviors did teachers or members observe? By which students? What examples could they provide? We also asked participants about their motivations for participating; their experiences with the program, including difficulties or costs to them; and we invited their advice to the program staff.¹ Teachers also described their classroom demographics, courses, and preparation for and follow-up to the programs. Science Squad members described their education and career paths and the Science Squad's role in their career decision-making. We also asked participants to respond to a common critique of programs like this program, that a one-time intervention may not yield any longer-term effects. These protocols were also adjusted throughout to incorporate newly emergent issues into subsequent interviews. Teacher interviews lasted 20–30 min; Science Squad member interviews lasted 40–60 min. The interviewers clarified the independence of their role relative to the staff and reiterated the confidentiality and anonymity of the interview data.

Our choice of ethnographic interview methods is well suited to this study. Grounded in methodological traditions from sociology, anthropology, and social psychology and commonly used in education, ethnographic approaches can uncover and explore issues that shape individuals' thinking and actions. Especially when previous knowledge is limited—as in this topic—findings from ethnographic data may be used to generate hypotheses for experimentation or to develop constructs for survey studies. Modern software tools allow ethnographers to disentangle patterns in very large text data sets and report them by using descriptive statistics. The results from careful sampling and consistent coding of text data can be very powerful, and these features distinguish qualitative research from collection of anecdotes. Readers more familiar with experimental methods may find the article by Green and Britten (1998) a helpful introduction to qualitative research.

Our methods are detailed in Seymour et al. (2004) (also see Berg, 1989). Briefly, interviews were conducted by phone, audiotape recorded, and transcribed verbatim, with the respondent's written consent. As transcripts were searched for information bearing on the research questions, text segments referencing distinct ideas were tagged by code names by using N'Vivo qualitative software (QSR International, Doncaster, Victoria, Australia). Codes were not preconceived but empirical: Each new code marked a discrete idea not previously raised. Because the codes and passages were linked in the software, a data set was amassed for each interview group. Groups of codes that clustered around particular themes were given domain names, and these clustered codes and domains defined the themes of qualitative analysis. In essence, coding an interview transcript may be compared with disassembling a necklace and sorting the beads by color, shape, size, and other variables.

For the Science Squad member interviews, ∼400 codes were developed and sorted into 18 major domains, such as motivations for participating, personal benefits, personal costs, teacher benefits, student benefits, career impacts, graduate school history, advice to project staff, and gender issues. Although many broad domains (e.g., personal benefits) could be anticipated from the research questions and interview protocols, they and their subdomains were developed from the data, as each broad domain was categorized and analyzed to build explanatory accounts—for example, sorting Science Squad member benefits into the types described below, or developing explanations for the high representation of women on the Science Squad. For the teachers, ∼160 codes were developed and categorized into domains such as classroom demographics, observations of student interest, differences in student reactions, their classroom practices, and observations about Science Squad presentations and about the Science Squad program as a whole. The analysis was conducted by the independent evaluators, but throughout the analysis, emergent findings were shared with the project staff (J.G. and colleagues), and their input provided clarification, advice, and a “reality check” on the findings.

It is sometimes useful to count the frequency of use for certain codes across the data set, by using conservative counting conventions to avoid overestimating the weight of opinion. Together, these frequencies describe the relative weighting of issues in participants' collective report. They hypothesize the strength of particular variables and their relationships that may later be tested by surveys or other means. These counts are not, however, drawn from random samples of identically conducted interviews; they are not subject to statistical tests. In this article, we count speakers rather than codes—that is, we report the number of individuals making an observation in a particular category rather than the total number of observations in that category. Following the bead analogy, we count the number of necklaces on which one or more beads of a certain type occur, rather than totaling all the beads of that type.

STUDY FINDINGS: OUTCOMES FOR K–12 STUDENTS AND TEACHERS

Student Gains

Evidence of student gains comes from both teachers and Science Squad members, through their observations of student response to the programs—classroom behavior, engagement in activities, verbal and affective responses, questions asked, and answers given by students. Teachers were able to observe and report evidence that emerged over the longer term, such as student comments or questions in later classes, or further investigation of the presentation topic in a paper or project. Their prior knowledge of their students enabled them to notice responses such as enthusiasm from a student not usually interested in science. However, they only saw the response of students in their own classroom. Science Squad members' observations had countervailing advantages: they had no preconceived notions about particular students, such as whether they were “good students” or not, and they saw many more students in a much wider variety of classrooms. The high corroboration in the types of student gains reported by these two groups is thus notable. Their different perspectives account for differences in the extent to which each group observed these gains.

The student gains fall into three categories, summarized in Table 1 and discussed in more detail below.

Table 1.

Student gains reported by teachers and Science Squad members

Student gains	% of teachers reporting student gains	% of Science Squad members reporting student gains
Enhanced interest and engagement in science	88	92
New views of science and scientists	44	100
Understanding of science concepts and their relevance to real life	38	33

Enhanced Interest and Engagement.

Enhanced interest and engagement was reported by 14 of 16 teachers (88%) and 22 of 24 Science Squad members (92%). As evidence for these benefits, teachers reported student behaviors such as concentrating on the activities, asking questions, and stating their interest. No teachers reported lack of engagement or interest.

“It was hard to stop them from what they were doing. I think they were looking at bones, or they were looking at photos and they were trying to figure out a mystery … they wanted to talk more about it in their groups or look at things again.”

“One kid came up to me and goes, ‘Oh, will she come back? I really want to talk to her more about this—I'm so interested in evolution.’”

Several teachers reported that all their students were engaged—despite variations in grade level, course titles, audiences, and levels (e.g., introductory or advanced placement)—and ascribed this engagement to the inquiry style of the presentations.

“I have seen kids that are low achievers, that have low motivation, don't do very much … some of the few times that I've actually seen them engaged and excited, and doing something, [are] when Science Squad presenters were there.”

“The English as second language kids tend to do okay with it, because, typically, there are lots of hands-on materials. They don't, obviously, cognitively get every piece, because of the language barriers. But most of them will get a sense of what it's about.”

Others noticed the “leveling” effect of Science Squad presentations, enabling both high- and typically low-achieving students to succeed.

“I've always expected my Honors Bio classes to be a little bit more sophisticated in the questions they ask, and maybe in their ability to manipulate the experiment—and what has always been funny to me is, they're not. At a sophomore level, the kids, once they're put into that kind of environment, they pretty well function at the same level. They ask the same basic questions and they have the same confusions during the presentation.”

“It hit the child who's at the top end, that doesn't necessarily get stimulated, down to the one at the bottom end that's like, ‘Oh, something else over my head.’ I'd say it was creative to the point where it involved lots of different abilities, and interests.”

Science Squad members also observed students' interested responses to their presentations. They attributed student interest to their choice of inherently interesting topics and the engaging, hands-on activities and authentic science materials and equipment that they brought. They hoped that this interesting one-time experience would spur students' longer-term interest in science: “I would do things on primate behavior and primate evolution and human evolution and spark some interest in things that they hadn't really thought about before.”

“It was very hands-on, and they were required to participate in ways that they aren't normally participating in their daily school activities. … They couldn't just sit there and read the book or be taught something up on the board. They actually had to do it in order to see what the results were. … My impression is that they didn't have that opportunity as much in their day-to-day activities, and so it was more of a discovery process for them. And when they found the end, they're like ‘Wow!’—just really impressed that they could figure it out.”

Also important was the novelty of a presenter different from the regular teacher. Both teachers and Science Squad members noticed that students perked up and paid attention to someone different from their everyday teacher, someone perhaps younger and more “cool.”

“With very few exceptions, you'll notice the people are paying more attention. They are kind of curious, 'cause who is this person, you know? And they're not gonna act up, because they don't know how the person's gonna respond yet. They haven't figured them out. (laughs) So, the person, him- or herself, the presenter, is in a lot of ways an attraction from the get-go.” (Science Squad member)

New Views of Science and Scientists.

A second category of gain, also broadly observed, was changes in students' views of science and scientists. Seven teachers (44%) and 24 Science Squad members (100%) reported this student gain. Moreover, many teachers stated that they deliberately used the Science Squad to provide scientific role models for their students. They wanted students to see that science was something real people did and enjoyed and that science provided many career and education options. This was something they could not easily provide in their role as teacher.

“I think when they see someone from outside of school, knowing that this person is in the real world going to school, going for their degree, either a graduate degree or work—just being outside of school and actually being interested in science, enjoying science, enjoying kids, I think that means a lot to them.”

“One of the things that's nice about Science Squad … is [to] make them aware that people get paid for doing these things. You know, that people get paid to go study lemurs in Madagascar, and people get paid to figure out how people died. And that those are career options—they get paid for hiking in. ‘Cause sometimes kids just don't get that.”

Science Squad members, too, described seeing changes in students' stereotypes about science and scientists. A fun and successful science experience could challenge students' belief that science was boring.

“When the really cool kid in the class comes up and says, ‘Oh wow, that was cool!’ you really feel like you've accomplished something. You've made this something acceptable, socially acceptable to the high school students, so that it's okay to participate in it and have fun participating in it … and, yeah, to me that's a huge success.”

By sharing their enthusiasm about science and their personal stories of entering science, Science Squad members were able to broaden students' views about what scientists were like, who can be a scientist, and the many paths to pursue within science.

“A lot of kids have a really stereotypical image of what a scientist is. And so being exposed to an actual scientist, or somebody that's involved in science, I think broadened their views a little bit about what it meant to do science. And maybe didn't make them think in such a small box. [That] lots of people, in lots of different places, can do science. And that science covers a really broad, broad variety of topics that they might not have thought about before.”

“I think having somebody who's sort of like them come back and say, ‘Look, I did this, and this,’ it sort of opens it up. ‘Cause otherwise you get used to your little narrow world of, go to high school, and maybe go to college, stay here and get married. You don't really see a lot of other possibilities role-modeled for you.”

This benefit extended not only to members of the same group as the presenter but also to other groups. For example, stereotypes held by both boys and girls were dispelled when a woman Science Squad member presented.

“The girls in particular really enjoy Science Squad, because most of the presenters are women. And I think that a lot of the ones that are maybe thinking about science for later on, just think it's really neat to see somebody, a female, doing something that they might want to do later on. But I also think a lot of the boys think it's really interesting that it's all these girl scientists that come in. It might change their image of who a scientist is, a little bit.” (Science Squad member)

The majority of presenters were women (20 of 24 interviewees), and some were members of ethnic minorities underrepresented in STEM fields. We discuss the high representation of women elsewhere (Laursen et al., 2005) and in a forthcoming article.

Presenters' perspectives could help students understand the concrete steps needed to pursue a career in science and could make these steps seem realistic and possible.

“Often … the teachers would ask us to talk about what it's like to be a student at the university. What's expected of students at a university? What is it like to go to a university to study? They would ask us to talk about our career aspirations—were we going to become a research scientist or a professor? And what sort of steps did it take to go from being a high school student to becoming a professor at a university? What sort of opportunities had that allowed us in our lives?”

“I think the students, especially in the lower-resource schools, just aren't even aware of what their options are, in terms of what kind of education does it take to get a certain job. And so just kind of helping to get a more realistic view on things, that you don't need a Ph.D. to do everything in science.”

Both teachers and Science Squad members emphasized the importance of sharing career ideas with students, information not always accessible to all students, as this Science Squad member pointed out.

“A lot of students have said, ‘Oh, I never thought about doing this.’ And I would always ask, ‘Okay, how many of you are interested in doing this, or doing that? Why? Why not?’ And you'd be surprised at some of the answers. A lot of times their counselors tell them they can't!”

Understanding of Science and Its Relevance.

Finally, teachers and Science Squad members reported gains in students' understanding of science—concepts, skills, and new connections. However, consistent with the short duration of the intervention, this benefit was much less widely reported than gains in interest and new views of science—only eight of 24 Science Squad members cited it, and six of the teachers. Most often these learning gains were related to scientific skills, such as collaboration, critical thinking, and problem solving, rather than specific subject matter content.

“Sometimes analyzing, if they got results back from a lab, sometimes looking at the results collectively as a class and trying to interpret them … critical thinking skills, trying to think a little bit more.”

Also important were the connections between science and everyday life that students made.

“We educated them about pertinent, germane scientific issues going on today. Like, there's some genetics work that was done in some of the classes, some genetic testing-type stuff, like, ‘What do you do if this disease runs in your family? You could get a genetic test, and this is how it works.’”

“To me, what they really got out of it was that there was science in their lives. That there was science that was actually not that far away from them. I think people see, and kids, too, see stuff on the news about whatever cutting-edge science, whatever's sexy enough to show on TV, and that's always pretty far removed from what you can see, feel, touch, smell, taste, outside your door.”

The relative paucity of learning gains as a student outcome observed by the adults is notable—but it is also not the primary objective of the program. This paucity is also consistent with the short duration and “special topic” nature of the presentations (even if aligned with the school curriculum through the teacher's efforts). Other studies also find that, although students may learn new vocabulary, the larger impacts are on their interest and enjoyment of science (CRS, 2006). These findings support our argument in the Introduction that student learning assessment is not the most appropriate outcome measure for programs of this type. However, the learning gains that were reported are well aligned with community-sanctioned science learning goals for students as represented in the national and state standards, particularly those emphasizing the abilities and understandings of inquiry (NRC, 1996).

Overall, the gains observed by adults among K–12 students emphasize development of interest and new views of science. These findings are consistent with those from studies of student outcomes for short-duration outreach programs that have used other methods (Bruce et al., 1997; Bottomley et al., 2001; CRS, 2006). For example, Bruce et al. (1997) found that even very young students found one-time scientist presentations special and memorable, as seen in their drawings and conversations, and were more likely to identify scientists as people like themselves. New views of scientists and novelty were student benefits reported by Trautmann et al. (2002). These gains also were those sought by the BSI staff in their program design.

Importantly, the K–12 student gains are all related to science literacy. Although these gains cannot be said to define science literacy (a complex construct), they do contribute to development of science literacy by engaging students' interest and providing views of science as an accessible, human activity (Eisenhart et al., 1996; Bybee, 2005). Hands-on, personally relevant learning experiences have been shown to be effective in inspiring students, especially girls, to continue to take classes in science and possibly pursue a career in the field (Fancsali, 2002; Eisenhart and Edwards, 2004). Students' gains are in domains where scientists' contributions have been argued to be greatest (Alberts, 1993; Bower, 1996; Bybee and Morrow, 1998; Colwell and Kelly, 1999; Laursen, 2006).

Teacher Gains

Teacher gains included those reported by teachers themselves and by Science Squad members who made observations or conversed with teachers about their gains. Again, the high degree of corroboration between the types of gains reported by these two distinct groups of observers increases the reliability of the findings. When asked about the benefits of the program to themselves, teachers often described benefits that were in fact realized by their students. Because these gains helped teachers be more effective in their work of teaching students, it was difficult for teachers to separate their own gains from their students'—they valued the student interest and learning that they observed and saw it as supporting their work; thus, they described this as a benefit to themselves as professionals.

As outsiders, Science Squad members were more easily able to identify benefits to teachers that were separate from enhancement of their classroom work. All 24 members, and four teachers, identified gains in teachers' understanding of science topics and ways to teach them effectively. Because teachers are responsible for a wide range of knowledge, their knowledge in any one area may not be deep, they may be teaching out of discipline, and it is difficult for them to keep up with cutting-edge science.

“With teachers, I feel like definitely it's more my role just to help them stay current in what's happening in science. The recent advances in science, and to provide them as many tools as I can, to teach those concepts in the classroom.” (Science Squad member)

“Experts coming in and talking about topics that I really wouldn't be able to do for my class. So I think that's the best benefit. We all learn and the kids get a lot out of it, because I don't know those topics.” (teacher)

Science Squad members reported examples of helping teachers clarify particular concepts and providing technical advice to improve experiments or activities. They saw teachers benefit from seeing new approaches to teaching familiar content. Sometimes teachers gained specific ideas for classroom activities to use again, and other times simply a new perspective.

“They can also see how another person presents some concepts that they may be familiar with teaching in the past, such as the scientific method, or they may not be, like gel electrophoresis. So they get to learn new things, too, another person's presentation style or new concepts, experiments, and activities.”

“And particularly because human evolution can be a little bit of a testy topic that some teachers aren't sure how to negotiate, I think they thought it was very helpful for us to give an idea of how to structure some of those lessons, and how to deal with something that might be a little touchy for some students. Because, depending on your background, you may have never taken an anthropology course … I think there really was some new content.”

These benefits came to teachers through seeing how a content specialist approached the lesson, by direct participation in the activities, and by observing their students' reactions to the lesson.

“The majority of teachers were actually pretty involved in the project. They were getting, on the very surface of it, at least the same thing that the kids were getting. They were getting some science, and they were doing some science, and you could kind of see the gears turning.”

Science Squad visits sometimes had lasting effects on classroom practice. Several teachers reported using Science Squad activities on their own later. Most teachers in this sample of frequent users reported that they already used inquiry pedagogies in their own teaching. Thus, only a few reported changes to their teaching style as a result of seeing inquiry-based teaching modeled in Science Squad presentations. However, teachers did report that observing Squad presentations reinforced their belief in the importance of active learning, because they saw their own students benefit from the hands-on, minds-on approach used by the Science Squad. The opportunity to step back and focus on student learning was valuable in itself. Although each Science Squad visit represents a single opportunity to observe students, the benefits reported by teachers are consistent with those cited by professional development opportunities, such as lesson study, that include collegial observation (Stigler and Hiebert, 1999).

“I guess it just reinforces for me why hands-on labs are good. I do a lot anyway, but it just continues to reinforce it, especially when I sort of observe. When I observe more intently how kids are listening or not listening to the content part, it makes it easier to realize why it's so important to do the lab part. It's kind of interesting too, to me, ‘cause sometimes I'll sit and really listen to how they give instruction and think about how I do it myself. So it provides some reflective time for me.”

Science Squad members' reports of their conversations with teachers, follow-up requests by teachers, and other teacher interactions indicated a broader degree of influence on teachers' teaching styles than did the teacher interviews. This difference is consistent with the differing perspectives of the two groups: Squad members interacted with a more varied group of teachers, not just the inquiry-supportive group represented by the frequent users.

In addition to their own professional learning, teachers benefited from a break in the routine. The Science Squad visit provided a respite from the everyday grind of planning and leading lessons. Last, teachers gained a variety of types of emotional support, including validation of their work through the support of a university program, collegial interactions with Science Squad members, and the intrinsic pleasures of learning with their students.

Negative Outcomes for Students and Teachers

We found little evidence of negative outcomes for students and teachers. In large part, the negative evidence reported was about missed opportunities or gains not made that might have been made in other circumstances. For example, a few teachers reported that their students did not consider Science Squad presenters as role models, because their contact time was too short to establish a meaningful personal connection or because the students were too young to view the presenter as a possible “future self.” An occasional case of poor fit occurred when a (usually novice) presenter aimed a presentation too high or too low for the student audience. Science Squad members reported classroom sessions to be less effective when teachers did not adequately prepare or supervise students. Most Science Squad members reported experiencing such situations, but as a small fraction of their total experience. Presenters and teachers alike reported that good advance communication clarified expectations and reduced the likelihood of problems.

The most frequent problem reported by teachers was the inability to schedule a particular presentation or preferred date because it was already booked. This is a negative for the individual, although it reflects the success of the program as a whole. Our data do not show whether scheduling was a significant issue for those who stopped using the Science Squad, or, conversely, whether reports of scheduling problems were greater among these teachers who wished to use the Science Squad often. A Web-based scheduling system was introduced recently, after our interviews, and the project reports that this has reduced scheduling problems substantially.

STUDY FINDINGS: PROGRAM ELEMENTS THAT SUPPORT BENEFITS FOR K–12 TEACHERS AND STUDENTS

Teachers were particularly insightful in identifying features of Science Squad presentations that led to the positive student outcomes they observed, and Science Squad members echoed these views. Often teachers discussed these features as factors in their decision to invite the Science Squad back to their classroom year after year. Key features included the following:

• Equipment and materials that enabled science learning experiences that were authentic and often novel for students (cited by 69% of teachers). Teachers valued the opportunity for students to experience new areas of science—for example, forensic botany or DNA fingerprinting—and use real scientific tools, such as bacterial cultures inoculated with a virus, or primate bones and skulls.

  “One of my major roles was bringing in concrete things to the classroom—either slides that I had of unusual environments, or lab demonstrations, microscope preparations…. The schools that I worked in, I felt, would not—the students would not have seen live organisms. They would not have seen these type of preparations under the microscope.” (Science Squad member)

• Science topics that were interesting, contributed to the curriculum, and taught both students and teachers (63% of teachers). Teachers valued Science Squad members' scientific expertise and the relevance and currency of the topics they chose.

  “In biology, you cannot be knowledgeable about everything, so that there are people who know a lot about diabetes, or know a lot about phosphorescence, or something of that nature, and they come as experts to your class. They would know more on that particular subject than I would, so it's a good thing to have them come.” (teacher)

• Style of presentation (50% of teachers). Teachers mentioned the hands-on and inquiry approaches that were integral to the Science Squad presentations, and the fact that Science Squad members were well prepared to lead their activity and adapt it to their audience.

  “The biggest impact is being able to provide the students with exposure to a variety of labs that we otherwise couldn't do. Because the labs are what makes science live…. The greater the variety of lab topics, the greater the possibility of reaching kids for science.” (teacher)

• A break from routine (50% of teachers). Teachers appreciated the way a novel presenter could provide new perspectives and bring students to a high level of alertness and interest.

  “I think it's really good for my kids to really be exposed to a lot of different people and not just hear one point of view, not just hear one teacher. I think they love having someone different talk to them.” (teacher)

These features gave rise to the student benefits observed and were valued by teachers. In parallel with their students, teachers benefited when they were actively engaged in the presentation themselves, participating as learners, and supporting the presenter's work with their students. By contrast, when teachers did not actively participate, Science Squad members reported that the presentations were less effective, and benefits to both students and teachers lessened. Emotional benefits to teachers arose from collegial interactions with the visiting Science Squad member—planning and follow-up, having lunch together—and their awareness of the effort and interest of Science Squad members in coming to schools and doing a good job.

In sum, both the pedagogy and content of the Science Squad presentations were crucial to the Science Squad's effectiveness in the classroom. These reflect choices made by the program—aspects of program philosophy, selection and preparation of Science Squad members, and program structure—that are discussed below. Teacher engagement was a critical variable in mediating the effectiveness of Science Squad presentations. Teachers were able to further enhance the benefits of the Science Squad benefits by preparing students for the session and following up on it later, making connections to their curriculum, and adapting effective activities for their own teaching.

STUDY FINDINGS: OUTCOMES FOR SCIENCE SQUAD PRESENTERS

Gains Reported by Presenters

In addition to the positive outcomes observed for K–12 teachers and students, the program also had substantial benefit for the Science Squad presenters. These benefits include the following types (with the percentage of members reporting any gain in each category in parentheses) and are discussed in more detail below:

• Gains of skills—in teaching, communication, and management (83%)

• Gains in understanding, particularly of issues surrounding education and diversity (92%)

• Personal gains, including growth in confidence and intrinsic or emotional rewards (83%)

• Career gains (96%)—primarily in the form of transferable skills and understanding, but also résumé enhancement, career networking, and materials used in later work. Career gains also include effects on members' career paths, such as clarification and confirmation of career plans or new career ideas based on the Science Squad experience.

Gains in Skills.

Gains in teaching skills were reported by most Science Squad members (83%). These included component skills, such as communication and organizational skills, but they were counted as teaching skills when described by interviewees as applied in a teaching context. Members gained skill in explaining and demonstrating complex scientific ideas to a variety of audiences, finding simple language and vivid analogies. They came to understand their own science better through explaining it to others with less background knowledge.

“I certainly had to learn a lot about the subjects that I was teaching about, so that I had enough knowledge to teach it to others. And I had to learn enough about it so that I could teach it at different levels … from kindergartners to seniors in high school.”

An important step was learning to adapt their presentations to the different audiences they encountered:

“… being able to gauge the level of the audience's understanding, ‘cause you can present the same activity to a seventh grade class and a 12th grade class, and there will be a vast difference in their background and their previous knowledge and their energy levels, their maturity. And so all those factors can be adjusted for, because you're giving the same material and content each time, but the way that you present it or the order that you present the concept or the things that you emphasize, or kind of just breeze over … can be different.”

Members developed both a recognition of the importance of adjusting to their audience, and the ability to do it quickly in a wide range of settings. They gained skill in using interactive teaching strategies and developed a belief in their value. They developed practical teaching skills, such as lesson planning, estimating time, developing low-cost and age-appropriate materials, and managing classroom dynamics.

“The content's still essentially the same content, but if you do things a little bit different, it can change the dynamic as you go through the day. Just your physical interaction with them, how close are you, are you up at the board, are you at their desks, are you picking people as volunteers to do things, or are you just trying to hurry through the introductions so that they can do their part of the project? And there are a lot of little pace things that I think I got a feel for … more in Science Squad than I did as a lab TA.”

Perhaps most importantly, participants were able to link these skills together to develop a coherent, personal teaching style. Even those with previous teaching experience found in the Science Squad an opportunity to refine, reflect on, and improve their teaching style.

“On the concrete side, it developed my teaching style. I mean, you're in the classroom two full days a week. And it doesn't matter if you have a cold, if you're crabby, or whatever, you have to be on for that five or six hours, five or six periods of that day. So it definitely helped me to develop my teaching style, and my comfort zone, find my way of presenting ideas, and what works and what doesn't work.”

Although communication is an element of teaching, 15 members (63%) reported gains in communication skills that they distinguished from teaching skills. This included increased confidence in their public speaking abilities, lowered anxiety, and the ability to select appropriate visuals and language—skills seen to apply widely beyond the classroom. In addition, five members (21%) reported gains in organizational skills, such as managing a complex schedule and planning ahead.

Gains in Understanding.

Gains in understanding were a second large category of benefit to Science Squad members, encompassing a range of issues affecting public education: diversity, social inequities in educational access, articulation between K–12 and higher education, student learning and development, and the work of teachers and schools. Gains in understanding of diversity and equity issues were particularly strong, reported by 17 members (71%). As they interacted with students and teachers in schools served by the Science Squad, members were exposed to broader diversity than they had encountered in their upbringing or in the university and were disturbed by the inequities they saw.

“My experience growing up was one where I was in a community where there wasn't any ethnic diversity. So my experiences in the Science Squad really opened my eyes …. I mean, the classrooms that we were going into were a better fit for what the general public in the United States is made up of, as far as ethnic diversity.”

“… Just things like the simple fact that the students can't be given a book to bring home. I'm just appalled. I just don't see how those students will ever be competitive with students from more affluent communities, where they have those resources.”

They came to realize the importance of high-quality education for changing students' economic and career prospects, and to understand the potential impact of cultural differences in the classroom—for example, in the different ways students interacted with their teacher. One interviewee articulated particularly well how greater awareness of diversity could also increase motivation to solve these social problems.

“The Science Squad provides an opportunity to get your hands dirty, getting in there and trying to do something, or at least experience what it is like under those conditions. So if the interest is just enough interest to experience the different issues and social issues, cultural issues, educational issues in the cities, then the Science Squad provides that. And if it's someone who wants to do a little bit more, then at least they have an opportunity to try to make a difference in some way.”

Members also reported gains in understanding how students learn at different developmental stages, and how to reach diverse learners. Greater understanding of the nature of teachers' work and school systems helped them to appreciate the difficulties faced by teachers and respect the teachers who persisted under challenging conditions. Understanding the varied backgrounds of their future students was helpful to those planning to teach at the college level.

Personal Gains.

Gains in confidence were reported by 17 members (71%). These were not gains in general self-esteem, but confidence in their ability to do the work at hand—to communicate science to others, to manage a classroom, to represent their STEM discipline.

“It gave me more confidence in seeing myself as an anthropologist rather than just a graduate student, because I was going out and representing my discipline, to so many, in so many different arenas.”

Confidence gains transferred to their professional lives, providing new ideas about future work options and the ability to compete effectively for teaching positions.

“It also made me much more comfortable with teaching in general, and so more interested in maybe teaching at the college level, which is something that I hadn't given as much thought to, before.”

“I was absolutely comfortable going into any teaching situation and being able to teach—I mean, just off the top of my head without being familiar with the students or the setup…. And so I think that gave me a lot of confidence going in and being able to teach and give presentations during my job interview.”

In addition to confidence gains, members reported intrinsic emotional benefits: “I got a lot of personal warm fuzzies, which are important, you know.” It was gratifying for them to see students learning, having fun, and appreciating science. Members met students who remembered them, received thank-you notes, and felt inspired by seeing teachers do good work in difficult settings. They had fun with the students and enjoyed collegial connections with their Squad cohort and the BSI staff. Several members reported these “warm fuzzies” as a change from their research environment, where positive feedback was harder to come by.

“‘What did I get out of this?’—just a whole lot of satisfaction. Both personally, ‘cause I was happy to see kids smiling, to take a break from grad school, that kind of thing. And then also … you can bang your head and get nothing done in graduate school for months and months at a time. And this is something that was great, ‘cause every time you went in, it was instant gratification, as soon as the class was over.”

Career Gains.

Last, Science Squad members who had entered careers² reported a number of ways in which these benefits had transferred to their work. They applied their teaching and communication skills to their jobs, reused teaching materials developed for the Science Squad, saw positive reception of their résumés during job seeking, and developed useful professional networks.

In addition to these concrete career benefits, members gained understanding about their career options and interests that helped them to make better decisions. Squad members described a variety of ways in which their experience had confirmed, clarified, refined, or altered their career goals and paths. All interviewees were still working in STEM fields or STEM education—one-third (8) in higher education and one-quarter (6) in K-12 education or outreach. Importantly, 20 of the 24 interviewees (83%) reported some significant effect of Squad participation on their career path—a figure much higher than we expected. We describe our extensive data on these career path outcomes and the role of Squad participation elsewhere (Laursen et al., 2005) and in a forthcoming article. The extent and depth of career influence from Science Squad participation for these individuals, who are already highly interested in teaching, may also reflect the general lack of other experiences supporting career development in teaching for STEM graduate students (Smith et al., 2002).

Negative Outcomes for Presenters

Although Science Squad members were unanimous in stating that the benefits to them outweighed the difficulties, they did report three types of negative outcomes of their participation:

• Problems of time, travel, and logistics (reported by 71% of members)

• Emotional costs, such as discouragement and frustration (79%)

• Marginalization or lack of support by advisors and colleagues (79%)

Problems of time, travel, and organization stemmed from the challenging schedules that Science Squad members maintained. In addition to giving presentations and traveling to schools, members had to plan their calendar, respond to teacher requests, and prepare their materials and equipment. Logistical glitches were a fact of life, and traffic conditions or bad weather could add stress to travel.

Science Squad members reported that they kept their work time to match their paid commitment (typically half-time), but found participating on the Science Squad to require more than the nominally equal time commitment of a teaching assistantship. They had to use time efficiently to make progress on their research and satisfy their advisors. This was challenging after a full day in a school.

“People don't realize how tiring teaching is. It's not a physical tiredness necessarily, but your mind is just shot. Then you gotta sit down, and do your schoolwork, and prep your experiments and that sort of thing.”

Emotional costs were incurred when students did not respond to their efforts or when teachers did not support them. Some were discouraged by the low quality of science education in the schools they visited, and they felt their efforts could not ameliorate this daunting problem. Others expressed frustration at seeing new groups of students each day, without a chance to build relationships with students or see their progress.

Last, 19 members reported some level of marginalization or negative response to their participation in the Science Squad, or to the career choices that it implied, by their graduate advisors, other faculty, or peers. They received both overt and implicit messages that teaching, especially in K–12 schools rather than in the university, was less valued than research. Although marginalization may be a factor that deters some from applying, it did not deter this group of actual participants; however, they did exert some energy in coping with and responding to these negative messages. We discuss this issue in detail elsewhere (Laursen et al., 2005) and in a forthcoming article. Although some members reported changes in career path, such as ruling out the possibility of teaching high school science after trying it as a Science Squad member, they did not count this as a negative outcome but rather as positive clarification of their career interests.

STUDY FINDINGS: PROGRAM ELEMENTS AND PROCESSES THAT GIVE RISE TO PRESENTER GAINS

The interview data revealed the Science Squad experience to function as a particularly effective form of experiential learning about teaching. Repetition was one crucial element: By offering a few programs in many classrooms, Science Squad members could refine their presentation, experiment with different approaches, and observe students' responses closely.

“… Getting a chance to take the same lecture and approach it from a number of different angles to sort of trial-and-error to see what works, you just don't get that opportunity most of the time…. But when you're in that really intense—especially high school—setting, you've got four or five times a day, the same program…. So you really get a chance to hone some teaching techniques, … to just try the same package again and again, to try different angles.”

“Just getting in front of a classroom day after day … I got really comfortable with the material, so I was really able to—instead of worry about, ‘How am I gonna present this?’—I could really look at the kids and pay attention to them.”

Participants were also able to observe “a whole flock of different teachers” in many schools. Insight came from both positive and negative examples of how teachers presented material, interacted with students, managed classroom dynamics, and handled problems.

“I learned, I really learned, from observing the way teachers interacted with their students about ways that teachers could interact with young adults and get things across to them or ways that they handled classroom management situations or various interactions with their students. Things that worked and things that didn't work.”

In the best cases, this was accomplished not just by observing teachers, but by collegial interaction with them. This occurred in coteaching lessons, in discussing a lesson together afterward, and in collegial feedback from the teachers to the Science Squad presenters, as each brought mutual respect to the other's differing expertise.

“A lot of the things that I did … I really needed the teacher to circulate and help and talk to students and interact with them, and so in the really good situations, when teachers were good at the classroom management and good at being another facilitator, those were opportunities for me to observe things that worked well for them.”

“I hope in some cases I taught them some things, but just to have that conversation about what I did, why I did it, their interpretations of what worked and what didn't work, what they might have done differently [and] why…, [was] certainly valuable for me.”

Formal training sessions also contributed to skill development.

“The Science Squad is a team…. We really interacted quite a bit on a week-to-week basis. … I really liked learning a lot about classroom strategies and about pedagogical methods just from the kind of on-the-ground experiences that I had, and then sharing those with other Science Squad members. You really got the sense that you were all in it together.”

Together, their training, classroom repetition, observation of skilled teachers, and group discussions amounted to an intensive teaching practicum for Science Squad members. Members could apply their ideas to real teaching situations and then analyze and discuss them afterward. Even members with formal training in teaching found this combination beneficial.

“In that one year in Science Squad, I learned so much more … it's all stuff people had taught me in my ed classes, but they're skills, skills that you have to learn about watching students, watching their facial expression, really waiting long enough for people to answer questions. Those kinds of nuts-and-bolts skills develop much faster in that context. And I don't know if that's in part because I was primed for it with the two years of teaching, or exactly how that plays out … but it was a great opportunity for me to learn teaching skills, and with practicing, in many cases veteran, teachers there at my side.”

The structure of the program, with visits to a large number of classrooms, fostered these benefits, but it also had some negatives for Science Squad members, primarily the inability to develop lasting relationships with students and to observe their learning and progress. Programs that build in more sustained engagement in a single classroom likely address this—but also run risks of reducing impact in other ways, such as loss of the novelty and excitement of a visitor, and reach many fewer classrooms with the same resources. Moreover, programs where a university student works with one classroom all year have reported difficulties in relationships between the student and teachers—when long-term classroom planning and pedagogical skills become more prominent in the student's role, their science expertise may seem less relevant or appreciated, and lessons may become traditional rather than inquiry based (Coates, personal communication; Bruce et al., 1997). Tradeoffs such as this are discussed in the following section.

DEVELOPER'S ADVICE: PROGRAM DESIGN CHOICES

In this section, program features are shared by the program director (J.G.), whose long experience designing, running, and refining the program gives additional insight into how design choices affect the observed outcomes.

Recruitment, Selection, and Training of Presenters

Recruitment begins with information sessions led by program staff and current Science Squad members, assuming that the better informed the applicant, the better the applicant pool. Previous presenters are often the best recruitment tool. Although applicants well outnumber the available positions, a shortage of minority candidates reflects the lack of diversity in the university's graduate student population. The written application includes several essay questions, two letters of recommendation, and a curriculum vitae, and it is followed by personal interviews.

Careful matching of applicant characteristics and program goals selects for presenters who are likely to succeed with students and to benefit themselves. Motivation is critical: alignment with BSI's mission, desire to “give back” to the community, and dedication to communicating science. Also important are sensitivity to educational and cultural differences and an ability to communicate well with a variety of people. The staff considers past teaching experience and judges potential to be a role model for youth, to develop novel, engaging presentations in their scientific field, and the “fit” and appeal of those topics in the K–12 curriculum. Also valued are maturity, flexibility, and organization. Over time, this extensive review process has reliably yielded effective presenters.

Applications are typically due in February, with interviews in March. April notification means new members have secure funding and time to prepare. New members shadow current presenters to get a clear picture of the work, learn in situ, and spur concrete thinking about their own science presentations. The staff leads group sessions emphasizing science inquiry, best teaching practices, classroom management, and role playing. Monthly group meetings provide a collegial forum to practice presentations and troubleshoot challenges. The program staff aims to set a tone of accessibility, collegiality, and support for the presenters. Research findings concur, as Science Squad members report they appreciate and feel inspired by the support and professionalism of BSI staff.

Development and Support of Presentations

After selection and shadowing, staff members meet with individuals to discuss their ideas for presentations and share resources, including an annotated library of past presentations with outlines, activities, and notes about implementation and logistics: timing, pacing, differentiation for student groups, and solutions to common problems. Each member prepares four or five presentations in their area of expertise, which are publicized in August. From that menu, teachers request specific presentations for their classroom via e-mail or phone directly to each presenter. Presenters compile, prioritize, and schedule those teacher requests on their own.

Staff members help new Science Squad presenters translate their initial ideas into hands-on activities and think through the logistics of conducting them with multiple classes of 30 students in a day. A lab coordinator helps each member to identify and acquire the needed materials and supervises undergraduate assistants who help with lab prep, restocking consumables, and photocopying; lab and storage space is set aside for this. To stretch the supplies budget, the program shares with and borrows from other campus units.

Focusing on High-Need Schools

Because increasing diversity in science is central to the program's mission, Science Squad efforts are concentrated in greater Denver schools with high-minority (≥50%) populations, as determined from state enrollment data. Presenters are asked to prioritize such that 75% of their presentations go to these schools. This approach generally ensures that ∼50% of students reached are ethnic minorities.

Teachers and Science Squad members suggest that the student outcomes may be most important in these high-need schools. This choice also benefits presenters: Because they visit both resource-rich and resource-poor schools, they often report that the Science Squad experience is an “eye-opener” in revealing disparities in public education and students' preparation for college.

With the high demand, prioritization and scheduling can be a challenge. A policy limiting presentations to four per year per teacher helps to increase fairness and spread the benefit. Most teachers are understanding—although they advocate for their own students, they recognize others' needs and support the broader goals. That many teachers participate in other BSI programs (e.g., teacher workshops and outreach grant awards) probably also helps.

Teacher Engagement

Science Squad presentations work best when teachers are engaged and set an example for their students. Guidelines mailed before the session emphasize tips that “help make the Science Squad visit the best learning opportunity for students.” Teachers are asked to communicate their students' background and needs to the presenter in advance, prepare their students, introduce the presenter, and remain in the classroom at all times to assist and monitor student behavior. It is made clear that the Science Squad member is a specialist in science, not in classroom management: that responsibility remains with the teacher. Likewise, the guidelines indicate what the teacher may expect of the presenter, such as punctual arrival and respectful treatment of students. To inspire complete reading, important scheduling information is included. If guidelines are not followed, teachers may become ineligible for future Science Squad visits. This is rare but invoked if problems are recurrent. More commonly, the program director helps to resolve problems reported by teachers or presenters, which are typically misunderstandings spurred by incomplete communication and lack of time.

The Choice of Focused or Distributed Intervention

A significant decision for any outreach program is the choice to offer a focused intervention to a small number of participants, or a less extensive intervention distributed to a larger number of participants—“a lot to a little, or a little to a lot,” as one Science Squad member put it. Initially, the BSI staff experimented with both approaches, giving members the option to work in one school with one or a few teachers, in addition to the current, distributed model. Several challenges to the focused model became clear to program staff from evaluation data. Teachers' desires for help with their science curriculum were ongoing, but it was challenging for members to keep coming up with something new, especially outside their area of expertise. With their pedagogical role unclear, members could devolve into just another adult set of hands, doing jobs such as grading and supervising small groups, and the benefits of their subject matter expertise were lost. When students came to view the Science Squad member as a “regular” teacher, the benefits of novelty, specialness, and role modeling of a science career were diminished.

This issue was also investigated in the interview study. Program participants recognized the inherent tensions of distributed versus focused interventions but also offered arguments supporting a distributed intervention: the high numbers of students reached, the possibility of multiple encounters in a year, the intensity of learning during a special event, the potential to inspire, and the importance of a single counterexample to stereotypes, showing students that a woman can be an engineer or a Latino can be a biologist. It is also clear from the interviews that visiting a large number and variety of schools was crucial to presenters' development of teaching skills and increased understanding of diversity and education. Although development of some skills—such as classroom management and individualizing instruction—might be greater from a longer-term classroom placement, other fundamental and transferable teaching skills were strongly reinforced by exposure to many classrooms and the opportunity to practice and refine. Without repetition, such growth would not otherwise be realized in a time as short as a year.

Last, the longevity and impact of Science Squad members' work is increased by other program elements. In addition to contributing their presentations to the support library, members may lead a teacher workshop at the end of the year. Workshops highlight the strongest aspects of the presentations and offer background information to teachers who want to use these elements in their own teaching. The chance to work more extensively with a group of teachers benefits Science Squad members as well.

CONCLUSIONS AND IMPLICATIONS

We have used a qualitative approach to demonstrate that a scientist in the classroom outreach program has benefits for K–12 students and teachers. Teachers and presenters observe students' engagement and interest in science, exposure to new science learning opportunities, and changing ideas about what science is and who can do it. Teachers benefit by learning new content and new ways to teach it, and they feel supported by the presence of interested individuals from the university. We conclude that, when well run and carefully structured, scientist in the classroom programs can have a positive impact on students' interest in science and thus their eagerness to learn it. As noted in the Introduction, increased interest is argued to increase students' potential to pursue further education or a career in science. Long-term longitudinal studies to follow the actual educational and career outcomes of students exposed to such programs would be of interest but difficult to conduct. At the same time, the program reaches a large number of students with these benefits.

Perhaps more importantly, the program has substantial benefits for its scientist participants, a group of STEM graduate students who seek preparation for teaching as well as for scientific research. Finding insufficient opportunities within their own departments and degree programs, they discover the Science Squad as an opportunity to learn to teach well and creatively. A high fraction of Science Squad alumni pursue academic careers in institutions with a balance of teaching and research or apply their skills to other types of public science communication. For some, the Science Squad experience confirms their career goals and enhances achievement of them; for others, it introduces new possibilities for careers using their science expertise. In a few cases, Science Squad participation seems to support students to complete a graduate degree and pursue a professional science career who may otherwise be at risk to drop out.

Collectively, studies such as this study offer lessons for the design of classroom outreach programs, helping designers to anticipate potential consequences of certain design elements. For example, could the program be equally successful if offered by undergraduates? (e.g., Bruce et al., 1997.) This approach has appeal as a means to recruit and prepare students to be science teachers and to reduce costs. Our findings indicate some likely tradeoffs: The success of the presenters depends on their inquiry approach and disciplinary expertise, which in turn require sound science knowledge and confidence in it. Research experience seems to be crucial, because presenters must deeply understand the nature of science to be able to effectively communicate the fundamentals of inquiry—gathering and evaluating evidence, drawing conclusions, building and communicating arguments—that are common to all the Science Squad presentations, regardless of topic.

Likewise, some programs are based on the hypothesis that engaging graduate students in a longer-term interaction with a single school will lead to greater student content learning, or to formation of individual relationships rewarding to both students and scientist (e.g., Trautmann et al., 2002). Many of the programs currently funded by the National Science Foundation's GK–12 program are based on this model (National Science Foundation, 2006). But, as we have argued, repetition and exposure to a range of classrooms is crucial in presenters' rapid gains of teaching skills and understanding; thus, this benefit to presenters would likely be reduced in return. Programs that support practicing scientists could provide the needed scientific expertise from otherwise untapped talent pools, not just universities, but they also must solve the challenge of preparing scientists to develop interactive, hands-on presentations and make presentations age appropriate (e.g., CRS, no date; Project Astro, 2001). Our point is not to rank these program design choices but to emphasize that program designers must make appropriate design choices depending on their goals and that they should use research findings to anticipate the outcomes of their choices.

Last, our findings suggest issues for further research. More direct measures of student outcomes are obviously needed, and the qualitative data suggest some domains of student outcomes that might be fruitful for development of other types of assessment. Given the program goals and the experiences of teachers and Science Squad members, it is not surprising that affective and attitudinal gains are much more pronounced than more easily measured gains in science content learning. Long-term impacts of outreach or informal education are of great interest to practitioners who design and run such programs, but they are very difficult to measure. Although it is argued that the impact of a single event may be great, it would be daunting and expensive to design a study to control for every possible relevant factor. The ultimate benefit of such a study is also questionable: As much as we might like a magic intervention that must be administered only once, surely common sense would suggest that what is most likely to work is a series of interventions—related or not—throughout every child's school and youth years—some of which reach children at times when they are primed to benefit most. Society cannot rely on a single event to inspire a future scientist, but it must provide a range of opportunities for excellent science education, in school and outside it.