Improvement in Student Science Proficiency Through InSciEd Out

Abstract

Integrated Science Education Outreach (InSciEd Out) is a collaboration formed between Mayo Clinic, Winona State University, and Rochester Public Schools (MN) with the shared vision of achieving excellence in science education. InSciEd Out employs an equitable partnership model between scientists, teachers, education researchers, and the community. Teams of teachers from all disciplines within a single school experience cutting-edge science using the zebrafish model system, as well as current pedagogical methods, during a summer internship at the Mayo Clinic. Within the internship, the teachers produce new curriculum that directly addresses opportunities for science education improvement at their own school. Zebrafish are introduced within the new curriculum to support a living model of the practice of science. Following partnership with the InSciEd Out program and 2 years of implementation in the classroom, teacher-interns from a K–8 public school reported access to local scientific technology and expertise they had not previously recognized. Teachers also reported improved integration of other disciplines into the scientific curriculum and a flow of concepts vertically from K through 8. Students more than doubled selection of an Honors science track in high school to nearly 90%. 98% of students who took the Minnesota Comprehensive Assessments in their 5^th and 8^th grade year (a span that includes 2 years of InSciEd Out) showed medium or high growth in science proficiency. These metrics indicate that cooperation between educators and scientists can result in positive change in student science proficiency and demonstrate that a higher expectation in science education can be achieved in US public schools.

Introduction

Science and technology supports modern societies, including the United States (U.S.). More than 50% of economic growth in the U.S. since World War II has been due to scientific and technological innovation.¹ Despite the role of Science, Technology, Engineering and Mathematics (STEM) jobs in this economic success, they represent only 5.5% of the U.S. labor work force.² Meanwhile, global problems (disease, pollution, social strife) continue to increase despite the best efforts of science.³ U.S. students have fallen to 17^th of 34 developed countries in science and 25^th in math according to the Programme for International Student Assessment.⁴ Improving science education in the U.S. has the potential to generate more STEM-trained citizens to address major world problems with added positive impact on the economy following the most significant economic downturn since the Great Depression.

The National Research Council,⁵ the National Institutes of Health,⁶ the National Academy of Sciences (NAS 2007), the American Association for the Advancement of Science,⁷ and the National Science Teachers Association⁸ share a common voice with two key messages concerning science education: 1) America must address the current shortfalls in student science proficiency with efforts that include K–16 education, and 2) communities should look to meaningful partnerships between scientists and educators to catalyze this change.

Historically, intervention methods in science education have ranged from televised scientific presentations to after-school academies for teachers or students.^9–25 For example, over 3000 video lessons are freely available from the Khan Academy online.²⁶ Few examples of measured student learning can be found in these efforts, however. Student engagement is the most common endpoint of these interventions.

In contrast, Silverstein et al.²² describe a program in which over 100 New York Public School middle and high school teachers entered into scientific laboratory summer internships spanning 11 years. The internship included scientific inquiry paired with professional development activities. Following the effort, a 10% difference in student science proficiency is shown between students of the interns and control classrooms. This change brought overall student science proficiency to 54% of participating students (up from 44% prior to the intervention). This effort focused on student learning while also highlighting a secondary concern in science education: U.S. public schools have a lower student performance in science than in reading and mathematics. For example, science proficiency at the time of high school graduation, especially biology, lags that of other subjects in the same cohort of students. Benchmark ACT scores are used to predict likelihood of a passing grade in first-year college curriculum based on the area score on the ACT test. The same cohort of students with a 70% college readiness score demonstrate a less than 30% college readiness proficiency in biology scores.²⁷ These outcomes suggest an ongoing “Science Gap,” a measurable difference between the performance of students in science versus math and language arts.

InSciEd Out began in the spring of 2009 as a tripartite partnership between the Mayo Clinic (life science expertise employing the zebrafish model system), Winona State University (teacher education expertise), and Rochester Public Schools (teaching excellence). The program recognized the need to engage with lagging K–8 science proficiency scores (46% using the Minnesota Comprehensive Assessment scores for science) and a shortage of beginning teachers prepared to teach science in elementary or middle level classrooms. InSciEd Out represents a partnership paradigm through which curriculum change is empowered by intellectual and technical resources found within each partner organization. Put simply, teachers are not asked to run a laboratory, and scientists are not asked to manage a kindergarten classroom. Instead, both are asked to share their expertise for a common end: science excellence.

Purpose

The purpose of this article is to introduce the infrastructure and methods of Integrated Science Education Outreach (InSciEd Out), a broad effort to address student learning of the process and practice of science with an emphasis on the use of the highly accessible model vertebrate, the zebrafish Danio rerio. Evidence for the outcomes of InSciEd Out methods is offered not only to provide empirical value of the program, but also as an overview of the metrics in use for long-term evaluation. The intent is for this work to be read by science and education partners together, allowing a deeper understanding of the language and culture of each. The ongoing nature of this project is reflected through the use of present tense where appropriate in the text.

Methods

Participant selection

Pilot study funding was used to initiate and envision the InSciEd Out internship program at Mayo Clinic in partnership with Lincoln K–8 (a public choice school) from Rochester Public Schools. As a choice school with a specific technology and innovation mission, Lincoln offered InSciEd Out the flexibility necessary to explore and define the methodology of this intervention. 100% of Lincoln students and teachers have participated in this study, as it has been integrated in their mainstream curriculum. Schools that have partnered with InSciEd Out following the initial work with Lincoln were selected through an application process that evaluated the proportion of teachers willing to take part in the internship, administration's commitment to supporting those teams, and the likelihood of the partnership to serve previously marginalized students.

The forms used to guide this selection process are included as Supplementary Material 1 (Supplementary data are available online at www.liebertpub.com/zeb). This section includes a guide for administrators to develop an application for their school, an overview of InSciEd Out, and a list of requirements of participating teams.

Internships

InSciEd Out involves multidisciplinary teams of teachers from a common school into Mayo Clinic's research facility for a summer internship experience. In their first internship within the lab, participants learn more about genetics, development, and the Nature of Science (the “who” and “how” of science). Later, such knowledge is utilized by the teachers to create curriculum modules for students in grades K–8 aligned with the constructivist teaching philosophy of InSciEd Out.

The three tiers of InSciEd Out internships are: 1) Tier-1 is a 3-week experience where new teacher partners engage in a curriculum of science, Nature of Science, generative dialogue (explained more fully below), and the pedagogy of inquiry. An excerpt from the internship curriculum is shared in Supplementary Material 2. In this internship, teachers create a new curriculum module for use with their students that includes the resources and shared vision of their new science and research partners. This curriculum is not intended to replace their entire science curriculum but rather to address opportunities revealed through standardized testing of their students. 2) One-week Tier-2 internships include an introduction of teacher interns to research on theories of fixed versus malleable intelligence. ²⁸ This exploration opens the mindset of teachers to the ability of all students to learn. Next, teachers are introduced to examples of marginalization of students in American classrooms.²⁹ Through active participation in generative dialogues,³⁰ teachers explore examples of marginalized students in their district, school, and classroom. Finally, teacher interns engage in deconstruction³¹ of their previously created InSciEd Out curriculum, specifically targeting examples brought up in dialogue. They then reconstruct the curriculum with cultural awareness.³¹ 3) Tier-3 experiences are 1-week internships with a focused theme for revision of Tier-1 (or Tier-2) curriculum.

Critical mass for the initiation of culture change

The partnership of InSciEd Out began with internships wherein all faculty and staff from Lincoln K–8 were invited to participate. This included the principal, classroom teachers, subject area teachers, support staff, paraprofessionals, math and reading support, and English for Speakers of Other Languages (ESOL) teachers. Beginning with an integrated partnership allowed new curriculum to be developed in a horizontally integrated (sharing across math, language, science, etc.), vertically aligned (connected from one grade level to the next) manner. Further, Lincoln had enough faculty and staff involved to maintain a culture of change upon return to their own facility.

Modules

InSciEd Out modules created within the internship experiences range from a few lessons to a month-long unit integrated among Language Arts, Mathematics, Science, Physical Education, and Art. Modules include front matter that shares overviews of the unit from the point of view of different readers. For instance, teachers may seek a daily lesson plan view; curriculum administrators will seek a standards view; and education researchers may seek a view grounded in learning theory. For the purposes of InSciEd Out curriculum, the latter is supported by the theories of science learning shared by Roger Bybee. In short, this “5E” view includes a progression of teaching strategies that includes Engage, Explore, Explain, Extend, and Evaluate.³²

The Minnesota Comprehensive Assessment (MCA) is the state's measure of student proficiency in response to requirements of the No Child Left Behind Act. MCA proficiency testing has included Reading and Mathematics since 2005–06, and Science since 2008–09.³³ All new curriculum modules were developed by the teaching teams in direct response to areas of concern identified by MCA testing of their own students.

An excerpt from the rubric for a module is included in Table 1, and example topics and characteristics are shared from a selection of modules in Table 2. An excerpt from a grade 1 module is shared in Supplementary Material 3. The InSciEd Out team is working to subsequently release current vetted modules and the initial internship curriculum as open access e-media to facilitate broader adoption.

Table 1.

Excerpt from Curriculum Module Rubric

	Expected outcome
Area of Assessment
Science standards	Standards and benchmarks are identified and appropriate to the module; also lists standards from other subject-area disciplines integrated into the module.
Content objectives	Objectives are appropriate to the module and address recall and interpretation levels of knowledge (Bloom's: Knowledge, Comprehension, and Application.)
Language objectives	Identifies the academic language and procedural discourse embedded within each lesson in the module and overtly incorporates a form of communication authentic to scientific community (i.e., journal article, field notes, poster presentation, letter to legislator, webpage).
Horizontal integration	Curricular plan authentically integrates standards/objectives from other content areas in a manner that lends coherence.
Structural Components
5E lens of module	Clearly conveys 5E scope and sequence of the module.
Standards lens of module	State and National Standards included in the module are clearly documented. Treatment of 2061 Benchmarks is included to maintain National scope.
Daily lesson lens	Clearly conveys the module from the lens of chronological lesson plans.
Supporting documents (handouts, task cards)	Includes copies of all material (handouts, assessments, task cards) that will be distributed to students. Has detailed notes on how/when such materials will be used.

This rubric is used to evaluate new modules created by teacher interns. Along with areas of assessment consistent with the goals of InSciEd Out (top rows), structural components (bottom rows) are assessed to facilitate module accessibility by several different kinds of users (Teachers, Administrators, Volunteers, Education Researchers).

Table 2.

Curriculum Modules Created Through InSciEd Out

Title	Grade level	Zebrafish role	Topics of focus	Horizontal integration	Lessons
What is a scientist?	K	Adult zebrafish in a bowl are used in an exercise on the role of observation in the scientific method. Students observe zebrafish and build “I wonder” questions later addressed with scientist partners.	Identifying Scientists, Scientific Tools, and Skills	Science, Language Arts, Math	16
Introduction to environments	1	The environment of the zebrafish in their native India is discussed. Students explore a zebrafish aquaculture system and describe the similarities and differences in the lab environment.	Environment, Behaviors, Adaptations	Science, Physical Education, Language Arts	11
Life cycles	2	Students observe development from 1-cell through 4 days post fertilization using dissecting microscopes (See Figure 3C and D)	Stages of Development	Science, Physical Education	5
The role of camouflage	3	Students develop experiments where adult red and yellow Glofish are used to explore how a zebrafish may hide in its environment. Further, paper fish are used in an activity to hide in the school environment.	Biomes, Habitats, Pigmentation, Camouflage, Survival	Science	6
Fish behavior and adaptations	4–5	Students explore the behavior differences in adult zebrafish confronted with different water temperatures. Conversations are driven with scientists about the effects of animals in an aquatic system previously separated by temperature gradients meeting each other for the first time.	Collecting and Graphing Data, Adaptations	Science, Math	7
The impact of chemicals	6–8	Students first explore zebrafish development in the presence of alcohol, then design experiments that relate to their own experiences and concerns about chemical dependence.	Authentic Science, Development, Peer Review	Science, Language Arts	18

Listed modules include those that have been created and revised at Lincoln K–8 Choice School. Nine more are currently used at other InSciEd Out partner schools. Zebrafish figure prominently in most InSciEd Out modules.

Module support

To facilitate the transition of modules from the summer concept to classroom implementation, modules created as a result of InSciEd Out internships are supported by the InSciEd Out team and volunteers during the subsequent school year. This ‘vetting’ includes a review of the scientific components and education theory included in the module and a first run in the teacher's classroom. Following the vetting process, each module undergoes review by InSciEd Out leadership and teacher peers for science accuracy and educational efficacy. Revisions from that review are included in the module, and the new version is maintained in an online space for use by appropriate teaching teams. The team that creates the initial module and any grade-level equivalent teams who have also undergone an InSciEd Out internship can access the module prior to external distribution. An excerpt from a classroom module is found in Supplementary Material 3.

Module support needs are coordinated through the Mayo Clinic team. More recently, undergraduate students from the local University of Minnesota—Rochester (UMR) have been trained as volunteer/resource coordinators to support the needs of partner schools. Technological resources including microscopes, computers, and laboratory supplies are checked out to teacher partners through Winona State University's STEM Village, a shared resource center for the support of STEM education.

Scientists who participate in the support of InSciEd Out play similar roles both in InSciEd Out internship and in the partner classrooms during implementation of new InSciEd Out curriculum. Most importantly, scientists are not asked to lead classrooms or directly teach a section of the curriculum. Instead, they are brought in as a scientific resource. For instance, as students (or teacher interns) are learning what science and scientists are, they participate in an activity called “speed-dating” where the students meet many scientists for brief conversation and to ask any questions they may have in the fields of science. For teachers, the goal is that they will have a potential networked partner who can later be called when a classroom has questions in the area of expertise of that scientist. For students, the goal is for them to see similarities between themselves and the scientists. They can take this time to try on the role of scientist as a possible future for themselves. Scientists also visit with classrooms to aid in focusing experimental design for students starting research projects. The scientists help to identify possible pitfalls and find resources for the best chance for successful research by students.

Mayo Clinic's Center for Translational Science Activities (CTSA) provides financial support for the coordination of InSciEd Out activities through their program grant (UL1RR024150). Additional support for the coordination team is covered through the Stephen C. Ekker Program (also at Mayo Clinic). A consumable supply budget for science supplies used by schools during internship and vetting is provided by the CTSA grant. Following vetting, partner schools cover the consumables for their module curriculum.

Evaluation plan

The mission of the InSciEd Out program is to improve science education, and a key aspect of that mission is a commitment to evaluation of all elements, processes, and outcomes followed by program refinement and improvement. Any demonstrated change is assessed for tangible improvement or merely change, and any improvement or success of an intervention is supported with empirical evidence, including student learning. The InSciEd Out evaluation plan aims to assess each component as well as the overall outcome and impact of the program on individuals and communities. The InSciEd Out high-level logic model (Table 3) presents the four key components of the InSciEd Out program: Input, Activities, Output, and Outcome. Evaluation of the last three components (activities, outputs, and outcome) is articulated in Supplementary Material 4.

Table 3.

InSciEd Out Logic Model

Teacher survey

Of the teachers who have attended Tier-1 internships, 63 completed a survey between 2009 and 2010 covering a small amount of demographic data (class level taught, years experience, etc.) and views and practice with regard to teaching science. This survey is included as Supplementary Material 5. Of the 63 teachers included in the pre-survey, 18 are Lincoln teachers. 2010 internships included teachers from Lincoln along with other schools. The data were not segregated to maintain anonymity of the teachers surveyed. Teachers who participated in the 2011 and 2012 internships were from schools with pilot teams already having attended an earlier internship. Because of this “pre-knowledge” of InSciEd Out, we do not include their responses here.

Dialogue as scientific discourse

Higher order thinking processes are critical to effective science education.^34,35 Dialogue training provided in InSciEd Out is presented as a distinct form of scientific discourse in contrast to argumentation. Generative dialogue is the process of consciously and deliberatively reflecting in the company of others.³⁰ Participants learn to “risk” sharing their own views and understandings of science and the social issues tied to scientific research. The goal of generative dialogue is not for one voice to gain consensus, but rather for each participant to truly hear the others. Teachers engage in dialogues about pressing scientific issues throughout the internship in a series of 1–2 hour sessions. Early dialogues are initiated through topical readings in science. A favorite of the team is the collection Being Human: Core Readings in the Humanities edited by Leon Kass for use by the President's Council on Bioethics in 2003. Excerpts from the book are used to drive reflection on what society believes about science and scientists. Later in the internships, dialogue topics emerge organically through the needs of the intern groups. Some interns have successfully used the dialogue sessions to address communication gaps between teachers and administration. Others have sought deeper understanding of the ethics and challenges of specific fields of science (stem cell research, cancer).

Teachers embed generative dialogue within their curriculum modules as a strategy that will draw student preconceptions and misconceptions to the surface, build understanding and respect for alternative perceptions, and address concerns with the content or process of a particular activity. For instance, students can openly discuss the use of animals in research and come to an understanding of their own values as well as those common to the field.

The capacities of dialogue shared in InSciEd Out internships include: 1) listening consciously and generously; 2) voicing the truth of your own experience; 3) suspending judgment and checking the assumptions you have made about others; and 4) engaging with others in the spirit of inquiry. Vertical alignment of dialogue capacities across the K–8 spectrum is ongoing. These capacities drive reflective action and provide a context for ongoing flexibility ³⁰ in the classroom.

Pre-service teacher support of module curriculum

Winona State University (WSU) pre-service teachers (Junior and Senior undergraduate Education students) enter a parallel internship to support the in-classroom vetting and school-to-school dissemination of new curricula.

The theory of “situated learning” in education research includes a central principle whereby newcomers to a community of practice enter as “unqualified” but are gradually entrusted with more important roles.³⁶ Communities of practice are “groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly.”³⁶ The principle of situated learning for newcomers in a community of practice is referred to as ‘legitimate peripheral participation’³⁶ In the InSciEd Out model, pre-service teachers are brought into two distinct but interdependent communities of practice, the laboratory and the classroom. While in the laboratory, the pre-service teachers experience the actions and interactions that accompany the Discourse (the culture and language)³⁷ of scientific research. While in the K–8 setting, pre-service teachers experience the Discourse of education and classroom management. They were not expected to take on leadership of the laboratory or the classroom, but are rather professionally introduced to both. The nature of this peripherality³⁶ of pre-service teachers, simultaneously situated in lab and classroom, is intended to provide the pre-service teachers access to an increased level of engagement and understanding around what it means to be and become a science teacher.

Beyond content to practice

One of the biggest opportunities for improvement at Lincoln identified by previous MCA testing (data not shown) was in the area of Nature of Science.³⁸ Analysis of these scores with teacher teams revealed that goals and assessments for Science rarely focused on cognitive capacities beyond basic recall of facts. For example, extant curriculum prior to InSciEd Out did not include the latitude for students to ask and answer their own questions. Such a restrictive framework is not conducive to student learning of the process, nature, and philosophy of science. During module writing, effort is made to move beyond recall and comprehension of the presented content toward demonstration of higher order cognitive capacities such as application, analysis, synthesis, and evaluation of evidence.^39,40 In kindergarten, students are challenged to come up with a question to which there is not a known answer. A game of “Stump the Scientist” sets the students with an expectation that science is the process by which new knowledge is found. These questions grow into full scientific research as the students experience the scientific method in later curriculum. Grades 6–8 students engage in novel research projects in the content areas of InSciEd Out modules in their classrooms.

Zebrafish

Within the internships and embedded in the modules produced, the zebrafish (Danio rerio) is introduced as the model system of choice. The zebrafish has properties that make it amenable to scientific study. High fecundity, transparent and external embryonic development, genetic similarities to humans, size, availability of tissue-specific transgenics, and timeline of development all combined to make the zebrafish accessible to the teachers and students.⁴¹ Zebrafish have previously been used effectively in inquiry-based classroom activities.⁴² At every grade level in Lincoln's curricula, initial contact with zebrafish embryos is made through a projected live video feed of development from one cell through 3–4 days post fertilization. The transparency of the zebrafish embryo and resultant accessibility of early development is particularly effective in challenging student misconceptions about Life Sciences, cells, and heredity.⁴³ This new resolution of a living animal enhances the students' understanding of cells, organs, systems, and whole organisms as the products of a genetic process. Examples of the role of zebrafish in curriculum modules are shared in Table 2.

Zebrafish are also made available to students during the student synthesis, or extension, portion of each module. The short timeline of zebrafish development allows meaningful student inquiry that fits within the academic calendar. Initially, all fish and embryos necessary for the work were attained from the Mayo Clinic Zebrafish Core Facility (ZCF). Later, Lincoln teachers have competed successfully for grants and sought parent support to produce a student-run laboratory with a capacity to house 2000+ adult fish. Student zebrafish experiments range from developmental studies testing the effects of energy drinks and vitamin supplements on growth to evaluation of complex behaviors like addiction. This issue of Zebrafish includes 15 abstracts from Rochester teachers and students who have used the zebrafish in their classroom^44–58 and 2 additional full research articles resulting directly from the use of zebrafish within the InSciEd Out partnership.^59,60

New metrics

The MCA is a helpful learning metric, largely due to the availability of previous year scores. However, the science testing for the MCA is only performed in grades 5 and 8, and reporting of test scores is not completed during the same academic year. To assess student learning within modules for all grades in a more integrated manner, a progression of assessment tools was implemented to monitor student learning. These tools include talking drawings (Fig. 1A and B) that work within the range of developmental abilities of K–8 learners and facilitate the assessment of their conceptual understanding of a topic. Student understanding is demonstrated by drawing a picture, adding whatever text they're able to provide, and then “telling the story” of the drawing to a teacher, paraprofessional, or parent volunteer who transcribes the explanation.⁶¹

FIG. 1.

Student Talking Drawings. Grade 2 students were asked before (A) and after (B) a module on the Nature of Science to draw a scientist. Student work after the InSciEd Out module showed trends toward a less formal laboratory setting, inclusion of more teammates, and student identification as the scientist. (C) and (D) A grade 2 student notebook during observations of zebrafish development clearly shows that the student has a grasp on embryology.

Students are asked to draw a picture of their understanding of a particular topic or concept at the beginning of a module and then asked to draw a picture of their understanding at the end of the module. Table 4 includes the detailed rubric for assessing these talking drawings.

Table 4.

Rubric for Talking Drawings

	Developing	Competent	Exemplary
Content Objectives	Student work shows little or no evidence of understanding academic benchmarks associated with the module. □ Use observations. □ Use tools. □ Classify/Sort □ Monitor change.	Student work shows evidence of understanding academic benchmarks associated with the module. □ Use observations. □ Use tools. □ Classify/Sort □ Monitor change.	Student work shows evidence of understanding that meets and exceeds academic benchmarks associated with the module.
Academic Language	Student work does not make use of academic language associated with the module.	Student work makes use of some of the academic language associated with the module.	Student work makes use of most or all of the academic language associated with the module.
Nature of Science	Student work does not demonstrate a basic understanding of the nature of science: □ Ask questions. □ Make hypothesis. □ Use tools. □ Collect data. □ Make observations. □ Explain ideas/findings.	Student work demonstrates a basic understanding of the nature of science: □ Ask questions. □ Make hypothesis. □ Use tools. □ Collect data. □ Make observations. □ Explain ideas/findings.	Student work demonstrates a sophisticated understanding of the nature of science: □ Peer review. □ Publication. □ Presentation.

Each potential set of talking drawings requires a grade-level specific rubric. This rubric was designed for a module entitled “What is a scientist?”, which resulted in the drawings in Figure 1.

Statistical treatment

Effect size calculations (Fig. 2B) are the difference between group average scale score and state average scale score, all divided by the state standard deviation. Students were included in analysis if they were enrolled at the school and completed the MCA in both 2008 and 2011.

FIG. 2.

Student Scores Show Growth in Science Learning. (A) Expressed by yearly cohort, 5^th grade students at Lincoln after the InSciEd Out partnership showed increases in science proficiency testing versus years prior to the InSciEd Out partnership. (B) Expressed as effect size, a comparison between a school's scores and the state averages, Lincoln students showed improvement that outpaced the rest of the district. In the two columns on the left, representing Rochester Public School district, the dark blue bar is from 2008 and the red bar is from 2011. In the two columns on the right, representing only Lincoln K–8 choice school, the light blue bar is from 2008 (before Lincoln partnered with InSciEd Out) and the green bar is from 2011 (following partnership). (C) In the years between 2008 and 2011, Lincoln students showed a higher percentage of medium and high growth in science proficiency than other students in the district.

For student growth assessment analysis (Fig. 2C), statistical methods were utilized to determine the growth of student cohorts, controlling for the demographics of Gender, Limited English Proficiency, Special Education Status, Free or Reduced Price Lunch Status, Hispanic Ethnicity, and Black Ethnicity. A multiple linear regression model was used to generate standardized residuals, which were then categorized as “high” if >=0.5, “Low” if <=−0.5, and “Medium” otherwise and summarized by year and school. The three additional middle schools (not currently partnered with InSciEd Out) in the Rochester Public School District are included for comparison.

Trends showing increases in student science learning were noted for all three cohorts assessed to date at Lincoln (Fig. 2). However, due to Lincoln's small student numbers (∼400) relative to the broader Rochester district (16,000+), adequate power is not achieved to relate the values in terms of formal statistical significance for any one of these cohorts.

Results/Discussion

The results of the InSciEd Out partnership model were assessed using teacher, facilitator and student outcomes.

Teacher outcome data

Teacher outcome data included a self-assessment through a survey completed prior to the initial internship, after vetting of curriculum, and after any subsequent internships (Tier-2 and Tier-3). Preliminary survey data in Figure 3A show a snapshot of Rochester district teacher confidence in teaching science as they include teachers from multiple schools prior to internships with InSciEd Out. Lincoln teachers surveyed after vetting their curriculum (Fig. 3B) show a higher trend toward confidence. After Tier-2 and/or Tier-3 internships, teachers at Lincoln show a trend for even higher confidence (Fig. 3C). Teachers at Lincoln who have not attended the Tier-1 internships show a similar trend toward higher confidence. These pilot results are being used as hypothesis-generating data to establish a formal prospective pre- and post-evaluation of future teacher cohorts as a future component of the InSciEd Out program.

FIG. 3.

Preliminary Evaluation of Teacher Confidence in Science. Teacher responses to the statement “I understand science concepts well enough to be effective teaching elementary science.” As described in Methods, (A) includes all teacher interns from multiple schools in 2009–2011 as a “snapshot” of the Rochester district prior to InSciEd Out. Eighteen teachers from Lincoln are included within the 63 respondees. The Lincoln teachers who have completed their first year with curriculum derived from a Tier-1 internship show nearly unanimous improvement in their assessment of their own science knowledge (B). The same teachers after completing Tier-2 and/or Tier-3 internships respond further toward “strongly agree” (C). Teachers at Lincoln who had not yet taken part in the internship show similar assessment of their scientific knowledge. (D) and (E).

The challenge to obtaining a full complement of teacher outcome data was the inclusion of interns from previous partner schools in subsequent year internships. Many of these interns cannot be considered naïve to the InSciEd Out program or the content of InSciEd Out curriculum. Therefore, an assessment of the teachers' perception of their own growth in content knowledge and comfort with scientific curriculum was recently added. This assessment has been completed for the most recent intern group (n=8 teachers), and on a scale of 1–5, teachers reported a 1.3 point improvement in content knowledge to 4.3 out of 5 and a 1.6 point improvement in content comfort to 4.4 out of 5.

Teacher content knowledge as assessed through a professional portfolio project and reflective writings during the internships revealed specific trends: First, teachers have taken on the core process of peer review in their own behaviors and evidenced within their written work. Second, teachers have taken on the Discourse of science, utilizing posters and the elements of publication to document and improve their own personal science. Talking drawings (more fully explored in student section below) done by teachers have demonstrated core scientific and academic language acquisition by teacher interns, as well as an emergence of use of scientific methodology in problem solving. Also, teachers show an increased understanding of scientists as people, an important component of the Nature of Science.

Facilitator/Program Outcome Data

Facilitators of InSciEd Out have been assessed by an internship course evaluation throughout the program's development. Teacher interns are asked to rate the facilitation team on a 6-point scale ranging from Bad to Excellent in their delivery of the internship curriculum. With over 60 responses, the average score falls at 5.5, directly between “Very Good” and “Excellent.” In a newer facilitator evaluation instrument used with 37 interns thus far, a 5-point scale including Strongly disagree (1), Disagree, Neutral, Agree, and Strongly agree (5), are used as responses from teacher interns to a variety of questions (Table 5). Finally, on the same scale, in response to the statement “I am glad I took this course,” the average score was a 4.9.

Table 5.

Facilitator Metrics

Metric: “The teaching team…”	Avg. score
Is actively helpful when students have difficulty.	4.8 of 5
Appears sensitive to the students' feelings and problems.	4.6 of 5
Stimulates thinking.	4.7 of 5
Encourages students to ask questions, disagree, and express ideas.	4.5 of 5

Upon completion of a Tier-1 internship experience, teacher-interns complete a course evaluation, including extensive evaluation of the facilitation team (faculty) for the internship.

Student outcome data

Following partnership with InSciEd Out, school-wide changes at Lincoln K–8 School have been assessed for changes in science culture and improvements in science learning. The engagement metrics following intervention by the InSciEd Out team are shown in Figure 4.

FIG. 4.

Student Engagement Following InSciEd Out Curriculum. (A) Following partnership with InSciEd Out, grade 8 students showed increased registration for the subsequent Honors Biology course upon graduation to high school. (B) Voluntary participation in the local, extracurricular Regional Science Fair also increased following student experience with InSciEd Out. Black data indicate outcomes prior to partnership with InSciEd Out. Gray data indicate outcomes following partnership with InSciEd Out.

Engagement outcomes

Course selection by grade 8 students for subsequent advanced high science curriculum was used as one engagement metric. Grade 8 students showed increased registration for Honors Biology upon graduation to high school (Fig. 4A) from 37%–42% (years 2006–2009) to 86% (2009–2010), 89% (2010–2011), and 85% (2011–2012). Within the Rochester public school district, this is a notable outcome. Prior to the 2011–2012 school year, Honors Biology was a prerequisite course for other science electives in subsequent high school years. This meant that students who were not “engaged” in science upon entering 9^th grade could find the door closed to them.

Voluntary participation in the local, extracurricular Rochester Regional Science Fair was used as a second engagement metric for middle school students. Grade 6–8 students showed an increase in science fair participation, from 4%–11% of students (2006–2009) to 41% in 2009–2010, 57% in 2010–2011, and 83% in 2011–2012 (Fig. 4B). This level of science fair participation was the highest in the Rochester Public School District, outpacing the 11% participation rate for the second-highest school.

Additional science engagement was also demonstrated in other subject areas, including music and art. A world-drumming ensemble with a “Zebrafish Rap” has found air-time at Lincoln and other schools' student assemblies (Fig. 5A). In some cases, student extensions have included musical and performance art, as with the boys' singing group “Gone Fluorescent” (Fig. 5B). This outcome, as well as horizontal integration with other disciplines, is visible in the hallways of Lincoln. A mural art elective course has filled the hallway space with zebrafish themes (Fig. 5C). These outcomes demonstrate that the intent of InSciEd Out is not to take from the ever-diminishing time that students are given in pursuit of the arts, but rather to integrate science work in the space of the Arts to ensure the success of both.

FIG. 5.

 Student Science Literacy in the Arts. Students having experienced InSciEd Out curriculum have shared their new found science literacy within the context of many other disciplines including (A) World drumming, (B) Popular music, and (C) Art.

Learning outcomes

Talking drawings (Fig. 1) are used within InSciEd Out internship and the internship-inspired modules as a method of content knowledge assessment.⁶¹ Figure 1A and B show student talking drawings made before and after a new curriculum module. In this exercise, 1^st through 3^rd grade students were asked to draw a scientist. This representative example shares a pre-module drawing showing a more controlled environment with lab coat and few elements of individual personality (Fig. 1A). After completion of a curriculum module, student drawings tended to include many more personal artifacts and more people (science as a team) in a less rigid laboratory environment. In Figure 1B, one student is drawing herself and friends in the pursuit of science in a less formal setting than in Figure 1A. The detailed rubric for student talking drawings is shared in Table 4.

Standardized tests (Fig. 2) are an ongoing assessment for potential changes in science proficiency. Rochester Public Schools currently uses the Minnesota Comprehensive Assessment for science at grades 5 and 8, a test largely focused on content learning. Analysis of the first 2 years of data offers baseline data (pre-InSciEd Out) and two cohort datasets at grade 5 (Fig. 2A) indicating potential improvement post-InSciEd Out in the percentage of students achieving a passing grade on the MCA science assessment. This analysis and traditional presentation of public test data, however, does not correct for differences between cohorts or differential school to school mobility by students. We have addressed this using two alternative metrics. Effect size is a calculation that allows a comparison of schools normalized to results from the state of Minnesota (see Methods). Figure 2B indicates that student scores at Lincoln increased at a greater rate than the Rochester school district (and via the calculation, at a rate higher than the rest of the state). These 2 years' of MCA science data offers the opportunity to address student growth for one cohort (between grades 5 and 8, with 2 of the 3 years post-InSciEd Out). With the demographics of gender, race, and socioeconomic status considered, students in 8^th grade in 2011 showed increased growth, outpacing the rest of the district with regard to percentage of students showing typical or high growth (Fig. 2C). Although the numbers are encouraging, the relatively small student population at Lincoln compared to the larger numbers of students in the district means that each individual cohort score is formally a trend and has not yet achieved substantial statistical significance. Together, however, these data from assessments of three different cohorts at two different grade levels indicate that, with InSciEd Out partnership, students are outperforming expectations. Rochester Public Schools has committed to conducting full science assessment at each grade level for future detailed analysis of the full impact of InSciEd Out for content learning.

Content and process knowledge assessment

Current MCA science testing is designed to measure content knowledge. The InSciEd Out approach, however, prioritizes the process of science over content knowledge alone. For example, note in Table 1 that the language objectives and horizontal integration purposefully embed the products of an authentic science process within module expectations. Within InSciEd Out curriculum, students are challenged to experience some new content (e.g., zebrafish embryology) to further drive the process and practice of scientific research. In Figure 1C and 1D, student embryology data are shared. Students of all ages have demonstrated a capacity to learn zebrafish embryology and development (Table 2). This learning acts to support student-led inquiry. Substantial progress in student process knowledge beyond standardized testing can be found within the 15 abstracts and 2 primary research articles in this issue of the Zebrafish.^44–60

Conclusion

InSciEd Out has reached to date 140 teachers and over 2500 students following initiation with Lincoln K–8 Choice School, Rochester, MN, in 2009. Despite this success, we are well aware that education reform efforts often struggle with the limitations of sustainability and scalability. The next challenge is to support both current efforts while offering this opportunity for more school participation.

Zebrafish

The intention of the InSciEd Out team has always been to support the culture and practice of science in partner schools and communities. The zebrafish has revealed itself as a practical and effective component of that work. Students and teachers often refer to InSciEd Out simply as “the Zebrafish Project.” Certainly the timeline of development and the accessibility of the zebrafish have shown the capacity to support improved student science. Additionally, the biennial Conference on Zebrafish Development and Genetics in Madison, Wisconsin, has revealed an international community with an increasing mission in education partnership.

Efficacy

In Minnesota, the Minnesota Comprehensive Science Assessments are administered to only 5^th and 8^th grade students, limiting the resolution by which any cohort of students can be followed. InSciEd Out is partnered at multiple schools within the Rochester public school district. At several schools, however, partnership is at a single grade level, leaving proficiency tracking noncontiguous by year. In response to the need for better measurement at the local level, the Rochester public school district has implemented additional science testing at all grades beyond 2^nd. This regrettably increases student time away from class each year but is required to yield a clearer picture of student progress.

Teachers of InSciEd Out have demonstrated substantial growth in their understanding and practice of science. Improved instruments for capturing these trends will lead to an increased understanding of partnerships between science and education. Success of InSciEd Out will be determined in the long run less by standardized testing and more fully in the discourse of science, through the process of poster presentation, peer-reviewed publication, and public interaction of our students.

Sustainable partnership

Long-standing partnerships are relatively unusual in science education reform. An integral part of InSciEd Out is intensive and sustained teacher development. Almost all teachers participate in some form of professional development every year. Over the years, schools have devoted a generous part of their budget on teacher development. Because teachers are the single largest human resource in schools, sustainable staff development becomes an issue of human resource development. Nearly half of all teachers are dissatisfied with their opportunities for professional development and relatively few U.S. teachers engage in intensive professional collaboration around curriculum planning.⁶² In contrast, the multi-year, tiered process of InSciEd Out meets the criteria described by Darling-Hammond for sustainability of teacher professional development.⁶² Often in education, one teacher attends a professional development opportunity, then shares it with his or her own school. This process includes an expectation that a teacher can develop enough expertise in one training experience to not only internalize new strategies for use within their own classroom, but also to teach those strategies to their adult learner peers. Similarly, in the above-described methods of Silverstein et al.,²² a single teacher is often drawn out of a school to initiate science education reform for all grade levels in that school. Because these methods do not provide the infrastructure for that follow-up training, examples of long-term change through this design cannot be identified. The concept of critical mass starts with participation of all teachers from a school participating in a professional development program. Because their entire team has had a common experience, they are positioned to support each other in innovation. Scientists and InSciEd Out teaching staff provide coaching across multiple years, further supporting the culture of change.

Sustainable cost

The initial implementation cost per student averages to $75/student/year. However, most of this expense was incurred during the professional development phase. Of the supply cost, less than 5% was in consumables. $55,000 of the initial costs were to purchase reusable and shared tools to be housed in the STEM Village shared resource where teachers can reserve and use microscopes, laptops, and other scientific equipment. Due in part to the modest cost of biological materials such as zebrafish embryos, ongoing active learning material cost for the current students of InSciEd Out corresponds to less than $5.00/student/year.

Scalability

The level of partnership at the core of InSciEd Out raises concerns that there is a finite capacity for InSciEd Out-type of science education innovation. Just how many education partners can a single academic institution take on before there is a substantial decline in the quality of those partnerships? To address this concern, InSciEd Out is adopting a “cell phone tower model” for scalability. The current tripartite relationship serves as an example hub, or single cell phone tower. This inaugural hub has the capacity to partner with the educators and students of our surrounding ∼50 kilometer radius. InSciEd Out has begun building the infrastructure for additional hubs in the urban areas of Minneapolis/St. Paul. Similar to a cell phone coverage map, the capacity of InSciEd Out to scale-up will be directly measurable by the ability to bring together partnership hubs with the capacity to serve their own geographic area. Zebrafish community members may be well positioned to provide this additional capacity.

Science is a profoundly human enterprise. We believe that providing an educational environment full of authentic science removes barriers to student learning and untaps the inherent potential of our students in science, regardless of their background. The data to date from InSciEd Out support the idea that enabling the teachers to integrate science as the norm within regular curriculum is one effective means to achieve the long-term goal of excellence in science education.

Acknowledgments

We thank Team InSciEd Out for their unending support. We also thank the Mayo Clinic Zebrafish Core Facility for excellent zebrafish husbandry and student mentorship. This work was supported by NIH Grants NCRR/NCATS CTSA UL1RR024150 and DA14546 (including an ARRA supplement). Additional support was from RPS exemplary grants, Bush Foundation through WSU and Minnesota State Colleges and Universities, WSU Stem Village, parent donations, and local businesses including Fish and Pets (Greg Goodnow, owner).

Disclosure Statement

No competing financial interests exist.