Collaborative Curriculum Design as a Framework for Designing Teacher Professional Development that Produces the Content Knowledge for Teaching the Life Sciences

Abstract

Effective science teaching critically requires content focused professional development (PD), particularly in life sciences where content evolves rapidly. How subject matter knowledge related to teaching (SCK) is most effectively incorporated into PD has not been investigated. We studied how a professional learning community of high school teachers and scientists co-designing a bioscience curriculum produced the accompanying SCK-focused PD. SCK was level-specific but teachers could not generate it alone. Co-designing SCK with scientists was valuable to teachers, as evidenced by significant increases in their cognitive and attitudinal attributes toward the PD, in turn promoting change in practice and student learning gains, both within and outside the initial partnership. Surprisingly, social network analysis of how the collaborators interacted revealed that though the network was cognitively and effectively robust, it was behaviorally much sparser than anticipated for such a high functioning partnership, counter to commonly accepted PD best practices. We suggest that the scientist/educator facilitators who intentionally promoted collaboration in the context of distributed leadership were able to eliminate extraneous interactions, optimizing the process. The results are further evidence that developing content-focused PD relevant to 21^st century life sciences requires dismantling the institutionalized segregation between practitioners of science and teaching.

INTRODUCTION

Shulman (1986) characterized content as the ‘missing paradigm’ in teacher education, and indeed poor teacher subject matter knowledge negatively impacts student performance (Ball et al., 2008). The National Academy of Sciences has stated that ‘teachers need expertise in both subject matter and in teaching’. ‘Subject matter’ can be subdivided into knowledge of the topic and the specialized content knowledge required to teach the topic effectively (Ball et al., 2008). ‘Teaching’ has been subdivided into a discipline specific knowledge of teaching and students in the context of a curriculum as well as general knowledge of teaching strategies (Ball et al., 2008; Shulman, 1986).

Most professional development (PD) has as its goal to change teachers’ behaviors, beliefs, and attitudes with the purpose of enhancing student learning (Darling-Hammond et al., 2009; Desimone, 2009; Guskey, 2002). Content-focused PD programs can increase both subject matter knowledge and pedagogical skills, stimulating changes in teaching practice (Garet et al., 2001). Unfortunately, in the United States (US) not every PD program succeeds. According to a recent status report (Darling-Hammond et al., 2009), although a significant number of US teachers participate in PD, almost 60% are dissatisfied with the outcome. Key complaints that accurately reflect the current PD landscape included that the PD programs were too short, not sufficiently content-focused and participants had limited access to mentors (Darling-Hammond et al., 2009; Wei et al., 2010).

How can we ensure teachers are exposed to current content and develop the understanding necessary to translate it into practice? Currently accepted best practices indicate that successful PD should couple content with a practice-based focus on discipline specific pedagogy. Sustained mentorship of teachers is also considered desirable (Borko, 2004; Garet et al., 2001; Penuel et al., 2007). Despite this, school-based PD often focuses on pedagogical techniques with general applicability with the result that science teachers in particular often struggle to keep their content knowledge current (Astor-Jack et al., 2007; Darling-Hammond et al., 2009). This is of particular concern in life sciences where content evolves rapidly and the contingent pedagogy is correspondingly highly dynamic. Developing PD that integrates content with appropriate pedagogy is further complicated because the science practice community, generally the gatekeeper of emerging content, rarely interacts directly with teachers, let alone provides sustained mentorship. So, while scientists’ input would seem valuable for developing life-science PD for teachers, how to structure it effectively is poorly defined and understudied.

Pedagogical content knowledge (PCK)

Teaching for understanding requires not only familiarity with subject matter, but also the ability to ‘grasp and respond to the relationships between knowledge of content, teaching, and learning’ (Loughran et al., 2003). This concept of ‘pedagogical content knowledge’ (PCK), first introduced by Shulman (1986), refers to how teachers interpret and transform their subject-matter knowledge to facilitate student learning. Notably, PCK encompasses understanding of common learning difficulties and preconceptions of students in the context of the curriculum, as well as the most effective pedagogical approaches for teaching it (Ball et al., 2008). PCK has been most thoroughly investigated in the context of mathematics teaching in part because the underlying content is relatively stable, and computational skills are readily measurable. Of the few reports on how to develop science PD that is content and PCK focused, most have involved chemistry and physics (e.g. van Driel et al., 1998, Loughran et al., 2003) rather than life sciences, where content evolves more rapidly and the underlying PCK is correspondingly more dynamic (Ball et al., 2008).

This study was initiated in the context of a long-term collaborative curriculum design project undertaken with a cohort of Boston Public School teachers to develop a high school life science curriculum based on health and disease. Development of the Great Diseases (GD) curriculum has been described previously (Jacque et al., 2013). Teacher to teacher (peer) collaborations, often described as professional learning communities, communities of practice or networks (DuFour et al., 2004; Lieberman, 2000; Stoll et al., 2006; Wenger and Snyder, 2000), can have robust impacts on teacher practice including increased focus on professional growth and improved self-efficacy (Rock and Wilson, 2005). Collaborative curriculum design is one way in which a professional learning community of educators can interact with other professionals, such as science content experts, to utilize their diversity of knowledge, beliefs and experiences to accomplish a common goal (Bransford et al., 2000; Voogt et al., 2015; Voogt et al., 2011; Vygotsky, 1980). Our collaboration therefore seemed an ideal venue in which to address four questions about how the teachers and subject matter experts would identify the PCK necessary to implement the curriculum effectively, and develop the corresponding PD:

• What was the contribution of subject matter specialists (scientists) and teachers in identification of the PCK?

• How was the resultant PD designed to encompass both content and PCK?

• How did the experience impact teacher knowledge attitudes and beliefs?

• Did teachers need to participate in identifying PCK to use it effectively?

In order to impact practice, collaborations where participants are from disparate disciplines with different vernaculars can benefit from facilitators who can foster an environment in which differences will be acknowledged and productively explored, and who can provide support in clarifying goals and setting the tasks to accomplish them (Voogt et al., 2011). Likewise, distributed leadership together with robust professional communications and social linkages are also seen as critical elements of success (Borko, 2004, Hardré et al., 2013). We therefore used social network analysis to investigate a further question that would be critical for dissemination and scalability of this curriculum/PD development approach:

• Did the collaboration achieve distributed leadership and does it engage in robust communication and social linkages?

Theoretical framework: Investigating teacher change

Since the overall goal of this study was to investigate best practices for developing a PCK-focused PD program for the purpose of implementing a high school health science-focused curriculum, we could not apply models of teacher change and professional growth that are non-linear (e.g. Clarke and Hollingsworth, 2002) or linear, but have teacher change contingent on student change (e.g. Guskey, 1986, 2002). We therefore selected Desimone’s linear model, derived from conceptual and empirical studies, that enumerates the constituents of effective PD that will elicit the critical features of teacher change (increases in knowledge, skills, attitudes and/or beliefs) that in turn influence teacher practice and improve student learning (Desimone, 2009). Importantly, Desimone considers that to be meaningful, PD should take place in a specific context, such as a curriculum. Since PCK establishes the high-level knowledge required to teach science content effectively, we hypothesized that the process of developing a cutting-edge biology curriculum in collaboration with scientists would be an effective way for teachers to both update their content and develop the necessary PCK to teach it in the high school setting. We have therefore modified Desimone’s original model to explicitly include the contribution of content and PCK to teacher growth and to illustrate how we evaluated teacher growth as changes in knowledge, skills and attitudes, particularly self-efficacy towards teaching the curriculum (Figure 1).

Figure 1.

Desimone (2009) Model of eliciting teacher change

METHODS

The Tufts Center for Translational Science Education (CTSE) established the collaborative learning community in the context of an award from the Science Education Partnership Award program administered by the National Institutes of Health. The long-term goal of the project, entitled ‘A collaborative approach to biomedical science in the high school’ was to develop the PD support and materials for a year-long modular biosciences curriculum targeted to 10^th–12^th grade high school biology. The disease modules covered in the Great Diseases (GD) curriculum - Infectious and Metabolic Diseases, Neurological Disorders and Cancer give students the opportunity to actively engage with the cutting edge science behind diseases of global significance; each lesson has been fully aligned with the Next Generation Science Standards (Jacque et al., 2013). The curricular materials are freely available on line at http://sites.tufts.edu/greatdiseases/.

Participants

Teachers

Eleven teachers with a collective average of 10 years’ experience participated in the professional learning community for five years. All were science teachers from two of the largest Boston Public high schools - one a technical and vocational high school, and the second an exam school focused on college preparation. Five held a Bachelor’s degree in Biology; three a Master of Science in Biology, and seven a Master’s degree in Education. Most taught biology at all levels. They were PD veterans, each having attended an average of 20 PD workshops in the five years prior to the project. Three reported previous experience collaboratively designing high school curricula including a guide for teachers, but none had received specific instruction in developing PCK. They received a stipend for their participation at union rates (Supplementary Table 1).

Subject matter specialists (scientists)

Twenty-eight Tufts postdoctoral fellows and graduate students with specializations in infectious disease, neuroscience, nutrition science or oncology participated over the 5-year duration of the project (an average of seven content specialists per disease module). None had formal training in education. Graduate students received a stipend for participation at the same rate as the teachers.

Facilitators

Seven CTSE staff (biomedical scientists with a research focus in science education) acted as facilitators throughout the project (an average of three per disease module). CTSE managed logistics, structured and facilitated the co-design meetings, and coordinated development of curriculum materials. During the latter phase CTSE also coordinated the classroom pilots of the curriculum, designed assessments, and collected data.

Structure of the collaborative curriculum design process

The rationale and the timeline of the GD project has been described in detail elsewhere (Jacque et al., 2013). In the context of this study, work on each of the four modules was divided into two phases: The first involved the science content specialists collaborating among themselves to establish the relevant content for each disease module and then partnering with teachers to parse out the overall curriculum structure. This work was guided by the CTSE facilitators. The second phase involved teachers collaborating with science content specialists and the CTSE facilitators to establish the PCK required to effectively teach the curriculum, to use it to generate support materials, and to embed the support materials into ongoing PD for new users of the curriculum (Figure 2). Each phase took one full year per module; however, timelines overlapped so that, for example, phase two of the Infectious Disease module occurred concurrently with phase one of the Neurological Disorders module.

Figure 2.

Two phases of collaborative curriculum design to establish content knowledge and contiguous PCK.

Changes in teacher outcomes

The research model we used to measure teacher or student-level change in knowledge, attitudes and beliefs characteristic of effective PD was a within group, change-over-time, mixed methods design study. Quantitative data from teachers was gathered through regular online surveys that used a retrospective pre-test model in which respondents were asked to recall and compare a previous state (e.g., one year prior). This method, while effective at reducing Type II error, can sometimes inflate effect size. However, other studies have shown that in this context any overestimate is no more than 5–10%, and has no practical implications (Jacque et al., 2016, 2013). To minimize respondent manipulation or social desirability bias, the online attitudinal survey was presented on different web pages so that the respondents reflected on their earlier mental state on the pre-test page before advancing to the next page and answering the same questions regarding their current mental state. We anticipate that preventing respondents from seeing both responses in a side-by-side format may minimize the desirability bias that is most responsible for inflating effect sizes. Data were analyzed in either SPSS or R. Qualitative data was gathered through semi-structured 40-minute telephone interviews conducted by an independent evaluator. Seven of the teachers who had been active in the project for at least two years were interviewed for 40 minutes each. Teachers were given the topics in advance of the interview and their responses were recorded for transcription and line-by-line coding that paralleled the domains laid out in the semi-structured protocol. Codes were then grouped together across conversations and re-analyzed in NVivo.

Changes in student outcomes

Quantitative data showing student learning were gathered from pre-post-tests designed by CTSE facilitators in collaboration with the science content specialists, and reliability testing was performed with content specialists from outside the partnership. Tests comprised multiple choice and short answer questions together with a case study that required open-ended responses and had multiple questions per concept. Questions were designed to avoid a plateauing effect. Pre- and post-tests were graded by two independent reviewers and their scores averaged. Answers were graded stringently: credit was given for correct choices and points were deducted for incorrect choices. Student data were analyzed for pre- to post change using a Wilcoxon signed rank test, and effect size was calculated with Cohen’s D. In all cases student responses were anonymous.

Social network analysis of the collaborative curriculum design community

To determine whether the relationship patterns in the group predicted establishment of a robust community, social network analysis and exponential random graph modeling (ERGM) were performed at the end of each of the first three modules. Social network analysis (analyzed in UClNET) focused on overall density, which is the observed density in the links compared to the possible number of links; centrality, which assesses the frequency and extent of communication between participants; and power, which assesses the dependency of actors on other actors. Visualization of the data used NetDraw (Borgatti et al., 2002). Exponential random graph modeling (ERGM) distinguished between episodic and relatively sustained interactions using a binary threshold of 0 for interactions occurring once a month or less, and 1 for interactions that occurred 2–3 times a month or more, focusing on the number of ties and the number of reciprocated ties; the number of in2stars and out2stars, which is indicative of propagation of ideas; and the number of transitive triangles, which can be interpreted as the stabilization of relations over time that ‘solidifies’ the network (Supplementary Figure 1). ERGM takes these five local social configurations and rearranges the parameters to create a distribution of networks allowing convergence of the model to be assigned. This modeling is used to determine whether the observed network varies from a simulated set of same-sized random networks to a statistically significant extent. ERGM modeling was conducted in MPNet (http://www.swinburne.edu.au/fbl/research/transformative-innovation/our-research/MelNet-social-network-group/PNet-software/index.html).

RESULTS AND DISUSSION

This study investigated a group of high school biology teachers partnered with subject matter specialists (scientists) in the context of a collaborative curriculum development project. The structure of the partnership and the timeline of curriculum development have been described in detail elsewhere (Jacque et al., 2013). The teachers had a diversity of pedagogical practice that reflected the differing demographics of their schools that became available to the group. Conversely, the science content specialists, who had no formal training in high school pedagogy, brought a wealth of content knowledge to the collaboration. In the context of this study teachers and scientists combined their knowledge of content with their knowledge of teaching to identify critical PCK required for effective PD. Although we limit our description here to one of the four curricular modules (Metabolic Disease) we performed similar analyses for each of the other three modules (Infectious Disease, Neurological Disorders and Cancer) with similar results. Hence the model of interactions described here is replicable within the context of this extended project.

The primary goal of the curriculum design project was to translate cutting edge biomedical concepts into engaging and accessible biology curricula for high school students. For example, instead of teaching Metabolic Disease from the simple focus of cellular metabolism – glycolysis, the citric acid cycle etc. – as is commonly done in high school biology, the GD curriculum focuses on life-relevant questions critical to a ‘deep’ or essential understanding from the perspective of health and disease. These questions were generated by the content specialists in collaboration with the teachers. Thus Unit 2 ‘How does our body use food?’ examines in part how the cellular metabolism that maintains glucose homeostasis in the blood is altered as the physiological demands of different organs change, and therefore how nutrition needs to change in response. This approach requires integrating an understanding of cellular metabolism (how key nutrients are used to produce ATP) with an understanding of physiology (how each organ contributes to maintaining glucose homeostasis in the blood) in the context of why glucose homeostasis in the blood is critically important (to conserve glucose concentrations in the brain, which cannot itself manufacture glucose).

What contribution did subject matter specialists and teachers make to identification of the PCK?

Figure 3 shows how the partnership identified the PCK needed to teach this highly integrated approach. All the teachers and scientists routinely taught about cellular metabolism and were able to generate valid concept maps about ATP biosynthesis from glucose. The science content specialists had a more complete grasp of the critical content, and included key nutritional constituents other than glucose (such as fatty acids and amino acids) into their concept maps. However, they failed to integrate fatty acids and amino acids into the overall process of cellular metabolism and also did not integrate physiological homeostasis with cellular metabolism (Figures 3A and B). Teachers had a less complete grasp of content and also did not integrate the concepts. However, when the facilitators focused the group’s attention on how the content would address the question ‘How does your body use food?’ teachers were quickly able to identify where concept integration would be required at the high school level. They told the scientists that the content (cellular metabolism, use of nutrients, physiological homeostasis) should be refocused to move glucose homeostasis to the center of the concept map, and that it would also be necessary to consider how different physiological demands would drive metabolism of each nutritional component. Figure 3C illustrates the final concept map that evolved as a result of the dialog that emerged from the initial question posed by the facilitators.

Figure 3.

A,B: Initial concept maps developed by teachers (3A) and scientists (3B) to address the role of nutrients in maintaining glucose homeostasis

C. Final collaborative concept map developed by the teacher-scientist partnership that outlines the content knowledge needed to teach how nutrients influence glucose homeostasis.

How was the resultant PD designed to encompass both content and PCK?

In structuring the curriculum deliverables CTSE had initially anticipated that scientists would generate a ‘primer’ that would update and integrate teacher content knowledge in the context of the key unit questions from each disease module, while teachers would generate the comprehensive lesson plans and activity materials that would provide the pedagogical strategies to address the learning goals. However, it became clear that for PD to be effective we would also need to address the content knowledge element of PCK, namely common learning difficulties and student misconceptions. For example, this part of the Metabolic Disease module required a resource that both emphasized how to teach metabolism and physiology integrated with homeostasis, which is radically different from how the topics are normally taught at the high school level, and that addressed typical misconceptions about metabolism. For instance, while students will often recall that glucose is used in cellular respiration to produce ATP, they commonly misunderstand that when we ‘burn’ fat to produce energy it is also used in cellular respiration. As the concept maps had revealed, neither scientists nor teachers could not develop this resource on their own, since they lacked this particular perspective. As a solution CTSE developed an extensive narrative to accompany the lesson plans that provided explicit pointers for initiating and managing the in-class Socratic discussions on which the curriculum is based, together with examples of common misconceptions. Using the narrative in conjunction with the lesson plans optimizes implementation of the learning objectives by providing a model of how to teach the material effectively.

Through this iterative process in which the partnership articulated key questions, identified and developed the PCK, and finally memorialized the PCK in the form of the lesson support narrative was repeated for each lesson in all four modules, a total of about 150 lessons.

How did the experience impact teacher outcomes: knowledge, attitudes and beliefs?

After completing each of the four disease modules partnering teachers completed anonymous internet surveys to assess how their experiences had elicited the changes described in Desimone’s model (Desimone, 2009). Each survey was therefore broken down into four broad domains: cognitive, attitudinal, behavioral, and social.

The cognitive domain asked about new knowledge (facts, concepts, teaching skills) the teachers felt they had acquired as a result of the collaboration. One of the four constructs within the cognitive domain referred to content, while three referred to the use of that content in teaching i.e. PCK (Table 1). After completing the Metabolic Disease module teachers reported significant gains in in all three PCK knowledge constructs in the cognitive domain (Cohen’s d = 2.66, p<0.001, paired t-test). Hence, providing teachers with the opportunity to devise PCK in the context of a specific curriculum is an effective way to improve their recognition of the knowledge needed to teach about novel content.

Table 1.

Teacher knowledge of metabolic disease (MD) content. Teachers were asked to compare the current status of their knowledge of the metabolic diseases topic with their knowledge one year ago using a 1–6 Likert Scale (1 = very low; 6 = very high). The questions were reliable (Chronbach’s alpha 0.873) and the effect size was large (Cohen’s d = 2.66).

	Before AVG (SD)	After Yr 1 AVG (SD)
MD content	3.0 (1.00)	4.78 (0.67)**
Special skills used for teaching MD content (PCK)	2.89 (1.54)	4.67 (0.87)***
Mentality of thinking about MD (PCK)	2.88 (1.58)	5.00 (0.54)***
Core concepts in MD research (PCK)	3.00 (1.58)	4.67 (0.71)***
Knowledge of MD resources for teaching (PCK)	2.78 (0.83)	5.00 (0.71)***
Total	14.13 (5.41)	23.63 (1.92)***

The results for each construct were compared with a paired ‘t’ test.

^***= p<0.0001;

^**= p<0.001.

The attitudinal domain asked about how the new knowledge impacted self-efficacy and intention to change teaching practice. Teachers participating in the project were already interested in and attentive to the disease topics in the curriculum, and showed significant gains in both of the attitudinal constructs related to PCK (enthusiasm and confidence in teaching the topic) and a corresponding intention to change their practice by incorporating the novel curricular material into their teaching (p<0.001, paired t test for both), suggesting a linkage between them (Table 2). Moreover, efficacy and enthusiasm towards teaching also significantly correlated with perceived gains in knowledge (0.83, and 0.91, p<0.05, Pearson), while no other pairing correlated, suggesting that providing science teachers with the opportunity to participate in developing content and PCK-focused PD may affect self-efficacy, a powerful predictor of changes in practice. In contrast, there was no significant change in either their overall enthusiasm for teaching biology or in constructs not directly related to the collaboration (awareness of the federal agencies that work on disease, such as the Centers for Disease Control) indicating that the observed shifts in attitudes are specifically related to the perceived utility of the partnership rather than a non-specific increase in interest or response bias.

Table 2.

Teacher attitudes and intentions towards teaching the Great Diseases curriculum. Teachers were asked to compare the current status of their attitudes and intentions toward the Great Diseases with the same survey year ago using a 1–6 Likert Scale (1 = very low; 6 = very high).

	Before Participation AVG (SD)	After Yr 1 AVG (SD)
Enthusiasm for teaching the Great Diseases	3.82 (1.08)	5.09 (0.70)**
Confidence in teaching the Great Diseases	3.55 (1.13)	5.00 (0.77)***
Knowledge about the Great Diseases	3.27 (1.27)	4.64 (0.67)***
Intention to incorporate the Great Diseases in your teaching	2.80 (1.32)	5.00 (0.94)**
Interest in the Great Diseases	4.27 (0.90)	4.91 (0.70)*
Awareness of the importance of the Great Diseases	5.00 (1.00)	5.55 (0.82)*
Attentiveness to news about the Great Diseases	4.27 (0.90)	4.82 (0.98)*
Interest in learning more about the Great Diseases	4.18 (1.08)	5.00 (1.00)*
Prominence of the Great Diseases in your teaching	2.50 (1.27)	5.00 (1.00)*
Enthusiasm for teaching biology	5.18 (1.08)	3.60 (1.26)*
Awareness of the CDC (Centers for Disease Control)	4.18 (1.33)	4.82 (1.17)
Appreciation for the work of the CDC	3.91 (1.22)	4.64 (1.29)
Awareness of the NIH	4.27 (1.27)	4.73 (1.27)
Appreciation for the work of the NIH	4.36 (1.21)	4.82 (1.17)

The results for each construct were compared with a paired ‘t’ test:

^***= p<0.001;

^**= p< 0.001;

^*= p<0.01.

The semi-structured interviews provided a deeper exploration of the attitudinal shifts seen in the participating teachers. The interviews were coded to reflect seven domains, five of which were directly relevant to this study (Table 3). A selection of specific comments from teachers that related to the PCK domains described above are provided in Supplementary Table 2, further illustrating how individual teachers reflected on the experiences. Analysis of the teachers’ statements further confirmed that the survey constructs had been appropriately constructed.

Table 3.

Changes in teacher outcomes assessed from telephone interviews.

Code	Meaning	Findings
Network	Refers to collaborations among the teachers	Teachers found enormous value in working together in an innovative setting on challenging curriculum.
Learning (content)	Refers to instances in which the interviewee spoke of the learning benefits of participation	Teachers acquired a great deal of knowledge at a depth they found novel and valuable.
Translation (PCK)	Refers to the translation of project learning into curriculum	Teachers deeply appreciated the collaborative and diverse social space within which the ‘deep knowledge’ could be translated into useable curriculum.
Attitudinal Shifts	Refers to instances in which the interviewee spoke of attitudes being affected by participation.	The main attitudinal shift had to do with increased confidence (self-efficacy) in being able to teach more complex curriculum)
Change in practice	Refers to instances in which the interviewee spoke of the likelihood of continuing to us the curriculum	Teachers were highly confident they would continue to use the newly-developed curriculum.

Impacts on student outcomes

The primary goal of all PD is to enhance student learning. Thus, once the curriculum and ancillary materials for each module were finalized, one of the teachers from the selective public exam school piloted the lessons in a Biology II course. This teacher was selected in part because her students could provide a benchmark of achievement. The students’ performance on the Metabolic Disease module content knowledge test showed a highly significant increase pre-post (p<0.0001, Wilcoxon signed rank test, n=98) with a large effect size (Cohen’s D = 1.8).

Do teachers need to participate in identifying PCK in order to use it effectively?

We recruited an additional implementing teacher from a general urban high school with a high proportion of English language learners who had not participated in the curriculum design partnership to utilize the curriculum-related PD we had developed. The PD was delivered through the ‘modeling for fidelity’ mentorship program that provides virtual support as the curricular materials are taught in the classroom (Malanson et al., 2014). The same pre- and post-tests revealed that the initial level of student attainment was significantly lower than that of the selective exam school students (−1.5 vs. −8.8) perhaps reflecting the different demographics of the school. However, these students did achieve a highly significant increase in knowledge, including problem solving abilities, related to the Metabolic Disease curriculum (Cohen’s D = 0.978, p<0.0001, Wilcoxon signed rank test, n= 49). Evaluating student outcomes in multiple classrooms of non-participant teachers has shown significant increases in learning and importantly in self-efficacy related to learning the material (Malanson et al., 2014, Jacque et al., 2016). These results, which indicate that PCK-intense PD is beneficial to teachers even if they did not participate in initial development, lend support to the notion that this method of providing content and PCK-rich PD is scalable beyond the initial design partnership. However, they do point to a limitation in this approach, namely that ‘Modeling for Fidelity’ mentorship is itself labor intensive. In response to this consideration we are currently adapting this mentorship method into an online format that will be thoroughly evaluated once complete.

Did the collaboration achieve distributed leadership and does it engage in robust communication and social linkages?

Another key question for reproducibility of this PD design approach is the nature of the collaboration. Figure 4 is a sociogram illustrating the interconnectivity of the collaborative learning community (network) during phase 2 of the 1^st disease module. The participants are linked only if they communicated about matters relevant to the partnership more than twice a month outside the monthly meeting times (communication referred to any kind of verbal or written exchange).

Figure 4.

Social network analysis demonstrates distributed leadership among the professional learning community

Centrality analysis, which defines the number and frequency of each participant’s connections was not significantly different per school than the overall centrality, indicating an absence of troublesome holes in communication. ‘Power’, which refers to the extent that participants depend on each other was also not significantly different across the group as a whole, indicating an absence of isolated individuals who depended on a more limited number of connections. The sociogram’s centralization can be used as a proxy for whether leadership is distributed. The worst-case scenario, in which only one actor (such as the P.I. or project manager) acts as the conduit for connections to the others, is called a ‘star network’ and has a centralization value of 1.0. Our network’s centralization value was 0.12 (i.e. 12% of this worst-case scenario) indicating that the CTSE facilitators have successfully avoided establishing a hierarchical network. Social network analysis therefore confirmed that even at an early stage the group had coalesced into a cohesive network, with distributed leadership. Subsequent analysis confirmed that these characteristics were sustained over the 5-year duration of the project (data not shown).

Communication characteristics of participant interactions

Currently recommended best practices in establishing collaborative learning communities call for sustained networks that provide opportunity for frequent communication. Given the robust impacts our curriculum co-design project had on participating teachers, their students, and auxiliary teachers and students, we perceive our network as high functioning (Jacque et al., 2016; 2013; Malanson et al., 2014). However, little is known about the characteristics high-functioning networks display. We therefore characterized how the extent of communication between participants developed over time to the eventual successful conclusion of curriculum development. Supplementary Figure 1 defines the five parameters (or configurations) of exponential random graph modeling (ERGM) we evaluated each year for three years to determine the frequency of talk about relevant issues. The threshold we selected (> twice a month) represents relatively sustained relations consistent with current views of best practices. The results, presented in Table 4, were surprising: At year 3 of the partnership, after two disease models had already been developed and when the partnership was maximally engaged with the Metabolic Disease module described above, only 2/5 of the types of communication that we assessed were statistically different from that expected by chance, and only one of these (unidirectional communications between participants) was more than expected, whereas there were fewer than expected reciprocated ties. Neither the stability of the network nor the propagation of ideas had significantly increased over time. Thus, in contrast to commonly articulated best practices (Borko, 2004) the absolute quantity of communicative exchanges must not be a foundational attribute of our highly functioning collaborative learning community.

Table 4.

Results from Exponential random graph modeling (ERGM).

Year	Reciprocated ties	In2stars	Out2stars	Transitive triangles
Year 1	No	No	No	No
Year 2	No	Fewer	No	No
Year 3	Fewer	No	No	No

CONCLUSIONS

Ball, Thames and Phelps (2008) have said that: “teaching may require a specialized form of pure subject matter knowledge … ‘specialized’ because it is not needed or used in settings other than… teaching”. This study has shown that participating in the curriculum co-design project with scientists improved teachers’ ability to identify and develop the specialized PCK required to teach a novel life sciences curriculum. Moreover, the positive impacts on teacher practice and student outcomes led to several insights that may be useful for replicating this approach. Our experiences here with a life science curriculum suggest that the combination of evolving content with a specific teaching context requires content and pedagogy specialists to collaborate to identify the necessary PCK. Hence, for high school students to address the question ‘how does your body use food?’ in a cutting-edge manner pertinent to health and disease, both content specialists and teachers needed to significantly rework their own initial concept maps in order to integrate cellular metabolism and physiology with glucose homeostasis. Only once the final integrated concept map had been developed was the partnership able to identify the relevant PCK teachers would need to implement lessons having this innovative approach. Thus, direct interactions between practitioners of science and practitioners of teaching seem to be key in identifying PCK in areas like life sciences and health where fundamental content is rapidly evolving. To expect educators or non-content specialist curriculum developers to be able to keep abreast of this dynamic flux on their own is unrealistic. For example, when we began the Cancer module, the ‘hottest’ topic in research was cancer metastasis - the ability of cancer cells to break out of their primary location and migrate around the body. But the PCK required to teach about metastasis in the context of migrating cancer cells becoming resistant to chemotherapy is significantly different from the PCK required to teach about metastasis in the context of being able to harness the immune system response to kill the migrating cells, which is where the field has evolved in only 5 years – the duration of this study. Our results also show that a top-down approach in which scientists impart content and PCK to educators is not effective: It appears that PCK is specific to teaching at a particular level, and that scientists use different PCK than high school teachers. On the other hand, once teachers have acquired the necessary content they are able to work with the scientists to identify and develop the PCK for their own students. With respect to the pedagogical element of PCK, teachers could work independently of scientists, and often successfully adapted strategies for their own classrooms while adhering to learning objectives. However, neither scientists nor teachers are effective at generating the actual PD deliverables that serve as support material for the curriculum, in large part due to time constraints in participating in such an ambitious project. This task we most effectively accomplished by facilitators from the Tufts Center for Translational Science Education (CTSE) - a dedicated team of professional scientists with expertise in high school education and a familiarity with both vernaculars. Even more important, it appears that when it comes to student outcomes, teachers do not need to participate in developing level-specific PCK to use it effectively as long as appropriate PCK-rich PD with appropriate mentoring is available. In toto, the results indicate that to keep abreast with teaching cutting edge concepts in life sciences will critically require breaching the institutionalized segregation between practitioners of science and teaching.

Our final research question investigated what characteristics a successful teacher/scientist collaborative curriculum/PD design partnership should have. The literature on collaborative learning communities emphasizes both the necessity for teacher connectedness and the importance of networks. We know very little about how a high-functioning network ought to behave other than as a non-hierarchical framework with distributed leadership that allows it to capitalize on the diversity of knowledge and experience of each actor (Gronn, 2002) which our network displays. For this project, we assumed that, in keeping with the literature, a high-functioning network would also be characterized by regular communication between participants (Borko, 2004). However, while our outcome data supports the notion that the network is indeed highly functioning, communicative exchanges were less regular than one would expect to see by chance. Yet Table 5 confirms that our network had built the mutual respect, trust, and support that is necessary when sub-groups (teachers and scientists) have different vernaculars and express distinct orientations and concerns (Lee et al., 2011). This implies that the changes in attitudes (self-efficacy, appreciation for novel content, etc.) that impacted the teachers’ cognitive and attitudinal attributes were due to sustained connections expressed either affectively (teachers felt respected or encouraged) or cognitively (teachers were aware of one another) rather than behaviorally (teachers communicated extensively). Moreover, this loose network structure may have allowed teachers to adjust to any challenges they faced without running any social network penalty. Thus, we suspect that it is not so much the quantity of exchanges that matters, but rather the relevance of the exchanges that was of greatest importance for the network, and we suggest that an effective model of teacher/scientist partnerships requires critical attention to affective and cognitive payoffs.

Table 5.

Sample teacher quotes from telephone interviews about the network and the role of the facilitators.

A) Network

‘The best thing about it was working with teachers outside of the school within the GD project. I think probably the biggest challenge was the time aspect. We met fairly frequently throughout the semester and throughout the year but in between the actual face-to-face meetings I don’t think there is probably much collaboration or talking between partners and that is just a function of doing that in addition to your everyday teaching and so your time was limited but also not being in the same physical location so you can’t just like walk down the hall and ask questions or talk about things’. ‘It’s funny because a lot of PD is transient. You go in for like a week, a week and a half, and you get all these great ideas and plans and slowly over time the communication pipeline between you and the resource sort of fractures or your interest wanes or maybe logistical issues start to pop up and you’re having trouble keeping it going and then after a year or two. This one seems like it is going to be a pretty lifelong support mechanism. I mean I’m sure logistically it will be tougher as they take on more teachers to use the curriculum, as long as there is still a line of communication, it sort of like feeds on itself. So the more teachers are sort of invited to serve as ambassadors to the program, the more that they can take on some of the logistic burden of some of the other teachers’. ‘Teaching like I said, is a reflective practice. There is also, like any field, a degree of emotional benefit through commiseration and things like that. So between that and the reflective practice and talking to each about what worked, what didn’t, “what is your perspective on this system?” “Oh, wow, I always thought of concept a through this model or through this lens but it is interesting making you think of concept a through this other lens or other model. So, from a teaching and curriculum development perspective that’s useful. Yeah, so building up a collection of collaborators and people to communicate with is always useful’. ‘The most productive days were definitely the full days where you could just really sit down and wrap your head around stuff and start something and sort of see it through to completion as opposed to working on a piece of it and then dropping it for two weeks until you get back .… The full-day pieces were very useful and I know it can be challenging trying to find times that work for people. Possibly having had some summer days over summertime would have been something that would have been useful. But, yeah, I felt like it was an evolving process and I felt like initially they didn’t necessarily know exactly where they wanted to go with it but it really came together pretty quickly in terms of the format that they wanted to proceed with. I think A was extremely instrumental in helping that process’. ‘I just remember that every single semester you kind of look back on the semester before, like how we met, when we met, what were the strengths and weaknesses and we kind of tweaked it and we changed it. So if there is a way of like kind of – continuing the networks in between the face-to-face meetings, you know, within obviously realistic constraints I think that would probably be better times where you got a whole lot done and then there were some like dry spells or doing their own thing and then we came back and got a whole bunch done. Those I think are the more – there wasn’t the continuous level of collaboration.

B) Facilitation

‘I think what the tufts team did is that we kind of did the structural stuff but then they definitely filled in the details, the supporting materials and all that stuff and that was amazing because there is just no way we could have had time to do that on our own.…And obviously the resources that come with a university like tufts was great to have as well. I think those were exceptional. I think what I remember most, probably what came through, is just kind of finding that pattern, that niche that like of how to go about building a curriculum. I just remember that every single semester you kind of look back on the semester before, like how we met, when we met, what were the strengths and weaknesses and we kind of tweaked it and we changed it. So if there is a way of like kind of – continuing the networks in between the face-to-face meetings, you know, within obviously realistic constraints I think that would probably be better times where you got a whole lot done and then there were some like dry spells or doing their own thing and then we came back and got a whole bunch done. Those I think are the more – there wasn’t the continuous level of collaboration’. ‘We sort of knew what to expect every single time we would go and what the format was. It was not really ambiguous as to what we would be doing or what our end goal was. They did a really nice job of laying out the goals and then making sure that we saw things through to completion and they were always prepared on their end with their guest speakers, with diversifying who was working with us, with providing support in their post-doc volunteers. Their end was extremely well-structured, they did a great job in supporting us. I feel like we definitely ended up getting a lot more out of this than they did. It may not have been equally fair. We offered our insight but, boy, they gave us a whole lot in terms of content and new ideas and ways to think about things and having that opportunity to network with other people’. If we were thinking of doing a lab that – you didn’t have to like – if you wanted to do a lab basically what we needed to do is say, “Listen, this is like – we want to go into the lab and just kind of like having the lab to look at but we don’t have time to kind of think about exactly every step of the lab and how that should work out and what tools you need and all that stuff.” He was really good at kind of taking that general sketch and then filling in the details’.’ “I would say that I’m certainly more comfortable with more conceptual material than I was. I would say also maybe my understanding of teaching kind of a large concept, how to really explore a concept rather than just sticking to the facts I think in terms of like a large unit. I had not really had much experience with kind of unit design so I think that sitting down and looking at the essential questions and breaking down how to design lesson units I think was very useful’. ‘And if we had our own personal areas that we wanted to sort of strengthen the curriculum in but the beauty of it is that we did in the curriculum we did and that they were always there and willing and interested and motivated to add content and to also hear us discuss educational theory’.

We argue that the participation of facilitators who were themselves working scientists with a research interest in high school education ensured that the collaboration was efficient even though the network was relatively sparse behaviorally. That is to say, because the subgroups were likely to have many pressing demands, the facilitators could ensure time spent actually collaborating was streamlined and efficient so the need for frequent communication was minimized. The facilitators also minimized teacher co-regulation and encouraged them to adapt their lesson design to their individual classrooms. This had another easily identified pay-off: while the curriculum featured novel content, it used readily identifiable pedagogies, and so could be readily implemented outside the partnership. Thus, the second implementing teacher from outside the partnership could affect significant change in her students despite their differing demographics and the difference in PD training she received.

The outcomes we have described here support the notion that teaching evolving content such as life sciences in the high school classroom requires a strong affective/cognitive network able to harness scientists’ content expertise together with teachers’ pedagogical expertise to establish the PCK appropriate for developing the PD resources to teach it. Fortunately for scalability it appears that implementing teachers only need to access that PCK, they don’t need to participate in developing it to elicit positive student outcomes.

ABBREVIATIONS

PD

professional development

US

United States

PCK

pedagogical content knowledge

GD

Great Diseases

CTSE

Center for Translational Science Education

ERGM

exponential random graph modelling

ATP

adenosine triphosphate

MD

metabolic disease