The ASM-NSF Biology Scholars Program: An Evidence-Based Model for Faculty Development

Abstract

The American Society for Microbiology (ASM) established its ASM-NSF (National Science Foundation) Biology Scholars Program (BSP) to promote undergraduate education reform by 1) supporting biologists to implement evidence-based teaching practices, 2) engaging life science professional societies to facilitate biologists’ leadership in scholarly teaching within the discipline, and 3) participating in a teaching community that fosters disciplinary-level science, technology, engineering, and mathematics (STEM) reform. Since 2005, the program has utilized year-long residency training to provide a continuum of learning and practice centered on principles from the scholarship of teaching and learning (SoTL) to more than 270 participants (“scholars”) from biology and multiple other disciplines. Additionally, the program has recruited 11 life science professional societies to support faculty development in SoTL and discipline-based education research (DBER). To identify the BSP’s long-term outcomes and impacts, ASM engaged an external evaluator to conduct a study of the program’s 2010–2014 scholars (n = 127) and society partners. The study methods included online surveys, focus groups, participant observation, and analysis of various documents. Study participants indicate that the program achieved its proposed goals relative to scholarship, professional society impact, leadership, community, and faculty professional development. Although participants also identified barriers that hindered elements of their BSP participation, findings suggest that the program was essential to their development as faculty and provides evidence of the BSP as a model for other societies seeking to advance undergraduate science education reform. The BSP is the longest-standing faculty development program sponsored by a collective group of life science societies. This collaboration promotes success across a fragmented system of more than 80 societies representing the life sciences and helps catalyze biology education reform efforts.

INTRODUCTION

Professional societies can influence beliefs, behaviors, and cultures within a discipline. Through efforts such as sponsoring conferences and publishing journals, societies help gain visibility and legitimacy for their members’ work. In addition, societies define professional standards, educate members, raise public awareness, facilitate networking, and build disciplinary communities. Through these endeavors, professional societies have established themselves as advocates for new practices and promoters of change in their disciplines.

Improvement in science, technology, engineering, and mathematics (STEM) undergraduate education is a change that many U.S. organizations have called for in recent decades. Indeed, the National Institutes of Health, National Science Foundation (NSF), and other entities have spent billions of dollars to boost STEM education at this level (4). Much of the funding has gone to individual college and university investigators who seek to develop and incorporate emerging scientific content and innovative teaching approaches. Recently, however, support has been provided to fund national organizations that are poised to spread these efforts more broadly. In one example, the NSF supported the American Society for Microbiology (ASM) to establish the ASM-NSF Biology Scholars Program (BSP), an initiative with three primary goals:

1. Empowering biologists to be leaders in science education reform

2. Expanding and supporting a highly interactive community of scholars committed to scientific teaching and the scholarship of teaching and learning in biology

3. Catalyzing deep networks among life science professional societies to collectively engage in sustained undergraduate education reform

The BSP is premised on the belief that improvements in undergraduate biology education can result from evidence-based changes in teaching and that faculty are well-suited to collect the evidence (i.e., scientific teaching and discipline-based education research) (3, 5). Specifically, the program supports biologists to 1) implement evidence-based teaching practices, 2) assess student learning, and 3) participate in a teaching community that fosters disciplinary-level STEM reform.

Since its establishment in 2005, the program has drawn more than 270 participants from biology and multiple other disciplines. It is the longest-standing faculty-enhancement program supported by a professional society rather than a campus-based program housed within a department or center for teaching and learning.

To determine long-term outcomes and impacts from the program, ASM engaged an external evaluator to conduct a study of individuals who participated in the BSP. In brief, findings suggest that their BSP experience was essential to these faculty members’ professional development, as suggested in these comments:

BSP is a stepping-stone to so many things; it was the seminal point in my early career that has shaped the past four years.

BSP helps guide young professors in the development of their careers. For experienced professors, it enables us to analyze teaching and learning scientifically. Personally, it gave me a fresh look at my profession.

BSP is a great program because it supports individuals who are not at primarily research institutions but who are interested in improving undergraduate science education at their local colleges. Although I have been teaching for more than two decades, I still have a lot to learn and am still looking for ways to improve student learning. BSP has been instrumental in my being able to make the effort.

Provided herein is a description of the program and evidence that the BSP provides a model for societies seeking to advance undergraduate science education reform.

BACKGROUND

Established in 2005, the BSP offers biologists and other scientists a continuum of learning and practice centered on principles from the scholarship of teaching and learning (SoTL) (1, 2). Program activities take place through three independent yet intertwined professional development residencies—the Assessment, Research, and Transitions Residencies—, year-long experiences designed to increase faculty expertise in the following areas: assessment tools and resources; research design, methods and analysis; and science education writing and publishing.

Once accepted into the BSP, faculty members are known as “scholars” who, throughout their residencies, work on self-directed SoTL projects and receive facilitator-guided training and practice that align with seven interventions based on literature about how people learn (Table 1). During their residency, scholars are provided leadership training to assist with promoting the benefits of scholarly teaching in campus-based and national society programs. Additionally, BSP facilitators use regular feedback to improve the program.

TABLE 1.

Seven BSP-utilized interventions based on literature about how people learn and leadership.

Commitment. Candidates apply and are selected for evidence of personal commitment and institutional support. Self-directed learning. Candidates enter the program having identified a classroom challenge to address during their participation. Formal, guided instruction. Clear audience needs; learning goals, outcomes and approaches; and reflective activities are embedded throughout multiday in-person institutes. Peer mentoring and community building. Program supports critical friends, small groups, an alumni network, and an online learning community. Structured mentoring. Guided by facilitators, scholars prepare for formal learning by completing readings, research, and writing assignments before attending the in-person institutes. This work also helps to develop scholars to form a single community of practice. Scholars continue their formal learning by collectively participating in online meetings, soliciting feedback on new practices, mentoring each other, and engaging in leadership activities. Leadership. Formal leadership training for practice within the BSP society partners separates BSP from other faculty development programs. BSP paves a way for scholars to connect and contribute to life science disciplinary societies. Evolving role. Participant identities (e.g., as teacher, learner, teacher-scholar, mentor, role model, and leader) evolve throughout the BSP journey.

BSP = Biology Scholars Program.

Four overarching guidelines direct all BSP scholar projects. The projects 1) are based on a research question important to the faculty member’s understanding of how students learn; 2) occur in the context of the scholars’ home institutions, departments, and classes; 3) occur concurrently with teaching, i.e., the research project becomes part of the teaching and is not an “add-on” study; and 4) are used to modify and improve teaching to enhance student learning.

METHODS

The ASM hired a professional evaluator to study the overall effectiveness of the BSP and to answer 1) whether the residencies were enhancing participants’ knowledge and skills in practicing evidence-based teaching and 2) what is or is not working within the program. The overall evaluation plan used a mixed-methods approach that included online surveys, focus groups, participant observation, and analysis of various documents, including electronic postings, agendas, scholar projects, abstracts, and publications. Quantitative and qualitative data collection methods were used to capture impact and outcome data about the participants and their students (summative data). Besides collecting post-workshop evaluations with all scholars and collecting general demographic information (n = 272), a follow-up online study was conducted with four cohorts of faculty (n = 127) who took part in the program between 2010 and 2014, and representatives from 11 professional society partners. Approximately 260 scholars completed the post-workshop surveys for a 95% response rate, and 98 scholars completed the online survey for a 77% response rate. The following results and discussion come from these two data sources.

RESULTS AND DISCUSSION

The four cohorts of scholars described herein started their residencies in years 2010 to 2013 and include biologists from diverse institutions and disciplines, with the largest group coming from doctoral institutions (38%), followed by master’s-level colleges (28%) and bachelor-degree-granting institutions (21%). A small group (8%) hails from community colleges. Not surprisingly, scholars represent the diversity of the life sciences. When asked to describe their primary disciplinary affiliation, more than 55 scholars provided more than one affiliation (Table 2). Overall, scholars reported more than 25 sub-disciplines of the biological sciences as affiliations. This variety attests to the diversity of the life sciences and presents some challenges in reporting our information. The leading five sub-disciplines (cited by 10 or more participants) include biology, microbiology, molecular biology, genetics, and biochemistry. The next group includes cell biology, anatomy and physiology, developmental biology, evolution and ecology, and immunology (Table 3).

TABLE 2.

Demographics: BSP 2010–2013 participants (n = 134).

Residency	No. (%)
Research	67 (50)
Assessment	48 (36)
Transitions	19 (14)
Institution Typea
Doctoral	51 (38)
Master’s	38 (28)
Bachelor’s	28 (21)
Community College	11 (8)
Other	6 (4)
Life Science Disciplineb
Biology	36 (20)
Microbiology	35 (19)
Other	110 (61)
No. of Life Science Disciplines
One discipline	78 (58)
Two disciplines	40 (30)
More than two disciplines	16 (12)

^aCarnegie classification; some items may not total 100% due to rounding.

^bOf 134 scholars, there were 181 responses, with many scholars reporting affiliation with multiple subdisciplines of the biological sciences.

BSP = Biology Scholars Program.

TABLE 3.

BSP 2010–2013: Breakdown of scholar disciplines.

Disciplinea	No.	%
Biology	36	20
Microbiology	35	19
Molecular biology	17	9
Genetics	16	9
Biochemistry	13	7
Cell biology	10	6
Anatomy and physiology	9	5
Developmental biology	7	4
Evolution and ecology	7	4
Immunology	6	3
Education	5	3
Biotechnology	3	2
Biomedical sciences	2	1
Medical science	2	1
Neuroscience	2	1
Other	11	6
TOTAL no. of disciplines reported	26
TOTAL no. of responsesb	181	100

^aThis information is self-reported on the application as “discipline and/or professional field.”

^bScholars reported up to four disciplinary affiliations.

Results indicate that the program achieved the outcomes originally proposed. For reporting these results herein, outcomes are organized in five overarching categories: scholarship, professional society partners and impact, leadership, faculty professional development, and community. Study participants also identified a number of barriers that hindered their ability to assess student learning in the classroom.

Scholarship

BSP participants have experienced very successful outcomes with regard to scholarship. Scholar study respondents (n = 272) reported producing a large number of publications and presentations related to their education research during the program. For example, more than 200 classroom research and assessment projects were developed and implemented, and more than 150 publications about classroom research, assessment, and student learning were published. In addition, approximately 90 presentations were conducted at the annual ASM Conference for Undergraduate Educators (ASMCUE), along with a number of additional presentations conducted elsewhere.

Professional society partners and impact

From the outset, 11 professional societies joined the initiative as BSP partners and were committed to promoting scholar activities and providing and publicizing program opportunities. A list of these societies is presented in Table 4, along with the percentage of scholars who are members of each society. Although 45% of scholars are members of the ASM, slightly more than 30% are members of BSP partner societies. Nearly 7% are not affiliated with a life sciences professional society.

TABLE 4.

Number of BSP participants affiliated with society partners (n = 272).^a

Affiliation	No. (%)
American Society of Microbiology (ASM)	146 (45)
American Association for the Advancement of Science (AAAS)	24 (7)
Ecological Society of America (ESA)	16 (5)
American Society for Cell Biology (ASCB)	14 (4)
American Physiological Society (APS)	12 (4)
American Institute of Biological Sciences (AIBS)	11 (3)
Genetics Society of America (GSA)	10 (3)
American Society for Biochemistry and Molecular Biology (ASBMB)	10 (3)
Human Anatomy and Physiology Society (HAPS)	5 (2)
American Society of Plant Biologists (ASPB)	2 (1)
American Society of Human Genetics (ASHG)	1 (<1)
Society of Toxicology (SOT)	0 (0)
Non-BSP partner societiesb	50 (15)
No society affiliation	22 (7)
Total	323 (100)

^aData are self-reported; some scholars belong to more than one partner society; items may not total 100% due to rounding.

^bBiomedical Engineering Society, National Association for Research in Science Teaching, National Association of Biology Teachers, National Science Teachers Association, Society for the Advancement of Biology Education Research and others.

As part of this study, several society representatives were queried about their knowledge of BSP, of BSP participants from their societies, and of effects within their societies resulting from scholar participation. The representatives indicated awareness of their members who had participated in the BSP and awareness of their activities in the partner societies—especially those members who are serving in societal leadership roles. About 15% of scholars are serving in leadership roles according to a search of each of the societies’ websites as well as the representatives’ responses (see “Leadership” below).

Leadership

Although our data suggest that the development of scholars requires years in evidence-based teaching before they are deemed “experts,” resulting in invited talks and publications, results from this study suggest that BSP scholars are meeting this goal. Many are sought locally (Table 5) and nationally (Table 6) as assessment and education research experts.

TABLE 5.

Percentage of participants engaged in campus-level activities (n = 98).

Activity	Average %	Assessment %	Research %	Transitions %
Presented about assessment, classroom research, or another topic I learned about at BSP	68	70	65	75
Mentored colleagues and peers about assessment and/or classroom-based research	63	65	60	75
Engaged in activities at my campus’s Center for Teaching and Learning	55	60	45	50
Have been sought out as an assessment “expert”	42	50	35	50
Was recruited for a curriculum or assessment committee	21	25	25	25

TABLE 6.

Professional society activities attributed to BSP (n = 98).

Presented my classroom-based research and results at a national conference	70%
Served as a reviewer of manuscripts about teaching and learning in my discipline	15%
Was nominated to a committee or special interest group about education in my discipline	13%
Published findings from my classroom-based research in my society’s journal	5%
Reviewed curricula or other teaching materials	4%

Scholars are serving as experts on curriculum committees, as advisers and mentors in campus teaching and learning centers, and as advisers to other departments working on teaching and learning in interdisciplinary programs at their respective institutions.

BSP has served as a launchpad for education research frontrunners, with several exceptional scholar alumni going on to become mentors within BSP and for other programs. As mentioned above, about 15% of scholar participants have served, or are currently serving, in leadership roles, including as chairs or members of education committees of the Ecological Society of America, American Society for Cell Biology, American Society of Plant Biologists, and other societies. The 15% figure is consistent with findings from studies of earlier and later BSP cohorts. For example, 14% of scholars from the 2005 through 2009 cohorts were elected into leadership positions in BSP society partners. These elections may account for greater acceptance by BSP society partners to advance BSP’s mission and recruit members.

Faculty professional development

Scholar respondents were asked to rank their agreement with a series of statements about student learning, assessment, research, publishing, and teaching. As shown by average scores in Table 7, scholars are moving from a stage of becoming aware of the value of SoTL to actively practicing scholarly teaching, e.g., being intentional about their classroom practices, collecting data to inform their work, and reflecting on their findings.

TABLE 7.

Average score^a for statements by assessment, research, and transitions residency participants (n = 98).

Statement	Assessment	Research	Transitions
Statements Relevant to Student Learning and Teaching in General
I am more focused on student learning, rather than teaching	4.39	4.36	4.00
I understand that student misconceptions often negatively affect their learning about a topic	4.39	4.32	3.60
I am aware that students have a variety of learning styles	4.22	4.05	3.80
I believe that grades are an accurate representation of how well students learn something	2.87	2.86	2.40
Statements Relevant to Assessment
I am familiar with a variety of classroom assessment techniques	4.83	4.41	4.40
I am able to write learning objectives for the classes I teach	4.74	3.77	3.00
I am able to assess student learning using a variety of methods	4.74	4.27	3.60
I can align learning objectives and course activities	4.74	3.91	3.60
My teaching can be changed to improve student learning	5.00	4.43	3.80
Statements Relevant to Researching
I understand the necessity of IRB approval when conducting research in my classroom	4.22	4.82	4.20
I am aware of the pros and cons of various classroom research designs	3.35	4.18	3.60
I can conduct simple quantitative analyses on data I collect	3.57	4.23	4.00
I can identify a few statistical tests to perform on data collected from my students	3.57	4.18	4.00
I understand that discipline- or classroom-based education research can be rigorous	4.52	4.73	4.40
I can write a research question that would enable me to study some aspect of teaching and learning	3.87	4.55	4.20
I am able to conduct classroom-based research, from creating a research design, to final results and conclusions	3.30	4.36	4.00
Statements Relevant to Publishing
I am interested in disseminating my classroom-based research results	4.09	4.73	4.60
I collaborate with others to help me conduct classroom-based research	3.43	4.23	4.40
I am able to draft a manuscript to describe my classroom-based research study and results	3.30	3.95	4.40

^aScores are based on an agreement scale in which 1 indicates “strongly disagree” and 5 indicates “strongly agree.”

IRB = institutional review board.

Qualitative comments also illustrate the benefits scholars derived from BSP faculty development activities:

Even those of us who have been interested in improving our teaching our entire careers largely have to rely on training ourselves. Many of us are being given more and more responsibilities by our administration but fewer and fewer resources. Programs like the BSP can give us the focused training that we need to make great strides in our professional development.

BSP gave me a solid framework, the necessary language and jargon to make me an independent learner in this field…. I wouldn’t have known even where to start or understand others who are working in the field.

Community

Scholars in the program develop connections to other scientists, such as social and cognitive scientists, who become integral to their studies and help them collaborate across multi-disciplinary groups. Several participants reported on the importance of peers and the community to continue reform efforts.

BSP really helped me develop a research question and figure out the best methodology for answering that research question. I learned that the assessments I had originally thought about using wouldn’t work for the question I was trying to answer. I also would have been going it alone instead of having a cohort of people who were implementing their own research program.

I was conducting research already before the BSP, but I would not be as far along in completing/publishing my work, and would not have as much of a network of support for moving forward.

I have discussed and shared things from the Research Residency among my peers at my home institution and see collaborative research coming from this.

My knowledge would be less if I had not participated in BSP. I would not be capable of developing research and publishing without a collaborator in the classroom.

I, along with a colleague of mine, have developed a SoTL group at my institution. Every year, the group gets larger and larger! We currently have over 20 participants from a variety of disciplines.

Communities are also becoming established outside scholar institutions, as evidenced by the 2010 formation of the Society for the Advancement of Biology Education Research (SABER), an initiative dedicated to using scientific methodology to inform the practice of teaching and ensuring that all students learn science. Two of the SABER program’s three founders are BSP alumni. SABER fosters biology education research (BER) and its dissemination by defining the standards for BER practice, supporting the BER community through training and faculty development programs, and fostering collaborations among BER investigators. Many BSP alumni are integral members of SABER and contribute to its mission. Approximately 20% of SABER conference participants annually in 2013 and 2014 are scholar alumni, and many play multiple leadership roles on their campus, in their respective disciplinary societies, and in SABER. Clearly, the impact on evidence-based teaching in biology has been felt by BSP scholars.

BSP CHALLENGES

Despite overall positive results, there have been challenges for the BSP and its participants. For instance, although scientists understand and practice their disciplinary research very well, our studies suggest a tremendous learning curve for faculty members transitioning to SoTL and evidence-based teaching. The challenges they may encounter include ambiguous and ambitious project goals, faulty research designs, inadequate literature searches, incomplete data gathering, inconclusive findings, and lack of support from colleagues and institutions that prefer traditional teaching methods. The latter challenge was cited several times in the qualitative comments:

I have tried discussing student learning research with my colleagues who are from research-intensive institutions and they just don’t get it and don’t understand the importance of quality student learning at undergraduate institutions.

Currently many universities and research institutions that train young scientists do not recognize their responsibility to prepare scientists to teach the next generation. Until this culture changes, programs like the BSP will be necessary to step in and fill this need.

Scholars may also be unfamiliar with the theoretical basis of social science and education research or appropriate methodologies and resources to assess student learning. This knowledge gap is especially challenging for faculty from community colleges or small and medium-sized institutions to overcome due to limited access to 1) social scientists and education researchers (due to fewer or smaller departments), 2) seminars and professional development opportunities (due to heavy teaching loads), and 3) extensive literature (due to smaller library holdings and institutional memberships for publications).

Another BSP challenge has been the level of participation by scholars in the program’s online database of their work. Envisioned as a repository of scholarly work, the database has suffered from rapid development of competition in the form of institutional and publisher databases and e-communities. However, the BSP has remained a viable and robust population among the life science education leadership, and program alumni regularly appear as organizers and presenters for SABER, ASMCUE, and the Gordon Research Conferences. In addition, alumni contribute to education journals in the life sciences.

Finally, although many outward signs suggest that BSP scholars are becoming leaders in life science societies and playing a significant role in advancing scholarly teaching in biology, extensive effort, time, and support are required to gain buy-in and trust among life science professional societies. In this sector, professional societies are fragmented, with representation by more than 80 disciplinary societies. As a result, a major challenge for the life sciences is ensuring coordination and fostering cohesiveness to advance undergraduate education reform. Subsequently, additional effort, time, and support are required to coordinate BSP programs and communications to represent the views of its society partners.

CONCLUSION – FUTURE DIRECTIONS

By training faculty to assess their own teaching and emphasize evidence of student learning, the BSP has built a community of individuals who are passionate about advancing undergraduate education reform in biology. The following survey comment illustrates how alumni are poised to ensure the proliferation of SoTL:

This phenomenon is contagious and has tremendous potential for spread of effect. For example, at my institution, initially I was one of two biology faculty interested in education research. After the Research Residency portion of the BSP, I was able to share my research with my immediate peers (NTT, teaching faculty). Within a year, the teaching faculty had formed a group that received a grant to both redesign our introductory biology course and also research the process. Now some of the tenure-track faculty are also researching in their classrooms, both in our department and in other departments on campus. This atmosphere of researching in the classroom, in combination with Vision and Change, has our biology department currently reevaluating our curriculum.

The program’s efforts have established a model for academic communities to create or expand their own SoTL initiatives. For instance, in 2015, Penn State University conducted a three-day education research workshop for 52 life science professors at Fudan University in Shanghai, China. Workshop leaders included BSP alumni, and the event itself was modeled on the BSP Assessment Residency Institute.

In its future efforts, the BSP plans to continue to engage societal partners. While 11 partners have joined the program, more can be done to engage all life science societies and scientists in education reform. The program will explore mechanisms such as online platforms designed to reach a broader audience and serve as a means to inform, engage, and excite all biologists about improving their teaching.