Using the Theory of Planned Behavior as a Framework for the Evaluation of a Professional Development Workshop

Abstract

This purpose of this study was to use a theoretical framework based on several decades of attitudinal research to assess the intentions of Microbial Discovery Workshop participants to incorporate the inquiry activities presented at the workshop into their curricula, to evaluate the participants actual use of these activities after the workshop, and to uncover the barriers and enablers the participants faced in doing so. As a framework, the theory of planned behavior was ascertained to be an appropriate means of assessment and it was revealed that participants’ intention to use the workshop activities significantly correlated with their actual use. The participants’ attitudes toward using the activities influenced their use more than the participants’ perceptions of the social pressures that would influence their decision to use the activities or their belief as to how easy or difficult it would be to incorporate a given activity. The participants were found to be highly self-efficacious pertaining to their ability to implement the activities, but perceived self-efficacy was not a significant predictor of the participants’ intentions to incorporate the activities into their teaching-learning repertoire. The study also uncovered other behaviors the participants displayed as a result of attending the workshop consistent with the goals and objectives of the workshop organizers.

The Microbial Discovery Workshop (MDW) is part of an outreach program administered by the Committee on Precollege Education of the American Society for Microbiology (ASM) and funded by the Foundation for Microbiology, The Pfizer Foundation, Inc., and the ASM. It is a multifaceted program to demonstrate to educators at every level how to use microorganisms to stimulate interest in science and the microbial world. The hallmark of the program is hands-on, inquiry-based explorations featuring microbes that are designed to develop process, data analysis, and critical and creative thinking skills in students consistent with the National Science Education Standards (22). Components of the MDW program include national leadership workshops, regional workshops, planning grants for hosting a workshop, and a collection of appropriate curriculum materials.

The national leadership workshops, following a train-the-trainer model, prepare teams for regional outreach activities. Each team consists of an ASM scientist, who is often a college educator, and a precollege teacher. The teacher and scientist must apply as a team, increasing the likelihood that they will work together to promote use of inquiry-based activities featuring microbes in future endeavors. Workshops have been held yearly since 1996. The purpose of this study was to evaluate certain aspects of two workshops held in the summer of 1999 within the framework of the theory of planned behavior.

The theory of planned behavior (TPB) was employed as a means to measure the intentions of participants to implement the instructional strategies presented at the workshop and to uncover the perceived enablers and barriers the participants faced in making the decision to incorporate the strategies and activities into the curriculum. Pioneering research conducted by social psychologists Fishbein and Ajzen (13), Fishbein (12), and Petty and Cacioppo (23) provided the conceptual foundation and empirical evidence necessary to understand the relationship between attitude and behavior and to develop the TPB. The usefulness of attitude and behavior theories is based on three considerations: (i) they aid the understanding and prediction of behavior, (ii) they direct the creation of instruments to measure the variables that determine behavior, and (iii) they guide the development of belief-based intervention techniques. An overview of the TPB variables and the relationships among them is depicted in Fig. 1.

FIG.1.

Relationship between variables in the theory of planned behavior.

The TPB uses a measure of a subject’s attitude toward the behavior (AB), the social climate surrounding the subject (subjective norm, SN), and the subject’s level of perceived control over the ability to engage in the behavior (PBC) to predict behavioral intention (BI). The TPB is a multifaceted combination of factors and the circumstances that affect them and has been proposed as a means of becoming ever more specific in the prediction of behavior. The power of the TPB in predicting behavior can be tied to its specificity, in which the variables are addressed at the same level in assessing attitude as they are in addressing behavior. Variables related to attitude and behavior must consist of four specific elements: the action, the target of the action, the context of the action, and the time in which the action occurs. In this study, the action was the participants’ intended implementation of the MDW activities, the target of the action was their science students, the context of the action was the science curriculum, and the time frame was the first half of the 1999–2000 school year.

Numerous investigators have applied the TPB to science education in the areas of laboratory learning (25), intention to enroll in upper class science courses (7, 21), science fair participation (8), and teachers’ intentions to implement state standards (14). Other studies include teachers’ intentions to engage in collaborative reflective practice (9), teachers’ intentions to use investigative teaching methods in physics instruction (6), preservice teachers’ intentions to employ hands-on science azctivities during their first years of employment (17), Chinese preservice teachers’ intentions to teach about the environment (18), intentions of science teachers to use science investigations (16), teachers’ motivations to motivate students (15), intent of oral surgeons to participate in continuing education (24), and intentions of teachers to use microcomputer science laboratory interface materials (26). In each of these studies, attitude toward the behavior generally was the single most important predictor of behavioral intention.

The self-efficacy theory of Bandura (3, 4, 5) was also investigated in this study as a possible means of enhancing the predictive capacity of the theory of planned behavior. Like attitude and behavior theories, social cognitive theories assume that goal-directed behavior is a purposive action rooted in cognitive activity. The hypothesis central to these theories is that humans systematically utilize and process information and thereby self-regulate their behavior. Bandura’s (4) social cognitive theory is a global framework intent upon explaining self-regulation of action and suggests that behavior is influenced by three self-regulatory mechanisms operating in concert: (i) perceived self-efficacy expectations, (ii) outcome expectations, and (iii) personal goal setting. A measurement of self-efficacy was incorporated into the TPB equation used in this study based on the rationale that the variable of PBC, as defined in the TPB, lacked an adequate measure of perceived self-efficacy in its measurement.

METHOD

The study focused on four questions: (i) is the theory of planned behavior an appropriate conceptual framework for evaluating the participants’ intents to use the workshop activities, (ii) what are the enablers and barriers to the subsequent use of the activities, (iii) what is the role of participant perceived self-efficacy (SE) in using the activities and in predicting behavioral intention (BI), and (iv) what other behaviors did the participants display as a result of participation in the workshop.

The methodology used in this study is based on more than twenty years of research. The process by which the TPB is implemented is hierarchical and specific parameters and procedures are to be followed. To collect the data necessary to answer the questions, the investigator designed and administered five instruments (contact author for instruments).

1. A nine-item, open-ended Belief Survey (BS) to elicit past workshop participants’ salient beliefs that influenced their decision to implement MDW activities in their teaching repertoire. The BS was administered as part of a pilot study in which the subjects were participants of the 1996, 1997, and 1998 workshops (n = 86).

2. A 63-item Likert scale Behavioral Intention Questionnaire (BIQ) generated from the analysis of the BS. The questionnaire assessed behavioral intention (BI), attitude toward the behavior (AB), subjective norm (SN), perceived behavioral control (PBC), and perceived self-efficacy (SE). This survey was administered near the beginning of the 1999–2000 school year to the participants of the 1999 workshops (n = 27).

3. A 22-item Measure of Self-Efficacy (MSE) administered to the subjects of the study to assess their level of confidence in the implementation of each of the workshop activities. This survey was administered along with the BIQ (n = 27).

4. A seven-item Behavioral Checklist (BC) to assess participants’ attainment of workshop goals and objectives. This survey was administered at the end of the first half of the 1999–2000 school year to the 1999 workshop participants (n = 23).

5. A 21-item Level of Implementation Checklist (LOIC) to ascertain the extent to which the participants implemented the activities presented at the workshop during the first half of the school year. The LOIC was administered at the end of the first half of the 1999–2000 school year to the 1999 workshop participants (n = 23).

Responses to the open-ended BS items were collected, counted, and content-analyzed per the method described by Ajzen and Fishbein (1). Table 1 lists the salient beliefs derived from the BS. To reduce the effects of researcher bias when content analysis of the BS data was performed, a microbiology professor, who is also an educational researcher, was asked to independently collate, content-analyze, and select salient beliefs from a sample of the data. Inter-rater reliability, as determined by percentage of agreement, was 89% for the AB section, 80% for the SN section, and 88% for the PBC section.

TABLE 1.

Salient beliefs identified from the Belief Survey

Attitude toward the behavior (outcomes associated with use of MDW activities)

Activities are more fun and interesting

Students are more engaged and more likely to take ownership of learning

Activities align well with the curricula used by the educators

Activities are adaptable to varied learning styles and age groups

Activities promote cooperative learning among students

Activities foster awareness of microbial diversity

Activities model real science and science problem-solving

Activities are inexpensive to implement

Activities are easy to implement

Activities take extra time to implement

Activities lead to loss of control of the teaching and learning process

Activities mean less material can be covered

Subjective norm (persons associated with MDW implementation)

Students

Administrators or supervisors

Colleagues

Curriculum or course coordinator

Department chair

Perceived behavioral control (factors associated with MDW implementation)

Additional funds or supplies

Additional preparation time

Additional class or lab time

Availability of additional new MDW activities

More opportunities to communicate with others using MDW activities

More support from administration

Requirement to teach to a specific curriculum or standardized exam

The Behavioral Intention Questionnaire (BIQ) was created once the salient beliefs were ascertained from the BS. Two questionnaire items were written for each belief; one linked the outcome to the behavior, called the behavioral belief (b_i), and the other was an evaluation of the outcome of that behavior (e_i). For example, if a salient belief derived from the first open-ended question on the BS was determined to be “using MDW activities allows engagement of students in more hands-on activities,” the behavioral belief statement would be written as: “my students will be engaged in more hands-on activities if I use the MDW activities as opposed to other activities that I use.” The statement would be evaluated by the participants using a seven-point Likert scale ranging from extremely likely to extremely unlikely. The evaluation of the outcome statement would be written as: “engaging students in more hands-on activities is…” and would be evaluated on the scale from extremely good to extremely bad. For purposes of data analysis, responses were assigned values from −3 to +3 (scale midpoint = 0) and the values for the behavioral belief and the evaluation of the outcome were multiplied to weight each response. All responses within the AB section finally were summed to obtain the indirect measure of AB.

In the same fashion, referents associated with the SN were derived from the BS and for each referent a pair of statements was written. For example, since students were determined to be influential in the decision to use MDW activities, the subjective norm paired statements read:

1. My students think that I should use MDW activities as part of my science teaching.

2. I want to do what my students think I should do concerning my teaching decisions in science.

Statement 1 is termed the normative belief (nb_j) and statement 2 measures motivation to comply (mc_j). Each statement was evaluated on a seven-point bipolar scale ranging from extremely likely (scored as +3) to extremely unlikely (scored as −3). The sum of the normative belief pairs (nb_jmc_j) served as an indirect, belief-based estimate of subjective norm.

Likewise, each event associated with PBC resulted in an item pair on the BIQ. One item linked the event to the behavior (called the control belief, cb_k) and the other assessed the likelihood of occurrence of the event (lo_k).

Reliability of the BIQ was ascertained by administering the questionnaire to participants of shorter regional MDWs and past participants of MDWs (n = 43) twice within a 28-day period. Responses were grouped according to question clusters and compared. The test-retest scores for each cluster were shown to be significantly similar (p < 0.01, two-tailed). In addition, internal consistency of survey responses was ascertained by subjecting the test clusters to a Crohnbach’s alpha analysis. Results revealed high alpha levels among the clusters (α = 0.70 to 0.96) indicating that the questions within the clusters were measuring consistent constructs.

This study also included a survey to measure self-efficacy (MSE) to determine the role of self-efficacy (SE) in the prediction of BI. Two steps were taken to measure SE. First, the direct measure of SE consisted of three general questions on the BIQ. These questions assessed the extent to which the participants (i) believed they could do an excellent job in implementing the workshop activities with their students, (ii) believed they had enough microbiology content knowledge to successfully implement the activities, and (iii) believed they knew enough about inquiry teaching and learning to successfully implement the activities.

The second step to assess self-efficacy involved presenting a list of the activities that were presented at the workshops to the participants who were then asked to first indicate their intent to use the activity during the first half of the 1999–2000 school year and second to indicate their level of confidence in using that activity. The level of confidence was rated on a scale of 1 to 10, based upon recommendations by Bandura (3).

The BC was simply a checklist of behaviors based on the general objectives of the workshop and the specific objectives of the 1999 workshops. To develop the instrument, a list of behaviors encouraged by workshop participation was drafted and distributed to a panel of 15 experts who were past coordinators of national leadership workshops. Each panel member was asked to identify those behaviors that describe teachers who actively use the MDW strategies and make suggestions for improvement to the list. The final list ascertained the levels of (i) participant sharing of activities with fellow educators, (ii) participant sharing of activities at professional conferences, (iii) participation in the MDW listserve, (iv) interaction of teacher and microbiologist pairs after the workshop, (v) membership in ASM, (vi) submission of activities to the ASM MicrobeLibrary, and (vii) use of the activities presented at the workshop.

The final instrument was the Level of Implementation Checklist (LOIC), which was designed to determine if the MDW participants intention to use the activities as reported on the MSE correlated to their actual use. Use of each activity was ascertained along a continuum which included: implementation of the activity exactly as presented at the workshop, implementation of a modification of the activity, nonuse of the activity but possibilities for future use being identified, and nonuse of the activity with no possibility of future use being identified.

RESULTS

The analysis of data in this study progressed in a sequence of steps moving from the broad categories to the more specific categories. While the study began with the collection of salient beliefs and progressed sequentially through collection of behavior data, data analysis began with the behavior scores and their relationship to behavior intention scores which, in turn, led to the analysis of specific beliefs held by the 1999 MDW participants. Figure 1 depicts the overall relationship between the variables in the path of data analysis.

Behavior (B) was scored dichotomously as either performed or not performed according to responses on the Behavioral Checklist (BC). The participant was said to have performed the behavior if he either redesigned his curriculum around MDW activities, replaced some of his regular activities with MDW activities, or used some of the activities as supplements to regular activities. Twenty participants responded to the BC and 19 (95%) indicated that they had used the activities in some fashion as described above. Further analysis of the responses revealed that half of the participants used the activities as supplements to their regular activities, 35% replaced regular activities with MDW activities, and 10% redesigned their curriculum around MDW activities.

Behavioral intention (BI) was derived from analysis of the BIQ which revealed that 70% (16 of 23) of the subjects were quite likely to use some of the activities presented at the workshop during the first half of the school year. This response was weighted by a measure of the participants’ reports of their likelihood to carry out intended behaviors. The intention to use the workshop activities was ascertained at the beginning of the school year and, in this study, served as a predictor of participant behavior approximately 16 weeks later at the end of the first half of the school year or at the completion of the fall semester in the case of the college instructors. A positive and statistically significant correlation was shown to exist between the intention of using the activities and their actual use (r = 0.47, p<0.05).

Having established the relationship of behavioral intention to behavior as significant, data analysis moved to determine the source of behavioral intention by determining the relationship between the direct measures of the variables (AB, SN, and PBC) and BI. Based on the direct measures, attitude toward using the activities in the classroom during the first half of the school year (AB) was highly favorable (M = 8.57, SD = 3.04). Additionally, the participants did not feel particularly influenced by others who are important in their teaching decisions, as indicated by responses to the questions that measured SN (M = 0.22, SD = 4.54). Participants agreed, although not strongly, that they would have difficulty using the activities during the first half of the school year if they desired, as evidenced by the measure of PBC (M = −1.09, SD = 2.65). A significant correlation was established between the combination of the direct measure variable scores and BI (r = 0.52, p < 0.05).

Multiple regression analyses were conducted between the direct measures of the variables and the individual salient beliefs comprising the indirect measure to determine which belief contributed most to the formation of the variable. A hierarchical regression analysis (Table 2) of the variables on BI revealed that AB was the only variable that made a significant contribution to BI.

TABLE 2.

Hierarchical regression analysis of direct measure variables, AB, SN, and PBC, on behavioral intention

Variablea	B	SE B	β	p
Model 1
AB	0.9	0.33	0.51	0.00

^aVariables excluded were SN and PBC. n = 23. Model 1: R² = 0.26, F (1, 21) = 7.3.

Analysis of the direct measure of self-efficacy on the BIQ indicated that participants believed most strongly that they knew enough microbiology to successfully implement the activities presented at the workshop, but they also felt confident they knew enough about inquiry methods of teaching to incorporate the activities into their teaching. On average, they believed it was quite likely that they could do an excellent job incorporating the activities into their teaching (Table 3).

TABLE 3.

Descriptive statistics for the components of the direct measure of self-efficacy

Belief	M	SD	Minimum	Maximum
I could do an excellent job implementing the activities	1.87	1.39	−3	3
I know enough microbiology to implement the activities successfully	2.13	1.29	−3	3
I know enough about inquiry teaching methods to implement the activities successfully	1.91	1.04	−3	3
Sum of the belief measuresa	5.91	3.05	−9	9

^aThe sum of the belief measures was subsequently used as the Self-Efficacy (SE) variable in additional analyses.

It was hypothesized that the addition of the SE variable to the TPB equation would improve the ability of the direct measure variables to predict BI. The results of the regression analysis are shown in Table 4 and indicate that the addition of the SE variable did not enhance the prediction of behavioral intention. The principle predictor of behavioral intention in the TPB model remained attitude toward the behavior in this study.

TABLE 4.

Stepwise multiple regression analysis of the measure of the self-efficacy variable and behavioral intention on behavior

Variablea	B	SE B	β	p
Model 1
SE	0.05	0.01	0.65	0.00

^aVariable excluded was BI. Model 1: R² = 0.56, F(1, 18) = 13.38.

The SE variable was shown to be significantly correlated with actual behavior (r = 0.75, p < 0.01). Therefore, to investigate the ability of SE and BI to predict B, a hierarchical regression analysis was done. Table 4 shows that self-efficacy was a better predictor of behavior than behavioral intention in this study, although lack of variability in the data occludes this finding.

Lastly, to investigate the possibility that perceived self-efficacy might influence the participants’ intentions to use the activities presented at the workshop, a correlation analysis was performed between intent and actual use of each activity. It was found that MDW participants were highly self-efficacious and confidence in their ability to successfully implement the activity with their students had no influence on their intention to use 18 of the 20 activities presented at the workshop. A correlation analysis comparing the confidence level of the participants and the actual use of the activity showed that perceived self-efficacy had no influence on using any of the activities.

DISCUSSION

It was the intent of this study to first investigate the theory of planned behavior as a framework for evaluating MDW participants’ intentions to incorporate the workshop activities into their curriculum. Within the tenets of the TPB, three predictors are historically investigated: (i) the attitude toward the behavior, (ii) the subjective norm, and (iii) perceived behavioral control. Regression analysis of the data in this study demonstrated that the combination of the three variables was a significant predictor of behavioral intention. These measures were obtained through the answers to eight questions on the Behavioral Intention Questionnaire (BIQ), suggesting that the survey could be reduced in length and still gather important, but limited, information about the participants’ intentions to use the activities.

Not only was it found that the combination of variables measured in the study significantly predicted behavioral intention, but the measure of behavioral intention also was a significant predictor of actual behavior. The validation of intention as a predictor of behavior in this study continues to pave the way for the use of behavioral intention scores as a reliable measure of behavior in science education. Unfortunately, very few studies employing the TPB in science-related behavior have examined the correlation of actual behavior to behavioral intention. Crawley and Koballa’s investigation of Hispanic Americans’ intentions to enroll in high school chemistry, for example, found that intention accounted for only 10% of the variance in behavior (unpublished data). Stronger relationships were found in Myeong and Crawley’s (21) study of Korean students’ science track choices; it was shown that intention explained 56% of the variance in boys’ choices and 37% of girls’ choices. Smith (26) found that intention accounted for 40% of the variance when examining teachers’ implementations of computer interface technology in the classroom. In this study, intention explained 30% of the variance in behavior. These results may suggest that simply ascertaining the intention of participants to incorporate the activities into their teaching repertoire may be sufficient to determine that the activities are being used. It is clear that continued investigations to confirm the link between BI and B are indeed necessary.

Secondly, the investigator sought to elucidate the enablers and barriers to activity use. Each variables’ power in the prediction of BI was investigated and it was discovered that only attitude toward incorporating the activities (AB) played a significant role in the participants’ decisions to use them. This corroborates the finding of Crawley (6) who examined science teachers’ intentions to use investigative teaching methods after attending a five-week institute. Regardless of the availability of resources and time, intent to perform the behavior appeared to be totally under the control of the participants, with little need for social support or validation from those important to the participant in teaching decisions. The findings were also in agreement with those of Haney, Czerniak, and Lumpe (14), who found that the obstacles and enablers that teachers encounter matter less to them than their beliefs regarding the outcomes associated towards inquiry-based instruction as well as the use of microbes in inquiry-based instruction.

It was not surprising that SN and PBC had no influence on BI while AB did in this study. Item analysis of the BIQ showed that the participants had very positive attitudes towards using the MDW activities, they were not greatly motivated to comply with the wishes of others, and they were not likely to attempt to change things beyond their control. Marlow and Stevens (20) reported that teachers face a myriad of problems when implementing inquiry activities. They often feel that inquiries are too difficult for the student to handle, overly time consuming, successful only with high-ability students, and cause classroom disruptions. Along with these concerns, teachers are pressured to prepare students for standardized assessments which, to them, means teaching facts as opposed to developing understanding of concepts, principles, and theories. These detractors to using inquiry activities were also identified in the Belief Survey used in this study. Regardless of these barriers, however, MDW participants still used many of the activities despite the short period of time in which they were evaluated. According to the results of the study, their positive attitude played the only significant role in their decision to implement the activities and may suggest that teacher attitude is more important when it comes to implementing workshop goals then the wishes of others or restrictions such as lack of time or funds. It is possible that these results may change over a period of time. Loucks-Horsley and colleagues (19) reported that once teachers have begun teaching what has been learned at a professional development workshop, it is essential that they have time, resources, and support to discuss concerns with other and access support personnel. In this study and in Crawley’s (15), measurements were taken shortly after the workshop and point to a need for additional, longitudinal studies.

The lack of predictive power for the construct of PBC may have been integral to the design of the workshop activities themselves. All MDW activities are developed specifically to use readily available resources and supplies, with many of the supplies found at a grocery or discount store. Item analysis of the BIQ indicated that participants did not feel strongly that lack of supplies would hinder their use of the activities. They did express some concern about lack of time to implement the activities given the class or lab time they were allotted.

Item analyses of the belief statements that comprised the indirect measure of AB revealed that participants had a very positive attitude toward the activities, believing that they were more fun and interesting than other activities they use and that they align quite well with their current curriculums. In addition they were easily adaptable to different learning styles and somewhat adaptable across different age groups. The participants did not feel strongly that the activities modeled real science or promoted cooperative learning to a greater extent than activities that they already used. They did feel that they would have slightly less control of their students and that they would be able to cover less material if they used the activities, but neither was a significant detractor to implementing the activities. In general, there does not appear to be a need for the workshop coordinators to improve the activities from a customer satisfaction standpoint.

The third intention of this study was to investigate the effect of the addition of the self-efficacy variable on prediction of behavioral intention and behavior. Although the overall level of self-efficacy of the participants was high as reported on the BIQ, the level of self-efficacy regarding each individual activity as reported on the LOIC had no bearing on whether or not it was used. Additionally, incorporation of the SE variable into the TPB formula did not improve its predictive capacity; AB remained the sole significant predictor. It was also found that perceived self-efficacy was a better predictor of behavior than behavioral intention, suggesting that workshop organizers should take steps to promote participant

Finally, the study sought to uncover other behaviors that the participants displayed as a result of attending the workshop. Analysis of the BC revealed that workshop alumni embraced the goals and objectives of the workshop. Within 16 weeks of the workshop, participants were sharing the activities with others at professional conferences, communicating with their teacher-microbiologist partner, engaging in electronic conversations on a dedicated listserve, and sharing the activities with their colleagues.

In summary, although the TPB was shown to be an appropriate conceptual framework for the evaluation of workshop participants’ intentions to use the workshop activities, it is probably too cumbersome for practical use as an end-of-workshop questionnaire. Results of this study suggest, however, that intention can be predicted by directly measuring only the three constructs, AB, SN, and PBC, or by measuring SE. This would significantly reduce the length of the BIQ. Support for truncating the questionnaire can be found in a study by Dzewaltowski, Noble, and Shaw (11) who assessed only direct-measure variables in their study of intention to participate in physical activity. Their rationale was supported by a previous study (10) in which the indirect-measure variables were shown not to contribute significantly to the prediction of intention over the direct measures. Indeed, even TPB founders Ajzen and Madden (2) demonstrated no predictive differences in the direct measures over the belief-based indirect measures. The danger in elimination of the indirect-measure variables is the loss of valuable information concerning the enablers and barriers to the use of the activities. self-efficacy to ensure future use of the activities.

The theory of planned behavior, as it was employed in this study, did not assess whether teachers experienced a change in their approach to teaching science; it simply assessed whether they intended to use the activities and what factors influenced their decisions. However, because the MDW activities employ inquiry as the primary method for teaching and learning, and the teachers clearly used the activities, they may be closer to attaining the National Standards for Science Education (22) recommendations for a problem-solving, inquiry approach in the teaching of science than they were before the workshop. Future studies should examine exactly how the activities are used in the classroom, the longevity of their use, their impact on teaching style, and their influence on student learning.