Table 3.
Exam equivalence analyses
| Independent ratings | ||||||
|---|---|---|---|---|---|---|
| Discussed- consensus | All three agree | Two of three agree | Sequential ratings | Nonsequential ratings | ||
| a. Percentage agreement among Bloom's taxonomy raters. | ||||||
| Percentage of total ratings | 7.3 | 26.4 | 51.1 | 9.9 | 5.2 | |
| “Discussed-consensus” means that questions were rated independently and then discussed to reach a consensus; “Independent ratings” were not discussed among raters. “Sequential ratings” were questions that received three ratings that differed by one Bloom's level (e.g., a 2, 3, and 4); “Nonsequential ratings” were questions that received three ratings that differed by more than one Bloom's level (e.g., a 2, 3, and 5). | ||||||
| b. Weighted Bloom Indices | ||||||
| Spring 2002 | Spring 2003 | Spring 2005 | Autumn 2005 | Autumn 2007 | Autumn 2009 | |
| Midterm 1 | 50.8 | 48.5 | 45.3 | 58.9 | 54.4 | 53.7 |
| Midterm 2 | 36.1 | 51.6a | 51.6a | 46.8 | 50.8 | 54.7 |
| Midterm 3 | 50.2 | |||||
| Final (or Midterm 4) | 48.1 | 54.3 | 45.4 | 51.6 | 51.6 | 55.3 |
| Course averageb | 45.8 | 52.1 | 46.9 | 52.2 | 52.1 | 53.5 |
| c. PES values (predicted percent correct) | ||||||
| Spring 2002 | Spring 2003 | Spring 2005 | Autumn 2005 | Autumn 2007 | Autumn 2009 | |
| Midterm 1 | 70.8 | 73.0 | 71.9 | 67.8 | 64.9 | 66.0 |
| Midterm 2 | 73.0 | 68.0a | 68.0a | 72.1 | 67.7 | 67.0 |
| Midterm 3 | 68.5 | |||||
| Final (or Midterm 4) | 69.4 | 70.0 | 71.8 | 71.0 | 69.6 | 68.6 |
| Course averageb | 70.6 | 70.2 | 70.9 | 70.5 | 68.0 | 67.5 |
aIdentical exams.
bCourse averages were computed from data on all exam questions from that quarter. (They are not the averages of the indices from each exam.)