TABLE 2.
Rubric for scoring the EDAT (Sirum and Humburg 2011)a
| Rubric item | Rubric answer |
|---|---|
| 0 | Did not answer |
| 0.0 | Answered, but no points |
| 1 | Recognition that an experiment can be done to test the claim (vs. simply reading the product label). |
| 2 | Identification of what variable is manipulated (independent variable is ginseng vs. something else). |
| 3 | Identification of what variable is measured (dependent variable is endurance vs. something else). |
| 4 | Description of how dependent variable is measured (e.g., how far subjects run will be measure of endurance). |
| 5 | Realization that there is one other variable that must be held constant (vs. no mention). |
| 6 | Understanding of the placebo effect (subjects do not know if they were given ginseng or a sugar pill). |
| 7 | Realization that there are many variables that must be held constant (vs. only one or no mention). |
| 8 | Understanding that the larger the sample size or number of subjects, the better the data. |
| 9 | Understanding that the experiment needs to be repeated |
| 10 | Awareness that one can never prove a hypothesis; that one can never be 100% sure; that there might be another experiment that could be done that would disprove the hypothesis; that there are possible sources of error; that there are limits to generalizing the conclusions (credit for any of these) |
aEach item included in a student response to a writing prompt was given 1 point, except for the first two items (rubric items 0 and 0.0). All responses in which students did not answer were removed from analysis. The sum of responses for items 1–10 was the composite score; the sum of responses for items 1–4 was the basic understanding score; and sum of responses for items 5, 8, and 10 was the advanced understanding score, based on Shanks et al. (2017). The writing prompt for both pretest and posttest was: “Advertisements for an herbal product, ginseng, claim that it promotes endurance. To determine if the claim is fraudulent and prior to accepting this claim, what type of evidence would you like to see? Provide details of an investigative design.”