Table 2.
Original versus replication methodological comparison.
|
| |||
|---|---|---|---|
| ORIGINAL | REPLICATION | REASON FOR CHANGE | |
|
| |||
| Participants | Undergraduate students from Hong Kong University of Science and Technology and University of Colorado. | Participants from CloudResearch/Amazon MTurk. | Larger more diverse sample. Addressing sample concerns and allowing for exploratory analyses comparing effects across studies. |
|
| |||
| Study 1, 2 and 5 were done separately with different participants. | Study 1, 2 and 5 were done in the same survey with the same participant. | ||
|
| |||
| Delivery | Paper questionnaires | Online questionnaire using Qualtrics | |
|
| |||
| Questions | The original studies did not use any comprehension checks or instructional manipulation checks. | We used comprehension and instructional manipulation checks in our replication. | To ensure that participants read and understood the materials. |
|
| |||
| Materials | In Study 5, a class on opportunity cost was delivered to those in the education condition. | A passage about opportunity cost along with questions about that passage as instructional manipulation checks were presented. | To adjust to an online sample, we used a passage that participants read instead of a class. |
|
| |||
| Scale | In Study 5 the preference scale was originally from 1 to 9 and presented as such. | We adjusted the presentation of the scale to 4/0/4 instead of 1 to 9. | Avoid biasing participants in a certain direction. |
|
| |||
| Order of studies | Study 1 –> Study 2 –> Study 5 | Randomized the order of studies 1 and 2 only, but not study 5. Study 5 is presented last at the end of the experiment. | To address potential impact of presentation order. |
|
| |||
Note. Effect size for Study 1 and 2 is ϕc and for Study 5 – ; see “Effect sizes calculation” section in the supplementary materials.