Table 2.
Sentiments about experiments by vignette and population
| Negative sentiment | Positive sentiment | |||||||
|---|---|---|---|---|---|---|---|---|
| Experiment Aversion | A/B Effect | More people averse than appreciative? | More people rank AB test worst than best? | More people rank AB test best than worst? | More people appreciative than averse? | Reverse A/B Effect | Experiment Appreciation | |
| (A) Lay Sentiments About Healthcare Experimentation | ||||||||
| Catheterization Safety Checklist | ✓ | ✓ | ✓ | ✓ | ||||
| Best Anti-Hypertensive Drug | ✓ | ✓ | ✓ | |||||
| (B) Lay Sentiments About Covid-19 Healthcare Experimentation | ||||||||
| Ventilator Proning | ✓ | ✓ | ✓ | |||||
| School Reopening | ✓ | ✓ | ✓ | |||||
| Masking Rules | ✓ | ✓ | ✓ | ✓ | ||||
| Intubation Safety Checklist | ✓ | ✓ | ✓ | ✓ | ||||
| Best Corticosteroid Drug | ✓ | ✓ | ||||||
| Best Vaccine | ✓ | ✓ | ||||||
| (C) Clinician Sentiments About Covid-19 Healthcare Experimentation | ||||||||
| Masking Rules | ✓ | ✓ | ✓ | ✓ | ||||
| Intubation Safety Checklist | ✓ | ✓ | ✓ | ✓ | ||||
| Best Corticosteroid Drug | ✓ | ✓ | ✓ | |||||
| Best Vaccine | ✓* | ✓ | ||||||
Note. The A/B Effect refers to the difference between the average rating of the two interventions and the rating of the A/B test. Experiment Aversion refers to the difference between the lowest-rated intervention and the rating of the A/B test. The Reverse A/B Effect refers to the difference between the rating of the A/B test and the average rating of the two interventions. Experiment Appreciation refers to the difference between the rating of the A/B test and the rating of the highest-rated intervention.
Checkmarks (✓) represent a stastically significant effect at p < .05. In one case, the checkmark is followed by an asterisk (*). This indicates that while the effect reaches statistical significance, the effect size is very small and might have only reached significance due to the large sample size (three times as large as that for other vignettes).
Variables to the right of the thick vertical line are the reverse of those on the left. If no checkmark appears in either of the corresponding columns to the left and right of the thick vertical line (e.g., “More people rank A/B test worst than best?” and “More people rank A/B test best than worst?”), that means that there is no significant difference (e.g., there is no statistically significant difference between the proportion of people ranked that A/B test worst and the proportion of people who ranked the A/B test best).