Highlights
-
•
In simulation research, we are often interested in comparing the effects of more than one independent variable.
-
•
Factorial designs allow investigators to efficiently compare multiple independent variables (also known as factors).
-
•
An example and resources are described for using a two by two factorial design in simulation research.
Keywords: Simulation, Methods, Factorial design, ANOVA
Key Points.
-
•
Simulation researchers are often interested in the effects of multiple independent variables.
-
•
Factorial designs allow investigators to efficiently examine multiple independent variables (also known as factors).
-
•
Factorial designs allow investigators to examine both main and interaction effects.
In simulation research, we are often interested in comparing the effects of more than one independent variable. For example, during the COVID-19 pandemic, educators experimented with the use of different synchronous communication platforms (e.g., online video conferencing and text chat) for debriefing after virtual simulation experiences. To contribute to the literature comparing the effects (participant outcomes) of these different synchronous communication platforms and the ideal group size when using these platforms for debriefing after virtual simulation experiences, we may want to design a research study. In this study, we want to examine the effects of both the type of communication platform and the ideal group size, and to understand the interactions between platform and group size on the dependent variables (participant outcomes).
It would be helpful to compare the effect of the two different synchronous communication platforms used for debriefing virtual simulation experiences on selected outcomes such as learning and learner satisfaction. Similarly, it would be helpful to compare the effect of two different group sizes, perhaps two to four students versus 10 to 12 students per group, during these debriefing sessions. It might also be interesting to identify if there are any interaction effects between these two variables: the type of communication platform and group size. For example, we could ask the following questions:
-
•
Are learning and learner satisfaction higher in the small group for those participants using online text chat but equal across group size for those using video conferencing?
-
•
Are learning and learner satisfaction consistently higher in the large groups regardless of which synchronous communication platform is used?
The question is how can we do all these comparisons in one experiment? Factorial designs allow investigators to efficiently compare multiple independent variables (also known as factors). The scenario described previously represents a two-by-two factorial design where synchronous communication platform has two conditions or possible options: online video conferencing and text chat. Group size also has two conditions: small (two to four participants) and large (10-12 participants). This creates a total of four conditions: small group online video conferencing (A), small group text chat (B), large group video conferencing (C), and large group text chat (D). We illustrate all four conditions using a table like this one.
| Online Video Conferencing | Text Chat | |
|---|---|---|
| Small group (2-4 participants) | A | B |
| Large group (10-12 participants) | C | D |
Two-way analysis of variance (ANOVA) can be used to assess the main effects of group size and synchronous communication platform as well as possible interactions between these variables. Additional analyses can also be completed to explore whether there is a cut-point at which group size is optimized. It is also possible to include a comparison of an additional group size, perhaps 18 to 20 participants, or an additional synchronous communication platform such as voice over internet protocol. However, each additional variable decreases the sample size in each box and makes the statistical analysis potentially less reliable. If the total sample size for the example in the table was 100 learners, we would have approximately 25 participants in each condition for comparison (100 learners/4 conditions = 25 learners per condition). However, if we added another group size and another synchronous communication platform, we would dilute that sample size over nine conditions and only have 11 or 12 participants in each condition for comparison (100 learners/9 conditions = 11.11 learners per condition).
Factorial designs are a simple, yet elegant, way of comparing the main effects of multiple independent variables and exploring possible interaction effects. We hope this example of a two-by-two factorial design will inspire you to efficiently compare the effects of two variables, each with two conditions, on simulation outcomes. Happy researching!