Table 3. Timing summaries of web-based software by package, platform, and browser.
The Var(iability) measures are the inter-trial standard deviations of the various latencies for that configuration. The table is sorted by the mean of those variabilities (Mean Precision). The Lag/Bias measures are the mean latency values, for that configuration. In the case of audiovisual sync, a negative bias indicates the audio led the visual stimulus, a positive bias means the visual led the audio. Cells are colored pale green where times are “good” (arbitrary cut-offs of <=1 ms for precision and <=5 ms for lag). Cells are a dark pink where the timing is notably “bad” (>10 ms for precision, >20 ms for lag). An interactive version of the table can be found at https://psychopy.org/timing/2020/table3.html and this also provides links to plots of the data distributions.
| Package | Platform | Browser | Mean Precision (ms) | Reaction times | Visual durations | Audiovisual sync | |||
|---|---|---|---|---|---|---|---|---|---|
| Var (ms) | Lag (ms) | Var (ms) | Lag (ms) | Var (ms) | Lag (ms) | ||||
| PsychoPy | Win10 | Chrome | 1.36 | 0.39 | 43.95 | 0.67 | −2.08 | 3.01 | 65.32 |
| Gorilla | Win10 | Firefox | 1.84 | 1.11 | 24.83 | 2.67 | 1.35 | 1.73 | 88.27 |
| Gorilla | macOS | Firefox | 2.18 | 4.47 | 30.34 | 0.94 | 1.16 | 1.12 | 38.43 |
| PsychoPy | Win10 | Edge (Standard) | 2.22 | 2.03 | 42 | 0.93 | −2.28 | 3.69 | 56.19 |
| PsychoPy | macOS | Firefox | 2.65 | 1.17 | 67.01 | 3.38 | 0.24 | 3.4 | −10.21 |
| PsychoPy | macOS | Safari | 2.66 | 1.05 | 33.5 | 4.26 | 0.49 | n/a | n/a |
| PsychoPy | Win10 | Firefox | 2.76 | 1.96 | 40.97 | 2.42 | −2.61 | 3.9 | 58.93 |
| Gorilla | Ubuntu | Firefox | 2.76 | 4.71 | 24.71 | 2.35 | 2.05 | 1.23 | −30.61 |
| jsPsych | macOS | Safari | 3.39 | 0.66 | 31.31 | 4.39 | 3.09 | 5.11 | −23.48 |
| jsPsych | Win10 | Edge (Chromium) | 3.85 | 1.74 | 15.19 | 4.21 | 2.97 | 5.6 | 44.51 |
| Testable | Win10 | Firefox | 3.92 | 3.87 | 31.36 | 2.94 | 1.91 | 4.95 | 76.32 |
| PsychoPy | Ubuntu | Firefox | 3.97 | 1.57 | 42.5 | 4.97 | 1.02 | 5.36 | 190.45 |
| Testable | Ubuntu | Firefox | 4.05 | 3.97 | 31.57 | 3.25 | 1.84 | 4.92 | −44.36 |
| PsychoPy | Ubuntu | Chrome | 4.14 | 0.2 | 66.98 | 1.77 | 1.77 | 10.45 | 187.19 |
| Lab.js | macOS | Firefox | 4.2 | 0.97 | 16.38 | 8.61 | 19.51 | 3.01 | 4.26 |
| PsychoPy | Win10 | Edge (Chromium) | 4.24 | 1.04 | 46.01 | 3.03 | −3.36 | 8.66 | 63.3 |
| jsPsych | Ubuntu | Chrome | 4.63 | 3.23 | 48.29 | 4.33 | 4.05 | 6.34 | 27.73 |
| PsychoPy | macOS | Chrome | 4.84 | 3.22 | 35.29 | 6.47 | −0.3 | 4.82 | −6.86 |
| Lab.js | Ubuntu | Chrome | 5.12 | 8.27 | 31.79 | 1.19 | 2.34 | 5.91 | 198.05 |
| jsPsych | Ubuntu | Firefox | 5.12 | 4.11 | 31.38 | 4.84 | 4.05 | 6.43 | 10.55 |
| jsPsych | macOS | Firefox | 5.16 | 6.85 | 53.7 | 2.57 | 3.27 | 6.05 | −15.29 |
| Testable | Ubuntu | Chrome | 5.46 | 4.23 | 47.09 | 7.06 | −11.92 | 5.09 | 94.18 |
| Testable | macOS | Chrome | 5.52 | 3.99 | 41.94 | 7.43 | 6.34 | 5.13 | 67.2 |
| jsPsych | macOS | Chrome | 5.62 | 5.68 | 41.42 | 5.74 | 3.25 | 5.45 | −9.71 |
| Lab.js | Win10 | Firefox | 5.78 | 7.88 | 8.22 | 4.21 | 14.25 | 5.25 | 70.93 |
| Lab.js | macOS | Chrome | 5.79 | 3.26 | 20.3 | 8.78 | 6.51 | 5.31 | −0.21 |
| Gorilla | Win10 | Edge (Standard) | 5.96 | 4.95 | 40.23 | 7.15 | 4.95 | 5.79 | 70.93 |
| Testable | macOS | Firefox | 6.1 | 5.2 | 57.66 | 7.63 | 22.83 | 5.48 | 40.73 |
| Lab.js | Ubuntu | Firefox | 6.19 | 2.91 | 25.54 | 10.22 | 13.66 | 5.44 | 185.35 |
| jsPsych | Win10 | Chrome | 6.23 | 7.85 | 23.27 | 5.1 | 3.6 | 5.73 | 43.57 |
| Testable | Win10 | Edge (Chromium) | 6.8 | 4.11 | 15.99 | 8.34 | −5.39 | 7.94 | 73.79 |
| jsPsych | Win10 | Firefox | 7.38 | 8.37 | 25.7 | 7.04 | 15.32 | 6.74 | 32.32 |
| Gorilla | Win10 | Chrome | 7.89 | 3.58 | 25.6 | 5.03 | 4.24 | 15.06 | 98.84 |
| Testable | Win10 | Chrome | 8.08 | 7.88 | 23.96 | 8.38 | −5.9 | 7.98 | 72.57 |
| Lab.js | Win10 | Edge (Chromium) | 8.57 | 4.22 | 17.14 | 8.03 | −4.9 | 13.45 | 82.45 |
| Lab.js | Win10 | Chrome | 9.48 | 8.44 | 19.2 | 7.76 | −1.73 | 12.25 | 86.03 |
| Gorilla | macOS | Chrome | 9.76 | 5.3 | 35.31 | 5.36 | 3.4 | 18.61 | 32.03 |
| Gorilla | Win10 | Edge (Chromium) | 11.34 | 4.89 | 23.4 | 3.01 | 1.56 | 26.13 | 121.57 |
| Gorilla | Ubuntu | Chrome | 14.17 | 0.43 | 40.85 | 1.66 | 3.36 | 40.42 | 200.55 |
| Gorilla | macOS | Safari | 19.16 | 1.53 | 29.65 | 30.11 | 22.25 | 25.83 | 285.81 |