Table 3B.
Study | Research question and study design | Validity and reliability | Format | Piloting | Sampling | Distribution, administration and response | Coding and analysis | Results | Conclusions and discussion |
---|---|---|---|---|---|---|---|---|---|
| |||||||||
Sassower (2001) | * | * | *** | * | * | *** | ** | ** | ** |
Erba (2013) | *** | ** | *** | *** | ** | ** | *** | *** | *** |
Engelhart (2013) | ** | * | ? | * | *** | * | ** | *** | *** |
Shen (2018) | * | * | *** | * | * | * | ** | ** | *** |
Sylv’en (2020) | * | * | *** | * | *** | *** | ** | ** | *** |
Nguyen (2020) | ** | * | *** | ** | * | ** | ** | ** | *** |
Bach (2020) | * | * | ** | * | * | * | ** | ** | *** |
- Strong
- Moderate
- Weak
- Unable to determine.
Research question and study design: Was a questionnaire the most appropriate method?
Validity and reliability: Have claims for validity been made, and are they justified? (Is there evidence that the instrument measures what it sets out to measure?) Have claims for reliability been made, and are they justified? (Is there evidence that the questionnaire provides stable responses over time and between researchers?)
Format: Are example questions provided? Did the questions make sense, and could the participants in the sample understand them? Were any questions ambiguous or overly complicated?
Piloting: Are details given about the piloting undertaken? Was the questionnaire adequately piloted in terms of the method and means of administration, on people who were representative of the study population?
Sampling: Was the sampling frame for the definitive study sufficiently large and representative?
Distribution, administration and response: Was the method of distribution and administration reported? Were the response rates reported, including details of participants who were unsuitable for the research or refused to take part? Have any potential response biases been discussed?
Coding and analysis: What sort of analysis was carried out and was this appropriate?
Results: Were all relevant data reported? Are quantitative results definitive (significant), and are relevant non-significant results also reported? Have qualitative results been adequately interpreted (e.g. using an explicit theoretical framework), and have any quotes been properly justified and contextualized?
Conclusions and discussion: Have the researchers drawn an appropriate link between the data and their conclusions? Have the findings been placed within the wider body of knowledge in the field (e.g. via a comprehensive literature review), and are any recommendations justified?