Appendix 3.

NICE guidelines for critical appraisal of qualitative studies.²⁹

1. Is a qualitative approach appropriate?

Does the research question seek to understand processes or structures, or illuminate subjective experiences or meanings?

Could a quantitative approach better have addressed the research question?

2. Is the study clear in what it seeks to do?

Is the purpose of the study discussed – aims/objectives/research question/s?

Is there adequate/appropriate reference to the literature?

Are underpinning values/assumptions/theory discussed?

3. How defensible/rigorous is the research design/methodology?

Is the design appropriate to the research question?

Is a rationale given for using a qualitative approach?

Are there clear accounts of the rationale/justification for the sampling, data collection and data analysis techniques used?

Is the selection of cases/sampling strategy theoretically justified?

4. How well was the data collection carried out?

Are the data collection methods clearly described?

Were the appropriate data collected to address the research question?

Was the data collection and record keeping systematic?

5. Is the role of the researcher clearly described?

Has the relationship between the researcher and the participants been adequately considered?

Does the paper describe how the research was explained and presented to the participants?

6. Is the context clearly described?

Are the characteristics of the participants and settings clearly defined?

Were observations made in a sufficient variety of circumstances

Was context bias considered

7. Were the methods reliable?

Was data collected by more than 1 method?

Is there justification for triangulation, or for not triangulating?

Do the methods investigate what they claim to?

8. Is the data analysis sufficiently rigorous?

Is the procedure explicit – ie is it clear how the data was analysed to arrive at the results?

How systematic is the analysis, is the procedure reliable/dependable?

Is it clear how the themes and concepts were derived from the data?

9. Is the data ‘rich'?

How well are the contexts of the data described?

Has the diversity of perspective and content been explored?

How well has the detail and depth been demonstrated?

Are responses compared and contrasted across groups/sites?

10. Is the analysis reliable?

Did more than 1 researcher theme and code transcripts/data?

If so, how were differences resolved?

Did participants feed back on the transcripts/data if possible and relevant?

Were negative/discrepant results addressed or ignored?

11. Are the findings convincing?

Are the findings clearly presented?

Are the findings internally coherent?

Are extracts from the original data included?

Are the data appropriately referenced?

Is the reporting clear and coherent?

12. Are the findings relevant to the aims of the study?

13. Conclusions

How clear are the links between data, interpretation and conclusions?

Are the conclusions plausible and coherent?

Have alternative explanations been explored and discounted?

Does this enhance understanding of the research topic?

Are the implications of the research clearly defined?

Is there adequate discussion of any limitations encountered?

14. How clear and coherent is the reporting of ethics?

Have ethical issues been taken into consideration?

Are they adequately discussed eg do they address consent and anonymity?

Have the consequences of the research been considered ie raising expectations, changing behaviour?

Was the study approved by an ethics committee?

As far as can be ascertained from the paper, how well was the study conducted? (see guidance notes)