Ensuring validity and reliability in qualitative research

In the 10 years since this paper was originally published in EBN’s Research Made Simple series,¹ the debate around validity and reliability in qualitative research continues and centres on how to assess the rigour and trustworthiness of qualitative studies, which differ significantly from quantitative research. While validity and reliability are well-established concepts in quantitative research, their application to qualitative research is contested, and alternative frameworks have been proposed. We present an updated Research Made Simple: ‘Issues of validity and reliability in qualitative research’ to help the readers of EBN understand the nuances of validity and reliability in qualitative studies, and apply appropriate strategies to ensure the rigor of their work. This leads to more credible and trustworthy findings, which are essential for influencing healthcare practices.

Assessing the quality of research is crucial to ensure findings can be effectively applied to clinical practice and are based on reliable, valid and scientifically sound evidence. Without thorough evaluation, clinical decisions might be guided by flawed or biased research, which could lead to ineffective or even harmful patient care. In a previous article, we explored ‘bias’ across research designs and outlined strategies to minimise bias.² The aim of this article is to further outline what is meant by rigour, or the integrity in which a study is conducted, in relation to ensuring the credibility of qualitative research findings. Concepts including reliability, validity and generalisability, typically associated with quantitative research, will be considered, and alternative terminology in relation to their application to qualitative research will be offered. Strategies adopted by qualitative researchers to enhance the credibility of their research are also outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Qualitative research is frequently criticised for lacking scientific rigour with poor justification of the methods adopted, lack of transparency in the analytical procedures and the findings being viewed as merely a collection of personal opinions subject to researcher bias.^{3 4} For the novice researcher, demonstrating rigour when undertaking qualitative research is challenging because there is no accepted consensus about the standards by which such research should be judged.³ Assessing the reliability of qualitative study findings requires judgement about the ‘soundness’ of the research in relation to the application and appropriateness of the methods undertaken and the integrity of the final recommendations and conclusions.

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, debates continue about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research.3,5 In the broadest context, these terms are applicable with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures.⁵ However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alternative frameworks for establishing rigour are appropriate.⁴ Lincoln and Guba’s seminal criteria for demonstrating rigour within qualitative research are based on the four key tenants of ‘truth value’, ‘consistency’, ‘neutrality’ and applicability’.⁶ Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

Table 1. Terminology and criteria used to evaluate the credibility of research findings.

Application of quantitative research terms to qualitative research4	Alternative terminology associated with credibility of qualitative research5
Validity The precision in which the findings accurately reflect the data.	Truth value Recognises that multiple realities exist; the researchers outline personal experiences and views that may have resulted in methodological bias; clearly and accurately presents participants’ accounts and perspectives.
Reliability The consistency of the analytical procedures, including accounting for personal and research method biases that may have influenced the findings.	Consistency Relates to the ‘trustworthiness’ of the methods undertaken and is dependent on the researcher maintaining a ’decision-trail’; that is, the researcher’s decisions are clear and transparent. Ultimately an independent researcher should be able to arrive at similar or comparable findings. Neutrality (or confirmability) Achieved when truth value, consistency and applicability have been addressed. Centres on acknowledging the complexity of prolonged engagement with participants and that the methods undertaken and findings are intrinsically linked to the researchers’ philosophical position, experiences and perspectives. These should be accounted for and differentiated from participants’ accounts.
Generalisability The population sample and subsequent findings represent the population under investigation. The findings can be applied to other settings/contexts.	Applicability Consideration is given to whether findings can be applied to other contexts, settings or groups.

What strategies can qualitative researchers adopt to ensure the credibility of study findings?

Unlike quantitative researchers, who apply statistical methods for establishing the validity and reliability of research findings, qualitative researchers aim to design and incorporate strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

1. Accounting for personal biases which may have influenced the findings.⁷

2. Acknowledging biases in sampling and constant critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis.⁴

3. Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent.^{4 5}

4. Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented.^{7 8}

5. Including rich and thick verbatim descriptions across participants’ accounts to support the findings⁷ and reporting data saturation.⁹

6. Demonstrating clarity in terms of thought processes and rationale for decisions made during data analysis and subsequent interpretations.⁴

7. Engaging with other researchers to reduce research bias.⁴

8. Respondent validation by inviting participants to comment on interview transcripts and whether the final themes and concepts created adequately reflect the phenomena being investigated.⁵

9. Data triangulation,^{4 5} whereby different methods and perspectives help produce a more comprehensive set of findings.^{10 11}

Table 2 provides specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the experiences of people with kidney disease after the implementation of an online mindfulness intervention.¹²

Table 2. Strategies for enhancing the credibility of qualitative research.

Truth value	Reflection on own perspectives: Interviews coded and analysed by two members of the team with extensive qualitative research methodology experience and training; coding and preliminary themes reviewed by the wider research team/steering group. Representativeness of the findings in relation to the phenomena: The sample of 19 participants and a willingness to share their experiences in depth ensured the depth of data collected from differing perspectives adding to the creditability of the findings; Semi-structured audio-recorded interviews allowed for an iterative process of revisiting the data to check emerging themes and remain true to participants’ accounts of taking part in the intervention; Use of rich and thick verbatim extracts from participants assists the reader in making judgements about whether the final themes are true to participants’ accounts.
Consistency/neutrality	Achieving auditability: Transparent and clear description of the research process from initial outline, through to the development of the methods and reporting of findings. Emerging themes discussed with research team members who had subject and qualitative research expertise, in an open process where assumptions could be challenged and consensus reached. The consolidated criteria for reporting qualitative research (COREQ) checklist used to report the study.
Applicability	Application of findings to other contexts: Rich detail of context, the setting and the details of participants facilitated the evaluation of study conclusions and transferability to other populations.

In conclusion, it is essential for all qualitative researchers to employ strategies that strengthen the credibility of their studies during both the design and implementation phases. While a universally accepted set of terminology and criteria for evaluating qualitative research does not exist, we have provided a brief overview of several strategies that can improve the trustworthiness of research findings.