Advancing mixed methods in mental health research

Abstract

In the mental health research field, there is a need for effective and robust methodological approaches that are able to address complexity. In this article, we reassert the importance of mixed methods, an approach increasingly applied within mental health research, and examine the ongoing challenges in the field. We provide worked examples from our own research to demonstrate the diverse range of designs and benefits afforded by mixed methods approaches, along with personal reflections on barriers navigated. We outline pathways towards advancing mixed methods in mental health research, advocating for a shift from fragmentation to integration, improved publication routes for mixed methods studies and effective resourcing. To truly realise the potential of mixed methods, we call for a commitment from individual researchers, leaders, institutions, editors, funders and training providers to further these pathways.

Advancing mixed methods in mental health research

Given the inherent complexity and multifaceted nature of mental health, the evolving field of mental health research needs methodological approaches effectively matching this. Following a twentieth-century positivist surge in psychiatry, recent decades saw significant advancement in robust qualitative methods. This has paved the way for a growing application of mixed methods across the field, yet ongoing challenges limit capacity to realise their potential. Here, we highlight the value of mixed methods in mental health research, examine ongoing challenges, and outline pathways to advancing mixed methods in the field.

Mixed methods research formally integrates qualitative and quantitative methods in a single study or project (often understood to differ from approaches that include, but do not actively integrate, these methods). We believe that mixed methods offer a powerful approach, facilitating a holistic, complex understanding of mental health, a greater understanding of breadth and depth of phenomena and opportunity to build and refine conceptualisations. Mixed methods approaches have been increasingly applied within mental health research, especially in service and intervention research^{1 2} and nursing.^{3 4} However it is our view that mixed methods approaches are not yet routinely embedded across the field, and frequently suffer from a range of ongoing limitations where implemented. Indeed, many challenges noted in earlier writings on mixed methods in the mental health research field continue to resonate today.^{1 5} This includes communication challenges within teams, undervaluing of qualitative methods, and under-rationalisation and explanation of the use and design of mixed methods,^{2 5} the latter of which was noted as a critical issue in a 2021 review of community-based participatory mixed methods mental health research.⁶

Mixed methods research is typically undertaken within a pragmatist epistemology. Pragmatism is a school of thought concerned with meaning and action and conceptualises truth and knowledge as provisional rather than absolute. As a paradigm, it rejects dualist approaches and instead emphasises a pluralistic blending of perspectives to develop meaningful understanding. Thus, a pragmatic mixed methods approach challenges the premise that the differing epistemologies underpinning qualitative and quantitative methods make them incompatible. Within a pragmatist lens, there is instead a focus instead on finding workable approaches to combining methods in order to best make sense of a given phenomenon as needed.^{7 8}

Mixed methods research can function towards different purposes, leading to various conceptualised design types, including Greene et al’s seminal typology⁹ of:

• Triangulation, where a study seeks convergence across methods;

• Complementarity, where findings of each method enhance those of the other;

• Development, where findings from one method inform the other;

• Initiation, where the study aims to stimulate new perspectives or interpretations; and

• Expansion, where mixed methods increase the breadth of inquiry.

Accordingly, there are various characteristics within one’s design that should be given consideration, which inform and link to these different design types. Here, we briefly cover some fundamental considerations, rather than offering an exhaustive or detailed ‘how-to’; for further reading, we suggest conceptual and practical texts from the Journal of Mixed Methods Research’s virtual special issues on integration and quality, respectively.^{10 11} There are features of timing and degree of independence across strands; for instance, one might have a sequential design where one strand directly informs another, or a concurrent one where both methods are implemented entirely independently of one another. A further consideration is status; strands can be equal, or more weight may be given to one strand, which would likely translate into the balance of each strand’s respective contribution to integrated conclusions. The method of integration is a further decision, which should correspond to the specific design type adopted; for instance, in a sequential development design, integration would occur perhaps primarily through study-building, while in a concurrent triangulation design, integration might function through a comparative process across the findings.

We offer our own worked examples (table 1), not as exemplary but as insights into some of the design types, rationales, processes and our own reflections, to highlight the potential of mixed methods in, for instance, facilitating a holistic understanding of mental health, integrating breadth and depth of phenomena, and refining conceptualisations. As authors at varying stages of the early career researcher (ECR) journey, some reflections speak to this, which might be valuable for ECRs and for those supervising and mentoring them.

Table 1. Worked examples of, and reflections on, mixed methods mental health research projects.

	Case study A: Environmental air and noise pollution and adolescent mental health21	Case study B: Multiple risk exposure and early adolescent girls’ emotional symptoms22	Case study C: Trends in the recording of anxiety in UK primary care23	Case study D: Using virtual patients to explore the clinical reasoning skills of medical students24	Case study E: Development and validation of the social media experience measure25
Project aim	To better understand how air and noise pollution might impact adolescent mental health	To explore multiple risk exposure and its relationship with early adolescent girls’ emotional symptoms	To examine and understand trends in the recording of anxiety in UK primary care	To identify the data gathering patterns of medical students while using the virtual patient (eCREST) and how eCREST influenced these patterns	To co-develop, with young people, a comprehensive and freely available self-report social media experience measure that will be appropriate for young people
Rationale for mixed methods	To bring together physical and subjective/experiential aspects of the environment, guide methodological decisions and derive more insightful and relevant interpretations and recommendations	To offer a more comprehensive understanding of these processes in context within early adolescent girls’ emotional symptoms	To quantify recorded trends and to provide insight into possible reasons for the trends observed	To develop typologies of decision-making strategies and understand how these strategies develop	To inform measure development incorporating the experiences and perceptions of young people
Design	Exploratory sequential (ie, exploratory qualitative component refined a subsequent quantitative strand)	Concurrent parallel complementarity (ie, both strands undertaken independently alongside one another to enhance the findings from each)	Concurrent triangulation (ie, both strands undertaken simultaneously to converge and compare findings)	Exploratory sequential (ie, the quantitative strand was followed by a qualitative component to elaborate on qualitative findings)	Exploratory sequential initiation (ie, exploratory qualitative component initiated directions for the quantitative strand)
Qual strand	Interviews were conducted with 15 adolescents to explore their thoughts and feelings about air pollution, noise pollution, their local environments and environmental issues in general. These were co-analysed with young people using qualitative thematic analysis.	Semi-structured interviews were conducted with three adolescent girls experiencing multiple risk exposure and emotional symptoms and were analysed using interpretative phenomenological analysis to understand their lived experience and sense-making.	In-depth interviews were conducted with 15 General Practitioners (GPs) to explore their views on the coding and diagnosis of anxiety. Interviews were analysed thematically prior to analysis of the quantitative data, and therefore not influenced by knowledge of the quantitative findings.	Think-aloud and semi-structured interviews were undertaken with 16 medical students at one medical school. Thematic analysis revealed students used three data-gathering strategies and described how eCREST influenced their data gathering.	Focus groups were undertaken with 26 adolescents aged 11–15 to understand their experiences, motivations and perceptions of social media use, relating to mental health and well-being. This was followed by a Delphi study with adults with experience and expertise in adolescent social media use and/or mental health and young people with experience of using social media.
Quant strand	Epidemiological cohort data from the Study of Cognition, Adolescents and Mobile Phones (SCAMP) was analysed using multi-level multivariate regression to identify associations between exposure to air pollution and noise and psychological outcomes.	Structural equation modelling was undertaken with 8327 girls aged 11–12 years, comprising both self-report and demographic data, to examine risk and protective factors, the role of multiple risk exposure and stress as an underlying mechanism in how risk exposure affects girls’ symptoms.	Data from adults (n=2,569,153) registered with UK general practices that contributed to Clinical Practice Research Datalink (CPRD) between 2003 and 2018. Incidence rates and 95% CIs were calculated for recorded anxiety symptoms and diagnoses.	A feasibility randomised controlled trial was conducted across three UK medical schools (n=148). Medical students in the intervention group received the virtual patient cases and students in the control group received no cases. Their clinical decision-making skills were compared.	Qualitative analysis is currently being used to guide item development for a pilot measure of of adolescent social media experience, which will be piloted and psychometrically evaluated shortly. This is also being guided by ecological momentary assessment evidence.
Integration	The qualitative insights informed analytical decisions for the quantitative strand. In RT’s PhD thesis, the findings of each study were used to contextualise and interpret the other, and the understanding gained about young people’s experiences and priorities was crucial for making research, policy and practice recommendations.	Findings from each strand were synthesised after interpreting each strand individually to develop meta-inferences and create a more comprehensive understanding of these phenomena.	Analysis for the GP interviews and the CPRD data was completed separately, but the qualitative results informed the interpretation of the quantitative findings.	The decision-making strategies identified in the qualitative study were quantified using the trial data, based on the number and relevance of questions they asked the virtual patient. The prevalence of these strategies was compared between the intervention and control groups. The qualitative data provided further details of how eCREST shaped these strategies.	The qualitative data and analysis, together with the ecological momentary assessment evidence, directly form the basis for item development for subsequent psychometric evaluation.
Reflections	There were no published mixed methods articles on this subject that we could identify, and many relevant journals did not have clear guidance or sufficient word count for mixed methods. On the other hand, there were clearer places and precedents to follow for publishing these strands as separate pieces. This may have increased their separate impact but some of the overarching process, reasoning and meaning may have been lost in translation.	This was a PhD thesis (traditional format), and with equal weighting for the two strands, reflecting the complementarity approach. In publishing as papers, the approach adopted was to publish quantitative and qualitative strands separately, with the intention to then publish a mixed reflective piece. In practice, publishing the qualitative strand has been challenging, due to the small sample size which seems off-putting to many editors. As a result, only the quantitative findings are published to date and the meta-inferences are not present in the evidence base beyond the thesis, which appears a missed opportunity.	Reviewers considered this project as multi-method rather than mixed method, whereby true integration might have entailed conducting further interviews after the quantitative analysis had been completed, asking participants to share their views on the trends observed. As an ECR, there was a lack of resources to conduct this additional step. The project was published as a multi-method piece, with a limited pool of journals with appropriate word counts.	The research was less linear in reality. Initially, this was planned as an explanatory sequential project with the qualitative data aiming to elaborate on the quantitative findings. However, analysis of the qualitative data sparked further research questions and led to a reanalysis of the quantitative data. It was challenging to articulate this in one paper, so one paper published just the trial data, followed by a mixed methods paper with the qualitative analysis and reanalysis of quantitative data.	Though using qualitative work to inform item creation is standard in classical scale development (ie, engagement with the population and Delphi methods), it has not fully been embraced as ‘mixed methods’, with qualitative components often reported on with just a sentence or two in otherwise quantitative outputs. This project incorporated the complexity of adolescents’ views, rather than noting qualitative work as a minor point. A key challenge was timing, as the developmental approach required a tight schedule to ensure project progress. This could be difficult within qualitative work, particularly in complex systems (eg, working with schools to facilitate data collection, cyclical analytic processes).

ECRearly career researcher

The value of mixed methods in mental health research

Neither qualitative nor quantitative methods alone can fully illuminate the complexities inherent to mental health. Mixed methods offer a holistic lens, providing a multifaceted, multidimensional perspective that enhances richness, depth, validation and contextualisation of phenomena and extends intervention development. For example, Case Study A (table 1) demonstrates how mixed methods can provide a more comprehensive understanding of how environments affect mental health, with qualitative methods revealing important contextual factors not captured in previous quantitative research.

Mental health experiences are inherently personal, situated in complex life contexts, with substantive individual differences. Case Study B illustrates how mixed methods can capture this richness: statistical analyses of factors and mechanisms associated with adolescent girls’ mental health outcomes were complemented by in-depth exploration of individual lived experience, yielding nuanced, embedded insights into how such processes function and are understood.

In many areas of mental health research, a balanced understanding of breadth and depth is critical. Case Study C demonstrates a focus on depth and how detailed qualitative insights on stigma and General Practitioner (GP) coding decisions enriched understanding of quantitative GP-recorded anxiety diagnosis trends at population level. Conversely, in some contexts, greater emphasis on breadth via quantitative data is valuable, with complementary qualitative methods used to deepen process and contextual understandings. This can be useful in intervention studies and trials; in Case Study D, quantitative data demonstrated the efficacy of an intervention while qualitative data illuminated how the intervention influenced clinical decision-making skills.

Mixed methods investigation can range from exploratory to confirmatory, which is particularly valuable for mental health science. Exploratory research can, for instance, advance less-understood areas. As an example, Case study E shows how exploratory mixed methods can further understanding and (thus measurement of) adolescent social media usage in relation to mental health, creating opportunity for inspection of this phenomenon from different angles. Alternatively, confirmatory mixed methods research can refine understanding (eg, building more nuanced or contextually specific knowledge).

Readers may wonder whether such benefits can simply accumulate across an evidence base, with separate quantitative and qualitative work offering parts of the bigger picture, but formal integration offers distinct opportunities.¹² Integrated projects are designed to most effectively complement one another, and encourage active consideration of how insights come together. This can facilitate the bringing together of researchers with differing lenses and the facilitation of more cohesive understandings on shared interests, and in turn offer distinct benefits in advancing the wider evidence base.

Pathways toward advancing mixed methods in mental health research

Moving from fragmentation to integration

To realise the full potential of mixed methods within mental health research, we must overcome siloed thinking and practices. Our author team has all encountered resistance to the merits of mixed methods from colleagues, collaborators, funders and reviewers. Purist qualitative and quantitative divisions and mistrust over the value and rigour of mixed methods persist. This systemic fragmentation risks hindering the progress of the mental health research field and contributes to compromised research questions, team dysfunction, marginalisation, and stunted career development.13,15 This must be overcome at multiple levels. PhD students and ECRs interested in mixed methods should be offered focused support and mentoring, and researcher roles in mixed methods teams need not always be divided as separate ‘quantitative’ and ‘qualitative’ positions. Within teams, better integration requires a shared language and agenda, and space for curiosity and learning about less familiar methods. While some creative tension in mixed methods research teams can be an asset, group dynamics need careful management to prevent dysfunction.¹⁶ Shifting from a fragmented multidisciplinary model towards a truly interdisciplinary team can indeed be aided well by mixed methods and can facilitate better collaboration and deeper integration of research findings.¹² Recommendations for the creation of well-functioning mixed methods teams include strategic role appointments, establishing non-hierarchical shared leadership, effective team coordination, provision of support, and creative approaches to resource optimisation.¹⁷

Enabling mixed methods publication routes

There are numerous opportunities to better enable mixed methods publication routes. Efforts to challenge ‘publish or perish’ discourses and focus on slower, more impactful output development could be leveraged to encourage mixed methods outputs (mixed papers can mean one output, instead of two). Solutions to word count challenges are critical. Mixed method papers necessitate detail on two methodological strands (plus a third reflecting on mixed methods design/procedures), two sets of findings and multiple points of discussion of integration across these strands.¹⁸ This is difficult to achieve briefly and can lead to a ‘catch 22’ for authors through the publication process.¹² Journals lack tailored mixed methods parameters and/or clarity in such flexibility, and this can limit publication options, audience reach, and impact factor. We encourage editorial boards to offer statements on their values around mixed methods, signpost quality guidance and explore word count parameters. We encourage openness from researchers, reviewers and editors-in-chief to solutions supporting extended but concise word count options that aid publication feasibility and reader engagement. Solutions might include twin papers, prioritisation of content and relegation of details to tables (eg, key methodological aspects, qualitative quotes) or granular details to supplementary materials (eg, complex statistical analysis explanations).

Effectively resourcing mixed methods

Compared with other methods, there is limited mixed methods training from reputable providers. This impedes researchers’ knowledge, confidence and ability to undertake mixed methods research.¹² We recommend additional routine training for ECRs who are likely to encounter mixed methods research and teams in their career. Developing a common language and understanding of the principles and value of mixed methods research early could mitigate some challenges. Advanced training is needed to support senior researchers to navigate the complexities of working in multimethod and mixed methods teams and progress use of mixed methods.¹⁶Mixed methods demand greater resources than qualitative or quantitative approaches alone, requiring larger teams with skills in both methodologies or longer durations for true integration. Unfortunately, many funders and calls (that ECRs, in particular, may target) do not afford sufficient funds to facilitate this effectively. Furthermore, funding applications may not allow space for appropriate details for both methods, or a clear justification for mixed methods approaches, which may disadvantage mixed methods applications.^{19 20} We call on funders to provide targeted funding calls for mixed methods mental health projects that can advance our ways of working, and consider flexibility in word counts for justification and detailing integration.

Vision for advancing mixed methods in mental health research

Challenges persist that restrict the potential of mixed methods approaches in mental health research, and this has consequences for individuals, teams and the field. We aim to further invigorate recognition of the advantages and action pathways towards advancing mixed methods in the field: a shift from fragmentation to integration, improved publication routes for mixed methods studies, and effective resourcing. To achieve this vision and realise the potential of mixed methods, we call for a commitment from individual researchers, leaders, institutions, editors, funders, and training providers, each of whom has a crucial role in furthering these pathways.