A Software Feature for Mixed Methods Analysis: The MAXQDA Interactive Quote Matrix

Abstract

Mixed methods research relies on the integration of quantitative and qualitative data to develop meta-inferences. Despite an increase in techniques to facilitate integration at the methods and reporting levels, there is less practical guidance on how to conduct mixed methods integrative data analysis. Qualitative data analysis software is rapidly advancing to facilitate integrative analysis in mixed methods research studies. However, examples on how these features support analysis are relatively limited. The purpose of this article is to promote the use of qualitative data analysis software for integrative analysis in mixed methods research by describing the value of the major features available in software (i.e., code queries, crosstabulations, and quote matrices). In addition to a brief overview of these features, this article demonstrates the use of MAXQDA’s Interactive Quote Matrix with a real-life example of a completed mixed methods study and delineates the step-by-step process on how to use this feature to develop meta-inferences.

1. Introduction

Mixed methods research generally involves the collection and analysis of qualitative and quantitative data, and integration of the two. Integration is the defining feature of mixed methods research, and consists of the integration of qualitative and quantitative data, analysis, results, philosophical perspectives, or other aspects of research (Bazeley, 2018; Fetters, 2020). In responses to numerous calls for practical guidance, the field of mixed methods research has seen numerous advancements regarding integration including ways to conceptualize integration (e.g., strategies for integration), ways to report integrated results (e.g., visual joint displays or how to structure papers), and specific methods and procedures for integration at the planning and data collection stages. Nevertheless, few published articles have provided guidance on specific software features to facilitate integration at the analysis phase (i.e., integrative data analysis). We will follow Bazeley’s (2018) description of integration as, “the extent that different data elements and/or varied strategies for analysis of those elements are combined in such a way as to become interdependent (a two-way process) in reaching a common theoretical or research goal” (p. 10). The purpose of this article is to promote the use of software for integrative analysis in mixed methods research by describing the value of the major features available in qualitative data analysis software. We provide an illustration of these features, which culminated in the use of the Interactive Quote Matrix in MAXQDA, to conduct an integrative analysis explaining quantitative variation.

2. How Qualitative Data Analysis Software can Assist Mixed Methods Researchers

Software can assist mixed methods researchers in at least three major ways: 1) working with quantitative data, 2) managing and analyzing qualitative data, 3) integrating quantitative and qualitative data or results. Software used in mixed methods research typically consists of traditionally qualitative software programs that have added mixed methods features. Major quantitative software does not support efficient qualitative data analysis, or mixed methods integrative analysis. Some aspects of mixed methods analysis features are present in qualitative software, including Atlas.ti (Smit, 2021), Dedoose (Lieber et al., 2021), MAXQDA (Kuckartz & Rädiker, 2022), NVivo (Bazeley, 2021), and QDA Miner (Péladeau, 2021). Developers have responded to the needs and demands of mixed methods researchers, yet little is known about the uptake of software for mixed methods analysis. Based on our experiences reading empirical mixed methods studies, we rarely see mention of software for mixed methods analysis, suggesting the need for more guidance in using these features.

2.1. Working with Quantitative Data

Although data could be stored and analyzed separately in statistical and qualitative data analysis software, for integrative analysis the two strands come together and become interdependent. Having quantitative and qualitative data in a single software program is one of the major ways software can assist mixed methods researchers. First, software can enable storing quantitative data. One option available is to import variables and the data for each variable. If it is not possible to import quantitative data (i.e., it is not already stored elsewhere), manual entry is another possibility. Second, when importing quantitative data, it can be attached to qualitative data sources (e.g., an interview transcript). In other words, the software allows researchers to manage quantitative data, and in effect, build an integrated dataset. Finally, some software programs, such as MAXQDA, have built in statistical analysis tools that permit simple analyses, such as descriptive statistic computation, correlational analysis, and analysis of variance. The ability of a single software program to manage quantitative and qualitative data can greatly facilitate mixed methods integration.

2.2. Managing and Analyzing Qualitative Data

Using qualitative data analysis software to manage the qualitative strand of a mixed methods study is more common. Nevertheless, it may be helpful to review how the software can assist mixed methods researchers. As with the quantitative strand, software is helpful for managing and storing qualitative data within a single project file. Software is also useful through the analysis process, such as coding data and examining patterns to identify themes. To be clear, the software generally will not code data for you, but rather stores and manages codes to easily apply to segments of text or multimedia. One of the most useful features of software is the ability to retrieve and query data in different ways to address research questions. For example, researchers can focus their analysis on select codes at a time, on select data sources, or conduct more sophisticated queries like comparing codes among groups. Having quantitative data together can ultimately facilitate mixed methods analysis when combined with qualitative data.

2.3. Integrating Quantitative and Qualitative Data or Results

When qualitative and quantitative data are stored together within a software program, researchers can conduct an integrative analysis that combines data or results to address research questions. Software is particularly useful for merging as an integration strategy. Merging consist of comparing quantitative and qualitative results (Fetters et al., 2013) or relating the two in order to examine patterns (Guetterman & Fetters, 2022).

3. Examples of Mixed Methods Features in Qualitative Software

This section provides a general overview of how qualitative software supports merging integration through queries, crosstabulations, and matrices of quotes. We provide examples of each of these features, and how they support mixed methods integrative data analysis, through the use of a completed mixed methods study.

3.1. Background about the Example Study

There has been a substantial increase in the use of interprofessional teams –psychologists, nurses, physicians, social workers, pharmacists – when working with patients, particularly patients with complex care coordination needs. Interprofessional teams are more successful when the provider-provider communication is effective and empathic. Empathy is generally a set of ideas concerning how an individual reacts and relates to the experiences of another individual (Davis, 2018). In addition, genuine, empathic patient-provider communication is one of the most important indicators of patient experience (Epstein & Street, 2011; Ong et al., 1995) and helps reduce medical errors and other adverse events (Andel et al., 2012; Beckman et al., 1994; Levinson et al., 1997). Therefore, training health professionals on effective communication is necessary to improve patient and worksite health; the foundation of this communication is empathy. Teaching empathic communication is relatively under described in health professional pre-professional training. Although psychologists are widely trained on empathy and decoding verbal and nonverbal communication, other medical professionals are not. Therefore, identifying ways to teach and improve empathic communication can help improve patient-provider and provider-provider communication, and also support teaching and learning efforts across health professions.

The example study for this article stems from the fields of educational psychology within medical education, specifically focused on empathic healthcare communication. Guetterman and colleagues’ (2019) analyzed data from a multisite mixed methods randomized control trial focused on teaching empathic communication to medical students, providing training in verbal and nonverbal communication through virtual reality simulations. Virtual reality attempts to facilitate a personalized, authentic learning experience (Lane et al., 2013). Based on the cognitive theory of multimedia learning (Sorden, 2012), the intervention included interactive simulations with different patient stakeholders (e.g., patient-provider, provider-provider), focused on improving empathy for intercultural and interprofessional communication (Guetterman et al., 2019). Quantitative data from this study were from an objective structured clinical examination of communication skills related to empathy expression completed 1–2 weeks post-intervention. The quantitative analyses indicated that MPathic-VR intervention participants had improved proficiency in patient-provider intercultural communication and provider-provider conflict resolution.

Qualitative data for this study were from reflections written by intervention and control participants after completing their assigned study procedures. Participants were randomly assigned to one-of-five reflective essay questions focused on: (1) human interactions, (2) nonverbal communication, (3) important learning outcomes, (4) recommendations for improving MPathic-VR, and (5) functional aspects of the intervention (Guetterman et al., 2019). In aim of understanding the potential mechanisms of action of the intervention, qualitative data were analyzed to understand MPathic-VR participants’ experiences (n=206). Specifically, Guetterman et al. (2019) sought to answer: “How do medical student reflections about their experiences compare between low, medium, and high performers on the primary outcome measures of communication performance in the simulation and on the [objective structured clinical examination].” The authors concluded that variation between intervention-group low, middle, and high performers were attributed to nonverbal communication; in addition, high and middle range performers were interested in learning about communication (Guetterman et al., 2019).

3.2. Queries

Queries provide a way to search data in a specific way and obtain a report. Researchers might run a query to obtain all coded segments for a particular code or small set of codes. Having all segments of text allows researchers to read through all the data to help identify patterns and develop themes. Queries also facilitate exploring relations among codes when the researcher uses more complex coding queries, identifying segments of text that have overlapping or intersecting codes.

When quantitative variables are attached to the qualitative data source, the data can display along with the segment of text. For example, a variable such as age, demographic characteristics, scores on an assessment, or medical test results will display along with the segment. This can help with merging integration by examining related data together for coded text segments. Figure 1 provides a screenshot of the retrieved segments window from MAXQDA that includes several variables for that participant along with coded segments of text.

Figure 1.

An Example of Retrieved Segments from MAXQDA that Includes Quantitative Variables and Coded Segments of Text

Note. In this case, the quantitative variables included are the age, gender, post-test score (“global”), and categorical performance level (“OSCELMH”).

3.3. Crosstabulation

Crosstabulations can aggregate quantitative and qualitative data to explore relations or associations among qualitative and quantitative results. Also known as a contingency table, a crosstabulation typically arrays different groups or levels on two different variables as rows by columns. In terms of mixed methods analysis, crosstabulation allows arraying qualitative codes or themes by quantitative groups. For examples, codes could be rows, and quantitative groups or levels of a quantitative variable (e.g., low, middle, high scores on a standardized assessment) could be columns. The cells in a crosstabulation provide counts of the number of coded segments. Figure 2 provides this type of crosstabulation from MAXQDA. In this figure, the columns represent higher, middle, and lowering performing students on the objective structured clinical examination, and the rows represent the qualitative codes. The cells indicate the number of coded segments. For example, the researchers applied the Remembering Nonverbal Communication code 10 times for higher performing learners. The final column and final rows present a sum of the number of coded segments for that group or code. The horizontal arrows in Figure 2 illustrate the analytical process of focusing a code (i.e., row) to explore patterns among the higher, middle, and lower performing learners in the intervention (i.e., columns).

Figure 2.

Crosstabulation from MAXQDA of Quantitatively Higher, Middle, and Lower Performing Learners by Qualitative Codes

Note. The columns, from left to right (starting with OSCELMH-0), represent of low, medium, and high scorers from the objective structured clinical examination.

The vertical arrows illustrate the process of focusing on a quantitatively distinct group (e.g., higher performers) and looking down across codes to explore patterns. A limitation of crosstabulation is that frequencies do not include the full richness of the qualitative data. However, it may be helpful in exploring patterns of qualitative codes by quantitative groups as an initial step in integrative analysis to develop meta-inferences. Thus, crosstabulations can be a starting point or used in conjunction with quote matrices discussed further below.

3.4. Matrices of Quotes

Examining matrices of quotes are a potential next step in integrative analysis, given the limitations of crosstabulations. A quote matrix is organized like the crosstabulation, but the data within cells contain the coded text segments rather than counts. Through quote matrices, the full richness of the qualitative data are available to researchers in developing meta-inferences. Examining relationships and patterns among quantitative and qualitative results follows the same process as described with crosstabulations. However, the benefit of the quote matrix is understanding what groups of participants (based on a quantitative variable) said about a particular topic rather than how often it is said. In other words, while the crosstabulation is aggregated, the quote matrix provides detailed data. Both could be very useful to develop meta-inferences, but the rich detail of the quotes is essential. Although researchers could undertake this process manually, the mixed methods features available in software are very efficient. In less than one minute, researchers can select which quantitative groups to focus on or compare and obtain a quote matrix. That process could be repeated quickly depending on research questions. This article provides a detailed description of steps in this process. The integrative analysis illustration (Section 4) focuses on a type of quote matrix, called the Interactive Quote Matrix, available in MAXQDA software.

3.5. Additional Mixed Methods Features

Some programs have additional mixed methods, such as the ability to develop visual joint displays. Joint displays are an approach to conduct integrative analysis and to represent integration through a visual such as a table or figure (Guetterman et al., 2015). Dedoose software includes mixed methods charts. One unique example is its “codes x descriptor” chart which will display a bar chart of quantitative data by quantitative groups. Most salient to integrative analysis, the researcher can click on a bar in the chart to see the related qualitative text data (Lieber et al., 2021). In addition, MAXQDA software has programmed joint displays that will produce a table that integrates results through side-by-side comparisons or relating qualitative with quantitative results (Kuckartz & Rädiker, 2021). Side-by-side joint displays, which are related to merging integration, are some of the most common joint displays used in the literature (Guetterman et al., 2021). Other features include integrated statistical analysis, which is available in QDA Miner (Péladeau, 2021) and MAXQDA software (Kuckartz & Rädiker, 2021). The features mentioned here are not exhaustive, and software developers are continually adding new innovations.

4. An Illustration of the Interactive Quote Matrix for Mixed Methods Analysis

The Interactive Quote Matrix is a powerful tool for conducting mixed methods analysis available through MAXQDA software. The Interactive Quote Matrix is very similar to matrices of quotes, as previously discussed, but the matrix is dynamic and displays within a software window. Figure 3 shows a screenshot of an Interactive Quote Matrix. The qualitative code “Motivated to learn more” is selected, so the columns display the corresponding segments of text for each quantitative level. Researchers can interact with this quote matrix by selecting different codes of focus and reviewing corresponding coded segments. Because of the ability to move between codes and compare quantitative groups as new ideas or threads arise, the feature enables an iterative and integrative mixed methods analysis.

Figure 3.

Interactive Quote Matrix from MAXQDA showing Quantitatively Higher, Middle, and Lower Performing Students by Qualitative Codes

The Interactive Quote Matrix has several uses, depending on the research questions and how data are organized. It allows comparisons of groups of qualitative documents (e.g., interview transcripts, field notes). Researchers can define the groups based on any quantitative variables in MAXQDA. The variable can be continuous or categorical (see Section 4.3). In terms of integrative mixed methods analysis, the Interactive Quote Matrix has several potential uses: (1) comparing codes or themes among groups or levels of a quantitative variable, (2) explaining variation in outcomes, and (3) examining variation in instrument development and validation studies. The feature can help to manage the mixed methods analysis by organizing the data by codes and by quantitative groups. Following the steps discussed below may be helpful for developing meta-inferences systematically and subsequently writing narrative integrative results and developing joint displays. We illustrate the use of the Interactive Quote Matrix with Guetterman and colleagues’ (2019) MPathic-VR mixed methods intervention study.

4.1. Steps in Using the Interactive Quote Matrix for Mixed Methods Analysis

We propose seven steps for using the Interactive Quote Matrix in integrative data analysis, as enumerated in Figure 4. Although presented in sequential order, these steps are not always linear and the process could involve returning to previous steps. The aims, objectives, or research questions should drive the analytic process, so it is important to refer to them when beginning the analysis and continually throughout analysis.

Figure 4.

Steps in Using the Interactive Quote Matrix for Mixed Methods Analysis

4.2. Step 1: Code Qualitative Data

The Interactive Quote Matrix relies on having coded data. Therefore, at least some data must be coded before the matrix is useful. Certainly, additional coding could occur after obtaining the Interactive Quote Matrix; this typically occurs when the analysis generates new ideas or threads that might lead researchers to return to the qualitative data for additional coding. Because the tool displays coded segments of text, it will be empty if there are no coded data. In this example, all data were coded prior to accessing the Interactive Quote Matrix. The codes were grouped into broad categories to facilitate both coding and retrieval. These categories (e.g., Comments Related to the Intervention Itself, Useful Verbal and Nonverbal Communication Skills) appear as primary codes with their related codes as subcodes.

4.3. Step 2: Add Variables

The Interactive Quote Matrix establishes columns based on variables. While MAXQDA has some system generated variables, for a mixed methods study researchers will most likely add variables to the data. In MAXQDA, variables can be one of five types: Text, Integer, Decimal, Date/time, Boolean. An integer can be further specified as a categorical variable. Variables can either be imported along with their data using a variable import tool or entered manually. For the illustrative study on the MPathic-VR intervention, the research team imported the quantitative variables from a spreadsheet file that contained a unique identifier to match the name of the qualitative text for each learner participating in the intervention. The variables were the continuous scores on an objective assessment, a 3-level categorical variable representing the scores (i.e., low, middle, high), prior academic test scores, and demographics for each learner. The variables imported included all variables available to address the primary research question, as discussed further in step 4.

4.5. Step 3: Open the Interactive Quote Matrix

Within MAXQDA, the Interactive Quote Matrix is accessible through the mixed methods menu options. After clicking on the icon, a window appears to identify columns of interest. In addition, from an existing crosstabulation, users can build an Interactive Quote Matrix based on the crosstabulation by clicking an icon in the upper left corner.

4.6. Step 4: Identify Variables of Interest

The identification of columns is based on aims and research questions. For example, if the research question is related to explaining variation in quartiles, the columns should most likely be those quartiles. After accessing the Interactive Quote Matrix, a window appears (Figure 5) that displays variables that could be included as columns in the Interactive Quote Matrix. Users can simply drag variables over from “Variables” to the “Columns.” It is then necessary to specify the values of what will appear in columns by using the simple Boolean logic. For categorical variables, researchers can specify a specific value for a column. For continuous integer or decimal variables, the Boolean logic is available to specify a range (e.g., a column could be X > 10 and other could be X <=10).

Figure 5.

Specify Variables and Use Boolean Logic to Indicate Values of Interest for Columns in the Interactive Quote Matrix

For the illustrative study, mixed methods aim included the following primary research question, “How do medical student reflections about their experiences compare between low, medium, and high performers on the primary outcome measures of communication performance in the simulation and on the OSCE?” (Guetterman et al., 2019, p. 3). The team identified categorical levels of the outcome variable based on tertiles in the continuous scores. As shown in Figure 5, the Guetterman et al. (2019) study used the three levels of the objective structured clinical examination score (OSCELMH) to appear in the matrix: OSCELMH=0 was the lower performers, OSCELMH=1 was the middle performers, and OSCELMH=2 was the higher performers. These values represented the columns requested and was consistent with how students were categorized in the quantitative aim.

4.7. Step 5: Identify Codes of Interest

Researchers can choose to include all codes in the matrix, which will appear as rows, or activate select codes to focus their analysis. However, reviewing all codes in the project may not be necessary if some are not pertinent to integrative analysis. Regardless, the Interactive Quote Matrix will display coded segments for one code at a time. Thus, by design, the researcher focuses on codes individually and examines the text segments by quantitative group. This tool can help to manage the complex process of mixed methods analysis, which often involves many codes and variables. Rather than including all codes in the Interactive Quote Matrix, based on previous experience with this tool, the researchers focused on groups of codes at a time to manage the cognitive load of reviewing the text data and improve efficiency. In the example study, to identify differences in intervention experiences by communication performance, Guetterman et al. (2019) focused first on codes related to empathic communication (see Figure 2 for sub-codes) and then those related to the intervention strategy. We recommend analysts focus on codes that will address their specific research question, particularly when working with large data corpuses.

4.8. Step 6: Iteratively Review All Codes and Compare Columns

Researchers can then use the Interactive Quote Matrix to focus on one code at a time and compare coded segments across columns. When selecting a code, the columns populate in seconds with the relevant segments of text. This tool allows for the examination of patterns or relationships such as how a code differs or is similar by quantitative groups to develop meta-inferences. The process of reviewing the coded segments takes time and could be segmented to focus on a few codes at a time. After reading through segments of a code in each column, we recommend memoing to document ideas about meta-inferences. The process repeats, continuing to the next code, which enables a systematic approach to integrative analysis in reviewing all related data. Clicking on a code will refresh the data to include segments for that code. In this way, the Interactive Quote Matrix is dynamic. Reviewing a code could generate ideas and thoughts about a different code. With the Interactive Quote Matrix, the researcher can easily switch to another code without running a new report—the data refreshes automatically.

In the illustrative study, the researchers focused on one code at a time and read through the segments for higher, middle, and lower performing students to address the research question comparing experiences among differential performance levels. They selected codes intentionally that reflected both positive experiences and those suggesting potential improvement to capture the full range of experiences. For example, reviewing coded segments for the motivated to learn more and doubting nonverbals codes in addition to the codes under the Useful Verbal and Nonverbal Communication Skills category (Figure 6), it became apparent that high and middle performing students believed learning nonverbal communication was appropriate and described appropriate nonverbal communication strategies, while substantially fewer statements were present among low performing students. This led to an important meta-inference concerning the higher buy-in in learning health communication and being motivated to learn more among higher performing students. This may have initially been informed by the crosstabulation of codes, where more codes on motivated to learn were applied to documents from higher performing learners than lower performing learners.

Figure 6.

Examining Codes Across Quantitatively Higher, Middle, and Lower Performing Students to Develop Meta-Inferences

4.9. Step 7. Develop a Narrative or Joint Display to Report Results

The Interactive Quote Matrix, along with memos or notes written in conjunction, will be helpful in developing results for a manuscript or other report. The process of using the Interactive Quote Matrix could be considered a type of joint display analysis, which is a process of examining the relationship between qualitative and quantitative concepts and developing a joint display by organizing the results in a matrix or figure (Fetters, 2020). First, while examining the data in the matrix, researchers can copy and paste quotes to include in a narrative report or a joint display. In addition, the bottom of each quote displays a hyperlink to the transcript and specific segment of text. Clicking on the link will minimize the Interactive Quote Matrix and open the correct transcript to the specific paragraph. In doing so, researchers can review the document in context, such as reading other text before or after the segment or finding the source (e.g., site of the intervention participant).

Researchers in the illustrative study followed this process to develop manuscripts and joint displays. For example, Guetterman et al. (2019) included a joint display table that compared higher, middle, and lower performing structured examination scores across themes developed during the qualitative analysis (see Table 1). They developed the table using the Interactive Quote Matrix and preliminary versions of the table included quotes too.

Table 1.

Joint Display Developed Through Integrative Data Analysis Facilitated by the Interactive Quote Matrix

	OSCE Advanced Communication Assessment
Themes	Low (<.55)	Medium (.54 - .98)	High (> .98)
Useful communication skills	N/A	“Effective communication both verbal and nonverbal will be essential in getting the best care for patients”	“I thought that I was given helpful strategies for interacting with patients such as asking open-ended questions, validating feelings, and types of non-verbal cues to use.”
Remembering nonverbals	“Smiling and nodding is also important” (6%)	“Non-verbal cues can be very helpful. There are good times to nod and also times when it is not appropriate” “In emotionally charged situations, I realize that using non-verbal communication is very important.”	“Helped teach how to read facial expressions from people such as when the nurse was upset”
Motivated to learn more	N/A	“I would definitely benefit from more training such as this. I found myself hoping that there would be another simulation or two.”	“It would be interesting to go through other scenarios, and to see if this actually has a positive effect on my future interactions with patients”
Prefer humans	“hard to engage in non-verbal communication when you know you are just talking at a computer”	“I think that training for communication with patients is better done with live patients”	“your true response can only come from human to human interaction…program is much stronger at allowing a person to think about their verbal responses”

Note. Table from Guetterman et al. (2019) reused under Creative Commons Attribution License.

5. Discussion

Features in qualitative data analysis software – queries, crosstabulations, and matrices – can facilitate the development of mixed methods meta-inferences. MAXQDA provides an additional feature which may be leveraged by mixed methods researchers: the Interactive Quote Matrix that moves beyond quantizing qualitative data. In this article, we have made a unique contribution to the applied methodological literature on how to use the Interactive Quote Matrix to facilitate merging integration. We have also delineated a seven-step process on how meta-inferences can be developed using this tool. To our knowledge, this article is one of the few that discusses the Interactive Quote Matrix in mixed methods integrative data analysis.

The use of the Interactive Quote Matrix as described is limited to mixed methods integrative analysis focused on merging integration. Although this limits the tool’s applicability to other integration strategies, this method has significant potential in the conduct of mixed methods studies; merging integration is the most common type of integration strategy at the methods-level across multiple fields (e.g., DeJonckheere et al., 2018; Fàbregues et al., 2020; Guetterman et al., 2021). Other strategies to facilitate building and connecting integration will likely require different tools.

In addition to the limited use of the Interactive Quote Matrix for merging, there are several other limitations related to the scope and usability of this feature. First, researchers should be cautious on coding large chunks of data during Step 1. Coding larger segments will lead to matrices being much larger, which may impact cognitive load and, therefore, quality of the analysis. Instead, researchers should consider coding smaller segments and using the hyperlink strategy described in Step 7 to read additional text.

Secondly, the present version of MAXQDA (2022) does not include functions to document meta-inferences or analytic insights directly on the Interactive Quote Matrix. Researchers will need to document memos and meta-inferences on another document, or as a free memo (unattached to a specific transcript or code) in the system. This will also require researchers to be diligent in their recordkeeping, and record which code by variable matrix is related to which inference by writing a memo. One potential area for improvement within MAXQDA would be the option to save instances of the Interactive Quote Matrix to the Questions-Themes-Theories workspace. Saving these instances while iteratively working through the matrix could allow researchers to add “insights” to this workspace (e.g., memos for meta-inference development) attached to the matrix (VERBI GmbH, 2022).

Lastly, although qualitative data analysis software may store quantitative data to develop an integrated dataset, these software do not support more sophisticated quantitative data analysis (i.e., the researcher will be limited to more basic statistics). In seeking to ensure analytic rigor in both quantitative and qualitative phases, researchers should take caution with relying solely on descriptive statistics in their quantitative aim. This is particularly important as the complexity of quantitative analyses in mixed methods studies have been subject to critical attention (Bash et al., 2021; Ross & Onwuegbuzie, 2014). In the example study, Guetterman and colleagues (2019) categorized non-normally distributed variables to define low, middle, and high performers; these categorical variables facilitated application in the Interactive Quote Matrix.

The Interactive Quote Matrix technique described in this article presents a notable advancement for integration techniques facilitating joint display analysis. Joint display analysis has been described as “the process of studying, examining, or investigating how to bring together or link qualitative and quantitative data in one or a series of visual representations with the intent of (1) identifying commonality between the two types of data, and (2) gaining a more robust understanding of what both types of data mean together by drawing conclusions or meta inferences based on combined findings” (Fetters & Guetterman, 2021 p. 259). The Interactive Quote Matrix extends this idea to allow the researchers to systematically make comparisons of quantitatively different groups, defined by variables, for a different intent of identifying variation and differences. The tool is entirely consistent with examples of joint display analysis that emphasize the linkage of qualitative and quantitative data or results, the cognitive process of considering both forms of research and the related concepts, and the iterative nature of joint display analysis (Haynes-Brown & Fetters, 2021). We recommend further research to combine joint display analysis with software.

The fields of health science, education, psychology, and educational psychology specifically have been at the forefront of integrating quantitative and qualitative data through joint displays (Guetterman et al., 2021; McCrudden et al., 2021). The application of the Interactive Quote Matrix can further accelerate the use and reporting of joint displays in these fields. More advanced quantitative methods commonly applied in psychology – classification methods (e.g., latent variable mixture models, cluster analysis) (Petersen et al., 2019; Ulbricht et al., 2018), or social network analysis (e.g., homogeneity of the network [homophily], network density) (Gilman et al., 2022; Neal, 2020) – may be particularly suited for integration in the Interactive Quote Matrix. These analytic techniques provide methods to empirically define for quantitative variables without using arbitrary cutoffs or descriptive statistics.

6. Conclusion

We hope this article demystifies the use of qualitative data analysis software and merging integration, inspiring more mixed methods researchers to explore software tools to aid in their analysis. Software can facilitate an efficient and systematic process for integrative mixed methods analysis. MAXQDA is one of several tools that provide excellent, yet likely underutilized, features to assist with mixed methods analysis. The Interactive Quote Matrix is one tool that can help to address mixed methods aims that call for comparing themes across quantitative groups that can help identify subthemes by group. Alternatively, the tool is useful for explaining variation in outcomes or other variables. Software offers potential for aiding researchers in integrative data analysis. As these software features become more tailored to the needs of mixed methods analysis, researchers should describe how software (and related features within software) is used in their data analysis sections of their manuscripts. We also encourage more scholarship to provide guidance about specific software tools, including but not limited to Dedoose, MAXQDA, NVivo, and QDA Miner that can assist with integration in mixed methods research.