More than just great quotes: An introduction to the Canadian Tri-Council’s qualitative requirements

Abstract

Although at times misunderstood by the general research community, qualitative research has developed out of diverse, rich and complex philosophical traditions and theoretical paradigms. In the most recent Canadian Tri-Council policy statement on the ethical conduct of research involving humans, a chapter was devoted to a summary of methods and methodological requirements that characterize robust qualitative research, despite the diversity of approaches. To dispel common misperceptions about qualitative research and introduce the unfamiliar reader to these requirements, the work of a qualitative study on isoniazid preventive therapy for prophylaxis of tuberculosis published in AIDS is critiqued alongside each of the Tri-Council’s nine requirements.

To the clinician or quantitative researcher untrained in qualitative approaches, qualitative research may at first appear to lack rigour and significance. Those in the human sciences may argue that poor examples in quantitative research might also be appropriately judged as such. More convincing is the recognition that quantitative research is over-represented in Western research paradigms, and standards for critique of research disproportionately include quantitative language and expectations. Failure to distinguish between appropriate qualitative and quantitative requirements has perpetuated the misguided belief that qualitative research requires no particular skill. Open-ended questions on a questionnaire or the inclusion of interviews in a research design are at times held as sufficient conditions for research to be considered “qualitative”. If this were the case, mortgage applications would be research tools, and the hosts of Entertainment Tonight would be researchers.

Simply, the difference between qualitative and quantitative research is in the prefix: qualitative studies investigate the qualities of an experience, while quantitative studies measure and interpret numerical output. Depoy and Gitlin (1) outlined qualitative research as a naturalistic form of inquiry through the use of inductive reasoning, whereby patterns are created based on observations. They distinguished qualitative from quantitative or “experimental-type” research, which uses primarily deductive reasoning through the use of hypothesis testing (1). Mayan (2) described qualitative research as interpreting or making sense of the meaning people attach to their experiences which, rather than controlling particular variables, invites “context, complexity and ‘confounding variables’”. Despite these definitions, it remains challenging for those immersed in quantitative research to appropriately evaluate the means by which qualitative researchers derive meaning from experience.

In late 2010, the Canadian Tri-Council released a policy statement on the ethical conduct of research involving humans and, in it, a chapter on the nature of qualitative research (3). Although embedded in an ethical document, this chapter provided an introduction to many of the requirements and practices of qualitative research, including the existence of multiple methods, theoretical paradigms and philosophical traditions. Herein, we will use the terms ‘method’ to refer to the qualitative technique used in a particular study (eg, grounded theory, narrative inquiry), ‘theoretical paradigm’ to refer to epistemological concepts of knowledge generation held by the researcher or informing the qualitative method (eg, feminism, constructivism), and ‘philosophical tradition’ (also called methodology in qualitative research) to refer to the ontological suppositions from which the researcher is operating (eg, existentialism, Cartesian dualism). At times, these concepts overlap, as in the case of phenomenology, which is a philosophical tradition, and the phenomenological approach, which is a method informed by the philosophical tradition. The term ‘approach’ will refer to all three aspects of qualitative research.

To introduce the Tri-Council’s requirements to readers unfamiliar with qualitative research, we juxtapose each with a critique of an article on tuberculosis prophylaxis recently published in the medical journal AIDS(4). The article entitled “Barriers to implementation of isoniazid preventative therapy: A qualitative study” appears in the journal’s 2010 supplement on use of tuberculosis prophylaxis among people living with HIV (PLWH). Overall, Lester et al (4) succeed in their aim to describe barriers to isoniazid preventive therapy (IPT) implementation and provide information that is useful to clinicians and policymakers; however, their qualitative approach and reporting of the project lack the rigour expected of a comprehensive qualitative investigation. Herein, we remark on the components of qualitative research that were lacking in the report, although we are mindful that the authors were at a severe disadvantage, likely constrained by word limits and expectations of primarily quantitatively trained reviewers and editors. To aid reviewers and researchers who wish to broaden or improve their research toolkit, we have provided a number of suggestions that would help strengthen future proposals and manuscripts.

LESTER ET AL (4)

The study to which we refer is a four-page research article that reports the findings from a set of interviews and focus groups that were conducted with health care providers (doctors, nurses and lay counsellors; n=22) selected from five clinics identified from the database of a nongovernmental organization providing treatment guidelines and medications for people living with HIV and AIDS in South Africa. The clinics were divided into “‘routine’ and ‘low’ IPT implementers (defined arbitrarily as above or below 5% of HIV-positive patients taking IPT, respectively, as a way of including IPT prescribers and non-prescribers in interviews).” Interviews also were conducted with patients (n=20) from a “purposive sample of 10 clinics, selected based on accessibility and likely willingness to participate,” although the authors focussed primarily on the provider findings in the article. Lester et al (4) described analysis of transcripts as thematic and coded line by line. According to the authors, “themes focussing on reasons for low IPT prescription were analysed in greater depth.” The following four themes were identified:

1. Knowledge and experience: This theme referred to physicians’ unfamiliarity with IPT, either through lack of use or beliefs about its efficacy: “‘I attended several meetings and some authorities were saying it’s really not an established fact that INH will definitely protect’ – doctor, routine usage.”

2. Perceived operational barriers: This theme covered practical barriers that discouraged the use of IPT, including inducing drug resistance, increased safety of offering treatment for active disease in this population, and the difficulty with ruling out active tuberculosis (TB) among PLWH: “‘This is South Africa, the world headquarters of TB. Every other patient has TB. In the developed world, maybe [IPT] has a place, but here, in this setting, it’s treatment we need. Just treat for TB – what are you going to lose’ – doctor, low IPT usage.”

3. Actual operational barriers: Here the authors described barriers within the health care system, including off-site sputum testing/radiography and poor coordination between HIV and TB activities: “‘I have been trying to get in contact with the lab. You can’t assume that ‘oh we’ve not heard anything’ so it is negative. The problem is from their side, they can’t contact or communicate. And they lose specimens’ – nurse, low IPT usage.”

4. Patient-derived barriers: This theme discussed potential barriers to patients, although no patients interviewed had heard of IPT. Some allied health professionals noted lack of money for transport and pill burden as possible barriers, although doctors did not feel concerned about patient adherence: “‘They take what we give them. They want to have their lives saved’ – doctor, low IPT usage.”

In the discussion, the researchers focused on changing health care providers’ perceptions about IPT to promote its use, including suggestions to identify IPT clinical champions and to raise awareness about current literature downplaying the need for chest radiography to rule out active TB disease in asymptomatic patients and the lack of drug resistance findings to date (4).

Among the authors of the AIDS journal example (Lester, Hamilton, Charalambous, Dwadwa, Chandler, Churchyard and Grant) are many knowledgeable researchers involved in IPT clinical trials in South Africa. In fact, members of this study group authored six of the eight articles in the IPT supplement, as well as its introduction. Of the five articles not discussed herein, four are quantitative in nature and one is an editorial on the benefits of IPT, despite the failure of randomized control trials to demonstrate decreased mortality following IPT intervention. With this in mind, we will begin with an essential requirement presented in the Tri-Council’s document – that of reflective inquiry.

THE TRI-COUNCIL REQUIREMENTS

Dynamic, reflective, and continuous research process

A fluid process of engaging with data, which requires “a constant reflective approach and questioning by the researcher”

(3).

A common critique of qualitative research is its subjectivity as opposed to the quantitative aim of describing objective reality (5). In fact, subjectivity is not the inferior word that the natural sciences may believe. Based on theories of social construction, subjectivity is the recognition that a subject observing an object situates the object within an existing context based on previous experience, which arguably exists in all scientific research. As Malterud (6) explained, “a researcher’s background and position will affect what they choose to investigate, the angle of investigation, the methods judged most adequate for this purpose, the findings considered most appropriate, and the framing and communication of conclusions”. In a process by which to qualify such subjectivity, reflexivity or positioning of the researcher includes the identification of, and constant reflection on, one’s own training and the philosophical traditions and theoretical paradigms to which one adheres. While it may allow for the use of ‘I’ and ‘we’ in qualitative manuscripts, reflexivity does not give free rein to the researcher to abandon sound research methods or rigorous investigation. Reflexivity, properly integrated, helps to frame a research question, which is then used to identify an appropriate method and philosophical tradition, and informs the analysis and representation of findings.

In Lester et al (4) this reflexive element is missing. For example, the authors do not mention their quantitative lens within the text. Although a reader of the full AIDS supplement may recognize this fact based on the pattern of authorship, one who seeks out the article in isolation would miss the context from which the authors begin their investigation. To be clear, we do not mean to suggest that quantitative research expertise precludes an investigator’s ability to conduct qualitative research; many researchers embrace the use of mixed methods and recognize that while quantitative questions can answer the who, what and where, qualitative studies are best suited to the why and how of inquiry. Nonetheless, qualitative research must convey the rigour and complexity integral to its practice, and the same level of training needed to perform sound quantitative research is needed to perform sound qualitative research.

More importantly, Lester et al (4) did not discuss their preconceptions about the worthiness of IPT in practice. Preconceptions are not strictly a disadvantage in qualitative investigation; on the contrary, Malterud argued that preconceptions “can be valuable sources for relevant and specific research,” although she cautioned that investigators should be mindful not to confuse contextually based intuition with the knowledge acquired through rigorous qualitative research (6). That the authorship team are strong proponents of IPT among PLWH, a practice not without contention (7), is not at issue; it is the lack of reflection on and identification of the authors’ context, one that may have influenced specific areas of the inquiry and presentation of analysis, that misleads the naïve reader to context-free conclusions or misperceptions about qualitative inquiry.

Sandelowski and Barosso (8) argued that reflexivity also implies the ability to reflect “outward to the cultural, historical, linguistic, political, and other forces that shape everything about inquiry; and, in between researcher and participant to the social interaction they share”. These aspects of reflexivity are equally essential requirements of qualitative research and will be addressed in-depth below.

Diversity of approaches

There is no single approach in qualitative research. Different fields or disciplines, and even individual scholars within a discipline, have different perspectives on, and approaches to, the use of qualitative methods… [Methods] include, but are not limited to, ethnography, participatory action research, oral history, phenomenology, narrative inquiry, grounded theory and discourse analysis. The term ‘qualitative research’ covers a wide range of overlapping paradigms or perspectives

(3).

Reinforcing this requirement, de Laine (9) argued that dichotomizing qualitative and quantitative research is misleading, implying “internal consistency and mutual compatibility” between them; rather, she explained, qualitative research comes from a number of different disciplines and theoretical paradigms – each having its own strengths and weaknesses. De Laine continued to say that “some approaches are considered better suited to handle a specific question, [but] the extent to which the approaches can be used will partly depend on the form of the question” (9).

One of the most critical pieces lacking in the study by Lester et al (4) is a declared method. The authors referred to the use of “qualitative methods” to undertake the study, but described no method in particular. Was the approach ethnographic, examining the cultural contexts in which IPT would or would not be taken up? Was it phenomenological, in that it aimed to get at the essence of the experiences of clinicians working at an overburdened clinic? Perhaps it was grounded theory, and the researchers wanted to develop a theory that could explain the social context behind the lack of IPT use. Unfortunately, it is difficult to assess because the information given in the article does not fit well with, or lacks the primary characteristics of, any of these methods.

The authors also referred to the use of interviews and focus groups. Interviews and focus groups are not qualitative methods per se, but means by which to collect data, the practice and analysis of which can appear quite different depending on the selected method of inquiry. This is an important distinction between quantitative and qualitative research. While a questionnaire used in a case control study may be similar to one used in a randomized control trial, the design of interview questions in a qualitative study would appear quite different depending on the method selected. Similar to the epidemiology example, one would have to consider different analysis techniques depending on the method used for qualitative investigation. In the Lester et al study, the authors also do not identify any philosophical tradition or theoretical paradigm in their study design or analysis, suggesting a lack of experience with qualitative research. Finally, no relevant or analogous qualitative studies were referenced in the article (eg, Escott and Wally [10]), although familiarity with similar studies may have helped the authors to select and describe their own method and process.

Research goals and objectives

The intended goals of qualitative projects may include ‘giving voice’ to a particular population, engaging in research that is critical of settings and systems, or the power of those being studied, affecting change in a particular social environment, or exploring previously understudied phenomena to develop new theoretical approaches to research

(3).

In this requirement, the Tri-Council statement referred to the type of research questions that are best suited to human science inquiry. Charmaz (5) described the human science lens as seeking to understand phenomena that is context-dependent, viewing “facts and values as inextricably linked” and “truth as provisional,” which she juxtaposed to the positivist approach which seeks explanation and prediction, whereby “parsimony, generality, and universality” are emphasized.

Lester et al (4) identified their study aim to “describe barriers to IPT implementation from healthcare provider and patient perspectives,” which is conducive to giving voice to the populations of interest. They also established the importance of the topic and the dearth of qualitative research in this area. There are, however, issues with the investigators’ implementation of the study aim in that the voices of participants are not expressed in depth, nor do the authors attempt to develop any innovative theories in light of the data.

While some may argue that the authors critiqued power structures when they concluded that barriers to IPT implementation rest primarily on health care providers, as suggested in the value-laden contrast between theme 3 “perceived barriers” and theme 4 “actual barriers”, we would argue that Lester et al fell short in developing this argument (4, our emphasis). A thorough analysis of power relations between health care providers and patients would investigate the deeper contexts experienced by both physicians and PLWH, including race relations, positions of authority, contexts of poverty, and the interplay between Muthi (traditional forms of South African medicine) and Western biomedical cultures. Moreover, an examination of the system in which health care providers operate must be taken into account, including resource constraint and burden of disease. How, for example, does the context of TB prophylaxis change in a community in which “every other patient has TB” (4)? What of the risk of reinfection once the patient has been treated? In the discussion, there is mention of “some senior clinicians in South Africa [who] have publicly opposed IPT, and this is likely to have increased the reluctance of staff to prescribe IPT,” but this issue was not pursued with participants (4). Questions such as “Why is there a fear of isoniazid resistance?” or “Are there historical experiences that underlie these fears?” would be examples of questions that could tease out the clinical environment that may underlie these concerns. Instead, a simplistic conclusion was drawn, suggesting that if health professionals were better informed their practice would change, for example, when the researchers suggested that “prescribers should be made aware of the consensus from current literature that IPT does not promote isoniazid resistance” (4).

Another issue is the lack of voice afforded to PLWH. According to the text, the research team interviewed 20 clinic patients, which would normally equate to hundreds of pages of transcribed data and a study in itself. With the exception of one quotation, Lester et al (4) disregarded patient perspectives altogether, stating that “none of the patients had heard of IPT and therefore the main focus for analysis was of staff interviews”. This appears to be a missed opportunity to examine the patient realm. The report begs the question why have PLWH not heard of IPT? Are there other ways of expressing treatment of latent TB to patients? Are there power relations that prevent patients from gaining knowledge in a biomedical framework? Would patients indeed demand IPT or dismiss it if they knew of the full implications of IPT treatment? Where were the patients who had been treated and what were their experiences? Might their experiences have influenced others? Qualitative research demands the flexibility to respond to conditions that may, in fact, prove counter to researchers’ expectations. It would have been helpful to learn more about the research encounters with PLWH, for example, did participants fail to identify any potential barriers or were interviews simply cut short when patients had not heard of the treatment regimen? We as readers cannot evaluate the patient context because no information was provided, nor, to our knowledge, were these findings published elsewhere.

Although the researchers set out to describe a phenomenon, their findings seemed rather superficial, serving to reinforce an assumption that IPT would be implemented if all players were well informed. Human interaction is complex and answers are rarely so simple – especially in the context of HIV in South Africa. Qualitative approaches are not defined by the choice of data collection tool (eg, interviews or focus groups), but by the theoretical and philosophical paradigms that best fit the research question. Had they approached the research from a social constructivist paradigm, for example, the investigators would have been better poised to examine the role of historical, cultural and relational contexts, the consideration of which are important to understanding the complexities of offering IPT in this setting. It is with depth of inquiry that one approaches understanding. Probing for opinions and attitudes beneath the surface might have presented a more complex picture of issues surrounding IPT use.

Diverse, multiple, and often evolving contexts

The [qualitative] researcher’s priority is to answer the research question stemming from the study of [particular] individuals in a specific social setting at a specific time. Researchers sometimes engage in research that questions social structures and activities that create, or result in, inequality and injustice. Studies may involve participants who are in highly vulnerable circumstances because of the social and/or legal stigmatization that is associated with their activity or identity, and who may have little trust in the law, social agencies or institutional authorities

(3).

The Tri-Council’s requirement regarding diverse and evolving contexts relates to issues identified under research aims, such as contextualized relationships between health care providers and patients, but also to power relationships that exist between participants and the investigators. Côté and Turgeon (11) explained, “the reader must be provided with meaningful information concerning the characteristics of the setting and the individuals involved… and all other information needed to understand the phenomenon”. In the Lester et al (4) example, further elucidation on the process of the research and players is needed. Who were the interviewers? How did the participants respond to them? What was the power relationship between them? These issues are discussed more in depth under the section on consent process below.

Data collection and sample size

In qualitative research, greater emphasis [is] placed on depth of research [versus] breadth,” and researchers seek “diverse but overlapping data on a limited number of cases or situations… Samples and research sites in these studies are chosen because they are viewed as particularly useful or rich sources of information for furthering one’s understanding of phenomena of interest, and not because the results may prove statistically significant. Participants are selected for their potential to inform theory development, and often selection of participants is guided by emerging patterns over the course of the data collection

(3).

Again, qualitative researchers are in search of the quality of experience rather than probabilistic quantities. The intention is not to remark on an essential experience shared by all, but to articulate experiences embedded in human interaction. Samples can range from one participant in a phenomenological study to a few dozen in an ethnographic study; ultimately the decision requires consideration of multiple factors, including the study scope, nature of the topic, study design (method) and quality of data (12).

While it is difficult to comment on sample size in Lester et al (4) due to an undeclared qualitative method, their sample size description appeared to favour quantity over quality. The investigators described their method of sampling as “a purposive sample of 10 clinics based on accessibility and likely willingness to participate” (4). This point is of a particularly contentious nature to the qualitative researcher. Purposeful sampling is not, as the authors imply, a convenient process that maximizes the number of participants. As Côté and Turgeon (11) explained, “often, relatively restricted samples are painstakingly selected, based on the study goals and the methodology used”. As previously mentioned, interviews and focus groups generate a large amount of data; arbitrarily selecting a large sample size unrelated to methodology can result in superficial analysis rather than an in-depth understanding gained from a thorough knowledge of the data (6).

Moreover, the authors did not seek deviant or diverse cases; these are participants who lie on either extreme of the experience under investigation whose varied experiences may have helped to develop a more complete picture of the topic. In the article, IPT use is reported as low overall among the identified clinics, which were “arbitrarily” labelled routine-use clinics if more than 5% of PLWH were prescribed treatment (4). Rather than selecting clinics with higher usage, it may have been more appropriate to seek out individual healthcare providers who used IPT more regularly to balance out the views of low prescribers. With regard to PLWH, the investigators approached patients who were deemed amenable through an anti-retroviral clinic program and who had not previously taken IPT (4). By omitting both professionals and patients with reasonable experience with IPT, the reader cannot discern the favourable conditions under which IPT may be prescribed and taken.

Dynamic, negotiated and ongoing consent process

Entry into a particular setting for research purposes sometimes requires negotiation with the population of interest. In some cases, participants hold equal or greater power in the researcher-participant relationship; in other cases, researchers themselves may hold greater power when access to prospective participant populations is gained through gatekeepers with whom the researcher has established a relationship

(3).

Different qualitative methods and research questions result in varying power relationships. It is important to identify and reflect on these relationships and evaluate how power imbalance may influence participation and the data collected.

In the IPT study, the participant-researcher dynamic was not made clear. The researchers highlight that no one declined to participate, although investigators must remember that it is possible to limit one’s participation. The Lester et al study was funded by the Aurum Institute, a nongovernmental organization through which antiretroviral therapy is offered to PLWH with limited financial resources. The institute also provides treatment guidelines to clinics for the use of IPT, meaning that both the researchers and institute had an affinity for IPT usage. Could this have influenced which clinics were most likely to participate? Did patients accessed through health care providers at a centre providing antiretroviral therapy feel obliged to participate? Did the power imbalance between interviewers and patients influence willingness to discuss TB prophylaxis? Did health professionals paint a simpler picture of IPT under-usage rather than address politically charged issues? These questions are hard for any researcher to clearly discern, qualitative or otherwise; however, the act of constant reflexivity allows the researcher to note some of these suspicions and perhaps discuss or refute them based upon their interactions. With no background on which to rely, all the reader can do is hope that these power relationships were taken into consideration.

Research partnerships

The emergent nature of many qualitative studies makes the achievement of rapport with participants and feelings of interpersonal trust crucial to the generation of questions considered important or interesting by both parties, and to the collection of dependable data. Research often becomes a collaborative process negotiated between the participant(s) and the researcher, requiring considerable time spent initially simply figuring out the focus of the research

(3).

Often, qualitative investigators will invest much time in building trust and relationships with participants before data collection to ensure that research is relevant and useful to the population under study. As discussed above, the investigators did not describe relationship building with participants other than to identify that participants who were most likely to participate were selected. Again, investigator reflection in the text would have contributed to a better understanding of process.

Inductive understanding

“Many forms of qualitative research entail gaining an inductive understanding of the world of participants to acquire an analytic understanding of how they view their actions and the world around them….” with an “emphasis on gaining insights into participants’ perceptions of themselves and others, and of the meanings that participants attach to their thoughts and behaviours”

(3).

Reiterating Charmaz’s (5) distinction between constructivist and positivist aims of research, inductive understanding in qualitative research is meant to get to the heart of an experience. There are many ways to approach inductive understanding, including various forms of relationship building, data collection and investigator reflection, often particular to a qualitative method. Ethnographers, for example, commonly use a combination of participant observation and guided, yet conversational interviews that include both descriptive and structural questions to arrive at an understanding of categories of meaning in a particular cultural setting (13). These data are often supplemented with reflective field notes that chronicle the investigator’s experience of the research process. Phenomenologists may also use interviews with the purpose of describing meanings embedded in day-to-day experiences. Interviews of this sort are often unstructured and might begin with broad, experiential questions intended to build rapport and encourage narrative responses (13). While both types of interviews come from different philosophical traditions and seek to answer different kinds of research questions, they both seek to inductively understand and translate the world of the participant.

The reader is not invited into the realm of the participant in Lester et al’s (4) investigation. As DeSantis and Noel-Ugarriza (14) pointed out, “themes emerge from the data. Themes are not superimposed on the data by the researcher, who then embarks on a ‘hunt and seek’ process to find supporting evidence of their existence”.

Quotations used to represent the views of the collective in Lester et al did not extend beyond responses to predominantly descriptive questions about IPT barriers, and the authors admit that “themes focussing on reasons for low IPT prescription and methods for improvement were analysed in greater depth” inviting the reader to instead enter the world of the researchers as they ‘hunt and seek’ for cursory barriers that can be easily identified and overcome (4). The goal of the qualitative researcher is not to find the easy answer, but to identify patterns that are tacit or implied and “extracted by a careful mental process of logical analysis of content from all data sources” (15). Although the quotations were relevant to the authors’ identified themes, the themes served more of an organizational function rather than elucidating the results of the study. Qualitative research is more than just great quotes; as Sandelowski and Barroso (8) described, arriving at results in qualitative research involves “the hard work of locating participants’ views and lives in some intellectual, theoretical, or other disciplinary tradition”. Lester et al neither developed new theories nor built on existing paradigms to critically analyze the presence of IPT barriers; they simply reiterated their study aim interspersed with the presentation of organized data. While direct quotes from participants can enhance the reader’s understanding of findings, they are not meant to stand alone or simply reinforce the investigators’ own agenda.

As described above, there are a number of avenues of investigation that may have led to a deeper understanding of why health care providers questioned the use of IPT in the South African context and what factors prevent awareness of the regimen among PLWH. This deeper level of understanding would have been better achieved through greater familiarity with and discussion of theories of social and relational power structures and a flexible, reflective process.

Research results (and analysis and interpretation)

Generalizability of the results to other contexts and the representativeness of the sample may or may not be a concern in qualitative research. Transferability of results from one setting to another is often viewed as more of a theoretical issue than a procedural or a sampling issue

(3).

Many would argue that before one can consider the transferability of findings, one must appropriately analyze and make sense of the data. While the Tri-Council does not offer direction with regard to data analysis, common analysis techniques do exist. Again, the choice of analysis will depend on the qualitative method used, the philosophical traditions informing the method, theoretical paradigms endorsed by the investigator, data collection techniques, and the research question under study; nonetheless, certain analysis techniques are often highlighted in the literature. Côté and Turgeon (11) described one such approach (and the choice of Lester et al.), thematic analysis, as “the process of creating categories based on an analysis of statements by participants” involving the following steps: “transcribing the interview material in full, reading the material several times, selecting the units of significance or units of meaning, identifying general themes, and categorizing and classifying the data”. They also described the practice of validating interpretations through the use of multiple analysts (11).

As Malterud noted:

A thorough, well prepared, and well documented analysis is what distinguishes scientific approach from superficial conjecture. The researcher’s task is to organise, compare, and validate alternative interpretations. Only when the researcher can identify the systematic procedure that has been followed in this process, can it be shared with others.

(6)

Importantly, some authors caution that the description of analysis should not take precedence to the actual findings of the research, but that well presented findings will make clear the process of analysis (8). Lester et al (4) do not indicate who undertook data analysis or describe how it was undertaken. The analysis process was also not made clear through the elucidation of findings, as discussed above. The critic of qualitative research should be able to discern how patterns in the data were noticed and categories and themes created, which theoretical paradigms informed the interpretation of findings, if multiple researchers reviewed transcripts or categories to present alternative hypotheses, and whether findings were member checked by participants to ensure that the interpretation accurately reflected the experience of those interviewed, all of which add credibility to the findings (11,16). In Lester et al (4), no alternative hypotheses were offered and no contradictory data were presented. The reader is left wondering if the analysis undertaken was exhaustive or if it served to validate an agenda superimposed on the data.

With regard to the generalizability of findings, social constructivist theory argues that human interaction is complex and cannot be generalized or predicted; yet, there is the concept of transferability in human science, which recognizes similarities in human experience. Although a given context may be impossible to repeat, the lessons learned from one context might transfer to another. In their discussion, Lester et al refer to generalizability, by which they may have meant transferability. We would argue that the authors discuss the concept of generalizability in the context of furthering IPT implementation rather than exploring the participants’ understanding of the context, making it difficult to transfer the findings to another population.

DISCUSSION

Having introduced the reader to the Tri-Council of Canada’s requirements and practices for qualitative research, it is important to reiterate that many of the aspects of the Lester et al article that we have criticized might have been unavoidable. While according to its guide to authors, AIDS invites social science research, articles are limited to 3500 words for original research and 1500 for field notes, and it seems likely that the latter dictated the present study. To abide by the requirements outlined by the Tri-Council and reiterated by many qualitative experts, short reports such as these constrain an investigator’s ability to accurately describe research design, reflexivity, analysis and interpretive frameworks, and prohibit the inclusion of rich quotations that exemplify the experience of participants. In high-impact social science journals, such constraints do not exist. Original research articles in Social Science and Medicine have a cap of 8000 words, while Qualitative Health Research has no word limit for contributors.

One author suggested that medical researchers interested in qualitative research work with reviewers and editors to overcome the dilemma of constrained space versus quality research reporting (6). We suspect that Lester et al (4) were invited to submit to the AIDS journal supplement, and the editors dictated space allotment; No researcher would want to jeopardize such an opportunity, and yet, it is a disservice to the qualitative research world for an incomplete work to appear as qualitative research in a high-impact journal. We would suggest that, rather than placing the onus on authors to negotiate space and debate the language appropriate for qualitative research, editors for journals that accept both human and natural science research consider appropriate word allowances based on method, and select peer reviewers based on expertise in the described method in addition to content knowledge.

CONCLUSION

What the clinician or quantitative researcher must take away from this brief introduction to the Tri-Council’s qualitative requirements is that research in the human sciences is not simply achieved through the superficial search for corroborating quotations, but involves a number of diverse methods that developed out of a range of multidisciplinary theoretical paradigms and philosophical traditions. Although great quotes can enhance the reader’s understanding of the findings in qualitative research, the analysis of data requires additional layers of complex interpretation. To undertake such research, the human scientist must carefully consider the research question to determine which qualitative method would best fit with the aim of the research. Qualitative research also requires a reflexive and thoughtful process in which the investigator considers the effect of human interaction and how their own position and ideological commitments may alter or influence the data collected and the process of analysis. That is not to say that crossover does not and cannot exist. When a quantitative researcher wishes to enter the qualitative realm, we suggest that consultation with an expert at least guide the investigator to appropriate reading material so that the rigour and complexity of the practice not be undermined. The Tri-Council’s qualitative requirements chapter is a fitting place to start. Additional qualitative research should also be consulted to better understand the types of research questions, philosophical traditions, theoretical paradigms, and methods that allow the investigator to explore an experience or phenomenon appropriately and exhaustively.