Qualitative Secondary Analysis: A Case Exemplar

Abstract

Qualitative secondary analysis (QSA) is the use of qualitative data collected by someone else or to answer a different research question. Secondary analysis of qualitative data provides an opportunity to maximize data utility particularly with difficult to reach patient populations. However, QSA methods require careful consideration and explicit description to best understand, contextualize, and evaluate the research results. In this paper, we describe methodologic considerations using a case exemplar to illustrate challenges specific to QSA and strategies to overcome them.

Background

Health care research requires significant time and resources. Secondary analysis of existing data provides an efficient alternative to collecting data from new groups or the same subjects. Secondary analysis, defined as the reuse of existing data to investigate a different research question (Heaton, 2004), has a similar purpose whether the data are quantitative or qualitative. Common goals include to (1) perform additional analyses on the original dataset, (2) analyze a subset of the original data, (3) apply a new perspective or focus to the original data, or (4) validate or expand findings from the original analysis (Hinds, Vogel, & Clarke-Steffen, 1997). Synthesis of knowledge from meta-analysis or aggregation may be viewed as an additional purpose of secondary analysis (Heaton, 2004).

Qualitative studies utilize several different data sources, such as interviews, observations, field notes, archival meeting minutes or clinical record notes, to produce rich descriptions of human experiences within a social context. The work typically requires significant resources (e.g., personnel effort/time) for data collection and analysis. When feasible, qualitative secondary analysis (QSA) can be a useful and cost-effective alternative to designing and conducting redundant primary studies. With advances in computerized data storage and analysis programs, sharing qualitative datasets has become easier. However, little guidance is available for conducting, structuring procedures, or evaluating QSA (Szabo & Strang, 1997).

QSA has been described as “an almost invisible enterprise in social research” (Fielding, 2004). Primary data is often re-used; however, descriptions of this practice are embedded within the methods section of qualitative research reports rather than explicitly identified as QSA. Moreover, searching or classifying reports as QSA is difficult because many researchers refrain from identifying their work as secondary analyses (Hinds et al., 1997; Thorne, 1998a). In this paper, we provide an overview of QSA, the purposes, and modes of data sharing and approaches. A unique, expanded QSA approach is presented as a methodological exemplar to illustrate considerations.

QSA Typology

Heaton (2004) classified QSA studies based on the relationship between the secondary and primary questions and the scope of data analyzed. Types of QSA included studies that (1) investigated questions different from the primary study, (2) applied a unique theoretical perspective, or (3) extended the primary work. Heaton’s literature review (2004) showed that studies varied in the choice of data used, from selected portions to entire or combined datasets.

Modes of Data Sharing

Heaton (2004) identified three modes of data sharing: formal, informal and auto-data. Formal data sharing involves accessing and analyzing deposited or archived qualitative data by an independent group of researchers. Historical research often uses formal data sharing. Informal data sharing refers to requests for direct access to an investigator’s data for use alone or to pool with other data, usually as a result of informal networking. In some instances, the primary researchers may be invited to collaborate. The most common mode of data sharing is auto-data, defined as further exploration of a qualitative data set by the primary research team. Due to the iterative nature of qualitative research, when using auto-data, it may be difficult to determine where the original study questions end and discrete, distinct analysis begins (Heaton, 1998).

An Exemplar QSA

Below we describe a QSA exemplar conducted by the primary author of this paper (JT), a member of the original research team, who used a supplementary approach to examine concepts revealed but not fully investigated in the primary study. First, we describe an overview of the original study on which the QSA was based. Then, the exemplar QSA is presented to illustrate: (1) the use of auto-data when the new research questions are closely related to or extend the original study aims (Table 1), (2) the collection of additional clinical record data to supplement the original dataset and (3) the performance of separate member checking in the form of expert review and opinion. Considerations and recommendations for use of QSA are reviewed with illustrations taken from the exemplar study (Table 2). Finally, discussion of conclusions and implications is included to assist with planning and implementation of QSA studies.

Table 1.

Research question comparison

Primary study		QSA
What is the process of care and communication in weaning LTMV patients from mechanical ventilation		What are the defining characteristics and cues of psychological symptoms such as anxiety and agitation exhibited by patients who are experiencing prolonged critical illness?
What interpersonal interactions (communication contacts, extent and content of communications) contribute to weaning success or are associated with inconsistent/plateau weaning patterns		How do clinicians discriminate between various psychological symptoms and behavioral signs?
What therapeutic strategies (e.g., medications/nutrients, use of instruction or comfort measures, rehabilitative treatments) contribute to weaning success or are associated with inconsistent/plateau weaning patterns		What therapeutic strategies (e.g., medications, non-pharmacologic methods) do clinicians undertake in response to patients’ anxiety and agitation?
What social (patient, family, clinician characteristics) and environmental factors (noise, lighting, room size/arrangement, work pattern, workload) contribute to weaning success or are associated with inconsistent/plateau weaning patterns		How do physiologic, social and behavioral characteristics of the patient influence the clinician’s interpretation and management of anxiety and agitation? What contextual factors influence interpretation and management of psychological symptoms and behavioral signs?

Table 2.

Application of the Exemplar Qualitative Secondary Analysis (QSA)

Consideration	QSA Example
Practical Advantages	Capitalized on existing longitudinal dataset. Required additional time, space and effort not part of the original study budget. -Permitted us to conduct the study for far less cost than undertaking a new inquiry. Credibility established during primary study facilitated entre when we returned to collect supplementary data. Reduced respondent burden for vulnerable critically ill patients and their caregivers Maximized participants’ investment of time and expertise
Data Adequacy and Congruency	Thick rich description was available and examples of anxiety or agitation found in each of the 30 cases. Anxiety and agitation are important factors in weaning management. Interview questions, medical record data and bedside observations offered rich descriptions. Descriptions were of sufficient volume to fully explore the phenomenon.
Passage of Time	QSA undertaken several years after primary data were collected. Updated observations of clinical practice indicated that implementation of recommendations from professional organizations and evidence from research findings were inconsistent at the institutional level nor applied at the bedside (Aitken, Marshall, Elliott, & McKinley, 2009; Patel et al., 2009; Shehabi et al., 2008; Tanios, de Wit, Epstein, & Devlin, 2009; Weinert & Calvin, 2007). Determined that the topic and dataset remained relevant.
Researcher stance/Context involvement	PI of QSA was integral part of primary study and involved in majority of data collection (observations, interviews, medical record review). PI of QSA had active contact with setting Anxiety and agitation data presented opportunity for additional (secondary) research question
Relationship of QSA Researcher to Primary Study	The principal investigator (MBH) for the primary study was readily accessible. Questions about the data could be answered with ease. Contact with additional informants and additional primary data collection was facilitated by ongoing presence in the clinical setting.
Informed Consent of Participants	Institutional Review Board approved secondary analysis and new research questions. Exploration of the "sub process" of management of agitation-anxiety was covered in the broad language of the original consent. Given frequency of observed and described anxiety events, further exploration of an important aspect of patients’ families’ and clinicians’ experience of critical illness was justified.
Rigor of QSA	Audit trail of methodologic notes and analytic memos was recorded systematically to detail decisions and analytic thoughts of researchers and establish dependability Multiple data sources were cross-checked or triangulated to provide confirmability. Credibility was established through consultation with colleagues and experts to ensure that the analysis accurately reflected critical care practice. Regular analysis meetings were held to establish credibility and fittingness of findings (Morse & Field 1995).

Aitken, L. M., Marshall, A. P., Elliott, R., & McKinley, S. (2009). Critical care nurses' decision making: sedation assessment and management in intensive care. Journal of Clinical Nursing, 18(1), 36–45.

Morse, J., & Field, P. (1995). Qualitative research methods for health professionals. (2nd ed.). Thousand Oaks, CA: Sage Publishing.

Patel, R. P., Gambrell, M., Speroff, T.,…Strength, C. (2009). Delirium and sedation in the intensive care unit: Survey of behaviors and attitudes of 1384 healthcare professionals. Critical Care Medicine, 37(3), 825–832.

Shehabi, Y., Botha, J. A., Boyle, M. S., Ernest, D., Freebairn, R. C., Jenkins, I. R., … Seppelt, I. M. (2008). Sedation and delirium in the intensive care unit: an Australian and New Zealand perspective. Anaesthesia & Intensive Care, 36(4), 570–578.

Tanios, M. A., de Wit, M., Epstein, S. K., & Devlin, J. W. (2009). Perceived barriers to the use of sedation protocols and daily sedation interruption: a multidisciplinary survey. Journal of Critical Care, 24(1), 66–73.

Weinert, C. R., & Calvin, A. D. (2007). Epidemiology of sedation and sedation adequacy for mechanically ventilated patients in a medical and surgical intensive care unit. Critical Care Medicine, 35(2), 393–401.

The Primary Study

Briefly, the original study was a micro-level ethnography designed to describe the processes of care and communication with patients weaning from prolonged mechanical ventilation (PMV) in a 28-bed Medical Intensive Care Unit (Broyles, Colbert, Tate, & Happ, 2008; Happ, Swigart, Tate, Arnold, Sereika, & Hoffman, 2007; Happ et al, 2007, 2010). Both the primary study and the QSA were approved by the Institutional Review Board at the University of Pittsburgh. Data were collected by two experienced investigators and a PhD student-research project coordinator. Data sources consisted of sustained field observations, interviews with patients, family members and clinicians, and clinical record review, including all narrative clinical documentation recorded by direct caregivers.

During iterative data collection and analysis in the original study, it became apparent that anxiety and agitation had an effect on the duration of ventilator weaning episodes, an observation that helped to formulate the questions for the QSA (Tate, Dabbs, Hoffman, Milbrandt & Happ, 2012). Thus, the secondary topic was closely aligned as an important facet of the primary phenomenon. The close, natural relationship between the primary and QSA research questions is demonstrated in the side-by-side comparison in Table 1. This QSA focused on new questions which extended the original study to recognition and management of anxiety or agitation, behaviors that often accompany mechanical ventilation and weaning but occur throughout the trajectory of critical illness and recovery.

Considerations when Undertaking QSA (Table 2)

Practical Advantages

A key practical advantage of QSA is maximizing use of existing data. Data collection efforts represent a significant percentage of the research budget in terms of cost and labor (Coyer & Gallo, 2005). This is particularly important in view of the competition for research funding. Planning and implementing a qualitative study involves considerable time and expertise not only for data collecting (e.g., interviews, participant observation or focus group), but in establishing access, credibility and relationships (Thorne, 1994) and in conducting the analysis. The cost of QSA is often seen as negligible since the outlay of resources for data collection is assumed by the original study. However, QSA incurs costs related to storage, researcher’s effort for review of existing data, analysis, and any further data collection that may be necessary.

Another advantage of QSA is access to data from an assembled cohort. In conducting original primary research, practical concerns arise when participants are difficult to locate or reluctant to divulge sensitive details to a researcher. In the case of vulnerable critically ill patients, participation in research may seem an unnecessary burden to family members who may be unwilling to provide proxy consent (Fielding, 2004). QSA permits new questions to be asked of data collected previously from these vulnerable groups (Rew, Koniak-Griffin, Lewis, Miles, & O'Sullivan, 2000), or from groups or events that occur with scarcity (Thorne, 1994). Participants’ time and effort in the primary study therefore becomes more worthwhile. In fact, it is recommended that data already collected from existing studies of vulnerable populations or about sensitive topics be analyzed prior to engaging new participants. In this way, QSA becomes a cumulative rather than a repetitive process (Fielding, 2004).

Data Adequacy and Congruency

Secondary researchers must determine that the primary data set meets the needs of the QSA. Data may be insufficient to answer a new question or the focus of the QSA may be so different as to render the pursuit of a QSA impossible (Heaton, 1998). The underlying assumptions, sampling plan, research questions, and conceptual framework selected to answer the original study question may not fit the question posed during QSA (Coyer & Gallo, 2005). The researchers of the primary study may have selectively sampled participants and analyzed the resulting data in a manner that produced a narrow or uneven scope of data (Hinds et al., 1997). Thus, the data needed to fully answer questions posed by the QSA may be inadequately addressed in the primary study. A critical review of the existing dataset is an important first step in determining whether the primary data fits the secondary questions (Hinds et al., 1997).

Passage of Time

The timing of the QSA is another important consideration. If the primary study and secondary study are performed sequentially, findings of the original study may influence the secondary study. On the other hand, studies performed concurrently offer the benefit of access to both the primary research team and participants member checking (Hinds et al., 1997).

The passage of time since the primary study was conducted can also have a distinct effect on the usefulness of the primary dataset. Data may be outdated or contain a historical bias (Coyer & Gallo, 2005). Since context changes over time, characteristics of the phenomena of interest may have changed. Analysis of older datasets may not illuminate the phenomena as they exist today.(Hinds et al., 1997) Even if participants could be re-contacted, their perspectives, memories and experiences change. The passage of time also has an affect on the relationship of the primary researchers to the data – so auto-data may be interpreted differently by the same researcher with the passage of time. Data are bound by time and history, therefore, may be a threat to internal validity unless a new investigator is able to account for these effects when interpreting data (Rew et al., 2000).

Researcher stance/Context involvement

Issues related to context are a major source of criticism of QSA (Gladstone, Volpe, & Boydell, 2007). One of the hallmarks of qualitative research is the relationship of the researcher to the participants. It can be argued that removing active contact with participants violates this premise. Tacit understandings developed in the field may be difficult or impossible to reconstruct (Thorne, 1994). Qualitative fieldworkers often react and redirect the data collection based on a growing knowledge of the setting. The setting may change as a result of external or internal factors. Interpretation of researchers as participants in a unique time and social context may be impossible to re-construct even if the secondary researchers were members of the primary team (Mauthner, Parry, & Milburn, 1998). Because the context in which the data were originally produced cannot be recovered, the ability of the researcher to react to the lived experience may be curtailed in QSA (Gladstone et al., 2007). Researchers utilize a number of tactics to filter and prioritize what to include as data that may not be apparent in either the written or spoken records of those events (Thorne, 1994). Reflexivity between the researcher, participants and setting is impossible to recreate when examining pre-existing data.

Relationship of QSA Researcher to Primary Study

The relationship of the QSA researcher to the primary study is an important consideration. When the QSA researcher is not part of the original study team, contractual arrangements detailing access to data, its format, access to the original team, and authorship are required (Hinds et al., 1997). The QSA researcher should assess the condition of the data, documents including transcripts, memos and notes, and clarity and flow of interactions (Hinds et al., 1997). An outline of the original study and data collection procedures should be critically reviewed (Heaton, 1998). If the secondary researcher was not a member of the original study team, access to the original investigative team for the purpose of ongoing clarification is essential (Hinds et al., 1997).

Membership on the original study team may, however, offer the secondary researcher little advantage depending on their role in the primary study. Some research team members may have had responsibility for only one type of data collection or data source. There may be differences in involvement with analysis of the primary data.

Informed Consent of Participants

Thorne (1998) questioned whether data collected for one study purpose can ethically be re-examined to answer another question without participants’ consent. Many institutional review boards permit consent forms to include language about the possibility of future use of existing data. While this mechanism is becoming routine and welcomed by researchers, concerns have been raised that a generic consent cannot possibly address all future secondary questions and may violate the principle of full informed consent (Gladstone et al., 2007). Local variations in study approval practices by institutional review boards may influence the ability of researchers to conduct a QSA.

Rigor of QSA

The primary standards for evaluating rigor of qualitative studies are trustworthiness (logical relationship between the data and the analytic claims), fit (the context within which the findings are applicable), transferability (the overall generalizability of the claims) and auditabilty (the transparency of the procedural steps and the analytic moves processes) (Lincoln & Guba, 1991). Thorne suggests that standard procedures for assuring rigor can be modified for QSA (Thorne, 1994). For instance, the original researchers may be viewed as sources of confirmation while new informants, other related datasets and validation by clinical experts are sources of triangulation that may overcome the lack of access to primary subjects (Heaton, 2004; Thorne, 1994).

Summary

Our observations, derived from the experience of posing a new question of existing qualitative data serves as a template for researchers considering QSA. Considerations regarding quality, availability and appropriateness of existing data are of primary importance. A realistic plan for collecting additional data to answer questions posed in QSA should consider burden and resources for data collection, analysis, storage and maintenance. Researchers should consider context as a potential limitation to new analyses. Finally, the cost of QSA should be fully evaluated prior to making a decision to pursue QSA.