Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Apr 23.
Published in final edited form as: Res Nurs Health. 2007 Feb;30(1):99–111. doi: 10.1002/nur.20176

Using Qualitative Metasummary to Synthesize Qualitative and Quantitative Descriptive Findings

Margarete Sandelowski 1,†, Julie Barroso 2,‡, Corrine I Voils 3,4,§
PMCID: PMC2329806  NIHMSID: NIHMS45489  PMID: 17243111

Abstract

The new imperative in the health disciplines to be more methodologically inclusive has generated a growing interest in mixed research synthesis, or the integration of qualitative and quantitative research findings. Qualitative metasummary is a quantitatively oriented aggregation of qualitative findings originally developed to accommodate the distinctive features of qualitative surveys. Yet these findings are similar in form and mode of production to the descriptive findings researchers often present in addition to the results of bivariate and multivariable analyses. Qualitative metasummary, which includes the extraction, grouping, and formatting of findings, and the calculation of frequency and intensity effect sizes, can be used to produce mixed research syntheses and to conduct a posteriori analyses of the relationship between reports and findings.

Keywords: qualitative research synthesis, quantitative research synthesis, systematic review

The new imperative in the health and social sciences to be more methodologically inclusive has generated a growing interest in integrating the findings of qualitative and quantitative studies (Forbes & Griffiths, 2002; Harden & Thomas, 2005; Hawker, Payne, Kerr, Hardey, & Powell, 2002; Lemmer, Grellier, & Steven, 1999; Popay & Roen, 2003), otherwise known as mixed research synthesis. Mixed research synthesis studies are systematic reviews of empirical qualitative, quantitative, and mixed methods studies in shared domains of research aimed at aggregating, integrating, or otherwise assembling their findings via the use of qualitative and/or quantitative methods (Sandelowski, Voils, & Barroso, 2006). The central challenge in conducting such studies is managing the differences presumed to exist between qualitative and quantitative research. In this article, we address this challenge and advance the use of qualitative metasummary as a technique useful for synthesizing qualitative and quantitative descriptive findings.

METHOD

This article is based on work completed in our ongoing study to develop methods to synthesize qualitative and quantitative research findings in common domains of health-related research. We began the work of this project in an area of empirical research with great significance to nurses and other healthcare providers, namely, with studies of antiretroviral adherence in HIV-positive women of any race/ethnicity, class, or nationality living in the United States. These delimitations were set largely to ensure a sample methodologically diverse enough to permit, but not so topically diverse as to preclude, the methodological experimentation at the heart of the project. Our study thus far includes 42 reports (35 journal articles, 6 unpublished theses or dissertations, and 1 technical report), retrieved between June, 2005 and January, 2006. Of these 42 reports, 12 are reports of qualitative studies; 3 of intervention studies; 1 of a mixed methods study (qualitative descriptive and pilot intervention study); and 26 of varieties of quantitative observational studies.

Using guides we had developed or adapted for reading qualitative and quantitative research reports (Alderson, Green, & Higgins, 2004; Bennett, 2005; Sandelowski & Barroso, 2007; West et al., 2002), we extracted information from each report in the following domains: research purposes and questions, theoretical framework, method and design, sampling strategy, sample composition, data collection and analysis techniques, techniques to optimize validity or minimize bias, techniques to protect human subjects, findings, and discussion. In the case of intervention studies, information also was extracted concerning the intervention and tracking of participants. Each report was reviewed by all three of us who also serve as principal and co-principal investigators of the methods study featured here.

Although reports varied in the degree to which their signal (informational value) outweighed their noise (methodological flaws; Edwards, Elwyn, Hood, & Rollnick, 2000; Edwards, Russell, & Stott, 1998), no report was excluded for reasons of quality. Scholars have increasingly argued against the a priori exclusion of studies for reasons of quality (Conn & Rantz, 2003; Cooper, 1998; Higgins & Green, 2005, Section 6) or “censor(ship) by some a priori set of prejudices” (Glass, 2000, p. 10), and for the use of typologies (as opposed to fixed hierarchies) of evaluation distinctively appropriate to them (Ogilvie, Egan, Hamilton, & Petticrew, 2005; Petticrew & Roberts, 2003). Moreover, reports and the findings in them are more or less suitable for inclusion depending on the nature of the question asked and type of analysis conducted. For example, not all of the findings from a set of quantitative reports in a shared domain of research will be amenable to quantitative meta-analysis. Most importantly, in contrast to the typical systematic review process, our bias is toward inclusion, not exclusion, of reports (Sandelowski, Voils, & Barroso, in press). The long-term aims of our methods study are to enhance the utilization of all kinds of empirical research findings in practice and to expand the methodological options available to accommodate the diversity in health sciences research.

In the course of reading these reports, we noted the similarity in form and manner of production of findings between the results featured in the qualitative research reports and those appearing in several of the quantitative reports. In these quantitative reports, researchers presented descriptive findings in addition to (i.e., not overlapping with) the results of bivariate and multivariable analyses. We wanted to include these findings as they were not subject to traditional methods of quantitative research synthesis, such as meta-analysis. We, therefore, decided to experiment with using qualitative metasummary—an approach we had developed to synthesize findings in qualitative studies (Sandelowski & Barroso, 2003b, 2007)—to combine the findings in the 12 qualitative studies, the qualitative findings in the 1 mixed methods study, and the descriptive findings in the 6 of the 29 quantitative studies that offered such findings (i.e., total n of reports featured here is 19) in order to answer the research question they addressed: Which factors favor antiretroviral adherence and which favor non-adherence in HIV-positive women? The remainder of this article is devoted to describing this method and showing why and how it was used and how it can be used to synthesize—and, thereby, to make the most use of—findings from both qualitative and quantitative studies.

QUALITATIVE METASUMMARY AND SURVEY FINDINGS

Qualitative metasummary is a quantitatively oriented aggregation approach to research synthesis we developed to accommodate the distinctive features of the topical and thematic survey findings typically produced from qualitative descriptive studies (Sandelowski, 2000; Sandelowski & Barroso, 2003a, 2007). Largely derived from manifest content analyses of individual interview or focus group data, topical surveys emphasize inventories and the numbers of research participants stating a topic or of the topics themselves. The usual format of the topical survey is to name a topic, briefly define it, and illustrate it with a few examples or quotations. Of the 12 qualitative reports and report of the qualitative component of the 1 mixed methods study in our sample, 9 presented findings in the form of topical surveys.

In contrast to topical surveys are thematic surveys that convey a latent pattern or repetition researchers discerned in their data with a stronger emphasis on qualifying findings, as opposed to simply cataloging or enumerating them. Thematic survey findings reveal a discernible effort on the part of analysts to move away from merely listing topics participants brought up in interviews and toward describing themes, or the patterned responses analysts discerned in the topics raised (Sandelowski & Barroso, 2003a, 2007). Three of the qualitative reports we reviewed presented findings in the form of thematic surveys. (We address the characteristics of the one remaining qualitative study later in this article.)

Although informative and, therefore, deserving of inclusion in systematic reviews, topical and thematic surveys do not readily permit the use of more interpretive methods generally viewed as defining qualitative research synthesis (e.g., metasynthesis, meta-ethnography; Noblit & Hare, 1988; Sandelowski & Barroso, 2007). These synthesis methods require that primary research findings be more interpreted themselves: that is, that they constitute grounded theories, phenomenological descriptions, narrative explanations, and the like. Accordingly, we developed qualitative metasummary specifically to sum up primary qualitative research findings in the form of topical and thematic surveys. Qualitative metasummary results are by themselves valuable end-products of a systematic review, and they can serve as empirical foundations for the use of more interpretive qualitative research synthesis methods. Qualitative metasummary reflects a quantitative logic whereby higher frequency findings are taken to be evidence of the replication that is both foundational to validity in quantitative research and to the claim of having discovered a pattern or theme (Sandelowski, 2001) or “preponderance of evidence” (Thorne, Jensen, Kearney, Noblit, & Sandelowski, 2004, p. 1362) in qualitative research. Qualitative metasummary entails treating research reports as indexes of the studies conducted, and the research findings in these reports as indexes of the experiences of the persons who participated in those studies (Sandelowski & Barroso, 2007).

ACTUAL VERSUS NOMINAL METHODOLOGICAL DIVERSITY

What permitted us to apply a method originally intended to synthesize exclusively qualitative survey findings to the quantitative as well as to the qualitative survey findings in our sample is their close resemblance to each other in form and manner of production.

Comparable Form of Findings

Virtually all of the qualitative reports we reviewed present findings as lists of facilitators and barriers to adherence, advantages or disadvantages, or of reasons for taking and not taking drugs, which are comparable to the lists of responses offered in the reports of the six quantitative studies. Such lists constitute an either/or logic common in both qualitative and quantitative descriptive studies focused on participants’ views. Like the quantitative surveys, these qualitative surveys are characterized by findings intended to serve as indexes of a range of attitudes, intentions, and practices reported by participants, an analytic emphasis on condensing the manifest informational contents of data, and by quasi-statistical data summaries (e.g., grouped lists, frequencies, means). These similarities in form reflect similarities in the degree of interpretation of data, as both qualitative and quantitative surveys remain interpretively close to the data originally given by participants. As shown in the two examples in Table 1, what typically differentiates the findings in the qualitative reports from the descriptive lists in the six quantitative reports is the absence in the qualitative reports of the number of women stating each facilitator or barrier to, or reason for, adherence or non-adherence. Only three of the qualitative reports contain any explicit indication of the numbers of women in each response category.

Table 1.

Similarity in Form of Qualitative and Quantitative Descriptive Findings

Report (Type of Study) Descriptive Findings
Abel and Painter (qualitative descriptive) Of 6 women, perception of facilitators of adherence:

  • Belief in benefits (to live)

  • Use of memory/visual prompts and other reminder strategies

  • Caring, continuous, and accessible communications with health care providers

  • Availability of less complex ART regimens

  • Availability of actual live person by telephone to direct their questions

Of 6 women, perception of hindrances to adherence:

  • Emotional trauma of dx and tx of HIV/AIDS (chronicity and uncertainty)

  • HIV-stigma

  • Side effects

  • Complexity of regimens
Durante et al. (quantitative observational) Of 61 women, reasons for ever missing doses:

  • Forgetting (33, 54%)*

  • Away from home (32, 52%)*

  • Busy, change in daily routine (as separate items) (25, 41%)*

  • Ran out of meds (23, 38%)*

  • Asleep at dosing (20, 33%)*

  • Sick (18, 30%)*

  • Depressed, overwhelmed (13, 21%)*

  • Too many pills (12, 20%)**

  • Not feel sick enough (8, 13%)**

  • Felt drug toxic, not want others to notice (6, 10%)**

  • High on drugs (4, 7%)*

  • Fear of interference with other meds (3, 5%)**
Open in a new tab
*

Unintentional non-adherence

**

Intentional non-adherence.

Methodological Convergence

The descriptive findings in the qualitative and quantitative reports are also comparable to each other in that they all were produced from ostensibly diverse studies that were actually more methodologically similar than different. That methods are typically honored more in the breach than in the observance is evident in the limitations section that characterizes the conventional scientific report in which researchers discuss how and why their studies did not meet expected performance standards or do not permit certain conclusions to be drawn or generalizations to be made. In another paper, we describe the extent to which the methods researchers claim were used in studies were not the ones actually used, thereby reducing the methodological diversity across studies (Sandelowski et al., in press).

Convergence in sampling

One key area widely viewed as defining qualitative versus quantitative research is the difference between purposeful and probability sampling. Purposeful sampling is directed toward the selection of informationally representative and “deliberately biased” (Wood & Christy, 1999, p. 189) cases to draw “illustrative inferences” regarding “possibility” (p. 185), while probability sampling is directed toward the selection of statistically representative cases to draw statistical inferences regarding probability. Purposeful samples, therefore, tend also to be smaller in size than probability samples.

Yet, all but one of the 19 studies featured here were conducted with convenience samples that do not permit the inferences of either purposeful or probability sampling to be drawn. Samples also converged in the homogeneity of their composition, including women in primarily minority groups (mostly African American women) in low to middle income categories. Sample sizes in the 19 studies generally conformed to expectations for qualitative and quantitative research, with a median sample size of 27 in the 13 qualitative studies (including the qualitative segment of the one mixed methods study) and a median sample size of 95.5 in the six quantitative studies.

Convergence in data collection and analysis

Qualitative and quantitative research also are said to differ in how data are obtained. The minimally structured and open-ended interviewing style typically associated with qualitative studies allows an unlimited number and unspecified nature and direction of responses and, therefore, yields data with a wider range of responses concerning a target event. In contrast, the highly structured and closed-ended questionnaire typically associated with quantitative research circumscribes the number and specifies the nature and direction of responses and, therefore, yields data with a narrower range of responses.

Qualitative and quantitative research also are defined, in part, by differences in the focus of analysis. Whereas qualitative research is said to be oriented toward singularity, or the illumination of the complex particularities of the case, quantitative research is said to be oriented toward single-dimensionality, or ascertaining differences between specified groups on a selected and relatively small number of prespecified variables (Sivesind, 1999). Data analysis in qualitative studies is case-oriented, directed toward the intensive study of a limited number of cases and toward ascertaining the confluence or configuration of events that constitute any one case. Data analysis in quantitative studies is variable-oriented, directed toward the extensive study of a large number of cases and toward ascertaining the mutual influence of a selected number of variables (Ragin, 2000).

The qualitative and quantitative studies featured here generally conformed to expectations in that the qualitative studies yielded a larger number of unique findings. Of the 63 findings shown in Table 2, 26 were contributed by only one study; of these 26, 25 were contributed by qualitative studies, including the qualitative component of the one mixed methods study. (This difference also may be due to the difference in the number of qualitative and quantitative reports included in this analysis: 13 vs. 6, respectively.) Yet, the qualitative reports depicted a largely variable-oriented approach to analysis, emphasizing factors associated with adherence or non-adherence over the particularities of adherence in individual women or styles of adherence.

Table 2.

Illustration of Grouping, Abstracting, Formatting, and Calculating Frequency Effect Sizes (ES) of Findings

Finding ES%
General findings
 Adherence is a dynamic process; women alternated between ARV adherence and non-adherence (Roberts, 2003, S. Wood) 11
 Non-adherence can be intentional or non-intentional (Durante, Roberts, 2003) 11
 Always, mostly, and somewhat adherent women were differentiated by narratives in which different sources of authority and moral accounts prevailed (Sankar) 5
Favoring Adherence or Intentions to Adhere ES% Favoring Non-Adherence or Intentions Not to Adhere ES%
Personal Characteristics/Responses/Previous Experience
Acceptance of being HIV-positive (Sankar) 5 Denial of, ambivalence about, HIV; negative emotions/emotional trauma associated with chronicity and uncertainty of HIV (Abel, Durante, Fourney, Garcia-Teague, Jones, Powell-Cope,a Sankar, S. Wood) 42
Drinking, on drugs (Durante, Erlen, Garcia-Teague, Powell-Cope) 21
Knowledge/understanding of HIV and ARVs (Fourney, Sankar) 11 Lack of knowledge/understanding of HIV and ARVs, information overload (Fourney, Garcia-Teague, Jones) 16
Active stance toward adherence (e.g., agency located in self, personal responsibility; Sankar) 5 Passive stance toward adherence (e.g., agency located in clinic, God; Sankar) 5
Priority given to individual/body wisdom (Sankar) 5
Seeing a difference, other HIV-positive people live long and well on ARVs (Fourney, Powell-Cope) 11 Not seeing, not noticing any difference with ARVs (Fourney) 5
Having, or seeing others have, HIV-negative baby on ARVs (Misener, Siegel & Lekas) 11 Having, or seeing others have, HIV-negative baby without ARVs (Siegel & Lekas) 5
Taking alternatives to ARVs (e.g., vitamins, positive thinking; Roberts, 2003) 5
Beliefs, Intentions, Desires
Belief in effectiveness, advantages, and lack of harm of ARVs (Abel, Erlen, Fourney, Garcia-Teague, Misener, Richter, Schrimshaw,b Siegel & Lekasb) 39 Equivocal about effectiveness, and short- and long-term toxicity of ARVs (to self or fetus/infant; Durante, Erlen, Fourney, Garcia-Teague, Mellins, Misener, Richter, Roberts, 2000, Schrimshaw,b Siegel & Gorey, Siegel & Lekas,b Wilson) 61
Viewing ARVs as symbol of hope and survival, way to live longer (Powell-Cope, Richter, Schrimshaw) 16 Viewing ARVs as reminder of HIV/deviant status (Gant, Roberts, 2000, Wilson, S. Wood) 21
Fear of drug resistance, mixing or interference with other drugs (Durante, Garcia-Teague, Siegel & Lekas) 16
Belief that HIV will lead to dysfunction and death (Erlen, Fourney) 11
Viewing ARVs as way to control one’s fate/disease (Richter) 5 Viewing ARVs as racist, genocidal (African American women; Misener, Siegel & Gorey) 11
Wanting to have an HIV-negative baby (Richter) 5
Motivation to stay healthy (Garcia-Teague) 5
Obligation to baby (Siegel & Lekas) 5 Belief that AZT is unnecessary if mother is healthy (Siegel & Lekas) 5
Belief in power of positive thinking (Sankar) 5
Altruism, wanting to help other HIV-positive persons (Powell-Cope) 5
Having confidence that can take ARVs as prescribed (Fourney) 5 Not having confidence that can take ARVs as prescribed (Fourney) 5
Social Support/Interactions
Absence of reference to or concern about stigma (exc. in case of children; Sankar) 5 Not wanting others to notice or know HIV status (Abel, Durante, Erlen, Garcia-Teague, Jones, Powell-Cope, Roberts, 2000, 2003, Sankar, Wilson, S. Wood) 58
Having children (to live for them; Fourney, Gant, Sankar, S. Wood) 21 Having children (care giving competes with medicine-taking; Powell-Cope, Roberts, 2000) 11
Family/friends have positive view, are supportive, remind to take (Gant, Garcia-Teague, Misener, Sankar, S. Wood) 26 Family/friends have negative view, are not supportive, cause distress (Misener, Powell-Cope, S. Wood) 16
Largely scientific authority in support of adherence (Sankar) 5 Largely personal, popular, and/or conflicting authority in support of selective adherence (Sankar) 5
Spirituality (Powell-Cope) 5
Support groups (Gant) 5
Provider Relations, Health Services
Supportive, trustworthy, accessible, or demonstrably caring MD/provider (Abel, Misener, Powell-Cope, Sankar, Siegel& Lekas) 26 Unsupportive, untrustworthy, inaccessible, or demonstrably uncaring MD/provider (Garcia-Teague, Misener, Sankar, Siegel & Gorey) 21
Receiving understandable educational materials (Abel, Fourney) 11
Access to care and to providers to answer questions (Abel, Sankar) 11 Provider told not to take (Garcia-Teague) 5
HIV/General Health Status
Having no symptoms of HIV, feeling healthy (Gant, Siegel & Gorey, Siegel & Lekas) 16 Having no symptoms of HIV, feeling healthy or better (Durante, Fourney, Gant, Garcia-Teague, Mellins, Siegel & Gorey, Siegel & Lekas, Wilson, S. Wood) 47
Feeling sick, having symptoms (Richter, S. Wood) 11 Feeling sick (Durante, Garcia-Teague, Powell-Cope) 16
Positive lab results (CD4, RNA; Powell-Cope) 5
Medication Regimen
Having no or manageable side effects of ARV (Misener, Schrimshaw) 11 Side effects of ARV (Abel, Erlen, Fourney, Gant, Garcia-Teague, Jones, Mellins, Misener, Powell-Cope, Richter, Roberts, 2000, 2003, Schrimshaw,b Siegel & Gorey, Siegel & Lekas,b Wilson, S. Wood) 89
Having less complex regimen, or one that allows integration into routine schedule (Abel, Gant, Powell-Cope, Richter) 21 Having ARV regimen that is difficult to execute in routine daily schedule (e.g., forget, asleep; Abel, Durante, Erlen, Fourney, Gant, Garcia-Teague, Jones, Powell-Cope, Roberts, 2000, Wilson, S. Wood) 58
Having ARV regimen that is difficult to execute in non-routine schedule (e.g., vacation, away from home; Durante, Erlen, Garcia-Teague, Roberts, 2000) 21
Having aids & prompts (Abel, Powell-Cope) 11 Pills hard to take, too many (Durante, Gant, Garcia-Teague, Jones, Mellins, Powell-Cope, Roberts, 2000, S. Wood) 42
Ran out of meds, not fill script (Durante, Erlen, Garcia-Teague, Jones, Wilson) 26
Tired of taking, not want to take, sold ARVs (Garcia-Teague, Jones, Powell-Cope, Roberts, 2003, S. Wood) 26
Easier to get if vouchers, refills at methadone center (Gant) 5 Hard to get ARVs (e.g., no transportation; Powell-Cope) 5
Open in a new tab

Note: To conserve space, only the first author is listed except where two reports are from the same first author. ARV, Antiretroviral.

a

Only qualitative findings included from the Powell-Cope mixed-methods study.

b

Two reports from common parent study with overlapping samples, counted as one report; denominator is 18 for findings derived from both reports.

In summary, the actual conduct of the qualitative and quantitative studies in the reports reviewed made them less methodologically different than might be expected of studies designated with these labels and, thereby, opened up the possibility that a common approach could be used to synthesize their findings.

USING QUALITATIVE METASUMMARY IN MIXED RESEARCH SYNTHESIS STUDIES

When used to signify method, qualitative metasummary entails the extraction, grouping, abstraction, and formatting of findings, and the calculation of frequency and intensity effect sizes. When used to signify the outcome, qualitative metasummary entails the presentation and interpretation of synthesis results (Sandelowski & Barroso, 2003b, 2007).

Extracting Findings

After our initial reading of the 19 reports featured here, we extracted the findings from each report. We defined antiretroviral adherence finding as any data-based result researchers reported pertaining to women’s taking or not taking their antiretroviral medications as prescribed. Because findings in qualitative reports may be presented with other elements of reports (instead of by themselves in the results section of reports), the extraction of findings entailed separating them from researchers’: (a) presentations of data (e.g., quotations, incidents, stories) as evidence for those findings; (b) references to findings from other studies; (c) descriptions of the analytic procedures (e.g., coding schemes) they used to produce their findings; and from researchers’ (d) discussions of the significance of their findings. In the quantitative reports, extraction of findings entailed lifting out the descriptive portions of the results section from the 6 (of the 29) quantitative reports containing this information.

In order to offset the lack of information in the qualitative studies on numbers of women per response or prevalence of responses across women, and the convenience sampling in both the qualitative and quantitative studies, we extracted any data-based result researchers offered pertaining to women’s taking or not taking their antiretroviral medications as prescribed, regardless of sample size. As shown in Table 2, the Roberts and Mann (2003) study of one woman thus contributed six findings. Ignoring sample size here meets the qualitative research imperative of taking account of all data no matter how idiosyncractic, and not assigning greater analytic significance to findings derived from studies with larger sample sizes simply by virtue of those larger sizes. This is an example of how “qualitative (as opposed to quantitative) thinking” (Mason, 2006, p. 10) can be used to address the challenges of combining qualitative and quantitative data.

Grouping Topically Similar Findings

After extracting the relevant findings, we grouped findings we judged to be topically similar together, as shown in Table 2. Seeing all of the findings pertaining to one topic together preserves the complexity of the findings as given in the research reports and, thereby, optimizes the descriptive validity (Maxwell, 1992) of the grouping process. Grouping findings about the same topic enabled us to discern the relationships that existed among them. For example, as shown in Table 2, a recurring finding (constituting a confirmation) was that side effects favored non-adherence; findings in a bifurcated, contradictory relationship were that both feeling sick and feeling well with HIV favored both adherence and non-adherence.

Abstracting and Formatting Findings

After all of the relevant findings were grouped, we created concise but comprehensive renderings of them, eliminating redundancies while working to preserve the complexity of their content. As shown in Table 2, we then arranged these abstracted findings to show their topical similarity and thematic diversity, and then referenced each finding with the report(s) from which it was derived.

Arranging findings in this way has the advantage of revealing findings that are not there that might theoretically or logically have been expected. We had no clear instance of this in our findings, but offer the example of the finding that difficulty accepting HIV was a deterrent to adherence (see the first set of findings under “personal characteristics” in Table 2). This finding suggests an opposite: that acceptance of HIV would favor adherence. Had we not actually found the one report with this finding, we could not have assumed it even though it might make sense to do so. Although theoretically possible findings may be derived from actual empirical findings, they do not constitute actual findings in a metasummary.

Calculating Effect Sizes

To assess the relative magnitude of the abstracted findings, we calculated their frequency effect sizes (Onwuegbuzie, 2003; Onwuegbuzie & Teddlie, 2003). Frequency effect sizes are computed by taking the number of reports containing a finding (minus any reports derived from a common parent study and representing a duplication of the same finding) and dividing this number by the total number of reports (minus any reports derived from a common parent study and representing a duplication of the same finding). For example, the frequency effect size of 39% (rounded-off number) of the finding—”belief in effectiveness, advantages, and lack of harm of ARVs (antiretrovirals)”—was derived by dividing 7, or the number of reports with this finding −1 (two reports were derived from a common parent study with overlapping samples and were, therefore, counted as 1 report) by 18, or the number of total reports −1.

To ascertain which findings reports contributed to the final set of abstracted findings, we calculated the intensity effect size of each report. This information is useful for various a posteriori analyses: for example, to determine whether any findings were derived from largely “weaker” studies, which reports contributed most of the findings with the largest frequency effect sizes across reports, and which reports contained findings no other reports contained. The intensity effect size shown in column A of Table 3 for one report was derived by dividing the number of findings with effect sizes >25 contained in that report (6) by the number of findings with effect sizes >25 across all reports (12), or 50%. The intensity effect size shown in column B was derived by dividing the number of findings contained in that report (10) by the total number of findings across all reports (63), or 16%.

Table 3.

IllustrationofIntensityEffectSizes(ESs)of(A)FindingswithFrequencyEffectSizes >25%(n = 12) and (B) All Findings (n = 63)

Intensity ESs (%)
Report A B Formatted Findings With Frequency Effect Sizes
Abel 50 16 Favoring adherence
 Belief in effectiveness, advantages, and lack of harm of ARVs (Abel, Erlen, Fourney, Garcia-Teague, Misener, Richter, Schrimshaw, Siegel & Lekas; 39%)
 Supportive, trustworthy, accessible, or demonstrably caring MD/provider (Abel, Misener, Powell-Cope, Sankar, Siegel & Lekas; 26%)
 Having less complex regimen, or one that allows integration into routine schedule (Abel, Gant, Powell-Cope, Richter; 21%)
 Receiving understandable educational materials (Abel, Fourney; 11%)
 Access to care and to providers to answer questions (Abel, Sankar; 11%)
 Having aids and prompts (Abel, Powell-Cope; 11%)
Favoring Non-adherence
 Side effects of ARV (Abel, Erlen, Fourney, Gant, Garcia-Teague, Jones, Mellins, Misener, Powell-Cope, Richter, Roberts, 2000, 2003, Schrimshaw, Siegel & Gorey, Siegel & Lekas, Wilson, S. Wood; 89%)
 Not wanting others to notice or know HIV status (Abel, Durante, Erlen, Garcia-Teague, Jones, Powell-Cope, Roberts, 2000, 2003, Sankar, Wilson, S. Wood; 58%)
 Having ARV regimen that is difficult to execute in routine daily schedule (e.g., forget, asleep; Abel, Durante, Erlen, Fourney, Gant, Garcia-Teague, Jones, Powell-Cope, Roberts, 2000, Wilson, S. Wood; 58%)
 Denial of, ambivalence about, HIV; negative Emotions/emotional trauma associated with chronicity and uncertainty of HIV (Abel, Durante, Fourney, Garcia-Teague, Jones, Powell-Cope, Sankar, S. Wood; 37%)
Open in a new tab

Presenting and Interpreting the Results of Metasummary

Because our purpose in this paper is not to report the results of a metasummary, but rather to advance and illustrate its use, the highly abbreviated results we present here are primarily to show its possibilities. Whereas we emphasize the method of metasummary here, a full report of a research synthesis study in which this method was used would emphasize the metasummary itself and its implications for practice and future research.

The results of the metasummary we conducted show approximately equal numbers of findings reported favoring adherence (31) and non-adherence (29) in 7 topical domains (Table 2), but more findings reported favoring non-adherence with effect sizes >25 (9 vs. 3). Of the nine most prevalent findings favoring non-adherence, five pertained to aspects of the medication regimen.

In the case of three of the most prevalent findings, their study prevalence is offset by their current clinical irrelevance. Advances in antiretroviral regimens (involving fewer pills to take less often) have made the findings concerning the difficulty incorporating these pills into either a routine or non-routine schedule and of taking them less relevant than they once were (Barroso, Sandelowski, & Voils, 2006). In contrast, the relative lack of study prevalence of many findings shown in Table 2 (i.e., of the 54 of 63 findings with frequency effect sizes <25%) is offset by their potentially high clinical relevance: their value for understanding, and recognizing in individual patients, subtleties in medicine-taking. Table 2 shows several such sets of common factors that favored both adherence and non-adherence: having children, feeling healthy, and feeling sick. Having children favored adherence when children were viewed as a reason to stay alive and well and when mothers felt they owed it to their babies to take antiretroviral medication; but having children favored non-adherence when their care competed with self-care, threatened disclosure to children of their mothers’ HIV status, and when mothers felt they owed it to their babies not to take these drugs. Feeling healthy favored adherence when continued good health was seen as an outcome of adherence, but favored non-adherence when it was viewed as indicating no necessity for these drugs. Feeling sick favored adherence when women felt sick enough to need these drugs, but favored non-adherence when they felt too sick to take them.

A dissertation report (Garcia-Teague, 2002) contains the largest number of findings (intensity effect size of 30%) and findings with frequency effect sizes >25% (intensity effect size of 90%). This is not surprising, as dissertation reports do not have the space restrictions of journal reports. The Sankar, Luborsky, Schuman, and Roberts (2002) report contains the largest number of findings (9) appearing in only one report. This is the only qualitative report that offers analyses of data more interpreted (or moving farther away from the data as given by participants) than topical or thematic surveys and, therefore, offers findings less amenable to simple combination with other findings. The result of narrative/discourse (as opposed to content/thematic) analyses, the Sankar et al. findings focus on the narratives differentiating always, mostly, and somewhat adherent women from each other. (See Sandelowski and Barroso (2007) for more information on incorporating highly interpreted findings into metasummaries).

CONCLUSION

We found qualitative metasummary to be a useful method for synthesizing the results of qualitative and quantitative surveys of responses obtained from similar data collection and analysis procedures. As most studies have limitations, and these limitations often reduce the methodological diversity ostensibly existing between qualitative and quantitative descriptive studies, qualitative metasummary offers a way to include such studies in research synthesis projects and, thereby, preserve the valuable information in them for practice. Indeed, what we referred to throughout this article as qualitative metasummary—to indicate a method for aggregating findings appearing in exclusively qualitative studies—can now be referred to simply as metasummary to signal its utility for systematically aggregating both qualitative and quantitative descriptive findings.

We found metasummary to be useful also for the a posteriori analyses of reports now recommended as essential components of any research synthesis project. The method allows reviewers to assess the impact individual primary reports and their findings had on the results of a synthesis. Such analyses can help clinicians to evaluate the utility of synthesis results for practice, and researchers to recognize the theoretical and methodological trends that have shaped the study of a target phenomenon.

Like any method for systematic review and research synthesis, metasummary is entirely a reflection of the judgments reviewers make throughout every phase of a review and synthesis project. These judgments concern, for example, what constitutes “qualitative” versus “quantitative” research, what constitutes a “finding,” which findings are unique enough to be listed separately, which findings are similar enough to group together, and what categories they are seen empirically to represent. The results of metasummary are, therefore, as much artifice as they are indexes of the experiences and events under investigation. There is no escaping that the entire systematic review process is an inherently artificial and judgmental one (Sandelowski et al., in press). Yet making the artifice in systematic review as transparent as publication media permit will preserve the scientific integrity and clinical utility of the process.

Acknowledgments

The study featured here, entitled Integrating qualitative and quantitative research findings, is funded by the National Institute of Nursing Research, National Institutes of Health (5R01NR004907, June 3, 2005 to March 31, 2010). We also acknowledge Career Development Award MRP 04-216-1 granted to Dr. Voils from the Health Services Research and Development Service of the Department of Veterans Affairs. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Footnotes

Contract grant sponsor: National Institute of Nursing Research in NIH; Contract grant number: 5R01NR004907.

Contract grant sponsor: Career Development Award; Contract grant number: MRP 04-216-1.

References

Included in metasummary: Qualitative report; +Quantitative report; *+Mixed methods report

  1. Abel E, Painter L. Factors that influence adherence to HIV medications: Perceptions of women and health care providers. Journal of the Association of Nurses in AIDS Care. 2003;14:61–69. doi: 10.1177/1055329003252879. [DOI] [PubMed] [Google Scholar]
  2. Alderson P, Green S, Higgins JPT. Chichester, UK: John Wiley & Sons, Ltd; 2004. Mar, Cochrane reviewers’ handbook 4.2.2 [updated quarterly]. In The Cochrane Library, Issue 1, 2004. Retrieved 5/11/04 from http://www.cochrane.org/resources/handbook/hbook.htm. [Google Scholar]
  3. Barroso J, Sandelowski M, Voils CI. Research results have expiration dates: Ensuring timely systematic reviews. Journal of Evaluation in Clinical Practice. 2006;12:454–462. doi: 10.1111/j.1365-2753.2006.00729.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bennett JA. The consolidated standards of reporting trials (CONSORT): Guidelines for reporting randomized trials. Nursing Research. 2005;54:128–132. doi: 10.1097/00006199-200503000-00007. [DOI] [PubMed] [Google Scholar]
  5. Conn VS, Rantz MJ. Research methods: Managing primary study quality in meta-analyses. Research in Nursing & Health. 2003;26:322–333. doi: 10.1002/nur.10092. [DOI] [PubMed] [Google Scholar]
  6. Cooper H. Synthesizing research: A guide for literature reviews. 3. Thousand Oaks, CA: Sage; 1998. [Google Scholar]
  7. Durante AJ, Bova CA, Fennie KP, Danvers KA, Holness DR, Burgess JD, et al. Home-based study of anti-HIV drug regimen adherence among HIV-infected women: Feasibility and preliminary results. AIDS Care. 2003;15:103–115. doi: 10.1080/0954012021000039806. [DOI] [PubMed] [Google Scholar]
  8. Edwards A, Elwyn G, Hood K, Rollnick S. Judging the “weight of evidence” in systematic reviews: Introducing rigor into the qualitative overview stage by assessing Signal and Noise. Journal of Evaluation in Clinical Practice. 2000;6:177–184. doi: 10.1046/j.1365-2753.2000.00212.x. [DOI] [PubMed] [Google Scholar]
  9. Edwards AG, Russell IT, Stott NC. Signal versus noise in the evidence base for medicine: An alternative to hierarchies of evidence? Family Practice. 1998;15:319–322. doi: 10.1093/fampra/15.4.319. [DOI] [PubMed] [Google Scholar]
  10. Erlen JA, Sereika SM, Cook RL, Hunt SC. Adherence to antiretroviral therapy among women with HIV infection. JOGNN: Journal of Obstetric, Gynecologic, and Neonatal Nursing. 2002;31:470–477. doi: 10.1111/j.1552-6909.2002.tb00070.x. [DOI] [PubMed] [Google Scholar]
  11. Forbes A, Griffiths P. Methodological strategies for the identification and synthesis of “evidence” to support decision-making in relation to complex healthcare systems and practices. Nursing Inquiry. 2002;9:141–155. doi: 10.1046/j.1440-1800.2002.00146.x. [DOI] [PubMed] [Google Scholar]
  12. *.Fourney AM. Adherence to HIV/AIDS therapies among low-literacy populations: The ALP project. Unpublished doctoral dissertation. University of Texas-Houston Health Science Center; Houston: 1999. [Google Scholar]
  13. Gant LM, Welch LA. Voices less heard: HIV-positive African American women, medication adherence, sexual abuse, and self-care. Journal of HIV/AIDS and Social Services. 2004;3:67–91. [Google Scholar]
  14. +.Garcia-Teague LA. Factors associated with appointment and medication adherence in low-income Latinas living with the human immunodeficiency virus. Unpublished doctoral dissertation. University of California; Los Angeles: 2002. [Google Scholar]
  15. Glass GV. Meta-analysis at 25. 2000 Retrieved 12/24/2005 from http://glass.ed.asu.edu/gene/papers/meta25.html.
  16. Harden A, Thomas J. Methodological issues in combining diverse study types in systematic reviews. International Journal of Social Research Methodology. 2005;8:257–271. [Google Scholar]
  17. Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising the evidence: Reviewing the disparate data systematically. Qualitative Health Research. 2002;12:1284–1299. doi: 10.1177/1049732302238251. [DOI] [PubMed] [Google Scholar]
  18. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions (4.2.5) 2005 May; Retrieved January 15, 2006 from www.cochrane.org/resources/handbook/hbook.htm.
  19. Jones DL, Ishii M, LaPerriere A, Stanley H, Antoni M, Ironson G, et al. Influencing medication adherence among women with AIDS. AIDS Care. 2003;15:463–474. doi: 10.1080/0954012031000134700. [DOI] [PubMed] [Google Scholar]
  20. Lemmer B, Grellier R, Steven J. Systematic review of nonrandom and qualitative research literature: Exploring and uncovering an evidence base for health visiting and decision making. Qualitative Health Research. 1999;9:315–328. [Google Scholar]
  21. Mason J. Mixing methods in a qualitatively driven way. Qualitative Research. 2006;6:9–25. [Google Scholar]
  22. Maxwell JA. Understanding and validity in qualitative research. Harvard Educational Review. 1992;62:279–300. [Google Scholar]
  23. Mellins CA, Kang E, Leu CS, Havens JF, Chesney MA. Longitudinal study of mental health and psychosocial predictors of medical treatment adherence in mothers living with HIV disease. AIDS Patient Care and STDs. 2003;17:407–416. doi: 10.1089/108729103322277420. [DOI] [PubMed] [Google Scholar]
  24. Misener TR, Sowell RL. HIV infected women’s decisions to take antiretrovirals. Western Journal of Nursing Research. 1998;20:431–447. doi: 10.1177/019394599802000403. [DOI] [PubMed] [Google Scholar]
  25. Noblit GW, Hare RD. Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage; 1988. [Google Scholar]
  26. Ogilvie D, Egan M, Hamilton V, Petticrew M. Systematic reviews of health effects of social interventions: 2. Best available evidence: How low should you go? Journal of Epidemiology and Community Health. 2005;59:886–892. doi: 10.1136/jech.2005.034199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Onwuegbuzie AJ. Effect sizes in qualitative research: A prolegomenon. Quality & Quantity. 2003;37:393–409. [Google Scholar]
  28. Onwuegbuzie AJ, Teddlie C. A framework for analyzing data in mixed methods research. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage; 2003. pp. 351–351. [Google Scholar]
  29. Petticrew M, Roberts H. Evidence, hierarchies, and typologies: Horses for courses. Journal of Epidemiology & Community Health. 2003;57:527–533. doi: 10.1136/jech.57.7.527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Popay J, Roen K. Unpublished report for the Social Care Institute of Excellence. Lancaster University; UK: 2003. May 23, Synthesis of evidence from research using diverse study designs: A preliminary review of methodological work. [Google Scholar]
  31. Powell-Cope GM, White J, Henkelman EJ, Turner BJ. Qualitative and quantitative assessments of HAART adherence of substance-abusing women. AIDS Care. 2003;15:239–249. doi: 10.1080/0954012031000068380. [DOI] [PubMed] [Google Scholar]
  32. Ragin C. Fuzzy-set social science. Chicago: University of Chicago Press; 2000. [Google Scholar]
  33. Richter DL, Sowell RL, Pluto DM. Attitudes toward antiretroviral therapy among African American women. American Journal of Health Behavior. 2002;26:25–33. doi: 10.5993/ajhb.26.1.3. [DOI] [PubMed] [Google Scholar]
  34. Roberts KJ, Mann T. Barriers to antiretroviral medication adherence in HIV-infected women. AIDS Care. 2000;12:377–386. doi: 10.1080/09540120050123774. [DOI] [PubMed] [Google Scholar]
  35. Roberts KJ, Mann T. Adherence to antiretroviral medications in HIV/AIDS care: A narrative exploration of one woman’s foray into intentional nonadherence. Health Care for Women International. 2003;24:552–564. doi: 10.1080/07399330390199537. [DOI] [PubMed] [Google Scholar]
  36. Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000;23:334–340. doi: 10.1002/1098-240x(200008)23:4<334::aid-nur9>3.0.co;2-g. [DOI] [PubMed] [Google Scholar]
  37. Sandelowski M. Real qualitative researchers do not count: The use of numbers in qualitative research. Research in Nursing & Health. 2001;24:230–240. doi: 10.1002/nur.1025. [DOI] [PubMed] [Google Scholar]
  38. Sandelowski M, Barroso J. Classifying the findings in qualitative studies. Qualitative Health Research. 2003a;13:905–923. doi: 10.1177/1049732303253488. [DOI] [PubMed] [Google Scholar]
  39. Sandelowski M, Barroso J. Creating metasummaries of qualitative findings. Nursing Research. 2003b;52:226–233. doi: 10.1097/00006199-200307000-00004. [DOI] [PubMed] [Google Scholar]
  40. Sandelowski M, Barroso J. Handbook for synthesizing qualitative research. New York: Springer; 2007. [Google Scholar]
  41. Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis studies. Research in the Schools. 2006;13:29–40. [PMC free article] [PubMed] [Google Scholar]
  42. Sandelowski M, Voils CI, Barroso J. Comparability work and the management of difference in research synthesis studies. Social Science & Medicine. doi: 10.1016/j.socscimed.2006.08.041. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Sankar A, Luborsky M, Schuman P, Roberts G. Adherence discourse among African-American women taking HAART. AIDS Care. 2002;14:203–218. doi: 10.1080/09540120220104712. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Schrimshaw EW, Siegel K, Lekas HM. Changes in attitudes toward antiviral medication: A comparison of women living with HIV/AIDS in the pre-HAART and HAART eras. AIDS and Behavior. 2005;9:267–279. doi: 10.1007/s10461-005-9001-6. [DOI] [PubMed] [Google Scholar]
  45. Siegel K, Gorey E. HIV-infected women: Barriers to AZT use. Social Science & Medicine. 1997;45:15–22. doi: 10.1016/s0277-9536(96)00303-6. [DOI] [PubMed] [Google Scholar]
  46. Siegel K, Lekas H, Schrimshaw EW, Johnson JK. Factors associated with HIV-infected women’s use or intention to use AZT during pregnancy. AIDS Education and Prevention. 2001;13:189–206. doi: 10.1521/aeap.13.3.189.19747. [DOI] [PubMed] [Google Scholar]
  47. Sivesind KH. Structured, qualitative comparison: Between singularity and single-dimensionality. Quality & Quantity. 1999;33:361–380. [Google Scholar]
  48. Thorne S, Jensen L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: Reflections on methodological orientation and ideological agenda. Qualitative Health Research. 2004;14:1342–1365. doi: 10.1177/1049732304269888. [DOI] [PubMed] [Google Scholar]
  49. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, et al. Evidence Report/Technology Assessment No. 47, AHRQ Pub. No. 02-E016. Rockville, MD: Agency for Healthcare Research and Quality; 2002. Systems to rate the strength of scientific evidence. [PMC free article] [PubMed] [Google Scholar]
  50. Wilson TE, Ickovics JR, Fernandez MI, Koenig LJ, Walter E. Self-Reported zidovudine adherence among pregnant women with human immunodeficiency virus infection in four US states. American Journal of Obstetrics & Gynecology. 2001;184:1235–1240. doi: 10.1067/mob.2001.114032. [DOI] [PubMed] [Google Scholar]
  51. Wood M, Christy R. Sampling for possibilities. Quality & Quantity. 1999;33:185–202. [Google Scholar]
  52. Wood SA, Tobias C, McCree J. Medication adherence for HIV-positive women caring for children: In their own words. AIDS Care. 2004;16:909–913. doi: 10.1080/09540120412331290158. [DOI] [PubMed] [Google Scholar]

RESOURCES