Abstract
Longitudinal qualitative research (LQR) is an emerging methodology in health behavior and nursing research. Researchers are turning to LQR to understand experiences across time as well as identify facilitators and inhibitors of health/illness behaviors and transitions. Currently, a lack of information exists to guide researchers on LQR techniques and considerations. Our objective was to provide a methodological resource for health behavior and nursing researchers conducting LQR. LQR may be applied to understand any human experience, as well as the sequalae of the experience and is well suited for studying transitions and developmental or behavioral changes. Conducting LQR is resource intensive and requires flexibility and complex analyses. We discuss multiple components of LQR such as design considerations, analysis options, and our lessons learned. Despite complexities, LQR provides the opportunity to understand experiences across time within an individual and among a group resulting in holistic, in-depth understandings beyond a cross-sectional time point.
Keywords: longitudinal qualitative research, qualitative research, methodological resource, nursing, health behavior research
Introduction
Longitudinal qualitative research (LQR) is an emerging methodology in health behavior and nursing research—fields focused on generating evidence to support nursing practices as well as programs, and policies promoting healthy behaviors (Glanz et al., 2008; Polit & Beck, 2017). Because human experiences are rarely comprised of concrete, time-limited events, but evolve and change across time, the use of LQR offers an innovative option to capture this natural history. The advantage of LQR over cross-sectional designs in health behavior and nursing research is that LQR provides a unique understanding of experiences across time, turning points, critical time points in transitions as well as the facilitators or challenges that support or undermine behaviors aligned with health/illness and life course transitions (SmithBattle et al., 2018). In pursuit of understanding the natural history or trajectories of human experiences, LQR generates in-depth data on the social and behavioral aspects of transitions that are less evident through cross-sectional or quantitative data alone. However, the broader nursing and health behavior research literature is deficient in resources offering theoretical, methodological, and analytical guidance on conducting LQR. To fill this gap, we developed a methodological resource to guide planning and decision making in LQR for health behavior and nursing researchers by pulling from our experiences and other disciplines such as education where more literature exists on conducting LQR. Many of the examples presented here are based on our research team’s LQR applied to better understanding the transition from pregnancy to postpartum among women living with HIV in South Africa and Kenya (i.e., K23MH116807 ELT; K01MH112443 JAP). Depending on the goals of the research team, this resource may be used in its entirety or by section.
This resource includes the following sections relevant to conducting LQR:
Background
Philosophical assumptions of LQR
Methods of LQR, including design strategies and data collection.
Analysis of LQR data, including an overview of several analysis options.
Results of LQR, including how to ensure trustworthiness of findings.
Discussion of challenges in LQR.
Table 1 includes a select group of LQR studies to serve as examples of different uses of this methodology to date.
Table 2 provides design considerations and our lessons learned.
Box 1 demonstrates the application of theory in LQR.
Box 2 presents a potential LQR design based on our research.
Table 1.
Title/Author(s) | Discipline | Setting | Population | Study Objective | Design/Timing | Analysis | Results and Commentary |
---|---|---|---|---|---|---|---|
Acceptability and Utilization of Three Nutritional Supplements during Pregnancy: Findings from a Longitudinal, Mixed-Methods Study in Niger Clermont, Kodish, Seck, Salifou, Rosen, Grais, and Isanaka, 2018 (Clermont et al., 2018) |
Public Health, Nutritional sciences and Biobehavioral health | Maradi region, Niger | N = 84 pregnant women, their families and healthcare providers | To examine the factors affecting acceptability and utilization of three different nutritional supplements among a rural population in southern Niger. The longitudinal aspect of this study aimed “to generate an understanding of change in women’s attitudes and experiences with supplement consumption over the course of the pregnancy.” | Two phases over a 6-month period including individual interviews, focus groups, and quantitative spot checks used to calculate supplement adherence. Phase 1: Individual interviews with Pregnant women n = 84, their husbands n = 18; their mothers-in-laws n = 9, and mid-wives n = 3, focus groups were also conducted with pregnant women n = 17 and health assistants n = 9. Household spot checks were also conducted at baseline n = 90. Phase 2 (4–6 months after Phase 1): repeat interviews (n = 39) and focus groups (n = 12) were conducted in late pregnancy or early postpartum and n = 50 spot checks were also conducted. | Mixed methods; “inductive approach” drawing from Grounded Theory; Authors report inductively generating an exhaustive list of codes across participants. Emergent themes were identified, and coded text was added to a thematic matrix stratified by key study variables (including supplement type) “for enhanced pattern identification.” Quantitative data from the “spot checks” (random checks to count how many portions of the supplement the women had on hand compared to how many they had been given) were used to indicate if the women were under-consuming, over-consuming or correctly-consuming the supplements. |
The qualitative results consisted of themes found to affect consumption according to supplement type with exemplar quotes. One section of the qualitative results was dedicated to describing 2 changes they observed at the group level between Phase 1 and Phase 2. The integration of Quantitative and qualitative results is not well described. The authors description of the analysis plan lacks clarity on how Time 1 and Time 2 data were analyzed longitudinally to establish the change across time (over the course of the pregnancy). |
The qualities of an effective mentor from the student nurse’s perspective: findings from a longitudinal qualitative study Gray and Smith, 2000 (Gray & Smith, 2000) |
Health and Life Sciences and Nursing Education | Scotland, UK | N = 17 nursing students | This publication presents a subset of findings from a study aimed at discovering the effects of mentorship on nursing students following educational policy reform. In this work, the researchers specifically aim “to capture changes in the students’ perspective of their mentor over time.” | Five “informal” in-depth interviews across 3-years (n = 10); diary entries submitted to the research team after completing each clinical rotation (n = 17) | “The constant comparative method of analysis”; In vivo coding was used to represent events and ideas in the participants own words. Coded data from the transcripts and memos were reduced to categories and then collapsed into five “major categories.” Using these major categories and other supporting literature, a “core category” or “theory” was developed. “The professional socialization of HE Diploma in nursing students.” | The authors present a summary of the participants views on mentorship before and after they worked with mentors in the clinical setting. They also included the participants changing views of the role of the mentor and what constitutes good or poor mentorship. Grounded Theory approach informed the analysis, however the aspect of establishing change using their longitudinal data is unclear. Moreover, it is not clear how the findings they present relate to the analysis they describe as the findings bear no clear connection to the 5 “major categories” or “core category” which they report were the outcomes of their analysis. |
Mechanisms and perceived mental health changes after a livelihood intervention for HIV-positive Kenyans: Longitudinal, qualitative findings Hatcher, Hufstedler, Doria, Dworkin, Weke, Conroy, Bukusi, Cohen, and Weiser, 2019 (Hatcher et al., 2019) |
Global public health | Migori County, Kenya | N = 54 adults living with HIV experiencing food insecurity and willing to join a patient support group | To explore “participant definitions of mental health, perceived changes in mental health due to the intervention, and mechanisms through which the intervention may have influenced mental health outcomes.” | In-depth interviews were embedded within a livelihood intervention pilot. Change was assessed indirectly by repeating the same questions after 7–9 months, as well as directly by asking about positive or negative changes in emotions. Time 1: n = 45 intervention participants interviewed 3–5 months after enrollment; Time 2: n = 31 of the 45 intervention participants and n = 9 control participants interviewed at 12 months after the start of the intervention | “Inductive-deductive team approach”; A coding framework was developed in advance according to the interview guide. Interviews were coded in large chunks under broad codes. Multiple authors examined all the excerpts under the broad code of “mental health” and applied sub-codes (“fine codes”) using inductive techniques while prioritizing the “views and voices of participants over pre-identified constructs.” Three researchers wrote analytic reports describing the sub-codes including representative quotes and divergent quotes. These reports were then used to generate theoretical constructs related to mental health change. | The authors present perceived mental health changes after participation in the intervention (i.e. reduced anxiety) with exemplar quotes from participants from before and after the intervention. They also describe the perceived mechanisms (related to the intervention) which changed the participants mental health. The mechanisms (i.e. improved food security) are some of the constructs which emerged from the analysis they describe. The description of analysis does not include how Time 1 and Time 2 interviews were handled in the analysis or how trajectories and instances of change were established. The results largely draw on how participants perceived change in Time 2. The authors preface their results with, “All quotes presented here include participant demographics, and where possible, a longitudinal perspective is explored.” |
Cross-sectional profiling and longitudinal analysis: Research notes on analysis in the longitudinal qualitative study, “Negotiating Transitions to Citizenship” Smith, 2003 (Smith, 2003) |
Social Science | Leicester, United Kingdom | N = 110 young people | The aim of the “Negotiating Transitions to Citizenship project was, “to investigate young people’s understandings and experiences of citizenship, including their sense of citizenship identity, and to consider how and why these changed over time.” | Baseline questionnaire and three interviews occurring at 1-year intervals (Time 1 n = 110; Time 2 n = 74; Time 3 n = 65) | “Cross-sectional profiling and longitudinal analysis”; Each interview was coded using predetermined codes from the first wave interview guide as well as emerging themes. Data on participants’ experiences and views were then summarized using “thematic and content analysis.” Profiles were created to show findings for each issue for each participant where all together the profile showed findings for the whole group. (See Cross-sectional profiling in Analysis of longitudinal qualitative data above for more details) | Smith describes adapting interview guides to revisit aspects of participants’ individual narratives while maintaining common themes to ensure uniformity in the data. Smith also recommends progressively narrowing the focus after each round of inquiry, so as not to end up with an unmanageably broad data set making it difficult to examine any one area of change. Results of the LQR are not presented as the intention was to discuss an analysis method developed for a LQ study on citizenship. In the Analysis of Longitudinal qualitative data section (see above), we have attempted to clearly explain specific steps which could be used to apply the cross-sectional profiling approach Smith describes. |
Trajectories of episodic disability in people aging with HIV: a longitudinal qualitative study. Solomon, et al., 2018b (Solomon et al., 2018b) |
Rehabilitation Science | Southern Ontario, Canada | N = 24 adults age 50 years, who were diagnosed with HIV more than 6 years prior | “To understand how the episodic nature of HIV and the associated uncertainty shape the disability experience of adults aging with HIV over time.” | Four semi-structured interviews occurring at 5-month intervals (Time 1 n = 24, Time 2 n = 24) | “Longitudinal analyses” guided by Episodic Disability Framework; A codebook was developed using 3 initial interviews and then all first-round interviews were coded. A cross sectional summary profile was then developed for each participant. The Episodic Disability Framework was applied to code subsequent interviews. All transcripts of the three subsequent interviews were then reviewed by investigators who documented the participants progression in the areas identified by the framework. Based on the changes or lack of change documented across the different areas, longitudinal summary profiles were independently developed by two investigators and then merged. The profiles were compared and contrasted across participants and common trajectories of episodic disability were identified. | Authors present 4 main trajectories of living with episodic disability: “decreasing disability over time, increasing disability over time, stable disability over time, and significant fluctuations in disability over time.” The application of a time/ change-oriented theoretical framework helped to orient the research objectives and analysis. The detailed account of how their analysis was carried out is also a strength of this work. Findings from this study were presented in two papers written by the same authorship team and published in the same year. The second paper (Solomon et al., 2018a) reports results only for women in the sample (n = 10) and focuses less on trajectories and more on what exactly contributes to uncertainty in the context of disability for women in their sample. The two papers do not cite one another. |
Muslim Women’s Narratives About Bodily Change and Care During Critical Illness: A Qualitative Study Zeilani and Seymour, 2012 (Zeilani & Seymour, 2012) |
Nursing and Palliative and End of Life Studies | Jordan | N = 16 Jordanian Muslim women who had undergone a minimum of 48 hours in hospital intensive care | The purpose of this work was “To explore experiences of Jordanian Muslim women in relation to bodily change during critical illness.” Where “the focus is specifically on the women’s experiences in relation to their bodies during their critical illness and the changes of these over time.” It is a presentation of findings drawn from a PhD study which “sought to describe Muslim women’s experiences of critical illness in Jordanian ICUs.” | One to three interviews at 3-month intervals (Time 1 n = 16; Time 2 n = 11; Time 3 n = 2) | “Influenced by Riessman’s method of narrative analysis (Riessman, 1993)”; All transcripts were translated and back translated from Arabic to English. Notes were made as field notes and reflective diary entries throughout data collection and analysis. The author read and reread transcripts to become familiar with and quality check the data. Detailed reflexive accounts of each interview were then written, these included non-verbal interactions and compared and contrasted the first and follow-up interviews. Reflexive accounts of each interview were then compared and contrasted and a list of codes was developed. All transcripts were coded. Similar codes were then grouped into sub-categories and sub-categories to main categories. | In a 2010 publication, the authors describe the “main categories” that emerged from their analysis related to women’s pain and suffering (Zeilani & Seymour, 2010). They later (2012) published the article referenced here which describes the theme “bodily experiences” presented as three categories: “the dependent body, the social body, and the cultural body” with example case studies. The presentation of change across time is most apparent in the description of “the dependent body” but the case studies show progression of each category between interviews. Although not stated explicitly, the methods read as though the analysis was done after data collection was complete. It is also not clear if anything different was done with the analysis between the two publications. However, they mention “applying the concept of total pain” in 2010 and the concept of “body work” in 2012. These authors demonstrate one way of disseminating extensive LQR findings through multiple publications with specific foci. They are transparent about publishing multiple manuscripts from this data by referring to their 2010 work throughout. |
Table 2.
Technical Item | Issue Confronted | Lesson Learned/Suggestions |
---|---|---|
Establishing Timepoints | Before establishing a timeline for the research study, there is a need to understand the timeline and critical events tied to the phenomenon of interest. | • Calman et al. (2013) recommend either using prior research, or a theoretical framework to establish the timeline for the LQR. • In our own research on HIV treatment across the pregnancy to postpartum transition, we relied on evidence describing the timeline for postpartum disengagement in HIV care to select our interview timepoints. Knowing when disengagement typically occurs during the postpartum period allowed us to select interview timepoints just before, and after that time. Understanding when infants were first tested for HIV and when the results were disclosed to their mothers was also important to our study’s timeline. • There may also be a need to revise timepoints based on preliminary findings. This requires flexibility and open-mindedness in order to optimize the depth and quality of data. |
Eliciting a rich narrative | Not everyone who meets inclusion criteria and consents to participate is truly willing and able to share their experiences with the research team. Some participants may give short or impersonal answers or may not fully understand the questions. For example, when asked how they feel about something they may simply respond, “I don’t know” or “I felt bad” and not elaborate much, even with probing. | • Becoming familiar with the target population, language and culture is a helpful first step, especially when trying to inquire about sensitive subjects or decide on effective interview techniques. Some populations may feel more comfortable sharing in a group setting while others may be more private. For example, we found our participants described certain experiences more freely during a focus group than during one on one interviews. • Consider pre-screening participants with one or two sample questions to assess if they are willing and able to share their thoughts and feelings about the experience of interest. • We suggest preliminary testing of guides and interview techniques when possible. • We found timely debriefings with our team after data collection and subsequently reading transcripts as soon as they were available guided adjustments to our lines of questioning as well as interview techniques. |
Sampling Techniques | Participants are selected based on a shared experience. But one’s experience of something is often heavily influenced by one’s personal characteristics. A challenge of LQR is to select a representative sample ensuring various points of view, while still having participants similar enough to allow connections and patterns in the experience to emerge. | • Carefully consider the population of interest and how different characteristics of potential participants might influence their experience of the phenomenon of interest. For example, if the population of interest is postpartum women, the experience of a single mother might vary from that of a woman with a stable partner. A sample with similar characteristics (age, parity, etc.) are more likely to have more similar experiences of change than participants who are very different from each other yet experiencing the “same phenomena.” • A structured purposive sampling method can be used to recruit along key axes of diversity (age, parity, education, relationship status) to facilitate the emergence of change across time and across the sample. • We found utilizing a clinic’s infrastructure and staff to identify and approach potential candidates was helpful. |
Retention | It is challenging to retain participants in a study over a longer period of time. | • Regular follow-up phone calls with participants in between interview encounters as well as offering financial incentives has worked very well for us. We also worked closely with a clinic to know when appointment dates were, if participants did or did not show up, if any transfers occurred and to coordinate appointment visits with data collection. • Establishing a strong trusting relationship with the participants also helps with retention. Having the same staff member conduct all interviews and follow-up phone calls between interviews can help build this trusting relationship. |
Relationships with research team | Developing trust and rapport. Recognizing when interactions are therapeutic. Study staff require support for emotionally draining work. | • The power of LQR lies in the relationships that build across time between the research team and the participants. This includes a mature respect for participants and their experience and a commitment to honoring their story through the research objectives. We invest in these relationships by thoroughly reviewing all the field notes, and interview transcripts of our participants prior to each encounter. This allows us to reference their prior concerns, feelings and situations as well as convey to them that their unique story matters to us. • These relationships may unintentionally change a participant’s experience. Participants may feel like the interview really helped them, even though it was not designed to be an intervention. It is important to acknowledge the relationship’s impact on the participant’s experience, if any. • In some cases, researchers, or their assistants in the field, may need extra support to process the emotional strain from working with participants who are sharing their personal experiences of suffering and vulnerability. Our interviewers found relief from this strain through team debriefings and discussions especially after difficult/sad interviews. |
Guide Development | What to consider when developing guides for in-depth interviews with the target population or key informants and focus group discussions. | • When using a guiding theory, operationalizing the concepts of the theory can help identify key areas of inquiry. • When possible, allow time between data collection points to interpret data as well as revise and seek ethical approval(s) for subsequent guides. • Keep the guides focused on collecting information that will answer the research questions while remaining open to understanding the complexities of the answers. • Tailor later guides to each participant when feasible; use data collected in prior interviews to personalize questions as well as to ask for clarification of or elaboration on previous responses or to inquire about how experiences may have changed since the previous interview. • Alternatively,Kneck and Audulv (2019) recommend considering a more consistent interview guide in terms of content and format. They caution that variations in guides can lead to misinterpretations of change. For example, a new question about community support in a follow up interview might stimulate a participant to state new information about the importance of the community support they have received. This could be interpreted as a change in the participant’s support system. However, the community support might have been equally as important during the initial interview when they were not asked about it and thus, did not mention it. |
Data Management | Large data sets and fluid procedures make it easy to lose sight of the study objectives or get side tracked with other questions that emerge. Managing a large quantity of data in multiple forms including: transcribing, translating, turning notes into coherent text and organizing is a major challenge. There is also a need to consider cyber security while sharing data online with remotely located research staff. |
• Constant discussion with the research team helps to balance openness to unexpected findings and the potential for new inquiry with a focus on the original research questions. • Put systems in place to securely store and manage data within the research team. • With the team’s available resources in mind, set up a detailed timeline of preliminary analysis plans including the potential coding schemas. • We used a software program to house all the raw data and perform coding and a secure server for file sharing with our transcription and translation teams. |
Fidelity | Ensuring interviews conducted cross-culturally in several languages are of high quality and answer the study questions. | • Extensive training for qualitative interviewers can ensure their understanding of the research objectives is deep enough to allow the development of meaningful translations of the study’s concepts, and questions which can be clearly understood within the language and/or cultural contexts of the participants. • Post-interview briefings help to identify questions or concepts that pose difficulties due to misunderstandings or challenges with translation. For example, we found our initial translation of “an empowered woman” was not well understood in the Dholuo language during our first interviews. After discussing in-depth with our locally based research assistant, we adjusted our line of questioning and used different words and examples to describe empowerment which were better understood in subsequent interviews. • Timely team evaluations and quality checks of translations throughout data collection are also important. |
Ethics | Protecting the privacy of participants can be difficult in LQR, especially in long term projects involving detailed personal data where it would be difficult to have complete anonymity or when data is expected to be used for future secondary analyses. | • Standard methodologies for ensuring confidentiality in cross-sectional research should be applied with LQR including utilizing participant identification numbers rather than participant names, storing documents that contain participants names (such as consent forms or locator information) securely and separately from all other study data, removing names and places from interview transcripts, and never reporting participant names in manuscripts or other dissemination materials. • We found that participants may feel differently across time about their participation or not recall study aims and thus reviewing the consent prior to each session as well as touching base between sessions to keep them engaged is important. |
Institutional Review Boards (IRB) | Investigators may need to modify interview guides, add new scales or surveys, collect different demographic data about participants, or even increase the sample size—this can be challenging as ethical review board/IRB processing times and requirements vary across institutions and organizations. Researchers can wait from a few days to several months for approval of even minor changes, depending on geographic location and/or multiple research sites. | • Thinking ahead as best as the team can—and prepare for anticipated changes whenever possible. It is also important to have a good understanding of how the IRB you are working with operates (i.e. when they meet, how to communicate with them and typical processing times). • We have found it helpful to reiterate multiple times in the initial IRB application that our study was iterative with changes to interview guides dependent on data collected in ongoing interviews. Thus, in the initial IRB application we only sought approval for our first interview guide and indicated that we would seek IRB amendments prior to each subsequent interview. Depending upon researchers’ experiences with their own IRBs this may be an acceptable approach, or IRBs may require interview guides for all time points during the initial application review process. |
Dissemination | Deciding at what point to share your findings and whether or not to publish baseline data. | • There may be very rich findings in the baseline data, but the risk of publishing it may jeopardize the power of the longitudinal results. The benefit of publishing baseline data as exploratory research is to encourage rigorous analysis of the initial status of participants’ lives before analyzing for change (Smith, 2003; Thomson & Holland, 2003). |
Box 1. Potential Theoretical Framework for LQR and its Application.
Transition Theory (TT)
(Chick & Meleis, 1986; Meleis et al., 2000) is a well-suited model to describe health behaviors that occur across time and across a transition, such as diet and physical activity across the transition from pregnancy to postpartum or drinking behaviors across the transition from adolescence to early adulthood. TT allows researchers to characterize and describe transitions, such as those described above, and define the relevant personal, community, and societal characteristics that may facilitate and inhibit health behaviors necessary for a successful transition.
Application:
In our recent study of HIV treatment adherence among pregnant and postpartum women we utilized Transitions Theory (TT) to examine maternal motivations, behaviors, and social contexts from the 8th month of pregnancy up to a year postpartum in Cape Town, South Africa (Pellowski et al., 2019). During the design process, TT was chosen as an overarching theoretical framework for the study and influenced the sample of interest (recruiting women during pregnancy as opposed to only focusing on the postpartum period). TT also guided the main sections of each interview agenda with a focus on the concepts that the theory posits to be the most influential on health behaviors across this transition (e.g. personal meanings, cultural beliefs and attitudes, socioeconomic status, preparation and knowledge, partners, families, and community stigma). Additionally, the four interview time points were carefully selected to align with what TT describes as critical points and events: 1) late pregnancy was chosen to capture preparations for the transition, 2) 6–8 weeks postpartum was chosen to capture reflections on birth and initial impacts of the newborn on daily life, 3) 4–6 months postpartum was chosen to capture early infant HIV testing, the end of exclusive breastfeeding/introduction of other foods, and maternal transfer of HIV care services, and 4) 9–12 months postpartum was chosen to capture possible disengagement from HIV care and the end of the breastfeeding period for many women in this context. Finally, TT was used to guide the analysis, which utilized an inductive approach. It is of note that the application of theory to an LQR project does not restrict the types of analyses that can be utilized (e.g. inductive vs deductive approaches). In this example, TT was used as a guiding framework to ensure that all phenomena relevant to the transition from pregnancy to postpartum for women living with HIV were captured in data collection. During data analysis, the main objective was not to support or refine the theory (deductive approach), but rather to understand and make generalizations about the unique experiences of women with the constructs defined by TT serving as general reference points (Pellowski et al., 2019).
Box 2. An Application of the Serial Interview Approach from NIH K23MH116807.
A sample of 30 perinatal women were interviewed during their 3rd trimester of pregnancy. They were asked about their breastfeeding intentions, expectations and past experiences with breastfeeding. Based on the findings from the initial interview the women were interviewed again at 6 weeks postpartum. The 6-week interview focused on how the reality of breastfeeding compared to the intentions and expectations the women had prior to giving birth, while looking for newly identified inhibitors and facilitators of breastfeeding behavior. Based on these findings, the researchers tailored a third interview guide to be used when the women reached 5–6 months postpartum to inquire how breastfeeding behaviors evolved as infants grew while exploring new contexts, such as mothers returning to the workplace.
Background
What Makes Longitudinal Qualitative Research a Distinct Methodology?
We have already introduced LQR as an emerging methodology. However, depending on one’s understanding of what a research methodology entails, LQR may appear to be something too broad or flexible to be considered a distinct methodology in and of itself (McCoy, 2017). This seems especially true when LQR is held up against long established qualitative methodologies (with more prescriptive methods) such as Grounded Theory (Glaser & Strauss, 1967; Glaser, 1978), Ethnography (Pelto, 2013) or Phenomenology (Colaizzi, 1978). Further confusion surrounding LQR’s classification as a methodology may stem from the substantial overlap of qualitative techniques and procedures between methodologies (Hermanowicz, 2013). Indeed, many LQR studies include the use of data collection techniques or analysis procedures commonly used in other qualitative methodologies.
We propose, however, that LQR exhibits all of the defining characteristics of a unique qualitative methodology (Carter & Little, 2007), including distinct research objectives, foundational assumptions, and well-developed explanations of the methodological and analytic principles as outlined in the following sections. Central to qualitative research, while some procedures or techniques may overlap between methodologies, the research objectives, assumptions, and principles of the chosen methodology should justify the procedures/techniques used (Carter & Little, 2007). For example, LQR may not simply apply a Grounded Theory analysis plan because Grounded Theory analysis procedures do not account for change across time (a primary objective of LQR). However, an inductive thematic analysis (as is applied in Grounded Theory) might be used in LQR to cross-sectionally analyze baseline data in order to identify emergent themes from the initial research encounter. Similarly, LQR studies may employ ethnographic data collection techniques such as observing behaviors across time. However, while ethnographic studies aim to understand a cultural phenomenon or behavior from the viewpoint of participants (De Chesnay & Abrums, 2015) LQR aims to establish a shared understanding of how and why the phenomenon or behavior changes across time. Thus, the management and analysis of data in LQR is inherently different from other methodologies.
What are the Unique Objectives of Longitudinal Qualitative Research?
LQR’s distinction is in its aim to understand an experience or behavior(s) across time; explicitly seeking to answer, “how did this change?” “how is this different?” “why did this change?” and/or “what remains the same?” (Saldaña, 2003). LQR designs have been applied in a variety of research areas including, transitions in human development (Schmidt et al., 2019), the experiences of incarceration (Cooper et al., 2015), aging (Oosterveld-Vlug et al., 2013) and the progression of chronic illness (Namukwaya et al., 2017), as well as behavioral research investigating medication adherence (Salter et al., 2014; Weiser et al., 2017) and breastfeeding (Doherty et al., 2006; Jardine et al., 2017). LQR may be applied to understand any human experience, as well as its sequalae and is particularly well suited for studying transition periods and developmental or behavioral changes across time. LQR may also be applied to inform the development of health behavior theories or interventions and may be used to understand if a policy or program was effective, why or why not and in what contexts might similar results be expected (Lewis, 2007; see Table 1 for selected examples of study objectives).
Philosophical Assumptions of Longitudinal Qualitative Research
Although the origins of LQR are not strictly defined, there are several assumptions that comprise the philosophical underpinnings of the methodology. First, LQR is based on the assumption that two key concepts—time and change are contextual (Saldaña, 2003). While LQR often occurs over months to years, it is not only the chronological passing of time that creates meaning, but rather how the individual experiences that passage of time (Saldaña, 2003). Different points in a person’s life may make the experience of time qualitatively different from another—passing quickly when one is busy or having fun and slowly when one sits quietly. In addition to personal level variation in the experience of time, the cultural context may also influence the interpretation of time. For example, the acceptable age for marriage or first-time parenthood can vary greatly across cultures, which may impose different time-based milestones or experiences in one’s life.
Since time and our human experiences within it are both contextual, the change we experience across time is also contextual (Saldaña, 2003). Change may not be a linear or ordered journey from one state to another with a definitive end point. Thus, the depth of transitions may not be captured when change is viewed in isolation either as a single unit of analysis or as a solitary episode. LQR assumes the need to explore the complex, haphazard and potentially contradictory ways change emerges and to conceptualize the pathways in which these complexities in experiences and behaviors exist across time (Pettigrew, 1990). Overall, LQR assumes change is multi-faceted and holistic where continuity, patterns, idiosyncrasies, and contexts are key components (Pettigrew, 1990).
The second assumption in LQR centers on the human experience being a construct of the participants’ personal reflections and the researchers understanding of them, allowing multiple realities to exist simultaneously (Balmer & Richards, 2017; McCoy, 2017). Furthermore, the construction of these experiences relies on the notion that participants are willing and able to articulate their experiences in a way that can be understood by the researcher (Baillie et al., 2000). In qualitative research, and LQR in particular, participants share their experiences and researchers listen, analyze, and interpret these experiences. Researchers may present their findings back to the participant for their evaluation or ask the participants about the same experience again at a later timepoint to evaluate how their experience or their feelings about it may have changed. Through this process, the essence of the experience across time is established for each participant (Balmer & Richards, 2017; McCoy, 2017).
Methods of Longitudinal Qualitative Research
There are no gold standards or fixed rules for data collection in LQR. In general, LQR applies either prospective or retrospective designs that include two or more data collection sessions using qualitative techniques (e.g., interviews, observations, multi-qualitative methods) over a specified time frame (Saldaña, 2003). Yet, the defining principles of data collection in LQR go beyond having data collected at multiple time points. The chosen data collection techniques in LQR must also ensure the quality of data collected as well as cater to the researchers’ abilities to systematically manage and thoughtfully analyze these data across time (Smith, 2003). The researcher’s ongoing assessment of data coupled with the flexibility to make adjustments are hallmarks of LQR methods.
General Design Strategies
Designing LQR studies that effectively capture change is not straightforward. Two overarching complexities are, 1) the length of time needed to be considered longitudinal is not definitive and 2) a universally accepted definition of change does not exist, making it challenging to identify change processes or outcomes across time a priori (Pettigrew, 1990). These complexities are key, however, for researchers to work through as they consider the change they are seeking to understand and the corresponding outcomes. Some design strategies to consider include theoretical frameworks, target population and size, setting and personnel (see Table 1 for selected examples of LQR designs, see Table 2 for additional design considerations and personal lessons learned).
Theoretical framework.
A theoretical framework is chosen based on the research objectives (See Box 1 for an example of a theoretical framework and its application in LQR). Theoretical frameworks are particularly helpful in identifying concepts relevant to the phenomenon of interest and how these concepts may change across time to influence behavior (Chinn & Kramer, 2011). A theoretical framework should be chosen at the outset of project planning and inform 1) sample(s) of interest, 2) content of data collection (e.g. questions/probes developed for in-depth interviews), 3) timing of data collection and, 4) plans for data analysis. Researchers can then operationalize and explore concepts from the framework by asking: How could we define and measure these concepts in the context we are interested in? What information would help us describe and understand these concepts across time? In addition, researchers must remain open to new concepts and pathways that emerge from their data.
Identifying the target population and sample size.
Participants in longitudinal studies are selected based on their shared experience of the phenomenon of interest (Saldaña, 2003). Yet, an individual’s experience is distinct and close observers (friends, family, or caregivers of the individual) can also lend valuable insight (Johansen et al., 2013). Moreover, LQR does not limit the unit of analysis to individual participants. Data might also be collected from focus groups, families, or groups of co-workers (Johansen et al., 2013; see Table 1 for other examples). Thus, researchers must carefully consider who to collect their data from and how many units of analysis (individuals, focus groups, families, etc.) are needed to adequately address the research aims (Kneck & Audulv, 2019). In LQR, in particular, researchers must also anticipate a certain level of attrition because over time participants may migrate, die, or simply lose interest in participating in the study (Calman et al., 2013; Kneck & Audulv, 2019). One approach researchers may use to determine sample size is estimating the number of cases needed to reach saturation (Hennink et al., 2017), which for a phenomenology design is typically 10–12 participants (Polit & Beck, 2017). Saldaña (2003) recommends LQR studies start with more participants than you anticipate needing to ensure data saturation is reached, especially if a study takes place over two or more years. Because the context, study design, population, and setting are study specific, determining a certain number or percent to overestimate on sample size is best left to the research team’s judgment, which is based on the stability of their target population. In a systematic review of LQR in nursing, attrition was either a major limitation (20% of studies estimated 50% attrition) or a major strength (30% of studies had 0% attrition; SmithBattle et al., 2018). Given these extremes, during the planning phases of the research, attention should be given to understanding sample characteristics including potential barriers to long term participation.
Setting.
A number of considerations are helpful when determining study setting in LQR. First, the venue must be convenient for the participants over the study period such as one close to the participants’ home or a venue the participants frequent such as their health clinic. Second, if the research team is conducting their study within a clinic or hospital where participants are patients, gaining the support of the clinicians and administrators prior to the start of the study and maintaining strong relationships throughout the study period is key to a collaborative, lasting partnership. Support from stakeholders ensures the desired space is reserved, the study does not disrupt the patient flow, and that the research encounters can be coordinated with participants’ regularly scheduled appointments. Third, the study team needs a private, quiet and secure location where participants will be able to focus on the interview questions while feeling relaxed and comfortable enough to fully express thoughts and experiences. This will also mitigate interruptions and background noise which may distract the participant and detract from capturing clear audio recordings. Fourth, supplying refreshments, child care, and easy access to restrooms may lead to a better experience for participants. Finally, if the researcher chooses to collect data in the homes of participants, the added value of observing participants in their own environment must be weighed against the challenges of working in a less controlled setting (more distractions, interference from other people in the home, potential safety concerns for the researcher, etc.) as well as privacy concerns (particularly when discussing stigmatized diseases or behaviors). Whatever venue is chosen, to the extent that it is feasible, maintaining the same venue throughout the duration of the LQR provides important design consistency and familiarity for participants, which may help retention. Some of these items may be relevant for cross-sectional studies as well, however, we have found that accounting for the aforementioned considerations are of paramount importance in LQR as they nurture long-term participation.
Personnel.
LQR is labor intensive as collecting, organizing and analyzing data is time consuming. Researchers should plan ahead, mapping out the time required for each phase, strategically selecting who will carry out each task and which tasks are best executed collaboratively. Many different skills may be required including, interviewing, conducting focus groups, videography, transcribing audio, translating transcribed text, organizing and managing data and finally conducting the analysis. In addition, there are other demands on staff time including, 1) reviewing and quality checking initial data, 2) revising subsequent interview protocols and guides and 3) maintaining contact with participants between study sessions. The research team should consider the different skills each staff member brings to optimize effectiveness of the study procedures. For example, team members with knowledge of the local language and culture who conduct interviews may also provide invaluable insight into the interpretation of data and its meaning beyond the strictly literal translations of the interviews. Additionally, planning for the same study team member(s) to interact with participants at each data collection point optimizes rapport and trust and aids in retention efforts—particularly when the LQR is occurring over long periods of time (months and years; Nevedal et al., 2018). Managing the ebb and flow of workloads across data collection time points requires the thoughtful organization and adaptability of project coordinators in collaboration with principal investigators.
Steps for Data Collection
Step one: Operationalize concepts, including time and change.
Conceptually, the notion of time may be different between participants or from the research team’s design expectations. To alleviate this potential disharmony, Pettigrew (1990) suggests that the research team clearly operationalize the concepts of time and change at the outset of the study (as discussed in the “Philosophical Assumptions of Longitudinal Qualitative Research” section above). In some cases, the “baseline” (starting point) from which the change/transition of interest begins may not fall within the first interview. For example, when looking at the experience of living with HIV, the baseline might be when the person was first diagnosed with HIV (i.e., years prior) or rather the first time they engaged in treatment sometime after their diagnosis. Change may also be absent across time, which may reflect positive or negative behaviors (maintaining medication compliance vs. maintaining unhealthy habits; Lewis, 2007; Saldaña, 2003).
Step two: Type of data to be collected.
LQR data may originate from interviews with members of the target population, or with key informants such as family, friends, clinicians or other stake-holders. Data may also come from short answer surveys, focus group discussions or direct observations (Johansen et al., 2013). Initially, data may be in the form of audio recordings, videos, pictures, drawings or field notes. In some cases, LQR studies are embedded in randomized control trials or mixed-methods studies where various types of data were collected. For example, a study on depression might use an established screening tool to assess depression scores at each encounter prior to conducting in-depth interviews with participants. There are no restrictions or limitations to type or quantity of data collected, only the a priori considerations of the desired contribution from each data source, data management and data analysis plan.
Step three: Study approach.
There are several approaches to consider in longitudinal qualitative inquiry. The primary approach used in LQR is serial interviews (Calman et al., 2013; Murray et al., 2009). This approach utilizes emergent issues or themes from one interview to inform the line of inquiry used in subsequent interviews. The time between data collection points allows the research team an opportunity to review the data and modify interview guides (Smith, 2003). Subsequent interviews can then be designed to build on rather than duplicate the previously collected data. Importantly, process notes/interview summaries and frequent debriefing of interviews is key to ensuring subsequent interviews are on target (See Box 2 for an example of a study on breastfeeding behaviors using the serial interviews approach).
Step four: Triangulation of data.
This step is meant to validate preliminary findings and ensure data completeness and trustworthiness. There are several ways to triangulate data. For example, findings from interviews with key informants, or focus group discussions can be compared to findings from in-depth interviews with individuals to compare completeness and consistency in findings. Another option is to conduct a final exit interview with each participant (Saldaña, 2003). The purpose of a final exit interview is to present the researcher’s findings to the participant for feedback. Questions may be asked to confirm or disconfirm preliminary assertions, themes or trends. Participants may also be asked to reconstruct their experience within the study timeframe. Similarly, questions could be focused on areas of uncertainty or missing details revealed by the analysis and interpretation. These final insights can assist the research team in confirming their description of change across time using a collaborative, reflective and flexible approach (Pettigrew, 1990).
Analysis of Longitudinal Qualitative Research Data
Longitudinal qualitative data analyses attempt to transform data into explanations and insights which address the original research objective—understanding an experience or behavior across time. Analysis in LQR is challenging on many levels given the large amounts of data to analyze (Lewis, 2007; Pope et al., 2000; Smith, 2003), the multiple types of data such as field notes, interview summaries, surveys, transcripts or even videos (Miles et al., 2014) as well as the challenge of describing how the experience may change across time within participant and among a group.
The research team is tasked with managing data collection, revision/development of subsequent interview guides and possibly even initiating data analysis while data collection is still ongoing (McLeod & Thoon, 2009; Pope et al., 2000). This is especially challenging because carefully transcribing (and when necessary translating) data is time consuming and it is not always feasible to allow ample time in between data collection time points for analysis to be completed (McLeod & Thoon, 2009). Some studies are chronologically time sensitive such as those seeking to understand distinct developmental time periods that would not be captured if data collection were postponed to a later date—early parenthood for example. In these cases, detailed process notes or summaries of individual interviews and frequent debriefings with study staff may be crucial for informing subsequent rounds of data collection. Bearing in mind the aforementioned challenges, what follows are the central analytic principles and procedures for LQR analyses (see Table 1 for selected examples of LQR analyses).
Step 1: Consider the Analysis Approach
The analysis of LQR data can be carried out using a variety of different approaches with the precise methods used often evolving alongside the data collection (Saldaña, 2003). Applying a deductive and/or inductive lens is often a good starting place. Using a deductive approach, researchers begin with a theory or framework in mind and analyze their data to identify specific findings that lend support to, clarify, or refine the theory/framework (Burnard et al., 2008). If applying an inductive approach, researchers start from their original observations and seek to find patterns or make generalizations about their data eventually using their findings to create a theory or framework, establish pathways, or to develop themes or categories related to the phenomenon of interest (Burnard et al., 2008). Researchers can also fall somewhere in between relying on predetermined codes or a framework to organize their data while still trying to identify new patterns or generalizations emerging from the data (see Box 1 for an example of this).
Researchers should also consider if their research objectives are best suited to a diachronic or synchronic analysis approach. Synchronic analysis implies analysis is simultaneous (synchronized) with data collection or occurring as a cross-sectional analysis after each wave of data collection (Nevedal et al., 2018). Synchronic analyses are common in LQR because data collection and analysis are often a fluid process where initial and ongoing analyses are imperative to inform subsequent data collection encounters (Balmer & Richards, 2017; Calman et al., 2013; Pope et al., 2000). Researchers must stop and ask, “what do we know so far?” “what have we missed?” and “what do we need to know more about to fully understand this experience?” The next round of inquiry is then directed accordingly (Pope et al., 2000). As mentioned, in some instances, synchronic analysis may be less feasible due to time constraints or less important for achieving the study objectives. In these cases, researchers may opt for a diachronic approach, meaning they wait to conduct their analysis until after all data has been collected. Of note, researchers may also choose to conduct both synchronic analysis (cross sectional, after each research encounter) and diachronic analysis (longitudinal, using all data once data collection is complete).
Step 2: Setting Up an Analytic Roadmap
Regardless of the chosen approach, an analytic roadmap outlining the specific steps of analysis is critical to providing direction given the complexity of LQR data. As the study progresses, the initial roadmap may change, and when this happens documenting how the path taken differs from the original plan is needed. A clear and auditable “trail of decisions” (Guba & Lincoln, 1981, as cited in Sandelowski, 1986, p. 33) can establish the dependability of results in qualitative research. Thus, recording when and how decisions about conducting the analysis were made is important for the research team’s reference as well as future reporting of results (see Trustworthiness of Longitudinal Qualitative Research below). The roadmap documentation should include: detailed explanations of what was done, when, and why as well as what did and did not lead to meaningful findings.
Step 3: Familiarization and Coding
After converting raw data (audio recordings, field notes, etc.) into coherent text, the next step of most analytic roadmaps is to read and reread transcripts to become familiar with the content, start identifying potential themes, and assess data quality and effectiveness of the interview guide. For some researchers, highlighting excerpts and adding comments or descriptive memos is also useful during this time, whether by hand or with qualitative data analysis software. Discussing initial data and data quality within the research team is also a part of this process. This is especially important in research teams where different members conducted the interviews and others are leading the analysis. Constructive feedback from the team can provide direction and suggestions for the interviewer in the next round of data collection while the interviewer can offer insight about the interactions with the participants (such as their tone or body language) that may not be fully evident to team members reading the transcripts.
After team discussions on data quality, the next step is often applying codes to the text. This could be a predetermined list of codes or one that emerges from the text. There are many different types of coding schemas such as descriptive coding, versus coding, or in vivo codes that one can apply to suit their analysis (for a comprehensive review on types and procedures for coding see Saldaña, 2009). In addition, one can apply the long table or manual approach to code data or use a qualitative data analysis software (Polit & Beck, 2017). Regardless of the type(s) of codes or method by which the coding is done, the objective is to inductively and/or deductively apply codes (labels) to segments of data for the purpose of grouping and organizing thematic segments as well as highlighting exemplar excerpts.
In LQR, there may be one or more members of the research team coding data. Having multiple members of the team coding has several advantages. First, this allows for the inter-rater reliability or the degree of agreement between coders to be assessed. Higher inter-rater reliability shows that codes were applied consistently and supports the rigor and trustworthiness of the study (see trustworthiness of LQR below; Tracy, 2010). Moreover, when more than one team member is coding there is opportunity to discuss discrepancies in the application of codes. This guides the team in developing codes with more complete and articulate definitions as well as develops a deeper common understanding of the meaning of each code (Miles et al., 2014). In addition, when various team members code transcripts inductively (without a predetermined code list) multiple perspectives may emerge and be useful, both in terms of capturing all of the possible emerging codes and also in terms of distinguishing between an individual coder’s interpretation of the text and the participants intended meaning (Pope et al., 2000). Conversely, some researchers prefer to have one member do all the coding. An advantage of this approach is that one person can be fully immersed in all the data which may optimize consistency in the analysis. It may also be a pragmatic decision; for example, when an ethnographer embedded in their field site conducts all the data collection and proceeds to do the analysis, this may result in a consistent, comprehensive and thoughtful telling of an experience (Saldaña, 2003).
Step 4: Describing Cross-sectional Data
Analysis of coded data in LQR frequently begins as a cross sectional analysis of the first round of data collected and can include repeated cross-sectional analyses as the researchers work to understand the experience at each timepoint of data collection (Nevedal et al., 2018). Cross-sectional analyses are often conducted using techniques borrowed from other methodologies such as thematic analyses, where coded data are grouped into common sub-themes, sub-themes are grouped into themes and themes into broad categories. Importantly, a meaningful analysis must subsequently attempt to develop a longitudinal (across time) description of the themes or experiences (Nevedal et al., 2018). As the analysis moves from cross sectional to longitudinal it evolves from descriptive (i.e., describing the changes observed) to exploratory (i.e., uncovering the causes and consequences of change or lack of change across time) (Kneck & Audulv, 2019; Lewis, 2007).
Step 5: Exploring Longitudinal Data
The final analytical leap from descriptive cross-sectional to exploratory longitudinal is often poorly described in LQR (Calman et al., 2013; Nevedal et al., 2018). This is likely because, until recently, neither prescribed nor clearly explained analysis plans for longitudinal data have been documented (Sheard & Marsh, 2019). Within the LQR methodology, researchers are developing variant and sometimes discipline specific analysis techniques consistent with the objectives, assumptions, and principles of LQR (Carter & Little, 2007; Sheard & Marsh, 2019). Such analysis plans primarily aim to find patterns of change across time and include: Longitudinal Interpretive Phenomenological Analysis (see McCoy, 2017), the Pen and Portrait Technique (see Sheard & Marsh, 2019), and the Trajectory Approach (see Grossoehme & Lipstein, 2016), which are described in detail elsewhere. In addition, there are the following approaches we describe in detail below:
Longitudinal analysis approaches
Framework Analysis (Lewis, 2007).
Framework analysis organizes data into one table for each participant (or other unit of analysis) which can then be used to find patterns across participants, across time, and across various identified themes. Patterns might be similar behavioral changes, similar feelings about an experience, or related changes in themes across time. For example, a change in a participant’s understanding of their own health condition may be closely linked to the services they are inclined to access (Lewis, 2007). The rows of the table (sometimes referred to as a framework or matrix) are labeled as the participant encounters (one row for each encounter) while the columns of the tables are topics or themes identified from the theoretical framework, the interview guide, or the initial readings, coding and/or thematic analysis of data (see Table 3). Additional columns can be left open for emerging themes (Lewis, 2007). The table is filled in with summaries from each participant in each cell as applicable. Kneck and Audulv (2019) suggest using descriptive summaries during this phase so as not to make any “analytic leaps” too early in the analysis. This process helps remedy the challenge that arises should there be a misinterpretation of data early on in the analysis process upon which future analyses are then based—making it challenging to look back and identify where the misinterpretation occurred. The cells of the table may also include salient words or phrases cut and pasted directly from the transcripts. Reading down the columns the researcher can explore the themes across time, while reading along the rows of the tables the researchers can explore the linkages between themes at a given timepoint. Researchers may also “zig-zag” through the tables to identify other patterns or trends (Lewis, 2007). As these fully populated descriptive tables are explored and analyzed, the researchers can create a second “analysis matrix” where each row represents one unit of analysis and the columns continue to represent the topics/issues/themes of interest. The analysis matrix is then populated with the researcher’s interpretations of how each theme changed (if at all) across time, for each unit of analysis (individual, focus group, family, etc.; Grossoehme & Lipstein, 2016).
Table 3.
Phase 1 (One table for each participant) | |||||
---|---|---|---|---|---|
| |||||
Theme 1 | Theme 2 | Theme 3 | Emerging theme | Summary | |
Baseline | What is happening with the participant at this timepoint? What requires follow up? | ||||
Time 2 | |||||
Time 3 | |||||
Time 4 | |||||
Phase 2 (Combining participant findings across time by theme into one table) | |||||
| |||||
Theme 1 | Theme 2 | Theme 3 | Emerging theme | Summary | |
| |||||
Participant #1 | Description of theme’s change across time for the participant. | Overall change across time for the participant. | |||
Participant #2 | |||||
Participant #3 | |||||
Participant #4 | |||||
Summary | Trends in the theme’s change across time for the group. |
Cross-sectional Profiling (Smith, 2003).
Cross-sectional profiling develops descriptive summaries of each theme, issue, or topic identified for each participant whereby the participant’s thematic profile is developed further with every encounter (Smith, 2003). The summaries might also be arranged as tables with a separate table for each theme, each row representing a participant with a column for each encounter (see Table 4). A profile contains a summary of the researcher’s findings related to a specific theme for each participant for each encounter. Within each table, the individual participants (the rows of the table) may be organized in groups according to demographic characteristics, intervention vs. control, or outcomes. Initially, the first column(s) of the profile table (the participants experience of the theme at the first research encounter) guides further inquiry. For example, Smith (2003) identified ineffective lines of questioning related to one of their interview topics in a first wave of profiling and subsequently adjusted their approach. Once the profile is complete (contains summarized data from each participant at each time point), the researcher establishes the overall narrative of change for each theme for the entire group as well as the sub-groups. Then the individual narratives of change can be viewed relative to the narrative of the entire group or subgroup to which the participant belonged (Smith, 2003). In this way the researcher can understand patterns and facilitating or inhibiting factors for individual change as well as develop individual case studies of change within a particular theme. The case studies can then be explored in terms of theme’s findings for the whole group—is it an exemplar or deviant case, or is the change more or less significant than among other participants (Smith, 2003)?
Table 4.
Theme 1 | Time 1 | Time 2 | Time 3 | Participant Summary | Group Summary |
---|---|---|---|---|---|
Group 1 (Single Mothers) | |||||
| |||||
Participant #1 | Participant#1’s experience related to Theme 1. | Continuation of Participant#1’s experience related to Theme 1, building on Time 1. | Continuation of Participant#1’s experience related to Theme 1, building on Time 2. | Summary of Participant#1’s experience of Theme 1 across time. | Theme#1 as experienced by Single Mothers across time. |
|
|||||
Participant #5 | |||||
Participant #8 | |||||
| |||||
Group 2 (Mothers with a supportive partner) | |||||
| |||||
Participant #2 | |||||
Participant #3 | |||||
Participant #6 | |||||
| |||||
Group 3 (Mothers with an unsupportive partner) | |||||
| |||||
Participant #4 | |||||
Participant #7 | |||||
Participant #9 |
Case Histories (Thomson, 2007).
This type of analysis uses archives of data to construct accounts of change and continuity across time including the researchers understanding of why things happened the way they did (Thomson, 2007). Researchers use multiple data sources (interview transcripts, field notes, diaries, or notes from focus groups) and synthesize large amounts of information to develop a storyline for each case (individual or group) narrating change or continuity across time (see Table 5; Thomson, 2007). Case histories go beyond the descriptive level as researchers form a more analytic narrative of the case throughout (Henderson et al., 2012). Sheard and Marsh (2019) describe a similar technique which they refer to as the “pen and portrait analytic technique.” They recommend researchers focus the summaries on the information that is pertinent to the research questions—perhaps centering them around an important theme identified by the researchers. In this way the narratives help to focus the analysis rather than simply serving as an all-encompassing summary. Researchers then use the case histories or narratives to analyze trends. They can group individual case histories by demographics, intervention vs. control or outcomes looking for similarities and differences between the groups as well as exceptional cases within groups. Thomson (2007) describes putting individual case histories “in conversation with each other.” She tried to understand the differences and similarities from different perspectives such as the perspective of the individual versus the perspective of society. Using individual case histories, researchers may also seek to explain why two seemingly different cases have similar outcomes or why two similar cases have different outcomes (Lewis, 2007).
Table 5.
Time 1 | Time 2 | Time 3 | Summary | |
---|---|---|---|---|
Participant #1 | Participants narrative together with the researcher’s insights. | Continuation of the narrative with the researcher’s explanation of change including how and why it may have occurred. | ||
Participant #2 | ||||
Participant #3 |
Pattern-Oriented Longitudinal Analysis (Kneck & Audulv, 2019).
The Pattern Oriented Longitudinal Analysis (POLA) approach is meant to be applied in nursing research when there is a single phenomenon in focus for the duration of the study and where questions and interview formats are generally consistent at each data collection point (Kneck & Audulv, 2019). POLA focuses initially on describing each individual participant’s change across time and later looks for patterns of change shared among participants. The shared patterns are developed inductively rather than grouping participants into predetermined categories or outcomes (Kneck & Audulv, 2019). Researchers must think critically to define a shared pattern as well as to assess the sufficiency of data which supports the defining aspects of the pattern and its boundaries (the limits outside which cases no longer fit the pattern). The POLA approach also uses matrices to organize data often with a specific analytic question in mind. For example, “how did the participants thoughts about their disease change across time?” The matrices evolve along with the analysis from organizing individual data to organizing group data. Shared patterns may eventually be categorized into types of patterns such as “a consistent pattern,” “an episodic pattern,” “an on-demand pattern” or “a translation pattern” (Kneck & Audulv, 2019).
Collaboration During Analysis
In some cases, a researcher may carry out their LQR analysis independently. However, it is often necessary, and arguably advisable that researchers work collaboratively within a team to design and execute their LQR data analysis (Calman et al., 2013; Pope et al., 2000). Working in teams can be useful for establishing reliability in coding as well as in theme development. Team members of various backgrounds will inevitably have conflicting interpretations of data leading to necessary discussions where multiple perspectives are taken into account and researchers attempt to distinguish between what is the researcher’s interpretation and what is an actual finding (Kinnafick et al., 2014; Pope et al., 2000).
Results of Longitudinal Qualitative Research
The emergent nature of qualitative inquiry requires flexibility in research design, data collection and analyses. Defining the endpoint for analyses can be difficult and knowing at what point and in what format to disseminate your findings is equally challenging (Thomson & Holland, 2003). Likewise, identifying a “gold standard” or “rules” that must be followed to ensure rigor is also a challenge and potentially less relevant as LQR research may be enriched by diverse strategies tailored to address specific research questions. Indeed, Nevedal et al. (2018) credits flexibility in LQR as a key facilitator that fosters innovation and creativity.
Ultimately, researchers aim to present results that speak to their original research objectives and in LQR, this includes a deeper understanding of the experience of change across time. Common outcomes presented in LQR publications are themes (and how they change across time), intervention development/evaluation, or conceptual pathways. For example, Clermont et al. (2018) were able to identify themes that explained decreased utilization of nutrient supplements in pregnant women despite their stated high level of acceptance. Mean-while, Corepal et al. (2018) used their qualitative study to better understand how and why an intervention designed to promote physical activity was effective among a group of adolescents. Findings from another LQR study among people living with HIV in Kenya provided key information to understand how and why a livelihood intervention impacted health behaviors (Weiser et al., 2017). LQR may also identify changes in needs or levels of stress that can in turn be used to inform the development of supportive interventions (Murray et al., 2009). Findings from LQR may ultimately be used by providers and case managers designing interventions to support an experience or transition that occurs across time such as a person transitioning from aggressive curative therapies to hospice care or an individual managing a progressive chronic illness (see Table 1 for additional examples).
Trustworthiness of Longitudinal Qualitative Research
The outcomes of the LQR must also adhere to a standard of rigor and quality that ensures meaningful qualitative findings. One way to describe this is by using the principle referred to as trustworthiness of data (Polit & Beck, 2017). Trustworthiness of qualitative data is based on four principles, 1) credibility—how confident the researcher is in the truth of the data; 2) transferability—the extent to which the findings can be compared to similar populations in other settings; 3) confirmability—the degree to which research findings are based on participant narratives—the researcher was neutral in their analysis and interpretation and 4) dependability—the study design could be repeated with consistent findings (Polit & Beck, 2017). Strategies to ensure each of these principles are met have been thoroughly discussed in the nursing literature (see Polit & Beck, 2017). In LQR, however, other considerations may be necessary to achieve trustworthiness.
In LQR there is often the need to make ongoing decisions about processes and procedures throughout the study including revising study guides or protocols, even if midway into the study. Changes may be essential to effectively achieve meaningful data that can be used to develop new knowledge (Saldaña, 2003). That said, some qualitative researchers note that changing interview guides and formats can make it challenging to compare the responses of participants across time (Kneck & Audulv, 2019). In contrast, Saldaña (2003), argues that adjusting methods to enhance data richness allows the opportunity to gain greater descriptions that ultimately may serve a larger audience, thus satisfying the transferability principle of data trustworthiness. Transparency in reporting how and why decisions and changes to the study were made, is therefore vital to trustworthiness as it allows others to consider the decisions and changes that were made in conjunction with the researcher’s findings (Sandelowski, 1986).
Discussion of Challenges in Longitudinal Qualitative Research
Despite clear benefits of LQR, there are several noteworthy challenges. First, depending on the objective of the study and the nature of the change being observed, researchers may be balancing a number of different logistical and conceptual challenges. Whether the study is investigating a disease state versus a significant life change will result in different participant experiences that may need varying amounts of time to capture the essence of that change; the amount of time needed may be a feasibility limitation for some researchers in terms of securing long-term funding as well as retaining participants. A second and related challenge is the labor-intensive nature of LQR which requires adequate funding to maintain research staff throughout the study period. Third, ethical considerations may be different for LQR versus cross sectional. By nature of LQR trying to ascertain a change across time, some studies may focus on enrolling youth or adolescents to follow over a certain number of years. This will require the careful consideration of consent processes, including the participants ability to consent and understand the objective of the study. In addition, informed consent should acknowledge the potential (albeit unknown) effects of long-term participation especially among young people whose life changes may be more unpredictable than middle-aged adults. Likewise, in cases where a person’s condition deteriorates, perhaps due to end of life, the ability to reconsent may be lost (Murray et al., 2009). A fourth challenge that we note is LQR analyses are often poorly described in the literature making it difficult to follow the “recipe” (or even the thought processes) of other researchers with regard to how results were generated. This lack of explanation compromises the trustworthiness (more specifically the dependability) of the results.
Finally, a note about causality in LQR. Determining causality often requires longitudinal data to establish pathways where there is no doubt about the role of an independent variable on a dependent outcome. However, in terms of human experiences, causation is neither linear nor singular in many cases. Transitions are often impacted by multiple causes and may be better explained as “loops” versus “lines” (Pettigrew, 1990). Causation is also shown when isolating independent and dependent variables to account for any confounding. However, transitions and behaviors are marked by convergent interactions and interconnected variables across time. Thus, LQR is well suited to establish or verify patterns of interactions and complex pathways but is not meant to show causation.
Summary and Conclusion
In summary, LQR provides a unique and important opportunity to understand human experiences across time within an individual and among a group using a more holistic, in-depth approach than is possible with retrospective or cross-sectional research alone. However, conducting LQR is complex and time consuming given the inherent contextual considerations of time and change and the many challenges and considerations unique to LQR. Ultimately, the task of exploring change is most effective when flexibility and acknowledgment of the process is considered at the outset. The main process elements include, managing large amounts of data; flexibility in data collection techniques to respond to data quality; sensitivity to many possible types of change that may be occurring; determining whether and in what ways these multiple types of change interrelate with each other; analyzing how and/why these changes occur; and pulling everything together in a complete and coherent report.
Ultimately, researchers must consider these complexities and processes alongside their research objectives to determine whether LQR is an appropriate choice. Our aim was to provide guidance on methodological considerations to aid the decision processes and support well informed study implementation.
Acknowledgments
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by UCSF Center for AIDS Prevention Studies and UCSF Center for AIDS Research (ELT) and by the National Institutes of Health Grants K23MH116807 (ELT) and K01MH112443 (JAP).
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- Baillie C, Smith J, Hewison J, & Mason G (2000). Ultrasound screening for chromosomal abnormality: Women’s reactions to false positive results. British Journal of Health Psychology, 5(4), 377–394. 10.1348/135910700168991 [DOI] [Google Scholar]
- Balmer D, & Richards B (2017). Longitudinal qualitative research in medical education. Perspectives on Medical Education, 6(5), 306–310. 10.1007/s40037-017-0374-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burnard P, Gill P, Stewart K, Treasure E, & Chadwick B (2008). Analysing and presenting qualitative data. British Dental Journal, (204), 429–432. 10.1038/sj.bdj.2008.292 [DOI] [PubMed] [Google Scholar]
- Calman L, Brunton L, & Molassiotis A (2013). Developing longitudinal qualitative designs: Lessons learned and recommendations for health services research. BMC Medical Research Methodology, 13(14), 1–10. 10.1186/1471-2288-13-14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carter S, & Little M (2007). Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research, 17(10), 1316–1328. 10.1177/1049732307306927 [DOI] [PubMed] [Google Scholar]
- Chick N, & Meleis AI (1986). Transitions: A nursing concern. In Chinn PL (Ed.) Nursing research methodology (pp. 237–257). Aspen Publication. [Google Scholar]
- Chinn P, & Kramer M (Eds.). (2011). Integrated theory and knowledge development in nursing (8th ed.). Elsevier. [Google Scholar]
- Clermont A, Kodish SR, Matar Seck A, Salifou A, Rosen J, Grais RF, & Isanaka S (2018). Acceptability and utilization of three nutritional supplements during pregnancy: Findings from a longitudinal, mixed-methods study in Niger. Nutrients, 10(8), 1073. 10.3390/nu10081073 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colaizzi P (1978). Psychological research as a phenomenologist views it. In Valle RS & King M (Eds.), Existential phenomenological alternatives for psychology. Open University Press. [Google Scholar]
- Cooper HL, Caruso B, Barham T, Embry V, Dauria E, Clark C, & Comfort M (2015). Partner incarceration and African-American women’s sexual relationships and risk: A longitudinal qualitative study. Journal of Urban Health, 92(3), 527–547. 10.1007/s11524-015-9941-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corepal R, Best P, O’Neill R, Tully MA, Edwards M, Jago R, Miller SJ, Kee F, & Hunter RF (2018). Exploring the use of a gamified intervention for encouraging physical activity in adolescents: A qualitative longitudinal study in Northern Ireland. BMJ Open, 8(4), e019663. 10.1136/bmjopen-2017-019663 [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Chesnay M, & Abrums ME (2015). Nursing research using ethnography: Qualitative designs and methods. Springer Publishing Company. [Google Scholar]
- Doherty T, Chopra M, Nkonki L, Jackson D, & Persson L (2006). A longitudinal qualitative study of infant-feeding decision making and practices among HIV-positive women in South Africa. The Journal of Nutrition, 136(9), 2421–2426. 10.1093/jn/136.9.2421 [DOI] [PubMed] [Google Scholar]
- Glanz K, Rimer BK, & Viswanath K (2008). Health behavior and health education: Theory, research, and practice (4th ed.). Jossey-Bass. [Google Scholar]
- Glaser BG (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Sociology Press. [Google Scholar]
- Glaser BG, & Strauss AL (1967). The discovery of grounded theory: Strategies for qualitative research. Aldine de Gruyter. [Google Scholar]
- Gray MA, & Smith LN (2000). The qualities of an effective mentor from the student nurse’s perspective: Findings from a longitudinal qualitative study. Journal of Advanced Nursing, 32(6), 1542–1549. 10.1046/j.1365-2648.2000.01606.x [DOI] [PubMed] [Google Scholar]
- Grossoehme D, & Lipstein E (2016). Analyzing longitudinal qualitative data: The application of trajectory and recurrent cross-sectional approaches. BMC Research Notes, 9(1), 136. 10.1186/s13104-016-1954-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guba EG, & Lincoln YS (1981). Effective evaluatio., Jossey-Bass. [Google Scholar]
- Hatcher AM, Lemus Hufstedler E, Doria K, Dworkin SL, Weke E, Conroy A, Bukusi A, Cohen C, & Weiser SD (2019). Mechanisms and perceived mental health changes after a livelihood intervention for HIV-positive Kenyans: Longitudinal, qualitative findings. Transcultural Psychiatry, 57(1), 124–139. 10.1177/1363461519858446 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Henderson S, Holland J, McGrellis S, Sharpe S, & Thomson R (2012). Storying qualitative longitudinal research: Sequence, voice and motif. Qualitative Research, 12(1), 16–34. 10.1177/1468794111426232 [DOI] [Google Scholar]
- Hennink MM, Kaiser BN, & Marconi VC (2017). Code saturation versus meaning saturation: How many interviews are enough? Qualitative Health Research, 27(4), 591–608. 10.1177/1049732316665344 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hermanowicz J (2013). The longitudinal qualitative interview. Qualitative Sociology, 36(2), 189–208. 10.1007/s11133-013-9247-7 [DOI] [Google Scholar]
- Jardine EE, McLellan J, & Dombrowski SU (2017). Is being resolute better than being pragmatic when it comes to breastfeeding? Longitudinal qualitative study investigating experiences of women intending to breastfeed using the theoretical domains framework. Journal of Public Health (Oxford, England), 39(3), e88–e94. 10.1093/pubmed/fdw073 [DOI] [PubMed] [Google Scholar]
- Johansen AB, Brendryen H, Darnell FJ, & Wennesland DK (2013). Practical support aids addiction recovery: The positive identity model of change. BMC Psychiatry, 13(1), 201. 10.1186/1471-244X-13-201 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kinnafick F, Thøgersen-Ntoumani C, & Duda JL (2014). Physical activity adoption to adherence, lapse, and dropout. Qualitative Health Research, 24(5), 706–718. 10.1177/1049732314528811 [DOI] [PubMed] [Google Scholar]
- Kneck Å , & Audulv Å . (2019). Analyzing variations in changes over time: Development of the pattern-oriented longitudinal analysis approach. Nursing Inquiry, 26(3), e12288–n/a. 10.1111/nin.12288 [DOI] [PubMed] [Google Scholar]
- Lewis J (2007). Analysing qualitative longitudinal research in evaluations. Social Policy and Society, 6(4), 545–556. 10.1017/S1474746407003880 [DOI] [Google Scholar]
- McCoy LK (2017). Longitudinal qualitative research and interpretative phenomenological analysis: Philosophical connections and practical considerations. Qualitative Research in Psychology, 14(4), 442–458. 10.1080/14780887.2017.1340530 [DOI] [Google Scholar]
- McLeod J, & Thoon R (2009). Researching social change. Sage Publications. https://uk.sagepub.com/en-gb/eur/researching-social-change/book229849 [Google Scholar]
- Meleis AI, Sawyer LM, Im E, Hilfinger Messias DK, & Schumacher K (2000). Experiencing transitions: An emerging middle-range theory. Advances in Nursing Science, 23(1), 12–28. 10.1097/00012272-200009000-00006 [DOI] [PubMed] [Google Scholar]
- Miles M, Huberman M, & Saldaña J (Eds.). (2014). Qualitative data analysis: A methods sourcebook (3rd ed.). Sage. [Google Scholar]
- Murray SA, Kendall M, Carduff E, Worth A, Harris FM, Lloyd A, Cavers D, Grant L, & Sheikh A (2009). Use of serial qualitative interviews to understand patients’ evolving experiences and needs. The BMJ, 339(7727), 958–960. 10.1136/bmj.b3702 [DOI] [PubMed] [Google Scholar]
- Namukwaya E, Murray S, Downing J, Leng M, & Grant L (2017). I think my body has become addicted to those tablets’. Chronic heart failure patients’ understanding of and beliefs about their illness and its treatment: A qualitative longitudinal study from Uganda. PLoS One, 12(9), e0182876. 10.1371/journal.pone.0182876 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nevedal AL, Ayalon L, & Briller SH (2018). A Qualitative evidence synthesis review of longitudinal qualitative research in gerontology. The Gerontologist, 59(6), e791–e801. 10.1093/geront/gny134 [DOI] [PubMed] [Google Scholar]
- Oosterveld-Vlug MG, Pasman HRW, van Gennip IE, Willems DL, & Onwuteaka-Philipsen BD (2013). Changes in the personal dignity of nursing home residents: A longitudinal qualitative interview study. PLoS One, 8(9), e73822. 10.1371/journal.pone.0073822 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pellowski JA, Harrison AD, Colvin CJ, Brittain K, Rini Z, Cu-Uvin S, Myer L, Operario D, & Lurie MN (2019, October, 21–22). HIV treatment adherence among pregnant and postpartum women in Cape Town, South Africa: Using transition theory to analyze longitudinal qualitative data [Poster presentation]. Inter-CFAR Women and HIV Symposium, Chicago, IL, United States. [Google Scholar]
- Pelto PJ (2013). Applied ethnography: Guidelines for field research. Left Coast Press. [Google Scholar]
- Pettigrew AM (1990). Longitudinal field research on change: Theory and practice. Organization Science, 1(3), 267–292. 10.1287/orsc.1.3.267 [DOI] [Google Scholar]
- Polit D, & Beck CT (Eds.). (2017). Nursing research: Generating and assessing evidence for nursing practice (10th ed.). Wolters Kluwer Health. [Google Scholar]
- Pope C, Ziebland S, & Mays N (2000). Analysing qualitative data. The BMJ, 320(7227), 114. 10.1136/bmj.320.7227.114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riessman C (1993). Narrative analysis. Sage Publications. [Google Scholar]
- Saldaña J (2003). Longitudinal qualitative research. Rowman & Littlefield Publishers. [Google Scholar]
- Saldaña J (2009). Coding manual for qualitative researchers. Sage Publications. [Google Scholar]
- Salter C, McDaid L, Bhattacharya D, Holland R, Marshall T, & Howe A (2014). Abandoned acid? Understanding adherence to bisphosphonate medications for the prevention of osteoporosis among older women: A qualitative longitudinal study. PloS One, 9(1), e83552. 10.1371/journal.pone.0083552 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sandelowski M (1986). The problem of rigor in qualitative research. Advances in Nursing Science, 8(3), 27–37. [DOI] [PubMed] [Google Scholar]
- Schmidt B, Schoppe-Sullivan S, Frizzo G, & Piccinini C (2019). A qualitative multiple case study of the division of labor across the transition to parenthood in south-Brazilian families. Sex Roles, 81(5), 272–289. 10.1007/s11199-018-0999-0 [DOI] [Google Scholar]
- Sheard L, & Marsh C (2019). How to analyse longitudinal data from multiple sources in qualitative health research: The pen portrait analytic technique. BMC Medical Research Methodology, 19(1), 169. 10.1186/s12874-019-0810-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith N (2003). Cross-sectional profiling and longitudinal analysis: Research notes on analysis in the longitudinal qualitative study, ‘negotiating transitions to citizenship’. International Journal of Social Research Methodology, 6(3), 273–277. 10.1080/1364557032000091888 [DOI] [Google Scholar]
- SmithBattle L, Lorenz R, Reangsing C, Palmer JL, & Pitroff G (2018). A methodological review of qualitative longitudinal research in nursing. Nursing Inquiry, 25(4), e12248–n/a. 10.1111/nin.12248 [DOI] [PubMed] [Google Scholar]
- Solomon P, O’Brien KK, Nixon S, Letts L, Baxter L, & Gervais N (2018a). Qualitative longitudinal study of episodic disability experiences of older women living with HIV in Ontario, Canada. BMJ Open, 8(4), e021507. 10.1136/bmjopen-2018-021507 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Solomon P, O’Brien KK, Nixon S, Letts L, Baxter L, & Gervais N (2018b). Trajectories of episodic disability in people aging with HIV: A longitudinal qualitative study. Journal of the International Association of Providers of AIDS Care (JIAPAC), 17, 1–7. 10.1177/2325958218759210 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thomson R (2007). The qualitative longitudinal case history: Practical, methodological and ethical reflections. Social Policy and Society, 6(4), 571–582. 10.1017/S1474746407003909 [DOI] [Google Scholar]
- Thomson R, & Holland J (2003). Hindsight, foresight and insight: The challenges of longitudinal qualitative research. International Journal of Social Research Methodology, 6(3), 233–244. 10.1080/1364557032000091833 [DOI] [Google Scholar]
- Tracy SJ (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851. 10.1177/1077800410383121 [DOI] [Google Scholar]
- Weiser S, Hatcher A, Hufstedler L, Weke E, Shari DL, Bukusi EA, Burger RL, Kodish S, Grede N, Butler LM, & Cohen C (2017). Changes in health and antiretroviral adherence among HIV-infected adults in Kenya: Qualitative longitudinal findings from a livelihood intervention. AIDS and Behavior, 21(2), 415–427. 10.1007/s10461-016-1551-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zeilani R, & Seymour JE (2010). Muslim women’s experiences of suffering in Jordanian intensive care units: A narrative study. Intensive & Critical Care Nursing, 26(3), 175–184. 10.1016/j.iccn.2010.02.002 [DOI] [PubMed] [Google Scholar]
- Zeilani R, & Seymour JE (2012). Muslim women’s narratives about bodily change and care during critical illness: A qualitative study. Journal of Nursing Scholarship, 44(1), 99–107. 10.1111/j.1547-5069.2011.01427 [DOI] [PubMed] [Google Scholar]