Critical incident technique: an innovative participatory approach to examine and document racial disparities in breast cancer healthcare services

Abstract

Disproportionate and persistent inequities in quality of healthcare have been observed among persons of color in the United States. To understand and ultimately eliminate such inequities, several public health institutions have issued calls for innovative methods and approaches that examine determinants from the social, organizational and public policy contexts to inform the design of systems change interventions. The authors, including academic and community research partners in a community-based participatory research (CBPR) study, reflected together on the use and value of the critical incident technique (CIT) for exploring racial disparities in healthcare for women with breast cancer. Academic and community partners used initial large group discussion involving a large partnership of 35 academic and community researchers guided by principles of CBPR, followed by the efforts of a smaller interdisciplinary manuscript team of academic and community researchers to reflect, document summarize and translate this participatory research process, lessons learned and value added from using the CIT with principles of CBPR and Undoing Racism. The finding of this article is a discussion of the process, strengths and challenges of utilizing CIT with CBPR. The participation of community members at all levels of the research process including development, collection of the data and analysis of the data was enhanced by the CIT process. As the field of CBPR continues to mature, innovative processes which combine the expertise of community and academic partners can enhance the success of such partnerships. This report contributes to existing literature by illustrating a unique and participatory research application of CIT with principles of CBPR and Undoing Racism. Findings highlight the collaborative process used to identify and implement this novel method and the adaptability of this technique in the interdisciplinary exploration of system-level changes to understand and address disparities in breast cancer and cancer care.

Introduction

The purpose of this report is to present the origins and elements of critical incident technique (CIT) methodology, and critically examine its fit with a participatory research process to examine system-level race-specific inequities in breast cancer healthcare. Racial and ethnic inequities in the delivery of healthcare in the United States received national attention in 2003, with the publication of the Institute of Medicine’s (IOM) report, ‘Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare’ [1]. Healthcare disparities were defined by IOM as, ‘… racial or ethnic differences in the quality of healthcare that are not due to access-related factors or clinical needs, preferences, and appropriateness of intervention’ [1]. Findings from their meta-analysis showed that, even after controlling for income, insurance coverage, and healthcare access, racial and ethnic disparities in quality of healthcare persisted. The reasons for such disparities have not been well conceptualized or documented, and have varied by type of condition, socio-demographic variables, economic factors and various cultural preferences, attitudes and ideas about disease etiology, prevention and treatment [2, 3]. Cognitive and decision-making processes may differ by cultural and ethnic group, meaning that choices and preferences may not be mutually understood or acted on [4].

Racial and ethnic disparities in healthcare may also involve organizational factors within facilities and health plans or systems, including complex appointment or referral systems or long waiting times; simple lack of providers within any reasonable traveling distance or time; poor understanding of how best to mobilize local, community organizations that principally serve African American residents; and matters such as the racial and ethnic concordance (or lack thereof) between patients and clinicians that may have effects on patient care-seeking behaviors or satisfaction with care [5–10].

Adams and Balfour defined the notion that people can act in ways that are harmful to others without being aware of the negative impact they have on them:

… it is entirely possible to adhere to the tenets of public service and professional ethics and participate in even a greater evil and not be aware of it until it is too late (or perhaps not at all) [11].

Adams and Balfour [11] viewed this phenomenon as being rooted in an organization’s perspective, technical language and dehumanization. Perspective embodies the notion that to eliminate racism it is important for organizations to understand it from the perspective of those impacted and oppressed by it. ‘Technical language’, such as jargon, code words, or euphemisms, and dehumanization often enable healthcare providers to distance themselves emotionally from the real impact of policies and protocols; disconnecting service delivery from a sense of right and wrong [11, 12].

Hence, to understand and ultimately eliminate such inequities in healthcare, several public health bodies have issued calls for innovative methods and approaches [1, 13–15]. Recommendations have included engaging healthcare providers, administrators and patients themselves in cultivating new conceptual frameworks, guided by those from systems theory, that examine determinants from the social, organizational and public policy contexts to inform the design of systems change interventions. One such study, Cancer Care and Racial Equity Study (CCARES), was informed by an Undoing Racism™ framework [16] to identify structures built into cancer care systems that can make cancer care vulnerable to unintentional, structural and institutional bias, which may contribute to racial inequity in quality and completion of breast cancer treatment. Fifty African American and White breast cancer survivors, who had received their treatment at the same cancer center, were interviewed by a community-academic partnership using the CIT. Previously, the CIT method has been used as a tool to facilitate a community-based participatory research (CBPR) process [17], but has not been used in the context to examine racial disparities in cancer and cancer care.

Methods

In the following section, we first provide an overview of our participatory research partnership, the process used to identify the CIT followed by a description of the application of CIT to explore racial disparities in breast cancer care.

Setting and CBPR partnership

CCARES, funded by NCI in 2006, was designed and conducted by the Greensboro Health Disparities Collaborative (GHDC) of North Carolina. The mission of the GHDC is to establish structures and processes that respond to empower and facilitate communities in defining and resolving issues related to racial disparities in health. Comprised 35 community, academic and health professional members, GHDC had undertaken an 18-month planning process to complete training in Undoing Racism™ [16] and the CBPR approach [18]. The goal was to establish a common language, conceptual framework and principles for collaborating on research that would move GHDC forward in submitting an application for NIH funding R21 CCARES to address racial disparities in healthcare. The details of this planning process are described in detail in Yonas et al. [19].

The research approach adopted by the GHDC was to integrate the principles of Undoing Racism™ with CBPR to encourage local communities to participate in both the analysis and the development of system change solutions that promote racial equity in care through transparency and accountability within the system. Once funding was received for CCARES, all GHDC members completed research ethics training, certified by the University of North Carolina at Chapel Hill’s Institutional Review Board as Non-Traditional Investigators. In the next section, we describe the trainings and procedures used in CCARES to engage GHDC members, community and academic alike, in consensus driven decision process for conducting CIT interviews and analysing CIT responses.

Methodological approach: integrating CBPR with undoing racism and CIT

The authors, including academic and community research partners of the entire GHDC, reflected together within a large group and established an interdisciplinary methodological subgroup of eight collaborative members to determine the appropriate study design and data collection methods to investigate the research questions. This group met in person three times for approximately 2 hours each time in person to discuss and examine the pros and cons of various research methods. Community members of the methods subgroup weighed in on the critical internal validity issues of authenticity and trustworthiness of the different methods considered, ethical issues with regard to recruitment of participants, incentives, community concerns about research [19]. In-person meetings were supplemented with email communication and phone calls among members with one primary academic and community lead partner to further explore and address any subgroup questions or concerns. It was this iterative process over a period of 4 weeks of critiquing multiple methodologies that led us to propose a mixed methodological approach of CIT for exploring the complex interplay of organizational factors that impact on breast cancer treatment and continuity of care for African American and white women. GHDC members were overwhelmingly supportive, as expressed during large group collaborative discussion and methods-subgroup deliberations of the CIT interview methodology because it was thought to be uniquely suitable to accommodate principles from Undoing Racism™ and CBPR by integrating community and academic expertise and experience in every phase of the project development, data collection and analysis process. The decision to use the CIT approach was made collectively by members of the GHDC after exploring together a variety of traditional qualitative data collection (e.g. focus groups, in-depth interviews, participant observation) and quantitative data collection (e.g. patient or community surveys) methodologies. To maintain and ensure this balance, we established procedures integrating community, academic and health professional expertise into the capacity to implement and participate in the CIT process. As with all decisions within this CBPR initiative, all decisions were made through an open consensus driven decision-making process involving methods-subgroup team meetings and an open discussion among the entire academic and community GHDC membership for final discussion and approval. Planning meetings to design the interview field guide and appropriate incentives (Fig. 1) were consensus driven and allowed community sensitivity to balance academic rigor. Acceptable, confidential and convenient community locations for conducting CIT interviews were revealed through group discussions.

Fig. 1.

CIT interview guide.

Critical incident technique

First introduced by John Flanagan (1954) [20] to the US Air Force Aviation Psychology Program, CIT was developed as a practical and efficient method for interviewing pilots, their subordinates and supervisors, to gain their first-hand reports on critical incidents that caused satisfactory and unsatisfactory execution of flight performance. The findings served as a basis for informing protocol improvements, such as the design of flight instruments and active system-level changes. For example, from a total of 3020 critical incidents reported by 640 Air Force officers, 58 categories of critical requirements were derived inductively and then classified into major areas for flight checks to increase safety and reduce pilot error.

In health service research, CIT interviewing has been adapted to patient care settings and systems by using the process to elicit comprehensive and detailed descriptions from patients about behaviors during significant and decisive situations that could be related to changes in patient outcomes [21]. Health service investigators have used CIT interviews to gather sensitive and essential insights from patients with regard to the quality of hospital services, assessing their satisfaction or dissatisfaction with healthcare provider behaviors and services; and explaining circumstances under which their emotional needs were or were not met by the healthcare system [22–24]. Findings from these studies have informed an outcome measure of patient engagement with nursing care providers, self-care strategies for living with HIV infection and education programs for self-management of chronic illness symptoms [25–33].

In sum, CIT interviewing is a practical and flexible methodology for engaging patients in pointing to (i) specific social, cultural and clinical encounters with the healthcare system that contributed to their satisfactory or unsatisfactory completion of optimal care and (ii) key interventions for leveraging system-level change [34]. CIT incorporates key features of qualitative research, focus on behavior, use of a step-wise participatory process for identifying and interpreting incidents from the participants’ perspective, collecting information about the incidents, looking for common issues that may help to explain the incidents and determining ways to resolve them. In addition, CIT is an iterative multi-step approach involving a team of researchers in the data collection, data analysis and reporting phases. This team approach to each phase of the research provides a unique methodological process to collectively identify, characterize and analyse the influence of specific experiences, called ‘incidents’, perceived by patients.

Recognizing the potential for excessive response burden on patients, CIT interviewing consists of a set of simple procedures to help patients recall and pinpoint what they saw, heard, or felt without judging or expressing a personal opinion in a structured format. CIT questions are finely focused on identifying the patient’s unique interactive experiences or encounters with the care system and characterizing specific ‘incidents’ during each encounter that influenced the patient’s adherence to care. CIT is designed to identify and characterize these incidents and problems they create to help develop practical solutions. Because the CIT interviews are conducted one on one, the process allows patients to reveal the private and deeply felt aspects of their emotional responses to care which was particularly appealing to the members of the GHDC. The development of relationships between the interviewer and the patient and the focusing of questions on specific incidents provide deeper insight into the effects of the institutional processes on the patients’ experiences of care.

In short, as compared with in-depth interview, focus group interviews, or participant observation, which focuses primarily on ‘describing’ patients’ treatment experience [32], CIT methodology engages patients in ‘interpreting’ the positive and negative effects from their specific encounters with the institutional structures and protocols that are inherent to healthcare systems. In the next section, we describe one of the first studies of breast cancer disparities that used CIT interviewing, which was informed by the Undoing Racism™ framework [16] and followed the principles of CBPR [18].

Data collection

CIT interviewer guide

The interviewer guide, or template of questions used, evolved through several rounds of edits before it was finally approved by the GHDC members and the University of North Carolina Institutional Review Board (IRB). Members of the GHDC edited the questions and compiled the revisions into one document. The questions were broken into three time segments of a woman’s experience with breast cancer: diagnosis, treatment and follow-up. Each interview began by focusing on the most recent encounter a woman had with anyone in the medical field dealing with her breast cancer. The interviewee was asked if an encounter she described was pleasant (positive) or unpleasant (negative) (Fig. 1). Interview time ranged from 30 to 90 min. Each woman had the opportunity to participate in two interviews. The second interview, which used the same initial interview guide format, was designed to provide an opportunity to explore concepts and incidents generated in the first interview more deeply. Interviewees received, on the recommendation of the GHDC members and UNC-IRB, an incentive of $50 for completing the first interview, and $75 for completing the second interview.

CIT CBPR capacity development

CIT interviewer and methods training

During the fall of 2007, the seven members of the CCARES research team were trained by expert consultants from the American Institute for Research (AIR) on the CIT (www.air.org). Once trained by AIR, they organized 1-day training for seven female GHDC members, who were interested in understanding the CIT process and having the opportunity to practice the interviewing technique. Members of the GHDC strongly supported matching the race of the interviewee and the interviewer as an essential component of the research process to minimize barriers to communication and cultural understanding. Prior to the training, individuals were informed that the requirements for becoming an interviewer were being a woman, identifying as African American or White, having no prior diagnosis of breast cancer and feeling comfortable facilitating such an interview process. All of the women from the GHDC who participated in the CIT training were invited to become interviewers. In addition, community-based interviewers were instructed in the use of necessary materials including the mechanics of digital audio recorders and on the processes for taking notes and guiding the CIT interview. In the end, six African American and four White community-based interviewers were selected and prepared to conduct the CIT interviews. The group of interviewers was composed of women whose education varied from finishing their formal education with some college coursework to completion of graduate degrees. Some interviewers had experience working directly in the healthcare field (including a retired nurse and a retired physician), and others were more experienced in community organizing.

Participant recruitment

Principles of CBPR were used to inform a comprehensive and culturally sensitive process of participant selection, recruitment, enrollment and data collection. First, a telephone recruitment script and consent form were developed with direct input from African American community research partners and breast cancer survivors in a manner to help us establish trust and a level of comfort with potential research participants, reducing the risk of psychological or emotional stress. For example, it was noted that the telephone script must be warm and friendly, and start with the line ‘Hello, my name is XXXX and I am NOT asking for money …’ Mock recruitment calls, using the telephone script, were conducted with community research partners involved in the collection to promote maximum comfort, skill and quality of data collection. Second, a community research partner who was the president of her neighborhood association was able to locate current contact information potential study participants with out-of-date contact information. This community partner was able to use her unique knowledge of publicly available records of neighborhood associations to locate women to recruit into this study that significantly increased enrollment. Third, community research partners assisted with identifying local, confidential and convenient locations for conducting the CIT interviews which consisted of rooms at the local library, and at various rooms on the local healthcare system, the office of the research partner agency and for a few, the participants’ homes. Fourth, participatory analyses of CIT audio recordings were conducted in pairs, one community and one academic partner—ideally mixed by race. The intent was to ensure diverse perspectives as each pair interpreted, coded and reconciled final coding of the CIT interviews through co-coding, discussion and mutual agreement. Finally, as CIT respondents were selected through stratified random sampling from the cancer registry, their zip codes were reviewed by community partners with special knowledge about neighborhood diversity.

The responsibilities of the interviewers included making the initial recruitment outreach telephone call, explaining the consent form, obtaining a signature on the consent form, conducting and audio recording the two interviews after obtaining permission from the participant and giving each woman a resource booklet and incentive gift card. The interviewer and interviewee were matched initially by race. This racial matching of the interviewer with interviewee throughout the project and maintenance of the same interviewer whenever possible was designed to establish a relationship and level of comfort necessary for candid interviews. Each interviewer was encouraged to maintain communication with the project coordinator throughout the recruitment and interview progress.

CIT analyst training

Traditionally, only academic researchers conduct the analysis of qualitative data. However, during this CBPR study, community leaders were given the opportunity to participate as trained interview analysts. Listening to segments of a CIT interview, completing a Critical Incident Form (Fig. 2) and coming to consensus on the critical incidents involved were other examples of CBPR by the CCARES research team. Two CIT analysis trainings took place during which the CCARES research team practiced completing incident forms and coming to agreement on what qualified as an incident. To be consistent with AIR procedures for CIT analysis, CCARES research team members were paired to complete incident forms for each interviewer. Within each pair group, different perspectives were represented by either an African American and White analyst and/or an academic and community research partner who worked together to analyse and summarize the data. Once an interview was conducted by ‘Interviewer A,’ then ‘Analyst B’ would fill out Critical Incident Forms for the interview, and ‘Analyst C’ would listen to the interview and edit or add to the first set of Critical Incident Forms produced. All analyses were conducted and interpretations developed through consensus.

Fig. 2.

Critical incident tracking and documentation form.

This procedure continued until several sets of Critical Incident Forms were created, which no longer needed editing or revisions because both analysts were in agreement about the analysis. Incidents were organized and reduced according to categories such as (i) incidents impacting a decision to delay care, (ii) incidents related to a woman’s decision to discontinue care and (iii) incidents influencing a woman to continue her care as planned. The CIT method was found to be flexible enough to apply the principles of anti-racism and CBPR to successfully produce quality system-level data regarding the experiences of women experiencing breast cancer and care and follow-up.

There are a number of notable implications related to this research and integration of CIT with principles of CBPR. First, our CIT interview findings described subtle yet important racial differences on the impact from patient encounters with the various systems of care, such as negative interactions and communication with physician and hospital office staff, and positive interactions with medical oncology specialists. Second, our integration of community and academic expertise provides a model participatory research template for integrating the distinctive systematic data collection CIT method with principles of the CBPR approach to exploring disparities in health and healthcare. Findings indicated opportunities for intervention at the systems and patient care levels to prevent and improve disparities in breast cancer care in this region and findings with potential implications for breast cancer care more broadly.

Discussion

As a result of the participatory and partnered research approach, and guided by community partner and academic support and feedback, we believe that the CIT methodology was an ideal approach for identifying, illustrating and examining women’s experiences with the healthcare system for breast cancer care. This process, integrating principles of CBPR with CIT, maintained integrity in implementation of the research process and as a method helped to maintain transparency at all levels of the CBPR approach. All members of the GHDC were invited to participate in the CIT training, regardless of whether they qualified to be interviewers. Individuals who trained but did not interview were then available and provided support, suggestions and critical review of the interviewers. This component is especially important in maintaining the balance of power between community member emerging researchers and the academic researchers. Without a strong community voice, effective CBPR cannot occur.

Another important facet of our work in the Collaborative was adherence to Undoing Racism™ principles which both guided the way the diverse research team worked together (shared language, transparency and accountability) and provided a lens for understanding and interpreting issues of power and control over decisions affecting people receiving care in the health system. The monthly group meetings at which concerns of the interviewers regarding the process were openly discussed provided a forum for feedback on these principles as well. We found, as a community and academic partnership that CIT was a useful, engaging and flexible methodology to gather information on structural and institutional bias. CIT focuses on behavioral effects relevant to a specific outcome of concern, unlike other in-depth interviews in which descriptions of situations could monopolize the conversation. Using CIT trains the interviewer to lead the conversation in such a way that information of positive and negative practices within the medical institution can be gathered and objectively reflected on by the analysts. In the analysis process, the researcher is allowed to distinguish where in the system there are opportunities for improvement in the manner in which doctors, nurses, technicians and other healthcare personnel implement institutional protocols that may influence a patient’s decision to continue, delay, or discontinue her care.

Our use of CIT also replicates the usefulness of the method as found in research on the quality of nursing care [32]. In addition, Belkora et al. [21] found the CIT method to be an effective structure to integrate involvement from a large spectrum of the community stakeholders in the research process given the flexible design, and personalized approach. The authors though found that the time and training associated with the CIT approach to be a barrier to the participatory partnered research process of CBPR. Overall, our academic-community partners expressed strong support for the use of the CIT as facilitating community involvement in our CBPR project, where we used it to formally document the forces promoting and inhibiting successful achievement of community aims. We also found that the flexible CIT approach, together with consistent and transparent communication structures allowed our CBPR team to swiftly and effectively address some of the time and training barriers experienced previously by others using CIT.

We found the CIT to be a flexible method of discovering aspects of medical practice that leads to greater patient satisfaction and continuation of prescribed care. Our use of the CIT made it possible to gather infrequently revealed information on the reasons Black and White breast cancer survivors continue their medical care because we approached the CIT method using anti-racism and CBPR principles and allowed nontraditional researchers to assist with gathering and interpretation of the data.

Principles of Undoing Racism™ remind researchers to think outside of the traditional clinical research practices of uncovering information. Using community researchers with the CBPR approach brings a fresh perspective when juxtaposed with the experiences of the academic researchers. One of the principles is to allow people who have historically been powerless to have access to powerful roles and responsibilities, to balance power among all to bring justice [35]. We applied this principle in conducting CIT interviews by encouraging community partners to serve as interviewers and analysts instead of reserving these roles for academic partners.

Strengths and limitations

There were a several limitations identified through using this participatory research process. First, as recruitment began, we found that more time than was expected was required to make initial contact and then converse with the participants to enroll them in the study which was addressed by providing nominal compensation. Interviewers were training to provide continuous feedback to the participants on the relevance and depth of responses elicited to generate critical incidents, especially prior to the second interview in order to build on the established rapport and delve more deeply into previously revealed critical incidents. In addition, a number of the interviewers noted that patients appeared initially reluctant to express criticism of healthcare providers whom they like, especially if they continued to receive care from that provider. If the woman ultimately had a positive outcome, any negative events that had occurred earlier seemed to be ameliorated by their current state of feeling that they had been successfully treated.

Despite these limitations, there were a number of unique noteworthy strengths of this study process. The CIT methodology provided a unique opportunity for participatory study engagement and a comparative structure for breast cancer survivors to recall the details of exposures within the healthcare system over a long and significant period of their lives. Because all parts of the care process were under consideration, this discussion was open to review of all types of medical providers and office personnel throughout the spectrum of treatment settings involved in their care. The use of a second CIT interview with each survivor provided the interviewers with the unique opportunity to review the audio recording of the first interview prior to the second.

The support provided during and after the CIT training and the practice sessions allowed the interviewers to become adept in the methodology. Because the interviewees were confronted with peer researchers, the women found they could be relaxed and candid in their discussions of their care.

Conclusions

Central to the CBPR approach and the purpose of the CCARES research effort was the dissemination of findings to the oncology providers and staff at the local cancer center; the study participants themselves at a private dinner to ensure a safe environment for confirming or clarifying results and providing input on next steps; and the community-at-large through a community forum held at the local public library. Based on information obtained in the CIT interviews, the GHDC has been instrumental in forming a local chapter of Sisters Network (http://www.sistersnetworkinc.org) which is a national advocacy organization for African American breast cancer survivors.

CBPR is a valuable and effective approach for engaging the affected community in the research process at all levels, beginning with proposing the question and ending with dissemination of the findings. Our unique collaborative based on Undoing Racism™ principles stresses the need for balance between community and academic researchers and the benefits that ensue when all voices are given weight in the process of decision making.

Use of the CIT methodology blended well with CBPR, by emphasizing the need for the voices of the patients to be heard. Moreover, through CIT interviewing, our study’s participants revealed structures in the cancer care system that obstructed transparency and accountability for quality and completion of treatment for all patients. The GHDC has used the knowledge gained through CIT interviews to inform a 5-year, NCI-funded, systems change intervention study titled, Accountability for Cancer Care through Undoing Racism and Equity.

Funding

National Institutes of Health (1 R21 CA119979–01).

Conflict of interest statement

None declared.