A Novel Mixed Methods Approach Combining Geospatial Mapping and Qualitative Inquiry to Identify Multilevel Policy Targets: The Focused Rapid Assessment Process (fRAP) Applied to Cancer Survivorship

Abstract

Multi-level perspectives across communities, medical systems and policy environments are needed, but few methods are available for health services researchers with limited resources. We developed a mixed method health policy approach, the focused Rapid Assessment Process (fRAP), that is designed to uncover multi-level modifiable barriers and facilitators contributing to public health issues. We illustrate with a study applying fRAP to the issue of cancer survivorship care. Through this multi-level investigation we identified two major modifiable areas impacting high-quality cancer survivorship care: 1) the importance of cancer survivorship guidelines/data, 2) the need for improved oncology-primary care relationships. This article contributes to the mixed methods literature by coupling geospatial mapping to qualitative rapid assessment to efficiently identify policy change targets.

Background:

In today’s increasingly complex healthcare environment, research investigating multi-level interactions of population health problems is paramount. A broad body of literature over the past decade has shown that investigating one level of influence such as patients, without concern for its relational interactions with other levels including practices, hospitals, and the regulatory and payment policies, frequently provides information that is limited in its applicability to complex, real-world settings for health care improvement (Kessler & Glasgow, 2011; Paskett et al., 2016; Stange & Glasgow, 2013; Tomoaia-Cotisel et al., 2013). The socio-ecological framework, and several variations and adaptations of this model, have been used to characterize the overlapping, multi-level influences on individual level health behavior and health outcomes (Bronfenbrenner, 1979; Stange & Glasgow, 2013; Tomoaia-Cotisel et al., 2013). For example, in Bronfenbrenner’s seminal illustration of the model, he describes the multiple widening concentric circles of influence on a patient, as one moves out from an individual, to their family, community, medical homes, larger medical systems and ultimately, policies and cultural influences (Bronfenbrenner, 1979).

Similarly, practice-based field research informing primary care transformation efforts improves when the research design includes ways to study facilitators and barriers at the local, community, medical system, and policy levels (Crabtree et al., 2018; Levesque et al., 2018; Russell et al., 2018). Few tools, however, are available to researchers trying to understand how to synthesize multi-level contextual information to inform health policy and/or practice change (Gorin, Badr, Krebs, & Prabhu Das, 2012; Stange, Breslau, Dietrich, & Glasgow, 2012). We therefore developed a novel mixed methods approach, the focused Rapid Assessment Process (fRAP), to help researchers with limited resources understand multi-level contextual factors affecting health issues within practical time frames. In this article, we describe our mixed method, the focused Rapid Assessment Process (fRAP), which combines a quantitative exploration phase using geospatial mapping tools that then focuses a rapid qualitative assessment phase to efficiently and effectively identify policy change targets (see Figure 1).

Figure 1:

Multi-level framework for fRAP

• Model of the three levels, community, medical and policy, included in the fRAP method for cross-comparison, adapted from the socio-ecological model

The History of Rapid Assessment Procedures and Rapid Qualitative Methods and the novel focused Rapid Assessment Process (fRAP)

fRAP derives from rapid ethnographic methods that have been used for decades. Rapid Assessment Procedures (RAP) were originally developed by rural sociologists in the late 1970s; they used a method they called Rapid Rural Appraisal to assess the needs of struggling farmers in rural areas around the world (Chambers, 1981). In the 1980s, applied anthropologists adapted these participatory tools, now known as RAP, to help assess primary health care and nutritional needs in developing countries (S. Scrimshaw & Hurtado, 1987). More recent uses and refinements of RAP recognize that it represents an iterative process using multiple qualitative methods and is often now referred to as a Rapid Assessment Process in order to more quickly conduct “intensive, team-based qualitative inquiry” that allows researchers to “develop a preliminary understanding of a situation from the insider’s perspective” (Beebe, 2001).

Over the last decade, further adaptations of rapid qualitative processes have been developed. Dr. Beebe himself has created a new teams-based approach of insiders’ perspectives coined Rapid Qualitative Inquiry (RQI) (Beebe, 2014). Other research teams have developed a new method, entitled Rapid Qualitative Analysis (RQA) where the qualitative analysis stage is completed even more rapidly (Taylor, Henshall, Kenyon, Litchfield, & Greenfield, 2018). Finally, another new method, the Rapid Qualitative Evaluation Synthesis (RQES) was recently designed to improve health technology in Scotland by synthesizing qualitative data across studies (Healthcare Improvement NHS Scotland, 2019).

Compared to traditional ethnography, which is premised on long-term, in-depth engagement at a particular field site, RAP and its adaptions typically require only a short-term stint in the field (anywhere from a few weeks to two months) and use less observation with more purposefully sampled key informant interviews (Gittelsohn, 1998). RAP methods have been extensively applied, and have successfully shown how an outsider can effectively enter, interview and work with people on the ground (Gittelsohn, 1998; N. S. Scrimshaw & Gleason, 1992). Critics of RAP however have noted two major risks: (1) challenges with accuracy, cultural inappropriateness, and representativeness due to the limited time in the field; and (2) a lack of additional external sources of knowledge to contextualize the RAP findings (N. S. Scrimshaw & Gleason, 1992). Additionally, as we recognized through the development of this work, RAP and all of these newer adaptations are entirely qualitative.

Building from RAP’s strengths while also attempting to mitigate its limitations, we created a novel mixed methods that would include the addition of quantitative mapping tools to focus where the qualitative rapid assessment should be undertaken. We hypothesized that by including a quantitative geospatial mapping analysis phase, we could more quickly and effectively target those geographic regions that should be included for qualitative investigation. Additionally, by completing both quantitative and qualitative phases across multiple socio-ecologic levels of the community, medical and policy realms, we would be able to create a more robust assessment of the problem and its possible solutions. Furthermore, by adding the geospatial phase, we could include external data sources to contextualize the rapid qualitative findings, and provide for meaningful opportunities for the patient population to be represented in the process.

Within this backdrop, we have thus developed a modified approach that refines and focuses the RAP process with a new mixed methods and health policy lens. Our method, the focused Rapid Assessment Process (fRAP), aims to uncover contextual elements across the socio-ecological framework that influence a particular health issue, while minimizing the time and resources needed. fRAP was developed directly in response to the lack of available tools in the literature aimed at understanding multi-level contextual interacting factors affecting health issues within practical time frames and with limited resources. fRAP uses geographic information systems mapping, short-term field observations, and targeted, purposefully sampled key informant interviews, to investigate multiple interacting levels of the public health issue in question (Frels, Frels, & Onwuegbuzie, 2011; Teddlie & Yu, 2007). Through the integration and analysis of these multiple data sources, our method seeks to identify modifiable elements within communities, medical systems and/or policy arenas for future interventions that can impact public health outcomes.

Attempting to identify modifiable contextual elements of communities, health care systems and local and state policies has proven challenging for cancer researchers (Praestegaard et al., 2017; Shariff-Marco et al., 2015). Contextual elements that impact care are oftentimes difficult to identify given the complexity of communities and health systems today, and frequently troublesome to systematically study across different communities nationwide given the lack of standardization across unique environments (Coughlin & Smith, 2015; Stokols, 1996). The fRAP method addresses these difficulties by investigating three levels of contextual elements along the socio-ecological framework with both a quantitative and qualitative lens: patient and community microsystem level variables, medical network mesosystem level features, and county, state and national macrosystem level regulations and policies (Bronfenbrenner, 1979). In the following sections, we describe how we developed and applied fRAP in one case study on cancer survivorship.

focused Rapid Assessment Process (fRAP): phases of a new mixed methodology

As a novel mixed method, fRAP uses an explanatory sequential mixed methods design to effectively use geospatial mapping tools to focus and target geographic areas of interest for further qualitative inquiry. In an explanatory sequential design, a quantitative phase is conducted first, followed by a qualitative investigation (Ivankova, Creswell, & Stick, 2006). In our explanatory sequential fRAP method, the quantitative phase is the smaller, focusing phase for the larger and subsequent qualitative rapid assessment phase.

Phase I of the methods (f, quan) begins with quantitative geospatial analysis of the problem in question. Geographic information systems are used to create maps of county or zip code level data to highlight areas that showcase numerical differences in demographics across community, medical and policy variables. Comparison of geographic regions of interest is critical, as different geospatial regions begin to illuminate contextual differences that subsequent qualitative work will target and cross-compare.

Integration occurs at this branch point between Phase I and Phase II, by utilizing multi-level mapped descriptive analyses from the mapping phase to focus where and what will be collected in the next qualitative phase. Phase II of the methods (RAP, Qual) then utilizes rapid qualitative assessment techniques within the geographically targeted regions of interest. Qualitative processes include scoping internet review, participant observation fieldwork, key informant interviews and focus groups. A combination of these techniques is undertaken in counties or zip codes of interest, and data is collected, analyzed and cross-compared, both across multiple levels in one region, and additionally, across different geographic regions within the same level. This cross comparison of themes, both across levels and across regions, is critical for identifying contextual differences that are impacting the problem in question.

Phase III is the final phase of the methods, set to identify modifiable contextual elements at each of the three levels that are possible change targets to improve the problem in question. Through integration of the cross-comparative analysis of quantitative and qualitative data, modifiable elements that are present in community, medical and policy levels can be identified for subsequent action. Table 1 highlights the three Phases of fRAP, including procedures and activities in each Phase, as well as points of integration across the mixed methods.

Table 1:

fRAP the method

Phase I	Phase II	Phase III
Quantitative	Qualitative	Identification of Modifiable Contextual Elements for Policy Action
Procedures: Geographic Information Systems mapping tools	Procedures: Internet Scoping Review Participant Observation Key Agency Tours Key informant Interviews Focus Groups	Procedures: Cross comparative analysis of quan + qual datasets
Activities: Choose region of interest Choose level of mapping (i.e. county, zip code)	Activities: Undertake rapid qualitative assessment in multiple geographic units	Activities: Policy analysis of quan + qual dataset
Multi-level considerations: Create Community Assessment Profiles to document Community, Medical and Policy maps	Multi-level considerations: Create cross-comparative Excel spreadsheet of qualitative data themes across community, medical and policy levels, across geographic units	Multi-level considerations: Create multi-level list of modifiable contextual elements for further policy action

↑ Integration Points ↑

Cancer Survivorship as a Case Study:

This type of multi-level investigation generated by the fRAP method is particularly important for research on cancer survivorship, given the complex and multi-faceted elements of survivorship care. As cancer clinical outcomes improve alongside an expanding and aging population, cancer survivors as a group are rapidly increasing, with projections of 20 million cancer survivors in the US by 2026 (Miller et al., 2016). The 2006 Institute of Medicine’s Lost in Transition report highlighted the care needs of this population, and illuminated the complexity associated with the long-term management of cancer survivors (Hewitt, Greenfield, & Stovall, 2005). Almost a decade later, the National Institutes of Health called for multi-level intervention research across the cancer care continuum to confront this rapidly burgeoning number of cancer survivors (Taplin et al., 2012; Zapka, Taplin, Ganz, Grunfeld, & Sterba, 2012). These publications from the National Cancer Institute and others indicate that multi-level interventions are necessary, given the complexity of chronic disease management for cancer survivorship within our current health system, to achieve the highest quality cancer care possible (Rubinstein et al., 2017; Taplin et al., 2012; Zapka et al., 2012).

Despite this call, prior research in cancer care delivery has primarily focused on single-level interventions to address patient, community, clinician or system-wide factors to improve care quality through the survivorship years and to reduce mortality (Kim, Chang, & Kong, 2017; Meneses et al., 2018; Simmons, Pineiro, Hooper, Gray, & Brandon, 2016). While many of these single-level interventions were effective within controlled study environments, the real-world implementation of such strategies has proved challenging (Simone & Hewitt, 1999). Given that cancer care requires multiple levels of interaction between patients, families, communities, medical systems and policies as described in the socio-ecological framework, it is critical to investigate all of these levels and their interactions (Bronfenbrenner, 1979). Few studies to date have examined contextual elements across multiple levels to understand barriers and strategies for improving cancer survivorship care (Bava, Johns, Freyer, & Ruccione, 2017; Katz, Young, Zimmermann, Tatum, & Paskett, 2018; Sposto et al., 2016).

Methods:

Supplement to Parent R01 study on Cancer Survivorship Care in Primary Care

This case study was supported by the National Cancer Institute as a supplement to a larger project evaluating cancer survivorship within innovative primary care practices across the country (Crabtree et al., 2020; Rubinstein et al., 2017; Tsui et al., 2018). As mentioned above, we hypothesized that given the complexity related to cancer survivorship care, more work would be needed beyond just primary care practices to fully understand the contextual backdrop of communities, medical systems and policy and regulatory environments as they affected high-quality cancer survivorship care. Specifically, with our research team specializing in mixed methods approaches, we recognized the value of combining both quantitative geospatial tools with rapid qualitative approaches to understand multi-level features affecting cancer survivorship. We thus created the fRAP method, and received a grant from the National Cancer Institute to develop and pilot fRAP to investigate contextual features outside of a primary care practice’s four walls, assessing how multiple levels of community, the medical system, and the policies and regulations around each practice affect how they provide care for cancer survivors within their practice.

Application of fRAP to Parent R01 Study Data

For this descriptive case study of fRAP, we will describe the method as it was applied to the first practice we evaluated in the larger study. In this case example, fRAP was utilized to understand contextual features of the community, medical system and policy/regulatory levels in Erie County, New York that are facilitating or inhibiting high-quality cancer survivorship care. For purposes of this case study, “cancer survivorship” was determined to begin after treatment completion for cancer and include long-term cancer survivorship follow-up care (Rubinstein et al., 2017).

Phase Ia: Identification of geographic area of interest (see Figure 2)

Figure 2:

fRAP Methods

• Diagram of our fRAP methods, describing quantitative Phase I and its procedures, qualitative Phase II and its procedures, integration in Phase III and its procedures, with the end target of future policy change.

The fRAP method begins with identifying the geographic area of interest for the health issue in question. For this particular case example, we sought to investigate the contextual and environmental factors affecting comprehensive cancer survivorship care within a specific county (Erie County, NY), which includes a Patient Centered Medical Home practice that participated in the larger National Cancer Institute study. The county was chosen as our geographic area of interest for this study, instead of ZIP Code Tabulation Areas or drive-time boundaries, due to availability of open source quantitative variables of investigation at the county level. While this particular case example had locations partially determined through the larger study, fRAP is ideally suited to utilize Geographic Information Systems technology to assist with uncovering geographic areas of interest a priori. Thus, fRAP can harness the strength of geospatial methods to uncover geographic hot spots or cold spots, quantitative high or low outliers for the public health issue of focus, in order to then qualitatively focus the investigation in that geographic area of interest with the potential for highest success at uncovering modifiable elements. For example, in subsequent iterations of fRAP, we utilized geospatial mapping to identify counties and then ZIP code tabulation areas with higher cervical cancer mortality in Hispanic women in order to illuminate geographical regions of high need in which to focus (Kieber-Emmons, Miller, Crabtree, & Topmiller, 2018).

Phase Ib: Assessment of Quantitative Data (see Figure 2)

The second step of Phase I begins with the development of a novel tool, a community assessment profile (CAP), of the county in question. This community assessment profile includes multi-level data from multiple levels of the socio-ecological model, including the community, medical systems/providers, and policies and regulations. Examples of such data would include rates of the particular disease-specific mortality, geographic location of medical oncology specialists and tertiary care centers, and disease-specific regulations and laws specific to that geography. In Phase I, quantitative variables of interest are filled into the community assessment profile using quantitative geospatial mapping tools of health variables and social determinants of health (see Figure 3).

Figure 3:

Community Assessment Profiles

• Diagram showing the three levels of interaction, community, medical and policy, that are investigated through fRAP, with the quantitative and qualitative data of interest collected for each level.

We worked in collaboration with the Robert Graham Center (Washington, DC) and their HealthLandscape Geographic Information Systems mapping platform at the time to investigate publicly available health variables that may have relevance to cancer survivorship care. Many variables, from publicly accessible census and other survey datasets, can be typed as a community, medical or policy variable, and are available within geospatial platforms like HealthLandscape to create and visualize in a map. Additionally, platforms like HealthLandscape can map any survey or clinical data that is input by a research team. For this study, publicly available variables of interest were mapped and evaluated, and key variables to include in the final analysis were determined through a round-table discussion, drawing from previous literature in the area of cancer survivorship, with three members of the research team alongside geospatial and policy experts at the Graham Center. Final health variables of interest collected for this study included 1) educational attainment for high school diploma and college degree, 2) poverty levels under 100%, 138%, 200% and 400% of Federal Poverty Level, 3) race/ethnicity for white, black, Hispanic, Asian and American Indians, 4) percentages of insured, uninsured, Medicaid, Medicare, and private insurance, 5) all-cause cancer mortality rate/100,000 and 6) population/ mental health clinicians (including psychiatrists and psychologists). Additionally, we mapped key health care practitioner locations, including maps of oncology physicians, primary care clinicians, and mental health professionals (including psychiatrists and psychologists) in the county using the American Medical Association Health Workforce Mapper (American Medical Association, 2018).

Phase IIa: Integration and Qualitative Data Collection and Analysis (see Figure 2)

After creating the quantitative pictorial representations of the county health variables and providers described above, the fRAP method moves to the qualitative investigative stage of Phase II. Integration occurs at this step of the methods through the utilization of quantitative findings to tailor qualitative inquiry and key informant identification. Using the GIS information described in the county CAP, a targeted internet search begins to add county-level details related to key community assets and barriers and health systems identified through the mapping process.

The internet review continues to broaden outward from the quantitative variables, and Phase II can then include interviews with a few local experts, a quick code of the open-source literature, or, in this case study, analyzing existing qualitative data from the parent study of the area. Depending on data sources available for a particular health issue and research project, the qualitative preliminary data entry source for on-the-ground interviews and participant observation will vary. For example, in our current study utilizing fRAP to study cervical cancer mortality disparities in Hispanic women, our preliminary data source characterization began with the local community Area Health Education Center public health agencies, alongside policy review of key county and state-wide cervical cancer screening and treatment programs (Kieber-Emmons et al., 2018; Kieber-Emmons, Miller, Crabtree, Jaen, & Topmiller, 2019).

For Phase II in this case study, we focused our qualitative investigation on the GIS findings which directed the open source internet review, and review of fieldwork data collected within the larger study. The larger National Cancer Institute study methods collected in-depth qualitative data through 10–12 days of participant-observation research in 14 exemplar primary care practices (Crabtree et al., 2020). A large portion of this time on site in the practices allowed for collection of survivor perspectives and data on the delivery of cancer survivorship care. For this case study, we focused on all data that was collected for the Erie County practice recruited for the larger study. Thus, for this iteration of fRAP, our qualitative Phase II investigation included analyzing all participant-observation materials obtained during the 10–12 days of on-site field research. Using observational fieldwork and interview transcripts of patients, clinicians, and administrators collected for the parent study, the first author highlighted passages related to each of the study levels: community, medical and policy. From these highlighted passages, a list of keyword/phrases for the three levels were then collated across all qualitative data and linked to relevant quantitative mapping data to create a community, medical and policy keywords/phrases list. For instance, the medical level list included keywords related to specific oncologists, cancer treatment centers, and radiology sites for follow-up that are used in this county.

These preliminary lists of keywords/phrases for each level were then utilized as a starting point for further review. Using open-source internet websites, the first author investigated each level (community, medical and policy) with iterative rounds of internet searching. Internet review for each of the keywords/phrases identified was undertaken to describe each in detail as they related to cancer survivorship. For instance, if a particular oncology group was mentioned in the transcript and/or identified on the map, internet review of that group was done with regards to cancer survivorship on their website. Any information related to cancer survivorship was recorded and subsequent keywords or phrases that indicated where additional internet review on other sites would be needed were collected. This scoping approach frequently brought up additional information on a different site from the first internet site for which to investigate, e.g. a support group at the local hospital that the oncology group recommends for patients. Iterative rounds of internet review for each of these keywords or phrases were undertaken with continuing cycles of subsequent keywords and sites identified. Internet review was complete when saturation was reached, typically after more than 20 internet sites were assessed for each level, and no further keywords were identified, and no new information was found on internet review. Community Assessment Profiles were filled in for qualitative data collected through Internet review (see Figure 3).

The next step of the fRAP method in Phase II includes identifying what questions related to the health issue for each level are not discoverable or explained by either the quantitative county maps and variables, or the analysis of qualitative data and iterative internet review cycles. For this particular study, the team of researchers met bimonthly throughout the quantitative and qualitative data collection phase for debriefing and discussion of what questions still needed to be answered regarding cancer survivorship care from the community, medical system and policy/regulatory perspectives. For each of the three levels, one to three key informants were identified who the authors believed could share additional information through interviews beyond what was publicly accessible online. Preferred informants, who are considered to be most closely connected to the outstanding information needed, were highlighted for each level.

Next, the interview guide for depth interviews was created through the qualitative process described in “The Long Interview,” with five team members contributing to ensure representation of a diversity of previous experience (Crabtree & Miller, 1991; Dicicco-Bloom & Crabtree, 2006; McCracken, 1988). The first author then conducted key informant recruitment by email and/or phone outreach with the individuals on the list. Key informants willing to partake in a phone interview with the lead author were consented for participation in the study. After potential key informants agreed to participate in the research study, consent was obtained and 45–60 minute depth interviews were completed by the first author using the semi-structured interview guide. One interviewee from each level (community, medical systems, policy/regulatory) was included. All interviews were recorded and transcribed for accuracy of content.

Interview transcripts were analyzed using an “editing approach” as described in The Long Interview (Crabtree & Miller, 1991, 1999; McCracken, 1988). The editing organizing style for qualitative analysis is one of three idealized approaches to analysis for primary care qualitative researchers ((Crabtree & Miller, 1999). It contrasts with template and immersion/crystallization styles. Whereas a template approach uses a template or code manual for analyzing qualitative text and immersion/crystallization involves intensive use of extended personal and reflexive immersion in the text, the editing approach has the investigator exploring the data for information most pertinent to the research question and purpose and then generating an emergent code book with subsequent identification of themes and patterns. Grounded theory, content analysis, and hermeneutic interpretation are examples of the editing style (Crabtree & Miller, 1999).

In this method, using the “editing approach,” three members of the team read each transcript and recorded expanded observations into small text boxes added to the margins of the transcript of the interview content. These observations are less constrained than typical codes, and allow for greater richness to be obtained in the analysis stage. The three group members had bi-monthly meetings to discuss their personal expanded observations for each transcript. Consensus was reached by all three team members on interpretation of expanded observations within each section throughout transcripts. Expanded observations were then sorted and bundled by the PI into category and summary themes identified for each category. These category and summary themes were then independently reviewed by the other two team members for accuracy. Through this analysis, between three and seven summary themes were identified in the transcribed interviews.

Legitimation of results was obtained through the three primary approaches of the use of a team, reflexivity, and member checking (Cohen & Crabtree, 2008). The qualitative analyses were done independently by three different people and then consensus reached in team discussions. Prior to this analysis work, the team members explored past experiences, beliefs, and assumptions concerning the research area and used these to help identify and mitigate interpretive bias. The findings were also shared with participants in the qualitative phase for feedback.

While this data analysis process is not particularly more rapid than traditional ethnographic techniques, the benefit of fRAP is that it allows for more rapid turn-around precisely due to its design. By harnessing geospatial mapping to illuminate where to undertake qualitative work, fRAP allows for more limited qualitative data collection in targeted areas of each level, and in targeted geographic regions. By limiting the amount of qualitative data collection needed, while still ensuring accurate end results, the qualitative data analysis step is able to be accomplished more rapidly than traditional approaches.

4). Phase IIb Cross-Comparison Analysis (see Figure 2)

In Phase IIb, a cross-comparative analysis process is undertaken across levels and regions. An Excel spreadsheet is created to place themes from three levels across each regions studied. Excel is an ideal low-cost, world-wide, easily accessible software package that is suited to visualizing themes across levels and across different geographic regions at one time. Cross comparison of themes between interviewees from different levels within the same county were analyzed. Identification of similarities and differences in terms of the provision of cancer survivorship care were compared between interviewees. Contextual factors from each level, community, medical systems, and policy/regulation, that were brought up as facilitators or barriers to the provision of cancer survivorship care were highlighted.

This manuscript’s case study showcases the fRAP method applied to only one geographic region, with quantitative and qualitative data collected across the community, medical and policy levels in one county. However, the fRAP method ideally includes a robust and important cross-comparative analysis stage across geographic regions. A key feature of fRAP is not only comparison of themes within one geographic region across each of the three levels, but also comparison of themes across level-specific informants from multiple geographic regions. This approach provides a rich understanding of the contextual factors that are unique in one geographic setting or in one specific level across regions, as well as the contextual factors that are similar across multiple levels and regions.

5). Phase III Identification of Modifiable Elements for Policy Action (see Figure 2)

The final stage of fRAP, Phase III, is the culmination of all data results and the identification of modifiable contextual elements that are potential environmental policy change targets to improve the health issue in question. For this study, our aim was to identify possible modifiable contextual elements that could improve cancer survivorship care. Within the context of this case study and larger study, Phase III examples of these types of policy changes include electronic medical record triggers, insurance company recognition of cancer survivorship for care management incentives, and improved collaborative processes between primary care clinicians and their local cancer centers. Specific contextual elements identified in this case study will be described below. Importantly, for subsequent use of fRAP, for example within our current cervical cancer survivorship study, Phase III has been refined and is now more robust, with the use of iterative site visits to counties and Zip code tabulation areas highlighted through geospatial mapping, and cyclical rounds of integration between quantitative and qualitative data (see Figure 4).

Figure 4:

fRAP Cyclical Rounds for Cervical Cancer Study

• Diagram showing the Cervical Cancer Study, and iterative rounds of site visits, and multiple points of integration between cyclical qualitative and quantitative data

Results:

We briefly describe results from the fRAP method with the case study from Erie County, NY.

Phase I- Quantitative Data:

The GIS community assessment profile developed with the Robert Graham Center shows that Erie County has a higher percentage of white (79.1% vs. 65.1%), insured individuals (93.6% vs. 89.4%) compared to New York State as a whole, which could indicate a healthier population (see Table 2, Figure 5). However, Erie County also has a higher all-cause cancer mortality rate than New York State, of 183.3/100,000 compared to 160.1/100,000 (see Table 2, Figure 5). This incongruence between demographics and anticipated outcomes was highlighted in our assessment profile of this county and further investigated during the qualitative phase.

Table 2:

Social Determinants of Health for Erie County

	New York State	Erie County
Educational Attainment % General Education Diploma	26.9%	28.4%
Educational Attainment % College	19.1%	17.2%
Poverty <100% Federal Poverty Level	15.6%	14.7%
Poverty < 138% Federal Poverty Level	22.4%	21.1%
Poverty < 200% Federal Poverty Level	32.6%	31.2%
Poverty < 400% Federal Poverty Level	60.0%	62.5%
Race/Ethnicity: American Indian	0.4%	0.5%
Race/Ethnicity: Asian	7.8%	2.9%
Race/Ethnicity: Hispanic	15.6%	13.2%
Race/Ethnicity: Non-Hispanic-Black	18.2%	4.8%
Race/Ethnicity: Non-Hispanic-White	65.0%	↑ 79.1%
Uninsured %	10.6%	6.4%
Insured %	89.4%	↑ 93.6%
Medicaid %	22.5%	19.9%
Medicare %	15.4%	18.2%
Private Insurance %	65.7%	72.4%
All-cause Cancer Mortality (per 100,000)	160.1	↑ 183.3
Population per Mental Health Provider (includes psychiatrist and psychologist)	1068:1	1860:1

Arrows indicate outlier value above overall New York State values. 2015 Robert Graham Center Report from Health Landscape.

Figure 5:

Erie County Variables of Interest for Cancer Survivorship Care

• Map of New York state and specifically, Erie County, NY (outlined county on far left of the map), developed to highlight demographic and social determinants of health relevant for cancer survivorship care in this county. Variables showcased include % insured across all New York counties, alongside Erie County all-cause cancer mortality rates/100,000 and population per mental health provider compared to New York state rates.

Additionally, mapping of oncologist providers was also completed and indicates a large National Cancer Institute-designated Cancer Center that is present within the boundaries of Erie County, along with numerous other oncology practices, which could have implications on cancer survivorship referral patterns (see Figure 6). Erie County has multiple oncology providers compared to the surrounding counties which have few to none. Once again, this mapping detail was marked within the CAP for subsequent qualitative investigation within the internet reviews as well as the key informant interviews.

Figure 6:

Erie County Map of Oncology Provider Locations

• Map of all oncology providers located in Erie County, NY and its surrounding counties. This map highlights the large number of oncologists located within Erie County (outlined county in center of map) versus the lack of oncologists in surrounding counties.

Phase II – Qualitative Data:

Final keywords/phrases for Erie County for the community level (see Figure 7), derived from mapping and qualitative parent data, illuminated community organizations related to cancer survivors in the region such as the Erie County Cancer Services Program, Susan G. Komen of Western New York, and the local Cancer Center’s Community Engagement Office. The medical level (see Figure 7) included key hospitals and cancer specialists from whom patients received care at, such as the local Accountable Care Organization (ACO) and hospital system that the practice participates with, as well as the free-standing radiology office that the practice utilizes, and the local National Cancer Institute-designated Cancer Center. The policy level (see Figure 7) highlighted a local insurance company, the electronic medical record utilized by the practice, and state-level Department of Health employees.

Figure 7:

Keywords Identified by Level for Practice 3 (P3)

• Diagram of all keywords identified through Phase II for the community, medical and policy levels surrounding the Erie County PCMH practice (Practice 3 in the R01 larger parent study). Key informants for each level were then identified through the use of these main keyword lists.

For this case study, six key informants, two from each level, were identified as potential interviewees. Our preferred informant for both the medical and policy levels agreed to participate in our study. Our preferred informant for the community level declined our invitation; however, our second choice informant agreed to participate in our study. All three informants were interviewed, and the audiotape transcribed for analysis. For the community level, a health disparities community researcher was identified and interviewed. For the medical level, the Chief Executive Officer of a large health system was identified and interviewed. For the policy/regulatory level, the Chief Executive Officer of an insurance company was identified and interviewed.

An excerpt from the medical level interviewee analysis process is included (see Table 3). This excerpt allows for illustration of the expanded observations that were developed by group consensus, alongside the organizing concept and summary theme for the observations that all fell into this one grouping. Roughly four to seven summary themes were identified for each key informant transcript (see Table 4).

Table 3:

Example of Expanded Observations, Organizing Concept and Summary Theme

The oncology-primary care relationship needs to be strengthened for cancer survivorship care to be implemented.  •The adult oncology-primary care relationship is weak. Incentives for better collaboration through the ACO and PCMH give reason to be hopeful that there may be improved collaboration in the future. As a pediatrician, he feels that “childhood cancers” are an anomaly; for a variety of reasons, pediatric primary care doctors and specialists provide better cancer survivorship care.

He feels that the relationship between oncology and primary care is improving, partly driven by the “need that that has to occur” [perhaps due to the ACO?]

He is interested in “tools and processes” that will enhance collaboration between primary care and oncology.

He feels that there is a good “collegial” relationship between providers partly because of some oncologists working hard to ensure good communication.

He feels that the relationship between PCPs and oncologists is not as strong in the adult population.

He is explaining that part of the difference between children and adult populations in this city is that pediatric oncology is only offered at the Cancer Center and the Children’s Hospital, but in the adults, they have community oncologists and the academic oncologists and he feels that there is a “historical” divide between those “in the ivory tower” and those in “private practice.” [could this be referencing tensions re: Cancer Center’s history as per fieldnotes?]

He feels that this positive relationship between providers and oncologists in pediatrics in partly due to the fact that people are willing to go the “extra mile” because of the “emotional aspect” of cancer in children.

He describes how primary care feels that cancer patients get lost to oncology, “going into the black box,” and the providers don’t know what is happening with their patients anymore.

He indicates optimism that we are at a time where collaboration needs to improve. [possible reference to market changes?]

In response to the interviewer’s question about EMR, he explains that they have 20+ outpatient EMRs and a different inpatient EMR which don’t talk to each other and make transmitting information “cumbersome and time-consuming.”

He thinks oncology is changing and becoming more “collaborative” across the health system.

He can imagine the possibility of cancer survivorship fitting into this model nicely.

Table 4:

Summary Cross Comparison Themes for Erie County Key Informants

	P3 Community	P3 Medical	P3 Policy
Data/guidelines are important, lack of guidelines, hard to implement existing guidelines given pressures on PCPs	X	X	X
Financial metrics drive what gets done		X	X
Need improved PCP-oncology relationship/coordination	X	X	X
EMR changes may enhance cancer survivorship		X	X
Focusing on diverse communities and their survivorship needs	X
Advocacy with community cancer groups	X
Involving Survivors in Research	X
Lack of National Survivorship Focus, Advancing the Agenda	X	X	X
Survivorship in Online Tools			X

Compiling all themes from analysis of each of the three key informants’ transcripts, a cross comparison matrix of themes was created (see Table 4). While some themes were level specific, two major themes showed up in all three key informant interviews:

1) Guidelines and data are important, yet a challenging barrier and 2) oncology-primary care relationships are paramount and need to be strengthened to optimize care delivery.

Across all three informants, guidelines were discussed extensively as a prerequisite for improving survivorship care:

Medical Level Informant: “When we get into really that survivorship portion, we currently don’t have a level of surveillance in place to see if in fact we are standardizing the care as well as it could be, so my guess – though I don’t have data to prove it one way or the other, which I guess is part of the indictment, the fact that there is no data in that area– my guess would be is that we probably have more variation than we would ideally like to see.” (March 14, 2016)

Community Level Informant: “And so the primary care docs are pretty overwhelmed here with the patient population… anything that’s not a mandated thing that they have to fulfill for medical home certification or stuff that’s not coming directly down from the feds - some of the cancer screening stuff and prevention stuff - especially if it’s like a guideline that’s kind of in flux, or, you know, not completely standardized or hard and fast – they’re, you know, a little wary of maybe promoting it or doing as much with it.” (March 23, 2016)

Policy Level Informant: “I think right now, providers are overwhelmed with reporting requirements and metrics, and I think that when they think about cancer survivors in their practice, I think they’re happy to see them, grateful that they’re alive, and pretty much consumed with the illness burden and the gaps in care that remain for the rest of the population, because so much energy now has been focused on quality metrics and quality performance, and now that [Centers for Medicare and Medicaid Services] CMS is significantly tying compensation to that.” (April 20, 2016)

Furthermore, all three informants highlighted the need for improved primary care practitioner (PCP)-oncology relationships:

Medical Level Informant: “Complaints I hear from the primary care doctors is the patient gets absorbed by the oncologist…you know, they’ll describe it as going into the black box: ‘I never see the patient again. I don’t know what’s going on with them, and sometimes they don’t even have the decency to call me to let me know the patient expired,’ that type of thing.” (March 14, 2016)

Community Level Informant: “And then also some special attention being given to having specific staff or positions that focus on relationships in the community with clinicians – so building relationships with primary care offices, primary care practices, and physicians, and making sure that people understand what’s available at [cancer institute]: what are the resources.” (March 23, 2016)

Policy Level Informant: “There’s still a lot of growth, a lot of room for growth and coordination. Typically, physicians have seen [cancer institute] through two lenses. One is it’s a great institution that can provide real excellence in care. But they’ve also seen it as kind of a black hole for their patients they never return. And so that’s an image that I think [cancer institute] is working to change, and wanting to be more collaborative with the physicians.” (April 20, 2016)

This manuscript highlights only fRAP applied in Erie County; nevertheless, it is important to restate that the cross comparative matrix created in this Phase also includes community, medical and policy level themes across 6 more regions included in the larger study. The qualitative results described above for Erie County only showcase the cross-comparative analysis between the community, medical and policy interviews within Erie County. However, a robust and illuminative analysis of this data across levels and across regions was also undertaken and is highlighted in our discussion.

III. Phase III: Identification of Modifiable Elements for Policy Action

The goal of fRAP is to quickly and effectively identify modifiable barriers and facilitators contributing to cancer survivorship care within the county that could serve as targets for future intervention. As such, Phase III integrates quantitative and qualitative findings to identify these modifiable contextual elements through detailed analysis and linkages across the Community Assessment Profiles. For example, in this case study of Erie County, it became apparent through qualitative interview analysis at the medical level, that some aspects of that level create challenges for delivery of high quality cancer survivorship care and may be contributing to the high all-cause cancer mortality noted above, namely a preponderance of different oncology practices, a competing Cancer Center, and a myriad of different EMRs across hundreds of primary care practices. Four final modifiable contextual elements of interest for Erie County discovered through the use of fRAP in this study include:

• 1) The need to develop metrics/guidelines for cancer survivorship care within primary care (medical and policy level)

• 2) The importance of financial incentives for provision of cancer survivorship care, including survivorship care plans, but expanding to other financial incentives relevant to primary care as well (policy level)

• 3) The need for strengthened shared care arrangements between primary care and oncology (medical level)

• 4) The development of Electronic Medical Record updates to reflect cancer survivorship follow-up requirements, along with the creation of cancer survivorship ICD 10 coding and appropriate billing options (medical level)

A key element of Phase III includes dissemination of results for change back to the community and stakeholders who participated in the process. A final summary of findings of potential contextual change elements was developed for dissemination to our participants in the study from Erie County.

Discussion:

Our development and implementation of a novel mixed methods approach, fRAP, to examine cancer survivorship care implementation, was effective in focusing and elucidating modifiable multi-level factors that affect care delivery, more quickly and efficiently compared with traditional, mixed methods approaches. Through fRAP, we were able to identify potential, actionable multi-level contextual features impacting cancer survivorship care delivery in Erie County.

Contributions to the Mixed Methods Literature

Rapid, focused investigation of multi-level contextual factors contributing to public health problems is needed in today’s highly complex environment. Health services researchers are beginning to recognize that focusing research solely on care that occurs within a practice during a clinician-patient visit is naïve (Klein & Kozlowski, 2000; Resnicow & Page, 2008; Zapka, 2008). Complexity abounds in today’s healthcare delivery environments; the majority of healthcare is delivered within a specific community that has one or more particular hospital or ACO networks with a subset of insurance products and regional policy forces that create an environment that encourages a model of care that may or may not provide the highest quality for a patient (McWilliams, Hatfield, Chernew, Landon, & Schwartz, 2016). An understanding of the complex interplays between these levels can help to develop local and regional policy solutions for improved cancer care, enhanced long-term cancer survivorship with better quality of life, and decreased morbidity rates (Gorin et al., 2012; Stange et al., 2012).

As a novel mixed methodology, fRAP was specifically developed to embrace this complexity, creating a clearer understanding of how multi-level contextual features interact across place and across time to impact the problem in question. Importantly, fRAP allows for data collection across multiple levels, and from multiple perspectives, through its use of both quantitative and qualitative data as well as its application to community, medical and policy level features. As such, fRAP is the first of its kind to combine the use of geospatial mapping along with rapid assessment processes to efficiently and effectively uncover modifiable targets for policy reform by specifically recognizing the complexity of these multi-level factors.

As a mixed methods approach, fRAP has the potential for use by health services and policy researchers to quickly and reliably identify key modifiable contextual elements that are driving a health issue. Its goal is to provide health services or policy researchers with the capacity to identify policy interventions. It is designed such that much of the fieldwork can be done from a distance, thus minimizing the resources needed to conduct the study. It allows for the ability to include the population studied in the data analysis, even though it is done rapidly, by including the target population in mapped quantitative data and/or qualitative informant interviews. Furthermore, fRAP can be replicated and spread for multiple cross-comparisons across different regions.

Through multiple iterative rounds of fRAP, applied to a particular public health issue across different regions of the country, potential best practices can be identified for policy advocacy and implementation. Furthermore, while this study focused on fRAP as applied to cancer survivorship care, we have utilized this methodology to examine other aspects of cancer care along the cancer care continuum, including our subsequent and ongoing study evaluating cervical cancer mortality disparities in Hispanic women. Within this study on cervical cancer disparities, we have begun to apply fRAP to investigate more nuanced complexities within multi-level systems, such as trust and fear of undocumented and immigrant populations, with much success. Additionally, the methods can be applied to other public health issues outside of the field of cancer, including our subsequent and ongoing study utilizing the methods to investigate geospatial “hot spots” of poorly controlled diabetics, and “bright spots” of well-controlled diabetics within the primary author’s health system (Shaak, Topmiller, Kieber-Emmons, Careyva, & Johnson, 2019; Topmiller, Kieber-Emmons, Shaak, & McCann, 2020).

Cross Comparative Benefit of fRAP

A distinct and compelling feature of the fRAP method is that summary themes can be compared between key informants for one county (as described above in this case study), as well as between additional counties or regions. In this grant-funded study, cross-county analysis was also undertaken to evaluate whether contextual or environmental cancer survivorship care elements were replicated or discussed by “like” level key informants across counties throughout the country. Seven counties and twenty-one key informants, representing diverse areas of the country, were included in this fRAP method, and analyzed for cross-comparison. These seven geographies represented three regions of New York State, including upstate and downstate, two areas in Maine, and two areas in Colorado, including Denver and the surrounding suburbs.

While we only report on one case study here to illustrate fRAP, we did apply the method to all seven regions included in the study. Interestingly, across the seven regions, only Erie County informants discussed guidelines and metrics as important to survivorship care. This suggests that contextual elements of the healthcare environment in Erie County contribute to this highly replicated theme within the county that was not echoed by any informant in any other area of the country. Alternatively, informants from all seven practice regions discussed the need for improved primary care clinician-oncology communication and relationships. This universal theme seems to indicate an overarching problem of an uncoordinated healthcare system nationally.

Limitations:

Given the design of this research study within a larger study, with an investigator on-the-ground in a priori designated counties, this iteration of fRAP was specifically tailored to the larger study design (see Figure 8).

Figure 8:

fRAP Phase I and II, as applied for this NCI Case Study

• Diagram of the additional data source for fRAP for this cancer survivorship case study, showing the use of the R01 larger parent study ethnographic fieldwork in the primary care practice collected in Phase I and then analyzed in Phase II.

However, ideally fRAP utilizes Phase I geospatial techniques to first target and choose counties or zip codes for focus based on the identification of geographic hot and cold spots on quantitative variable mapping. Then, after the identification and investigation of the county of choice through GIS mapping of multi-level variables, Phase II entry into a county would most often begin with preliminary on-site qualitative participant observation and key informant interviews. The benefit of this approach is that it is a reproducible type of identification and entry point for fRAP when applied to any health problem across multiple geographic regions. In one of our subsequent grant-funded studies utilizing fRAP to study cervical cancer mortality disparities in Hispanic women, the authors developed and refined the fRAP process for entry in Phase II into the county through initial key informant interviews with local Area Health Education Center and public health department representatives. In another subsequent grant-funded study, fRAP is being used to study diabetes care within our network, using geospatial mapping of institutional electronic medical record data to identify zip codes of interest for Phase II qualitative investigation and patient focus groups.

A second limitation of this study was the ability to only include one interviewee from each level. While our larger study did include cross-comparison of individuals from each level across seven regions of the country, within each county, we may have included a stakeholder for a level whose perspective was not reflective of others of that level in that region. For our two subsequent studies utilizing fRAP, we have attempted to mitigate this concern by including 2 or more informants from each level in each geographic region.

Finally, while low and resource-poor countries may have challenges to undertaking fRAP in their settings, we believe that many aspects of our methodology can be modified in ways that it can still be utilized in those settings. In particular, Excel is a software package that is likely to be accessible in these settings. While geospatial mapping packages may be unavailable, maps of small villages and rural areas could be drawn with paper and pencil. Additionally, while the internet scoping review of data would be not be applicable in locales without internet, the qualitative work of participant observation, agency and site tours and key informant and focus group interviews could be undertaken in Phase II to still develop a robust cross-comparative database of qualitative themes.

Conclusions:

Long-term and sustainable policy interventions are challenging, complex and often span multiple levels of the socio-ecological framework. As the cancer care research community has noted, multi-level interventional research is needed to ultimately improve outcomes (CDC; Stokols, 1996). fRAP is an innovative health policy method aimed at understanding these complex, multi-level connecting and competing factors affecting health issues in our country today. Through the fRAP method, we hope to create a public tool or roadmap for how to tackle today’s most challenging health problems by illuminating effective and realistic policy change targets. Such a focused and rapid assessment of contextual environmental and policy elements is needed if health services research hopes to effect change and improve health.