Abstract
Background
Systematic screening of all colorectal tumors for Lynch Syndrome (LS) has been recommended since 2009. Currently, implementation of LS screening in healthcare systems remains variable, likely because LS screening involves the complex coordination of multiple departments and individuals across the healthcare system. Our specific aims are to (1) describe variation in LS screening implementation across multiple healthcare systems; (2) identify conditions associated with both practice variation and optimal implementation; (3) determine the relative effectiveness, efficiency, and costs of different LS screening protocols by healthcare system; and (4) develop and test in a real-world setting an organizational toolkit for LS screening program implementation and improvement. This toolkit will promote effective implementation of LS screening in various complex health systems.
Methods
This study includes eight healthcare systems with 22 clinical sites at varied stages of implementing LS screening programs. Guided by the Consolidated Framework for Implementation Research (CFIR), we will conduct in-depth semi-structured interviews with patients and organizational stakeholders and perform economic evaluation of site-specific implementation costs. These processes will result in a comprehensive cross-case analysis of different organizational contexts. We will utilize qualitative data analysis and configurational comparative methodology to identify facilitators and barriers at the organizational level that are minimally sufficient and necessary for optimal LS screening implementation.
Discussion
The overarching goal of this project is to combine our data with theories and tools from implementation science to create an organizational toolkit to facilitate implementation of LS screening in various real-world settings. Our organizational toolkit will account for issues of complex coordination of care involving multiple stakeholders to enhance implementation, sustainability, and ongoing improvement of evidence-based LS screening programs. Successful implementation of such programs will ultimately reduce suffering of patients and their family members from preventable cancers, decrease waste in healthcare system costs, and inform strategies to facilitate the promise of precision medicine.
Trial registration
N/A
Electronic supplementary material
The online version of this article (10.1186/s12913-018-3636-2) contains supplementary material, which is available to authorized users.
Keywords: Lynch syndrome, Implementation, Qualitative comparative analysis (QCA), Consolidated framework for implementation research (CFIR), Precision medicine, Universal screening, Colorectal Cancer, Economic evaluation, Decision analytic modeling, Configurational comparative method
Background
Rationale
The goal of precision medicine is to improve health outcomes by tailoring healthcare based on an individual’s genomic and other relevant information, including patient preferences [1]. One example is systematic screening of colorectal cancer (CRC) tumors to identify all patients whose CRC may be related to Lynch syndrome (LS) [2, 3]. LS is the most common form of inherited CRC and is also associated with significant risk for endometrial, ovarian, gastric, small bowel, and renal cancers, among others [4, 5]. LS most commonly results from the inactivation of the DNA mismatch repair (MMR) system. A first step in screening for LS in patients with cancer involves testing tumor tissue for signs of MMR deficiency in one of four genes (MLH1, MLH2, MLH6, and PMS2) [2]. Following a positive tumor screening test, additional testing for the presence of a germline pathogenic variant in one of the MMR genes is needed to confirm diagnosis of LS. A diagnosis of LS influences cancer screening and surveillance guidelines to be followed and treatment options for those already diagnosed with cancer. Patients with CRC who have LS benefit from treatment with immunotherapy [6] and have the option for more extensive colonic surgery to decrease the risk of metachronous malignancy, which is more common in LS [4, 5]. Additionally, prophylactic hysterectomy, and salpingoophorectomy can reduce risk of endometrial and ovarian cancer (90–100%) in women with LS [7].
Approximately one million people in the US have LS, of which only about 2% are aware [8, 9]. Therefore, most are not receiving potentially life-saving surveillance and treatment. In addition, at-risk family members, who are at 50% risk to also have LS, are not being identified. Cost-effective, evidence-based systematic screening strategies to identify cancer patients with LS are available and have broad recommendation for implementation [2, 10–16], yet LS screening is inconsistently applied within healthcare systems, if at all [17–19]. Systematic (also called, “universal”) screening of all newly diagnosed cases of CRC for LS is cost-effective by most measures [20, 21] and is classified by the Centers for Disease Control (CDC) as having top-tier evidence for reducing cancer morbidity and mortality and improving quality of life [11, 22]. LS screening was first recommended by the Evaluation of Genetic Application in Practice and Prevention (EGAPP) working group in 2009, and is currently recommended by multiple professional organizations, including the National Comprehensive Cancer Network (NCCN), and the Healthy People 2020 initiative [3, 10, 12–16, 23]. LS screening was also recently recommended by the Blue Ribbon Panel Precision Prevention and Early Detection Working Group to meet the goals of the Cancer Moonshot [9].
Contextual factors such as organization mission, patient population, and costs may all influence decisions to implement genomic technologies in healthcare systems [17, 24–27]. However, there is poor understanding of how these and other contextual factors impede or facilitate LS screening implementation in healthcare systems and under what circumstances [17, 18]. Factors specific to LS screening implementation may include: variable involvement of multiple key stakeholders and champions, availability of genetic counseling, genetic testing costs, and perception of value to the healthcare system [18, 19, 28]. Previous research has shown that institutional testing costs, number of CRC patients diagnosed per year, local prevalence of LS, and perceived cost of screening older cancer patients are key determinants for organizational decision makers and likely key barriers to LS screening implementation [28–30]. Additionally, there are multiple evidence-based protocols that provide different recommendations for LS screening [4, 10, 31]. These conflicting recommendations are not surprising in the face of evolving evidence, but they are likely contributors to variability in implementation. For instance, the NCCN guidelines suggest that if all CRCs cannot be tested, then testing should occur on tumors of patients diagnosed under age 70 and for patients with CRC over age 70 who meet complex criteria using family history or tumor characteristics associated with increased LS risks [4, 15]. This conflicts with evidence demonstrating that imposing age restrictions can result in substantive loss of LS index case finding, is challenging to implement, and assessment of family history and other risk factors often fails to identify individuals who are appropriate candidates for LS testing [8, 32].
However, LS screening is only successful if patients who screen positive (by tumor tissue testing) are informed and follow-through with confirmatory germline testing so that patients and their family members can access appropriate cancer treatment, screening, and prevention options. A functional tracking system and/or other processes to ensure program success is therefore an additional component of critical importance to LS screening. A study of several medical centers found that positive results were often not disclosed and poor rates of patient follow-through to germline confirmation occurred until such procedures were included in LS screening programs [19]. Finally, healthcare systems vary in implementation across a spectrum from no organized LS screening to optimal LS screening. The latter is defined as including implementation or maintenance of cost-effective tumor screening protocols, procedures to ensure disclosure of positive screening, and active processes to facilitate patient follow-through to germline genetic testing for diagnosis of LS [33–35]. An additional file shows a typical optimal LS screening program design [see Additional file 1].
Guiding framework
Due to the complexity of LS screening and the variability of healthcare systems in which screening is to be implemented, a systematic approach based on the principles of implementation science is needed to ensure the success and sustainability of LS screening programs. The Consolidated Framework for Implementation Research (CFIR) has been used successfully to evaluate variation in program implementation [36, 37] and to identify conditions necessary for patient follow-through to confirmatory gene sequencing in LS screening programs [19]. The CFIR describes several processes that may influence implementation success and guides assessment of contextual conditions that may serve as implementation barriers or facilitators at the individual, organizational, and external levels. The CFIR also guides data gathering and structuring for conducting configurational comparative analysis to identify which CFIR constructs and LS screening program-specific procedures are minimally sufficient and necessary for optimal implementation, and under which circumstances in complex health care delivery systems [19, 38, 39].
Objectives and aims
The goal of this project is to utilize the CFIR and other tools from implementation science to describe, compare, and explain variations in LS screening across multiple healthcare systems and create a comprehensive, customizable organizational toolkit for implementing and improving LS screening programs. To achieve this goal, this study has four specific aims:
Aim 1: Describe variations in LS screening, implementation processes, and contextual conditions facilitating or impeding implementation across multiple healthcare systems.
Aim 2: Explain current practice variation and identify contextual conditions that influence variation in LS screening, including follow-up procedures, and implementation processes to identify which combinations are minimally sufficient and necessary for optimal implementation.
Aim 3: Determine the relative effectiveness and costs of different LS screening protocols using decision analytic models developed from previous work [29, 30] and data specific to each healthcare system to demonstrate the relative effectiveness and efficiency of various LS screening protocols used by healthcare systems based on their local data.
Aim 4: Develop an organizational toolkit to facilitate LS screening implementation and optimization and disseminate to all sites. Further assess the toolkit’s utility for facilitating LS screening implementation or optimization then refine and for broader dissemination.
Methods/design
Overall study design
This study combines multiple methods of exploring implementation in complex healthcare systems to determine which implementation strategies are likely to work best in different organizational contexts. We will use traditional case-based, in-depth analyses of conditions that may serve as barriers and facilitators of LS screening (Aim 1), followed by cross-case and configurational comparative analysis to determine minimally sufficient and necessary conditions for optimal LS screening (Aim 2). These data will be used to populate economic evaluation decision analytic models to demonstrate how organizational context, available resources, and screening protocol impact organizational costs and program success (Aim 3). A toolkit will be created and disseminated to sites to guide implementation of new programs that are aligned with the organizational context and costs or to optimize existing programs (Aim 4).
Study setting
The unit of analysis is the clinical site through which LS screening is or can be implemented. We will study the contextual factors of 21 clinical sites across 7 healthcare systems purposively selected to maximize the number of clinical sites in various stages of implementing LS screening, as well as to maximize diversity of location, system structures, and patient populations (Table 1). Six healthcare systems (Kaiser Permanente Northwest and Colorado, HealthPartners, Harvard Pilgrim Healthcare, Meyers Primary Care Institute/Reliant Medical Group, Sutter Health-Palo Alto Medical Research Foundation, and Geisinger) are part of the Healthcare Systems Research Network (HCSRN), and one is a national non-profit faith-based system with locations in 18 states (Catholic Health Initiatives). Of the six HCSRN sites, Kaiser Permanente sites are integrated healthcare systems where patients are members, while the other sites are “open” systems where patients can be members, patients, or patient members (e.g. members receiving care in the system or outside the system, or patients receiving care who are not members of the health plan).
Table 1.
Lynch Syndrome Screening Program Typea | |||||||
---|---|---|---|---|---|---|---|
CRC Tumor Screeningb | EC Tumor Screeningc | ||||||
Healthcare System | Clinical Site (State) | No Program | CRC age cutoff | BRAF Reflex | PHM Reflex | EC age cutoff | PHM Reflex |
Geisinger | Geisinger (PA) | All ages | X | X | All ages | X | |
Kaiser Permanente (KP) | KP-Colorado (CO) | X | |||||
KP-Northwest (OR/WA) | All ages | X | X | All ages | X | ||
Sutter Health (SH) | SH-Palo Alto Medical Foundation (CA) | X | |||||
Meyers Primary Care Institute (MCPI) | Reliant Medical Group (MA) | All ages | |||||
HealthPartners | HealthPartners (MN) | All ages | X | X | All ages | X | |
Harvard Pilgrim Healthcare | Harvard Pilgrim (MA) | X | |||||
Catholic Health Initiatives (CHI) | Franciscan (WA) | All ages | X | X | All ages | X | |
Tri Health (OH) | All ages | X | X | All ages | X | ||
Mercy Des Moines (IA) | All ages | X | X | All ages | |||
Kentucky One (KY) | All ages | ||||||
Chattanooga (TN) | All ages | X | X | All ages | X | ||
Good Samaritan (NE) | X | ||||||
Lincoln (NE) | < 70 years | X | |||||
St. Francis (NE) | X | ||||||
St. Joes Bryan (TX) | X | ||||||
St. Vincent (AR) | X | ||||||
Centura (CO) | < 70 years | < 60 years | |||||
Alegent Creighton (OH) | X | ||||||
St. Alexius (ND) | X | ||||||
Mercy (ND) | X |
aRecommendation: screen all Colorectal cancer (CRC) and Endometrial Cancer (EC) tumors for mismatch repair deficiency
bRecommendation: reflex test positive screens for BRAFV600E point mutation and MLH1 Promoter Hypermethylation (PHM) testing to rule out somatic loss of function (sporadic cancer)
cRecommendation: reflex test positive screens for PHM to rule out somatic loss of function (sporadic cancer)
Data collection aims 1 and 2
A comprehensive case description of each organization, including practice variation and factors influencing implementation, evaluation, maintenance, and improvement of LS screening, will be created through qualitative interviews of organizational stakeholders and patients with newly diagnosed CRC. For sites with LS screening programs, additional interviews will be conducted with patients diagnosed with LS. Interviews of organizational stakeholders will be conducted centrally by trained staff at Geisinger. Interviews of patients will be conducted centrally by trained staff at Kaiser Permanente Northwest (KPNW). Consent will be obtained at the start of the interview process and patient interviewees will receive a $25 gift card upon completion of the interview.
Organizational stakeholder recruitment
Up to 10 organizational stakeholders per site will be recruited through purposive role-based recruitment and snowball sampling [40, 41]. The actual number of and specific key stakeholders interviewed will depend on the site’s organizational structure. Typical stakeholders relevant to LS screening include: pathologists, genetic counselors, gastroenterologists, oncologists, surgeons, healthcare administrators, and health plan leaders. Research staff from each site will reach out to initial stakeholders from their organization via email or other methods to alert them to the study and invite them to participate in a telephone interview. At the end of each completed interview, the interviewee will be asked to identify any additional organizational stakeholders necessary for implementing new processes generally and LS screening specifically (snowball sampling). Additional stakeholders will be sent an email indicating that they were nominated to be invited into the study and offered the opportunity to participate in a telephone interview.
Patient stakeholder recruitment
Consideration of patient needs is another CFIR construct that may impact implementation success; therefore, it is critical to gather their feedback about LS screening. Patient follow-through with germline confirmation of LS is required for tumor screening to impact downstream clinical care processes. Likewise, anticipation of adverse patient reactions by organizational decision-makers can be a barrier to implementation at the system level. Two different groups of patient stakeholders will be invited to participate in this study: [1] patients newly diagnosed with CRC (N = 10 per site) and [2] patients who have been notified of a positive LS screen result and were recommended for additional genetic counseling and testing to confirm a diagnosis (N = 25 total across all sites with LS screening). For patients with newly diagnosed CRC, study staff at each site will determine the best way to identify and contact patients 3–6 months post-diagnosis and offer the opportunity to participate in this one-time telephone interview. This group will illuminate local patient attitudes and opinions about LS screening for organizational decision-makers, while the diversity of these patients across all sites will provide insight into patient attitudes in general towards LS screening. Patients who have received a positive LS screen result will provide insight into patient experiences with a positive LS screen across different sites and different LS screening protocols.
Qualitative data collection
Based on prior work [19, 40–42], it is anticipated that most organizational stakeholders will participate in interviews, resulting in sufficient information from each site to create a comprehensive case description with consistent detail for comparison across sites. While the number of patients interviewed per site is small (n = 10), the total number of patients across all sites (n = 220) is large for qualitative research and will result in sufficient sample size to capture the range of experiences, expectations, and preferences of newly diagnosed cancer patients from diverse backgrounds and healthcare systems related to LS screening.
As used previously by this research team and others, semi-structured interviews will be conducted via telephone centrally as noted above by staff experienced in qualitative data collection [40–44]. A summary will be created immediately after each interview and reviewed with site investigators during regular study meetings. These summaries will be used to modify the sampling procedure or interview guides if needed, to create the initial coding schema, and to create the comprehensive case description of each organization. Such summaries allow for high-level analysis during on-going data collection, facilitate codebook development, and reduce the number of de novo codes requiring re-review and re-coding of transcripts during data analysis [45, 46].
The CFIR-guided constructs to be assessed through the organizational and patient stakeholder interview guides are detailed in Table 2. The organizational stakeholder interview guide has been developed and refined using the question bank available from the CFIR website [47] and will be further tailored to the position of the key stakeholder. For example, system leaders may be asked more questions about engagement of leadership, external influences, and reimbursement incentives [40]. The patient interview guide has been adapted from a prior study [44].
Table 2.
CFIR Domain | CFIR Constructs Specific to LS Screening |
---|---|
Intervention Characteristics | Adaptability of LS screening to local context |
Perceived difficulty implementing LS screening | |
Cost to the organization associated with screening | |
Outer Setting | Patient needs and resources |
Competitive pressure to implement screening | |
Impact of external policies on organization | |
Inner Setting | Organization structure |
Perceived organizational priority to implement | |
Implementation climate in organization | |
Characteristics of Individuals | LS knowledge and beliefs, perceptions of evidence |
Individual readiness to implement screening | |
Self-efficacy to complete actions in screening | |
Implementation Process | Planning process to implement LS screening |
Champions, opinion leaders, and other stakeholders | |
Tracking and feedback processes for LS screening |
Data analyses aims 1 and 2
We will use CFIR and the case summaries created for each site to guide cross-case comparisons with the purpose of describing contextual variation associated with where, when, and under which conditions different processes for implementing or improving LS screening might be successful [47]. The result will be a descriptive summary of patterns in variations across sites, which will provide the basis for selecting conditions for inclusion in the analyses for Aim 2.
Configurational comparative methods (CCM), such as Qualitative Comparative Analysis (QCA) and coincidence analysis (CNA) are particularly suitable analytic techniques for studying causal complexity in organizational implementation (e.g. multiple conditions may combine in various ways to cause the same outcome) [19, 38, 48–50]. CNA was selected for use in our study because, unlike QCA, it uses a different minimization algorithm that can identify underlying causal chains [51]. This is useful because we anticipate the presence or absence of some contextual conditions may impact one or more combinations of other contextual or implementation conditions that are minimally sufficient and necessary for the outcome under investigation. Data analyses for Aim 2 will consist of 4 main steps [38]: 1) code data for context and implementation conditions (CFIR constructs), 2) code implementation outcomes, 3) calibrate conditions and outcomes to create data matrix and 4) conduct CNA and interpret solutions to create a model of minimally sufficient and necessary conditions for defined outcomes (Fig. 1).
Coding LS screening contextual and implementation conditions (step 1)
All patient and organizational stakeholder interviews will be digitally recorded and transcribed verbatim. Transcripts will be uploaded into Atlas.ti (www.atlasti.com) for qualitative analysis. Transcripts will be initially coded using an a priori codebook developed from the semi-structured interview guides, interview summaries, and CFIR constructs. This first round of coding (Aim 1) will look for description of LS screening procedures, process of implementing LS screening, individuals involved and inner or outer setting as shown in Table 2. Emergent (de novo) codes will be added to any other relevant sections of transcript text not fitting the a priori codes. This coding will be an iterative process involving team members independently coding 2–3 transcripts at a time, then discussing their coding to adjust the codebook and subsequently to group codes into categories or themes. This process will continue until the code list is static, all transcripts are coded, and the categories and themes are finalized. Additional codes will note whether contextual conditions were reported to impact the site’s implementation processes or procedures and how. A minimum of two researchers will independently code and resolve disputes by consensus until all conditions and impact are coded. Coding will follow protocols detailed in the CFIR technical assistance website [47] .
Coding implementation outcomes (step 2)
Two implementation outcomes will be examined: [1] implementation of any type of LS screening and [2] optimal LS screening implementation. As noted above, conditions for optimal implementation include: implementation or maintenance of any systematic LS screening without limitations; quality tracking to confirm testing of all tumors and disclosure of all positive screens, and systems for facilitating patient follow-through with confirmatory gene sequencing. An additional file (see Additional file 2) shows how the domains and constructs of the CFIR will be utilized to code the stakeholder information from each site.
Sites conducting any tumor screening on all or a clearly defined subset of endometrial and/or colorectal tumors will be scored 1 and those that have not yet implemented routine screening will be scored 0 for the first implementation outcome (Sites with no program in Table 1 vs. sites with any type of program). Using data from interview transcripts, site summaries, and the ability of sites to easily access numbers of patients screened, proportion who screen positive and proportion of those who follow-through with confirmatory testing and counseling. At least two members of the research team will independently code the second implementation outcome on a scale of 0 (tumor testing with no follow-up procedures) to 5 (fully-functioning tumor screening with facilitation and confirmation of germline testing). Coding conflicts will be resolved as described previously.
Selecting and calibrating conditions and outcomes to create a data matrix (step 3)
Following standard practices for CCM, we will select which conditions to include using theoretical and empirical knowledge of the cases [52–54]. To complete this task at least two members of the research team will review the coded data and select a set of approximately 4 to 8 conditions hypothesized to impact each outcome. Final consensus will be reached through discussion among coders and other research team members who have substantive experience with LS screening. Our second implementation outcome and any non-binary conditions will systematically be calibrated using theoretical knowledge and recommended practices in order to indicate the degree to which the outcome or condition is present or absent [52, 54]. The calibrated conditions and outcomes for each site will be compiled into a numerical data matrix for use in conducting coincidence analysis (CNA).
Conducting coincidence analyses (step 4)
Coincidence analyses (CNA) will be conducted following standard procedures [51–54] using the CNA package in R (www.R-project.org) with the main goal of identifying causal chains of conditions that are minimally sufficient and necessary for each of the two main implementation outcomes [55]. Additional CNA will be conducted to identify which contextual conditions (if any) contribute to variability in implementation processes or procedures that are hypothesized or have previously been shown to be more efficient, cost-effective, or result in better patient ascertainment and completion of confirmatory genetic testing [19, 29, 30, 33–35].
The resulting CNA solutions will be interpreted by the research team to identify implications for optimal LS screening implementation. This will be combined with the economic analysis data from Aim 3 to form the basis for the toolkit to help organizational decision makers determine what implementation strategies are more likely to work in optimizing screening given their organizational context.
Procedures aim 3
In Aim 3 we will calculate expected outcomes central to evaluation of effectiveness and efficiency of LS screening in CRC and endometrial cancer (EC) patient populations. Previously developed models [29, 30] will be modified and updated based on recent developments, current evidence and guidelines. Models will be populated using local data to reflect site-specific conditions as determined from data collected from study site stakeholder interviews in Aim 1.
We will construct decision tree-type analytic models in Excel© software (Microsoft, Inc., Redmond, Wash.), with the @Risk© software add-on for Excel (Palisade, Inc., Ithaca, NY) to enable various complex analytic procedures. The decision analytic team will construct multiple decision tree models to represent all LS screening protocols the team considers currently viable. Model variables include estimates of annual incident CRC and EC cases by age strata, LS prevalence or the assumption of an equivalent rate for all populations, and cost of tests included in screening protocols from each site to populate decision analytic models. When institutional cost is not available, alternative methods such as the average test cost based on costs reported by other participating clinical sites, regional test cost figures if publicly available from testing companies, or Medicare reimbursement rates will be used. Finally, reliable estimates of site-specific LS prevalence may not be available; therefore, this model parameter may be estimated from the most current estimates for U.S. populations [56]. Because clinical sites, and scientific evidence are dynamic, we do not expect organizational resources, testing costs, or LS screening guidelines to stay static. Therefore, the decision analytic models developed will be designed in such a way to facilitate ongoing use and modification by organizations.
Analyses aim 3
For base-case and sensitivity analyses, the decision analytic models will calculate, for each site and in CRC and EC virtual cohorts (e.g. 500 cases per year): (1) sensitivity and specificity of each LS screening protocol in identifying LS cases including number of LS cases expected to be identified, (2) total costs for each screening protocol, (3) cost per CRC and EC case-screened, (4) cost-per LS index case identified, (5) incremental differences between guideline-based screening protocols including costs and LS cases identified, (6) and patient adherence to each screening protocol including number of CRC and EC cases lost to follow up. Additional modeling of different LS screening age cut-off policies using local-level data will be conducted for each site. Models will provide objective metrics, driven by local data, of the impact of applying age-cutoffs in LS screening implementation.
Procedures aim 4
Data from Aims 1, 2, and 3 will be used to generate a working organizational toolkit to guide implementation, maintenance, and optimization of LS screening. Additional information will be collected over the entire project period from monthly project meetings, communications from site investigators, pertinent data regarding site-specific screening changes, and external evidence or guideline changes for LS screening. These data will be recorded in a project specific database created for tracking such information related to implementation [57]. This tracking database will provide important information for the toolkit development should any sites begin to implement LS screening based on being interviewed for Aim 1, but prior to receiving the toolkit.
The organizational toolkit will be created based on the CFIR conceptual framework, the in-depth knowledge of LS screening programs and contextual factors of healthcare systems from Aim 1, the cross-site comparison and CNA results from Aim 2, and economic evaluation by decision analytic modeling with local data from Aim 3. This toolkit will be disseminated to all sites through site investigators and the tracking database will record to whom it is distributed, questions asked by those receiving the toolkit, and any immediate actions taken by the site.
Six months after distribution of the toolkit, additional qualitative interviews will be conducted with up to 5 organizational stakeholders at each site using the same processes described for Aim 1. Stakeholders from sites without LS screening and those with sub-optimal implementation will be interviewed about the utility of the tool to facilitate implementation and improvement. Stakeholders from sites with optimally implemented programs will be interviewed about the utility of the tool for improvement or adaptation to emerging evidence. The interview guide for this aim will be developed by the research team in parallel with the development of the toolkit.
Analyses aim 4
Interviews will be conducted, transcribed, and coded as described for Aim 1. Interviews will be coded for information on to whom the organizational toolkit was distributed at each site, questions that were asked by key stakeholders, and whether and how the toolkit was used by organizational decision makers to facilitate LS screening implementation and/or improvement.
Discussion
Dissemination
The final organizational toolkit for LS screening implementation, maintenance, and improvement will be modified based on the information learned throughout the study and from the pilot distribution to all study sites. The final product will include descriptions of most commonly included stakeholders and general processes needed for optimal LS screening, directions for processes and protocols that are more likely to work by contextual factors identified, and a generic decision analytic model for costs and effectiveness related to available LS screening protocols. This organizational toolkit will have an intuitive interface that allows for the input of local parameters for use by organizational decision-makers. This toolkit will be distributed through the Lynch Syndrome Screening Network (LSSN; www.lynchscreening.net) and through other channels that may be identified.
Innovation
Through in-depth assessment of contextual conditions impacting LS screening implementation across an unprecedented number of sites representing diverse healthcare systems, geographies, and patient populations served, this study will provide significant information for the Precision Prevention and Early Detection Working Group of the Blue Ribbon Panel to successfully address the Cancer Moonshot [9], and for the working group of National Academies of Sciences, Engineering, and Medicine (NASEM) to successfully facilitate their goal of broadly implementing LS screening [58]. This study will also contribute to implementation science more generally, as it combines multiple methods to extend beyond the typical “lessons learned” approach and utilizes a relatively new analytic approach to illuminate minimally necessary and sufficient conditions for LS screening implementation in different organizational contexts. Another innovation of this study is our ability to provider tailored information to each site by combining key stakeholder information with business case decision analytic models that can be populated with local, real-world data. The relevance of general societal cost to organizational decision-making has been questioned [59, 60] and prior studies indicate that organization-specific costs to screen and to detect LS cases for different protocols is critical information for health systems to make decisions about LS screening implementation [28–30].
To our knowledge, no studies have synthesized in-depth cross-site comparison of context, barriers and facilitators with local business case analyses into a comprehensive toolkit for organizations implementing and optimizing LS screening programs. Most studies to date have focused on strategies for initial implementation, rather than maintenance and improvement in the face of organizational context or changes in evidence. This study will fill that gap and provide insights into successful implementation in the era of precision medicine where evidence is evolving and costs and opportunities for genomic testing are rapidly changing.
Limitations and challenges
Healthcare systems and evidence are dynamic; therefore, it is possible that any of the participating sites may implement LS screening during the course of the study, or that sites currently implementing LS screening may optimize their programs prior to receiving the toolkit. This possibility may decrease the diversity of our sites in contextual or implementation conditions or outcomes; thus, changes will be tracked to ensure the integrity of the analyses. Additionally, new evidence [61] and declining costs of sequencing may lead to the replacement of currently recommended LS screening protocols with tumor sequencing or even germline sequencing of all newly diagnosed cancer patients. Because economic evaluation is an integral part of this study and direct sequencing cost comparison will be included in the organizational decision-making guide, this changing evidence can easily be re-evaluated by each organization’s decision-makers over time. Because this study is specifically designed to collect cost and contextual information throughout the course of the study, we will utilize changes made at any site over the course of the study as data to inform implementation, maintenance, and adaptation broadly. Therefore, this study will not only result in a toolkit to guide LS screening implementation in an era of dynamic and changing evidence and costs, but may also broadly demonstrate for the field of implementation science methods for conducting research in the real-world that are focused on optimization, maintenance, and adaptation to new evidence.
Summary and impact
This study addresses a major unmet need identified by the Blue Ribbon Panel to achieve the goals of the Cancer Moonshot and to improve our understanding of clinical implementation of complex interventions [9]. The overarching goal of this study is to determine factors associated with variations in LS screening implementation across multiple healthcare systems and to provide the organizational decision tools, or “recipes,” for implementation success in light of different organizational contexts and costs. The organizational toolkit that we produce will help health systems to maintain and optimize LS screening programs in the face of changing costs and evidence. Our toolkit based on implementation science may be adapted to help in implementing evidence-based screening for other genomic conditions or for implementing other types of complex genomics interventions into healthcare systems.
Additional Files
Acknowledgements
The collaborations of healthcare systems participating in this study are possible through the Healthcare System Research Network (HCSRN) and the Cancer Research Network (CRN). Geisinger, KPNW, KPCO, MPCI, PAMFRI, Harvard Pilgrim Health Care, and HealthPartners are all members of the HCSRN and the CRN. CHI is a previous member of the HCSRN. RS is a CRN Scholar. Additional guidance on study activities and dissemination is provided in collaboration with the Lynch Syndrome Screening Network (LSSN).
Funding
This project and publication are supported by the National Cancer Institute (NCI) twenty-first Century Cures Act - Beau Biden Cancer Moonshot R01CA211723. The content of this article is the sole responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Availability of data and materials
Not applicable.
Abbreviations
- CDC
Centers for Disease Control
- CFIR
Consolidated Framework for Implementation Research
- CHI
Catholic Health Initiatives
- CNA
Coincidence Analysis
- CRC
Colorectal cancer
- EGAPP
Evaluation of Genetic Application in Practice and Prevention
- KP
Kaiser Permanente
- KPCO
Kaiser Permanente Colorado
- KPNW
Kaiser Permanente Northwest
- LS
Lynch syndrome
- MPCI
Myers Primary Care Institute
- NASEM
National Academies of Science, Engineering, Medicine
- NCCN
National Comprehensive Cancer Network
- PAMFRI
Palo Alto Medical Research Foundation Institute
- QCA
Qualitative Comparative Analysis
Authors’ contributions
AKR is the project PI and leads the research team and developed the initial manuscript. JEH, ABH, MME, CL, JL, PP, RS, SYL are site PI for the participating healthcare systems and collaborated on the development of the original research grant and study protocol. DC wrote the CNA protocol and analysis section in collaboration with AKR and substantially edited the manuscript. JG developed the original economic models with MSW and collaborates on the model development with JH and SS. RG is a clinical partner and is involved in study design and manuscript review. IL is the project manager and edited the manuscript for clarity and consistency and references. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This project is approved by the Geisinger IRB (2017–0238), and the Sutter Health IRB (for Palo Alto Medical Foundation Research Institute) (2017.134EXP). The following sites have elected to cede IRB oversight to Geisinger IRB, which has been accepted and approved: KPNW, KPCO, CHI, HealthPartners, Meyers Primary Care Institute, and Harvard Pilgrim Health Care.
Patients and organizational stakeholders will be provided a study information sheet upon invitation to participate. Consent to participate in the interview will be obtained verbally after interviewer reviews the elements of consent and prior to asking any interview questions.
Consent for publication
Not applicable.
Competing interests
All authors declare they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Alanna Kulchak Rahm, Phone: 570-214-5093, Email: akrahm@geisinger.edu.
Deborah Cragun, Email: dcragun@health.usf.edu.
Jessica Ezzell Hunter, Email: Jessica.E.Hunter@kpchr.org.
Mara M. Epstein, Email: mara.epstein@umassmed.edu
Jan Lowery, Email: Jan.Lowery@ucdenver.edu.
Christine Y. Lu, Email: christine_lu@harvardpilgrim.org
Pamala A. Pawloski, Email: Pamala.a.pawloski@healthpartners.com
Ravi N. Sharaf, Email: ravisharaf@gmail.com
Su-Ying Liang, Email: liangs@pamfri.org.
Andrea N. Burnett-Hartman, Email: Andrea.N.Burnett-Hartman@kp.org
James M. Gudgeon, Email: jim.gudgeon@imail.org
Jing Hao, Email: jhao@geisinger.edu.
Susan Snyder, Email: srsnyder1@geisinger.edu.
Radhika Gogoi, Email: rpgogoi@geisinger.edu.
Ilene Ladd, Email: igladd@geisinger.edu.
Marc S. Williams, Email: mswilliams1@geisinger.edu
References
- 1.Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–795. doi: 10.1056/NEJMp1500523. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Palomaki GE, McClain MR, Melillo S, Hampel HL, Thibodeau SN. EGAPP supplementary evidence review: DNA testing strategies aimed at reducing morbidity and mortality from lynch syndrome. Gen Med. 2009;11(1):42–65. doi: 10.1097/GIM.0b013e31818fa2db. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Healthy People 2020 2014 [1/22/2016]. Available from: https://www.healthypeople.gov/2020/topics-objectives/topic/genomics/objectives.
- 4.Giardiello FM, Allen JI, Axilbund JE, Boland CR, Burke CA, Burt RW, et al. Guidelines on genetic evaluation and Management of Lynch Syndrome: a consensus statement by the US multi-society task force on colorectal Cancer. Am J Gastroenterol. 2014;109(8):1159–1179. doi: 10.1038/ajg.2014.186. [DOI] [PubMed] [Google Scholar]
- 5.Lynch HT, Snyder CL, Shaw TG, Heinen CD, Hitchins MP. Milestones of lynch syndrome: 1895-2015. Nat Rev Cancer. 2015;15(3):181–194. doi: 10.1038/nrc3878. [DOI] [PubMed] [Google Scholar]
- 6.Le DT, Uram JN, Wang H, Bartlett BR, Kemberling H, Eyring AD, et al. PD-1 blockade in tumors with mismatch-repair deficiency. N Engl J Med. 2015;372(26):2509–2520. doi: 10.1056/NEJMoa1500596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.McCann Georgia A., Eisenhauer Eric L. Hereditary cancer syndromes with high risk of endometrial and ovarian cancer: Surgical options for personalized care. Journal of Surgical Oncology. 2014;111(1):118–124. doi: 10.1002/jso.23743. [DOI] [PubMed] [Google Scholar]
- 8.Hampel H, de la Chapelle A. The search for unaffected individuals with lynch syndrome: do the ends justify the means? Cancer Prev Res. 2011;4(1):1–5. doi: 10.1158/1940-6207.CAPR-10-0345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cancer Moonshot Blue Ribbon Panel Report 2016. National Cancer Institute: 2016 10/17/2016. Report No.
- 10.Stoffel EM, Mangu PB, Gruber SB, Hamilton SR, Kalady MF, Lau MW, et al. Hereditary colorectal cancer syndromes: American Society of Clinical Oncology clinical practice guideline endorsement of the familial risk-colorectal cancer: European Society for Medical Oncology clinical practice guidelines. J Clin Oncol. 2015;33(2):209–217. doi: 10.1200/JCO.2014.58.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.CDC. Genetic Testing: Genomic Tests and Family Health History by Levels of Evidence 2015 [updated 8/18/20151/22/2016]. Available from: https://phgkb.cdc.gov/PHGKB/topicFinder.action?Mysubmit=init&query=tier+1.
- 12.Balmana J, Balaguer F, Cervantes A, Arnold D. Familial risk-colorectal cancer: ESMO Clinical Practice Guidelines. Ann Oncol. 2013;24(Suppl 6):vi73–vi80. doi: 10.1093/annonc/mdt209. [DOI] [PubMed] [Google Scholar]
- 13.Syngal S, Brand RE, Church JM, Giardiello FM, Hampel HL, Burt RW. ACG clinical guideline: genetic testing and management of hereditary gastrointestinal cancer syndromes. Am J Gastroenterol. 2015;110(2):223–262. doi: 10.1038/ajg.2014.435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Weissman SM, Burt R, Church J, Erdman S, Hampel H, Holter S, et al. Identification of individuals at risk for lynch syndrome using targeted evaluations and genetic testing: National Society of genetic counselors and the collaborative Group of the Americas on inherited colorectal Cancer joint practice guideline. J Genet Couns. 2012;21(4):484–493. doi: 10.1007/s10897-011-9465-7. [DOI] [PubMed] [Google Scholar]
- 15.Network NCC. NCCN Clinical Practice Guidelines in Oncology: Genetic/Familial High-Risk Assessment: Colorectal 2016 [updated 9/26/20165/1/2017]. Available from: nccn.org.
- 16.Rubenstein JH, Enns R, Heidelbaugh J, Barkun A. American Gastroenterological Association Institute Guideline on the Diagnosis and Management of Lynch Syndrome. Gastroenterology. 2015;149(3):777–782. doi: 10.1053/j.gastro.2015.07.036. [DOI] [PubMed] [Google Scholar]
- 17.Khoury MJ, Coates RJ, Fennell ML, Glasgow RE, Scheuner MT, Schully SD, et al. Multilevel research and the challenges of implementing genomic medicine. J Natl Cancer Inst Monogr. 2012;2012(44):112–120. doi: 10.1093/jncimonographs/lgs003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bellcross CA, Bedrosian SR, Daniels E, Duquette D, Hampel H, Jasperson K, et al. Implementing screening for lynch syndrome among patients with newly diagnosed colorectal cancer: summary of a public health/clinical collaborative meeting. Gen Med. 2012;14(1):152–162. doi: 10.1038/gim.0b013e31823375ea. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cragun D, DeBate RD, Vadaparampil ST, Baldwin J, Hampel H, Pal T. Comparing universal lynch syndrome tumor-screening programs to evaluate associations between implementation strategies and patient follow-through. Gen Med. 2014;16(10):773–782. doi: 10.1038/gim.2014.31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ladabaum U, Wang G, Terdiman J, Blanco A, Kuppermann M, Boland CR, et al. Strategies to identify the lynch syndrome among patients with colorectal cancer: a cost-effectiveness analysis. Ann Intern Med. 2011;155(2):69–79. doi: 10.7326/0003-4819-155-2-201107190-00002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Mvundura M, Grosse SD, Hampel H, Palomaki GE. The cost-effectiveness of genetic testing strategies for lynch syndrome among newly diagnosed patients with colorectal cancer. Gen Med. 2010;12(2):93–104. doi: 10.1097/GIM.0b013e3181cd666c. [DOI] [PubMed] [Google Scholar]
- 22.Khoury MJ, Feero WG, Valdez R. Family history and personal genomics as tools for improving health in an era of evidence-based medicine. Am J Prev Med. 2010;39(2):184–188. doi: 10.1016/j.amepre.2010.03.019. [DOI] [PubMed] [Google Scholar]
- 23.Recommendations from the EGAPP Working Group Genetic testing strategies in newly diagnosed individuals with colorectal cancer aimed at reducing morbidity and mortality from Lynch syndrome in relatives. Gen Med. 2009;11(1):35–41. doi: 10.1097/GIM.0b013e31818fa2ff. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wade JE, Ledbetter DH, Williams MS. Implementation of genomic medicine in a health care delivery system: a value proposition? American journal of medical genetics part C. Sem Med Gen. 2014;166c(1):112–116. doi: 10.1002/ajmg.c.31392. [DOI] [PubMed] [Google Scholar]
- 25.Williams MS. Perspectives on what is needed to implement genomic medicine. Mol Gen Genom Med. 2015;3(3):155–159. doi: 10.1002/mgg3.135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mittman B. Implementation science in health care. In: Brownson RCCG, Proctor EK, editors. Dissemination and implementation research in health. New York: Oxford University Press; 2012. pp. 400–418. [Google Scholar]
- 27.Snyder SR, Mitropoulou C, Patrinos GP, Williams MS. Economic evaluation of pharmacogenomics: a value-based approach to pragmatic decision making in the face of complexity. Public Health Genom. 2014;17(5–6):256–264. doi: 10.1159/000366177. [DOI] [PubMed] [Google Scholar]
- 28.Williams M. Delivery of personalized medicine in an integrated healthcare system. In: Ginsburg GS, Willard HF, editors. Genomic and personalized medicine. 2. New York: Elsevier Inc; 2013. pp. 340–352. [Google Scholar]
- 29.Gudgeon JM, Belnap TW, Williams JL, Williams MS. Impact of age cutoffs on a lynch syndrome screening program. J Oncol Pract. 2013;9(4):175–179. doi: 10.1200/JOP.2012.000573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Gudgeon JM, Williams JL, Burt RW, Samowitz WS, Snow GL, Williams MS. Lynch syndrome screening implementation: business analysis by a healthcare system. Am J Manag Care. 2011;17(8):e288–e300. [PubMed] [Google Scholar]
- 31.Heald B, Plesec T, Liu X, Pai R, Patil D, Moline J, et al. Implementation of universal microsatellite instability and immunohistochemistry screening for diagnosing lynch syndrome in a large Academic Medical Center. J Clin Oncol. 2013;31(10):1336–1340. doi: 10.1200/JCO.2012.45.1674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Cross DS, Rahm AK, Kauffman TL, Webster J, Le AQ, Spencer Feigelson H, et al. Underutilization of lynch syndrome screening in a multisite study of patients with colorectal cancer. Gen Med. 2013;15(12):933–940. doi: 10.1038/gim.2013.43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Daly Corinne, Rotenberg Carolyn, Facey Marcia, Baker Natalie A., Baxter Nancy N. Reflex Lynch syndrome screening by example: A review of existing programs. Journal of Clinical Oncology. 2015;33(3_suppl):543–543. doi: 10.1200/jco.2015.33.3_suppl.543. [DOI] [Google Scholar]
- 34.Cohen SA. Current lynch syndrome tumor screening practices: a survey of genetic counselors. J Genet Couns. 2014;23(1):38–47. doi: 10.1007/s10897-013-9603-5. [DOI] [PubMed] [Google Scholar]
- 35.Beamer LC, Grant ML, Espenschied CR, Blazer KR, Hampel HL, Weitzel JN, et al. Reflex immunohistochemistry and microsatellite instability testing of colorectal tumors for lynch syndrome among US cancer programs and follow-up of abnormal results. J Clin Oncol. 2012;30(10):1058–1063. doi: 10.1200/JCO.2011.38.4719. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR) Implement Sci. 2013;8:51. doi: 10.1186/1748-5908-8-51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Cragun Deborah, Pal Tuya, Vadaparampil Susan T., Baldwin Julie, Hampel Heather, DeBate Rita D. Qualitative Comparative Analysis. Journal of Mixed Methods Research. 2015;10(3):251–272. doi: 10.1177/1558689815572023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Thiem A. Conducting configurational comparative research with qualitative comparative analysis: a hands-on tutorial for applied evaluation scholars and practitioners. Am J Eval. 2017;38(3):420–433. doi: 10.1177/1098214016673902. [DOI] [Google Scholar]
- 40.Rahm AK, Boggs JM, Martin C, Price DW, Beck A, Backer TE, et al. Facilitators and barriers to implementing SBIRT in primary Care in Integrated Health Care Settings. Subst Abus. 2014:0. [DOI] [PubMed]
- 41.Schneider JL, Davis J, Kauffman TL, Reiss JA, McGinley C, Arnold K, et al. Stakeholder perspectives on implementing a universal lynch syndrome screening program: a qualitative study of early barriers and facilitators. Gen Med. 2015. [DOI] [PMC free article] [PubMed]
- 42.Harris JN, Liljestrand P, Alexander GL, Goddard KA, Kauffman T, Kolevska T, et al. Oncologists' attitudes toward KRAS testing: a multisite study. Cancer Med. 2013;2(6):881–888. doi: 10.1002/cam4.135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Hamilton AB, Oishi S, Yano EM, Gammage CE, Marshall NJ, Scheuner MT. Factors influencing organizational adoption and implementation of clinical genetic services. Gen Med. 2014;16(3):238–245. doi: 10.1038/gim.2013.101. [DOI] [PubMed] [Google Scholar]
- 44.Hunter JE, Zepp JM, Gilmore MJ, Davis JV, Esterberg EJ, Muessig KR, et al. Universal tumor screening for lynch syndrome: Assessment of the perspectives of patients with colorectal cancer regarding benefits and barriers. Cancer. 2015:n/a-n/a. [DOI] [PMC free article] [PubMed]
- 45.Beebe J. Rapid assessment process: an introduction. Walnut Creek: AltaMira Press; 2001. [Google Scholar]
- 46.Padgett DK. Qualitative and mixed methods in public health. Thousand Oaks, CA: Sage; 2012. [Google Scholar]
- 47.CFIR Technical Assistance Website Ann Arbor, MI: CFIR Research Team [1/30/2016]. Available from: http://cfirguide.org/.
- 48.Devers KJL, Nicole Cafarella; Burton, Rachel A; Kahwati, Leila; McCall, Nancy; Zuckerman, Stephen. Using Qualitative Comparative Analysis (QCA) to Study Patient-Centered Medical Homes2013 1/22/2016. Available from: https://innovation.cms.gov/Files/reports/QCA-Report.pdf.
- 49.Berg-Schlosser DDM, Rihoux G, Ragin B, Charles C. Qualitative comparative analysis (QCA) as an approach. In: BRCC R, editor. Configurational comparative methods: qualitative comparative analysis (QCA) and related techniques. Thousand Oaks: Sage Publications, Inc; 2009. [Google Scholar]
- 50.Kahwati LC, Lewis MA, Kane H, Williams PA, Nerz P, Jones KR, et al. Best practices in the veterans health Administration's MOVE! Weight management program. Am J Prev Med. 2011;41(5):457–464. doi: 10.1016/j.amepre.2011.06.047. [DOI] [PubMed] [Google Scholar]
- 51.Baumgartner M. Detecting causal chains in small-n data. Field Methods. 2012;25(1):3–24. doi: 10.1177/1525822X12462527. [DOI] [Google Scholar]
- 52.Basurto X, Speer J. Structuring the calibration of qualitative data as sets for qualitative comparative analysis (QCA) Field Methods. 2012;24(2):155–174. doi: 10.1177/1525822X11433998. [DOI] [Google Scholar]
- 53.Schneider Carsten Q., Wagemann Claudius. Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets. Comparative Sociology. 2010;9(3):397–418. doi: 10.1163/156913210X12493538729793. [DOI] [Google Scholar]
- 54.Legewie N. Anchored Calibration: From Qualitative Data to Fuzzy Sets. 2017. 2017;18(3).
- 55.Baumgartner M, Thiem A. Identifying Complex Causal Dependencies in Configurational Data with Coincidence Analysis 2015. 176–84 p.
- 56.Grosse SD, Palomaki GE, Mvundura M, Hampel H. The cost-effectiveness of routine testing for lynch syndrome in newly diagnosed patients with colorectal cancer in the United States: corrected estimates. Gen Med. 2015;17(6):510–511. doi: 10.1038/gim.2015.53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Rahm A, Hawkins R, Dearing J, Pingree S, Lomax J, McDowell H, et al. Implementing an evidence-based breast cancer support and communication tool to newly diagnosed patients as standard care in two institutions. Translat Behav Med. 2015;5(2):198–206. doi: 10.1007/s13142-015-0305-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.National Academies of Sciences E, and Medicine . Applying an implementation science approach to genomic medicine: Workshop Summary. Washington, DC: The National Academies Press; 2016. [PubMed] [Google Scholar]
- 59.Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Urdahl H, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess. 2004;8(49):iii–iiv. doi: 10.3310/hta8490. [DOI] [PubMed] [Google Scholar]
- 60.Schlander M. The use of cost-effectiveness by the National Institute for health and clinical excellence (NICE): no(t yet an) exemplar of a deliberative process. J Med Ethics. 2008;34(7):534–539. doi: 10.1136/jme.2007.021683. [DOI] [PubMed] [Google Scholar]
- 61.Hampel H, Pearlman R, Beightol M, et al. Assessment of tumor sequencing as a replacement for lynch syndrome screening and current molecular tests for patients with colorectal cancer. JAMA Oncol. 2018. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Not applicable.