Organizational Preparedness for Implementing an Evidence-Based Multilevel Intervention to Improve the Timeliness of Lung Cancer Treatment

Abstract

Background:

Delayed time to treatment is a key determinant of poor survival among patients with lung cancer. Factors affecting timeliness of lung cancer treatment are multilevel in nature. Most interventions target only one level of influence. Existing evidence-based interventions (EBIs) that work in one setting may fail to work in a different setting due to poor fit in the new setting or deviations from the original intervention. Our study evaluated the feasibility of adopting a Veterans Health Administration (VHA) evidence-based multilevel intervention (EBMLI) within an academic health system.

Methods:

The VHA EBMLI is a multilevel lung cancer care coordination program previously employed by the VHA targeting the individual-, provider team-, and organization- level. The intervention effectively reduced time to treatment for Veterans with lung cancer over 7 years with significant individual level reductions by about 50%.

We conducted a survey of providers across 5 sites on the acceptability, appropriateness, and feasibility towards the EBMLI using the following validated scales: 1) Acceptability of Intervention Measure (AIM, score range: 1–5); 2) Intervention Appropriateness Measure (IAM, 1–5); 3) Feasibility of Intervention Measure (FIM, 1–5); 4) Implementation Climate Scale (ICS, 0–4) and; 5) Implementation Leadership Scale (ILS, 0–4).

Results:

45 providers consented to participate in the survey and 36 (mean age 46±11 years) across 5 sites completed the survey. Most had MD or PhD degrees (46%) and were physicians (42%), had >10 years healthcare experience (49%) with 20±20 patients with lung cancer seen per week, of which 60% of providers saw only adult patients. Mean scores on the scales were: AIM (4.1±0.8), IAM (4.0±0.8), FIM (4.1±0.8), ICS (2.7±0.7), ILS (3.2±0.9).

Conclusion:

While the proposed EBMLI may be well received by providers and leadership, acknowledgement of contexts affecting organizational climate is needed to assure feasibility of the adaptation, implementation and sustainability of the VHA EBMLI.

BACKGROUND

Delayed time to treatment is a key determinant of poor survival among patients with lung cancer, especially for early stages lung cancer.^1–6 In the US, the death rate is 14% higher for patients treated after 90 days from diagnosis compared to those treated within 30 days.⁴ Yet, only 54% of ever-treated patients with lung cancer receive treatment with 30 days.⁴ Even worse, 21% of lung cancer-diagnosed patients never receive treatment.⁷ West Virginia (WV) ranks among the top five US states with the highest mortality rates for lung cancer. Twelve percent of patients with lung cancer in WV do not receive treatment,⁷ and treatment is delayed beyond 35 days for 39% of those who receive treatment.⁸ This combination of patients who do not receive timely treatment or any treatment constitute a significant number diagnosed with lung cancer in WV who are at risk for poor cancer outcomes.

Multilevel interventions target more than one level of influence concurrently or in close succession. Despite the multilevel nature of barriers to receipt of lung cancer treatment, most interventions that address cancer care delivery only target individual level factors.⁹ Evidence-based interventions (EBIs) are intervention strategies that have been shown to be effective. Existing EBIs that work in one setting may, however, fail to work in a different setting due to poor fit in the new setting or deviations from the original intervention during implementation in the new setting.¹⁰ To improve the timeliness of treatment for patients diagnosed with lung cancer within the West Virginia University (WVU) health system, we are adapting an evidence-based multilevel lung cancer care coordination intervention previously employed by the Veterans Health Administration (VHA) targeting the individual-, provider team-, and organization- level.^11,12 The VHA multilevel intervention: 1) provided a cancer care coordinator/nurse navigator as contact person for patients (individual level) and clinicians within a thoracic tumor board (provider team level) during diagnostic and staging process to identify, track, and expedite care from suspicion to definitive treatment; 2) set up a pulmonary tumor board (provider team level); and 3) reorganized workflow logistics address patient, provider, and health system barriers to treatment, respectively (organizational level) (Table 1).^11,12 In the Connecticut VA Healthcare System, time to lung cancer treatment reduced from 117 days to 52.4 days over 7 years.¹¹

Table 1:

Lung Cancer Provider Characteristics across Five Sites within the Academic Health System

CHARACTERISTIC	N (%)
Age (mean, SD)	45.78 (11.05)
Gender
Female	20 (44.44)
Male	14 (31.11)
Prefer not to say	1 (2.22)
Missing	10 (22.22)
Race/Ethnicity
Non-Hispanic White	27 (60.00)
Non-Hispanic Asian	5 (11.11)
Others	5 (11.11)
Missing	8 (17.78)
Highest Level of Education
Associate Degree	2 (4.44)
Bachelor’s Degree	6 (13.33)
Master’s Degree	3 (6.67)
Doctorate	5 (11.11)
MD	16 (35.56)
Others	4 (8.89)
Missing	9 (20.00)
Role in Healthcare
Physician	19 (42.22)
Nurse	12 (26.67)
Physician Assistant	2 (4.44)
Pharmacist, Therapist	3 (6.67)
Missing	9 (20.00)
Specialization
Oncology	22 (48.89)
Surgery	6 (13.33)
Other	11 (24.44)
Years of Experience
0–5 years	5 (11.11)
6–10 years	10 (22.22)
11–15 years	5 (11.11)
15+ years	17 (37.78)
Missing	8 (17.78)
Setting
Hospital	17 (37.78)
Clinic	18 (40.00)
Infusion center	1 (2.22)
Missing	9 (20.00)
Patient population
Adult	27 (60.00)
Mixed Age	10 (22.22)
Missing	8 (17.78)
# Patients/week (median, IQR)	15 (8–30)

OBJECTIVE

To evaluate the feasibility, acceptability, appropriateness of implementing an evidence-based multilevel lung cancer care coordination intervention aimed at improving the timeliness of lung cancer treatment and assessing the leadership and organizational climate for implementing the intervention.

METHODS

Study Design and Setting

This was a multisite survey study of provider teams who provide care to patients with lung cancer conducted across the WVU Cancer Institute (CI) network. WVUCI is a regional network of cancer centers affiliated with WVU Medicine Health System and located across locations in West Virginia, Pennsylvania, and Maryland, with 5 sites credentialed for clinical trials with the National Cancer Institute. The study was approved by the West Virginia University Institutional Review Board.

Sample and Procedures

An electronic survey was distributed to individual providers with provider teams via REDCap where all providers involved in the diagnosis and care of patients with cancer were invited to participate in the study. REDCap is a secure web application for building and managing online surveys and databases. Providers were invited to complete the survey in an introductory email. The introductory email advised that clicking on the survey link was documentation of consent to participate in the study. A $20 Amazon gift card was provided as incentive upon completion of the survey.

Measures

Data was collected using the following validated scales: 1) Implementation Climate Scale (ICS); 2) Implementation Leadership Scale (ILS); 3) Acceptability of Intervention Measure (AIM); 4) Intervention Appropriateness Measure (IAM); 5) Feasibility of Intervention Measure (FIM). The ICS is an 18-item scale with 5-level response options (Not at all (0) – Very great extent (4)) and collected data from providers’ on the support of WVU Medicine as an organization towards the adoption of evidence-based practices in general.¹³ The ILS is a 12-item scale with 5-level response options (Not at all (0) – Very great extent (4)) and collected data on providers’ attitudes towards adopting evidence-based practices in general.¹⁴ The AIM, IAM, and FIM are each 4-item scales with 5-level response options (Completely disagree (1) – Completely agree (5)) and collected data on the acceptability, appropriateness, and feasibility of adopting the lung cancer care coordination intervention.¹⁵ The AIM, IAM, and FIM were introduced with a brief description of the proposed EBMI: We are adapting an evidence-based multilevel intervention to improve timeliness to lung cancer treatment to be used across the West Virginia University Cancer institute (WVUCI) network. Multilevel interventions address multiple levels of needs concurrently or in close succession, improving the chances of successful outcome. The planned multilevel intervention is a lung cancer care coordination program which: 1) provides a cancer care coordinator as contact person for patients and clinicians within a thoracic tumor board during the diagnostic and staging process to identify, track, and expedite care from suspicion to definitive treatment; 2) modifies the pulmonary tumor board to be consistent with evidence-based processes; and 3) reorganizes workflow logistics address patient, provider, and health system barriers to treatment, respectively.

Additional demographic data were collected on: age, gender, race/ethnicity, highest level of education, type of provider, specialization, years of experience, practice site, type of practice, patient population, and number of patients with lung cancer cared for in one month.

Analysis

Demographic data were summarized using means (with standard deviations) and frequencies (with percentages) for numeric and categorical data, respectively. Mean scores for the set of items that make up each subscale (where applicable) were computed to obtain scale scores. The mean of the subscale scores or mean scores of the individual items (where there are no subscales) were computed to obtain the total scores along with the standard deviations of the means. Internal consistency of the scales and subscales were assessed using Cronbach’s alpha. Further, intraclass correlations of the individual items within subscales and the subscales within scales were assessed. Analysis was conducted using SAS 9.4 (SAS Institute Inc, Cary, NC, USA).

RESULTS

Forty-five providers consented to participate in the survey and 37 across the 5 sites attempted the survey. Completion rates for the different sections of the survey varied from 70% (N=27) for the AIM, IAM, and FIM, which were presented at the end to 92–100% (N=34–37) for the ICS, which was presented first. Table 1 shows characteristics of lung cancer care providers across the five sites within the academic health system. Mean (SD) age was 46 (11) years. Most were female (44.4%), non-Hispanic White (60.0%), with MD degrees (35.6%) and providing care as physicians (42.2%) or nurses (26.7%) within oncology (48.9%) or surgery (13.3%) specialties in a hospital (37.8%) or clinic (40.0%) setting. The majority of providers had 15 or more years of experience (37.8%) with an adult only patient population (60.0%).

Table 2 shows the mean scores, internal consistency reliability and intraclass correlations for each subscale and total scale. Mean scores for ICS and ILS were 2.70 (0.66) and 3.16 (0.93), respectively while mean scores for AIM, IAM, and FIM were 4.06 (0.82), 4.00 (0.80), and 4.06 (0.78), respectively. Within the ICS and ILS scales, mean scores for the subscales were moderate to high, ranging from 2.68 to 3.25 except for Rewards for EBP subscale within ICS which had a mean score of 0.75.

Table 2:

Mean Scores on Scales and Sub-Scales evaluating Provider and Organizational Preparedness

SCALE AND SUBSCALE	# RESPONDENTS	MEAN (SD)	ICC	Cronbach’s α
ICS (Range=0–4)	37	2.70 (0.66)	0.11	0.77
Focus on EBP	37	3.52 (0.60)	0.98	0.99
Educational support for EBP	37	2.68 (1.09)	0.70	0.88
Recognition for EBP	37	3.21 (0.77)	0.50	0.86
Rewards for EBP	34	0.75 (1.06)	0.79	0.93
Selection of staff for EBP	36	2.68 (1.06)	0.84	0.94
Selection of staff for openness	36	3.08 (0.77)	0.83	0.94
ILS (Range=0–4)	27	3.16 (0.93)	0.84	0.96
Proactive	27	3.02 (1.04)	0.87	0.96
Knowledgeable	27	3.25 (0.88)	0.92	0.97
Supportive	27	3.17 (1.02)	0.92	0.97
Perseverant	27	3.20 (1.03)	0.95	0.98
AIM (Range=1–5)	26	4.06 (0.82	0.89	0.97
IAM (Range=1–5)	26	4.00 (0.80)	0.91	0.98
FIM (Range=1–5)	26	4.06 (0.78)	0.83	0.96

AIM=Acceptability of Intervention Measure; EBP=Evidence-based practice; FIM=Feasibility of Intervention Measure; IAM=Intervention Appropriateness Measure; ICC=Intraclass correlation; ICS=Implementation Climate Scale; ILS=Implementation Leadership Scale;

Intraclass correlations for the total scales ranged from 0.84 to 0.91 except for the ICS scale which had an ICC of 0.11I. ICC for the subscales within the ICS and ILS were higher ranging between 0.50 to 0.98 for ICS and 0.87 to 0.95 for ILS. Internal consistency reliability was high for both the total scales and subscales ranging from 0.77 to 0.98 for the total scales, 0.88 to 0.99 for the subscales within ICS, and 0.96 to 0.98 for the subscales within ILS.

DISCUSSION

This study aimed to evaluate the feasibility, acceptability, appropriateness of implementing an VHA multilevel lung cancer care coordination intervention aimed at improving the timeliness of lung care treatment and assessing the leadership and organizational climate for implementing the intervention. High scores were obtained for feasibility, acceptability, and appropriateness of the intervention while moderate and high scores were, respectively, obtained for organizational climate and leadership attitude. Strikingly, moderate scores on organizational climate were driven by low scores on the rewards subscale.

The reward subscale of the ICS elicits financial incentives, bonuses, and compensated time for providers for the use of an EBP. It has been shown in other studies to score lower than other subscales in the ICS.^13,16,17 For example, clinicians who completed the initial survey for the development of the ICS scored an average of 0.72 on the rewards subscale compared to 1.68 to 2.79 for the other subscales.¹³ Our survey was conducted within an academic health system where best practices are the standard. The observed low scores on the rewards subscale may be a reflection of the lack of expectation for extra compensation for providing the best possible care to patients. Similar patterns have been observed among nurses and teachers.^16,17 The impact of low scores on the rewards subscale was reflected in low ICC on the overall ICS scale although high ICC were obtained within the subscales. The initial development of the ICS also support aggregation of the six subscales that make up the ICS.¹³

Additionally, the academic health system serves a largely disparate population. About 38% of WV population is considered rural with a poverty rate of 17%.¹⁸ This implies a high level of constraints at the local level as well as budgetary constraints at the policy level which may not support financial incentives for providers to implement EBP. However, the high scores on the other scales support evidence for positive attitudes by both leadership and providers for implementing our proposed multilevel intervention to improve the timeliness of lung cancer treatment. Next steps will involve adapting the VHA multilevel intervention to improve fit for our academic health system.¹⁰

CONCLUSION

High scores were obtained on the acceptability, appropriateness, feasibility, and leadership scales. Moderate scores were obtained on the ICS, driven by low scores on the rewards subscale. The observed pattern is consistent with other studies. While our proposed adoption of the VHA evidence-based multilevel intervention may be well received by providers and leadership, acknowledgement of contexts affecting organizational climate is needed to assure feasibility of adaptation, implementation and sustainability of the VHA intervention.

Keypoints.

• High scores were obtained on Acceptability of Intervention Measure, Intervention Appropriateness Measure, Feasibility of Intervention Measure, Implementation Leadership Scales

• Moderate scores were obtained on the Implementation Climate Scale (ICS), which was driven by low scores on the Rewards subscale