Clinician perceptions of Passport for Care, a web-based clinical decision support tool for survivorship care plan delivery

Abstract

Background:

The Children’s Oncology Group Long-Term Follow-Up Guidelines provide exposure-based risks and recommendations for late effects screening of survivors of childhood cancer. Passport for Care (PFC) is a web-based clinical decision support tool for generating a personalized survivorship care plan (SCP) derived from the Guidelines and user-entered exposures. We assessed PFC clinician user practices and perceptions of PFC impact on clinic workflow, Guidelines application, and survivor shared decision-making.

Procedure:

A 35-item REDCap™ survey was emailed to all PFC users (n=936) in 146 current and former PFC user clinics. Anonymous responses were permitted. Results were summarized and compared with a 2012 survey.

Results:

Data were available from 148 respondents representing 64 out of 146 PFC user clinics (minimum clinic response rate 44% excluding 49 anonymous responses). Generation of a personalized SCP was the most common application of PFC, followed by determination of surveillance recommendations and use as a survivor database. Twenty-five respondents (17%) felt data entry was a significant or insurmountable barrier to PFC application. Sixty-nine percent of respondents attributed PFC with a very high/high impact on Guidelines adherence in their clinical practice, compared with 40% that attributed PFC with having a significant impact on adherence in 2012 (p<0.001).

Conclusion:

The survey results provide valuable insights on patterns of SCP delivery and Survivor Clinic workflow. User-perceived benefits to PFC included facilitating clinician ability to follow Guidelines recommendations in clinical practice. Importantly, some barriers to resource utilization were also identified, suggesting a need for user-informed adaptations to further improve uptake.

Introduction

An effective transition from cancer treatment to survivorship is critical to the long-term physical and mental health of survivors.¹ Survivorship care plans (SCPs) facilitate this transition, summarizing cancer treatment, risk for late effects, and related recommendations for late effects surveillance, thereby serving as a roadmap for long-term follow-up care. While there is limited evidence that SCPs have directly improved health outcomes in survivors, among randomized studies that employ an SCP intervention, survivors in the intervention arm report greater adherence to medical recommendations, and greater satisfaction with care and the amount of information received.^2,3 Survivors of cancer in childhood and adolescence are at exceptionally high risk for suboptimal transition to survivorship care, due to 1) the complexity of diagnosis and treatment while under the guardianship of an adult who is responsible for medical decision-making, and 2) the transition from pediatric to adult-based care that often requires a change in medical providers and/or treating institutions. For survivors of childhood cancer in particular, an individualized SCP is essential to guide long-term follow-up care through the transition to adult-based care, as well as to promote lifelong healthy behaviors and efforts toward risk reduction.⁹

Passport for Care (PFC) was launched in 2007 to facilitate delivery of the SCP to survivors of cancer in childhood and adolescence. Leveraging a partnership with the Children’s Oncology Group (COG) Long-Term Follow-Up Guidelines (www.survivorshipguidelines.org),⁴ PFC was originally developed as a clinical decision support system (CDSS) for healthcare professionals that generates a personally-tailored SCP from clinician user-entered diagnosis and treatment data.⁵ Passport for Care algorithms derive the SCP from the most current version of the Guidelines, overcoming many barriers to Guidelines dissemination and implementation, and ensuring that PFC-generated SCP late effects risk and recommendations are accurate, current, and evidence-based. Passport for Care also serves as a HIPAA-compliant survivor database and a means for tracking and entering late effects of cancer treatment. Once the SCP has been generated in PFC it remains permanently affiliated with the patient, eliminating the need for future medical record abstraction and referral to the Guidelines during subsequent visits. When given access to the PFC survivor website (https://cancersurvivor.passportforcare.org), survivors and their families are able to view their personalized SCP and share it with their primary or subspecialty medical provider. This function supports survivor care coordination, a key ongoing gap in survivorship care delivery in the United States.¹⁰

The number of survivors with data in PFC is growing at an average rate of 4,000 survivors per year, now over 50,000 worldwide (Figure 1). In 2015, a separate PFC website was created to provide survivors (or their legal guardians if <18 years old) with secure access to their personal SCP.⁶ To assess clinician-user practices in Long Term Survivor Clinics, and the impact of PFC on clinic workflow, Guidelines application, and clinician-patient interactions, a 17-item survey was conducted in 2012 (results reported previously).⁷ Subsequent to this survey, a major application revision was completed in 2020 that included a graphical redesign, improved system security, and modernized website infrastructure. In addition, substantial changes were made to streamline the workflow, such as expanded drop-down menus for data entry and a patient report feature providing search and filter options by demographic features, disease subtypes, and specific exposures. We conducted a second follow-up survey in 2021 to assess clinician PFC usage patterns and to obtain feedback on the revised platform. In addition, the 2021 survey assessed clinician-user perceptions of PFC’s impact on survivorship care, and identified opportunities to further improve this resource. Here, we report results from this survey of former and current PFC user sites, comparing results to that of the 2012 survey.

FIGURE 1:

Passport for Care Enrollment Over Time

Methods

A 35-item REDCap™ survey was developed by the study team, including 17 items from the previous survey. Each PFC user clinic determines the number of site-specific user accounts, with a minimum of one user per site but without any specified limit. There are 936 individual clinician user accounts across all PFC user sites (average 6.6 per site, median 4, minimum 1, maximum 39). This survey was emailed to all 936 PFC user accounts, representing 146 clinics in the United States (n=135) and internationally (n=11). Clinics were encouraged to select a representative respondent from their site, but multiple responses from the same site were also permitted. Therefore, the response rate for the survey was determined by the number of clinics (assessed by identifiable respondents) that submitted a survey response out of the 146 total clinics. The sites that were surveyed included those that were both active and less active at the time of the survey, and a member from each of the 146 sites had logged in to PFC at least once. ‘Active’ sites were defined as those that had enrolled survivors to PFC within six months prior to the survey release date. Sites with 5 or fewer survivors in the database were reviewed and included as ‘Active’ only if they were sites that began using PFC during this six month time frame. The remaining sites were considered ‘Less Active/Inactive.’ Recipients of the emailed survey link were invited to respond as a representative for their clinic, but more than one respondent per clinic was also permitted. Two reminders were sent during the 30 day period that the survey was open. Respondents were invited to provide their email address at the survey end to facilitate remittance of a nominal gift card. To minimize response bias, respondents could also elect to remain anonymous. Once complete, the survey results were described using summary statistics, and compared with prior survey results using a Chi square test. This study was performed in accord with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Committee of Baylor College of Medicine (Approval period: 3/25/21–2/22/26, H-14099). This research involved minimal risk, and therefore met criteria for a waiver of requirement for written documentation of informed consent.

Results

Of the 146 clinics with a PFC agreement that were sent this survey, 106 clinics had enrolled survivors to PFC within the previous 6 months (meeting criteria for ‘active’). Of the remaining 40 clinics, 10 had enrolled survivors to PFC between 6 months and one year from the survey date, 26 had not enrolled any survivors for over one year, and 4 had logged-in to PFC but had never entered any survivor data. These 40 clinics were considered ‘Less Active/Inactive.’ Across all sites surveyed, active PFC Clinics entered data from a mean of 397.4 survivors in their PFC database (median 267.5), vs. Less Active/Inactive PFC Clinics, which had entered data from a mean of 48.5 survivors in their PFC database (median 19.5). A comparison of survivor distribution between Active and Less Active/Inactive PFC Clinics is shown in Figure 2. The total number of respondents was 148 out of the 936 approached: 104 (70%) provided an email address permitting identification of the 64 clinics that they represented (response rate >44%). There were 97 identifiable respondents from 58 Active clinics, and 7 identifiable respondents from 6 Less Active/Inactive clinics. There were an average of 1.6 respondents per clinic (range 1–4). The response rate from Active clinics far exceeded that of Less Active/Inactive clinics (54% vs. 16%, c²₍₁₎ = 16.78, p<0.001). Survey respondent characteristics are shown in Table 1.

FIGURE 2:

Passport for Care survivor enrollment by site designation as ‘Active’ or ‘Less Active/Inactive’

TABLE 1:

Survey Respondent Characteristics by Clinic Passport for Care Active/Inactive Status

	Total (N=148)	Active (N=97)	Less Active/Inactive (N=7)	Unknown (N=44)
Unique clinics represented	64	58	6	(unknown)
Respondent role, n (%)
Physician	46 (31)	29 (30)	4 (57)	13 (30)
Advanced Practice Provider	42 (28)	29 (30)	2 (29)	11 (25)
Nurse or Nurse Coordinator	48 (33)	31 (32)	1 (14)	16 (36)
Other	12 (8)	8 (8)	0	4 (9)
Respondent’s outpatient clinical time allocation (majority), n (%)
Childhood cancer survivorship	57 (39)	43 (44)	1 (14)	13 (30)
General pediatric hematology/oncology	57 (39)	34 (35)	5 (71)	18 (41)
Disease-based pediatric oncology (e.g., CNS tumor, leukemia)	26 (18)	14 (14)	1 (14)	11 (25)
Other	8 (5)	6 (6)	0	2 (5)
Respondent’s practice location includes a Long-Term Survivor (LTS) Clinic, n (%)
Present	127 (86)	85 (88)	5 (71)	37 (84)
Absent	21 (14)	12 (12)	2 (29)	7 (16)
For practice with an LTS Clinic, age range of survivors eligible to be seen, n (%)
Up to age 18–21 years	18 (12)	11 (11)	1 (14)	6 (14)
Up to age 39 years	34 (23)	21 (22)	1 (14)	12 (27)
No age limit	58 (39)	36 (37)	3 (43)	19 (43)
Other	34 (23)	26 (27)	2 (29)	6 (14)
Unsure	4 (3)	3 (3)	0	1 (2)

Clinic usage patterns and workflow integration

The most common application of PFC was to generate an SCP (n=131, 89%). Other common usages included determining tests needed for surveillance (n=105), as a method for accessing relevant Health Links (n=87), and as a database for survivors (n=83). Fifty-four (36%) used PFC at every survivor visit, 7 (5%) used it only at the first survivor visit to generate the SCP, and 64 (43%) used PFC at the first visit and then only as needed, such as when there is a revision to the Guidelines. A minority (n=3) utilized PFC only when it became time for the survivor to transition out of the program. Thirty-seven respondents (25%) used PFC in advance of, during, and after the survivor clinic visit, 19 (13%) used it before and during the survivor visit, and the remainder used it only prior to [40 (27%)] or prior to and following [11 (7%)] the survivor visit. The remaining 28% used PFC during the visit, after the visit, or both. Forty-eight percent of respondents (n=71) provided survivors with access to their personalized SCP by providing them with an access code to the PFC Survivor Website. Four percent of respondents, n=6, did not answer this question. The spectrum of how PFC functionality is applied by clinicians across all user sites in shown in Figure 3A.

FIGURE 3:

Passport for Care Clinic Workflow Integration

3A: Respondent estimates of how Passport for Care is utilized

3B: Estimated percentage of survivors seen in the respondent’s clinic and enrolled to Passport for Care, comparing survey results from 2012 and 2021.

*Health Links are late effects information written in lay language, developed by members of the Children’s Oncology Group Nursing Discipline.

In order to generate the personalized SCP, users must enter in PFC the baseline demographics, diagnosis, and treatment exposures. A survivor with a history of uncomplicated acute lymphoblastic leukemia was selected as an illustrative example of a case commonly seen in a Long-Term Survivor Clinic. In preparation for this type of patient visit, nearly half of respondents (43%, n=64) estimated more than 30 minutes would be required for both medical record abstraction and data entry, 23% estimated 21–30 minutes, 14% respondents estimated 11–20 minutes, 3% estimated 10 minutes or less, and 11% were not sure. Seven percent of respondents, n=10, did not answer this question. The timing of when clinicians abstracted and entered patient data in PFC varied substantially: 18 (12%) performed this task within the first year off therapy, 25 (17%) between one year off therapy and the first survivorship visit, 69 (47%) immediately prior to the scheduled survivorship visit, 4 (3%) immediately following the survivorship visit, and for 17 (11%) the timing was variable, with 6 (4%) not sure. Six percent of respondents, n=9, did not answer this question. PFC data entry was completed by the physician or advanced practice provider (APP) alone [31 (21%)], a nurse or nurse coordinator [43 (29%)], other clinic personnel [10 (7%)], or a combination of the physician or APP, nursing professional and/or clinic personnel as a shared responsibility [51 (34%)]. Nine percent of respondents, n=13, did not answer this question. Respondents in clinics with a physician or APP as solely responsible for PFC data entry were more likely to report data entry as a significant or insurmountable barrier to using PFC, compared with respondents in clinics that utilized other members of the team for this task [odds ratio (OR) 5.5, confidence interval (CI) 1.9, 15.4], p=0.002].

Figure 3B indicates the pattern of PFC usage patterns across respondent clinics, noting that while the majority of respondent clinics elect to enter all survivor data in PFC, many respondent clinics only enter data for a select proportion of survivors. Comparison of this pattern with survey results from 2012 suggests little change over time, indicating a relatively stable integration of PFC in the clinic workflow. Just over 40% of clinician respondents indicated enrolling all survivors in the PFC, whereas 30% enroll about 75% of survivors seen in their clinic. Respondents that enrolled fewer than 50% of survivors in PFC noted an intentional selection of more complex survivors for PFC data entry, as well as a lack of time and resources for data entry as a barrier to enrolling additional patients.

User satisfaction and perceived impact on survivorship care delivery

Clinician users reported a high level of satisfaction with the PFC CDSS, and that the availability of this service had a high impact on their ability to provide comprehensive survivorship care services that adhered to the current Guidelines. Seventy-two percent of respondents (n=106) felt that PFC has a ‘high’ impact on their ability to provide survivors with comprehensive information regarding their risk for late effects. Six percent of respondents, n=9, did not answer this question. Depicted in Figure 4 are additional comparisons between the 2012 and 2021 survey data on perceptions of PFC overall satisfaction, impact on practice adherence to the COG Guidelines, and impact on fostering conversations regarding the importance of long-term follow-up care. The percentage of respondents that were ‘very satisfied’ declined from 2012 to 2021, but the number that were ‘generally satisfied’ increased, so that there was no difference between the 2012 and 2021 surveys in the overall percentage of respondents who were either very satisfied or generally satisfied with PFC (Figure 4A). There was also no difference in responses from the 2021 survey when comparing respondents from active clinics to those that responded anonymously, so that their clinic PFC activity status was unknown.

FIGURE 4:

Perceptions of Passport for Care Impact on Survivorship Care Delivery 4A: How satisfied are you with Passport for Care? (n=94 in 2012; n=148 in 2021, data shown as percentage of respondents)

4B: How has Passport for Care impacted adherence to the COG Guidelines in your practice (data shown as number of respondents)

4C: What is the impact of Passport for Care on fostering discussions with survivors regarding healthy behaviors and the importance of long-term follow-up care? (data shown as number of respondents)

User feedback and recommendations for future updates

The survey presented users with a series of proposed modifications to the PFC tool based on unsolicited user feedback recommendations received in the prior 1–2 years. Respondents were asked to rank these modifications in order of importance/interest. The most highly ranked modification was an alternative means of entering exposure data by selecting the treatment protocol rather than the more time-consuming process of selecting individual chemotherapy agents from a drop down list. This modification would produce, from a single selection, a list of intended chemotherapy exposures by the selected protocol that could then be modified as needed for any deviations from the planned regimen. The second most requested change was inclusion of a doxorubicin dose equivalency calculator for estimating cumulative anthracycline exposure. Third was a mechanism for automated abstraction of demographic and treatment data from the electronic health record into PFC. Fourth was development of a mobile health application to allow survivors to access and review their SCP on their personal device.

Discussion

Evidence-based consensus guidelines such as the COG Long-Term Follow-Up Guidelines are critical to providing high quality survivorship care to survivors of cancer diagnosed in childhood and adolescence that reduce survivor morbidity and mortality.⁸ However, challenges remain with respect to Guidelines dissemination and uptake. PFC facilitates Guidelines application through automated extraction of relevant risks based on user-entered exposures to generate the personalized SCP, as well as providing assurance that the most up-to-date version of the Guidelines has been applied.

Results from this survey suggest a high level of PFC user satisfaction and perceived impact on the delivery of quality survivorship care services that adhere to current Guidelines, perceptions that were consistently maintained over a ten year time frame. Our results provide insight as to how pediatric oncologists deliver survivorship care services across at least 64 long-term survivor clinics worldwide. The majority of respondents deliver the SCP at the initial long-term survivor visit, but thereafter only if key revisions are made to the COG Guidelines. Presumably, these clinician users rely on the SCP that was generated at the initial LTS visit to inform future survivorship care.

Importantly, our results identified opportunities for improvement in PFC uptake and utilization. For example, the PFC Survivor Website provides survivors with online access to their personalized SCP, but is currently offered in less than 50% of PFC user clinics that serve survivors and their families. A future line of investigation seeks to identify barriers to dissemination and use of this feature. A key barrier to delivery of comprehensive survivorship care is the investment of time required for the initial medical record abstraction of treatment exposures that inform identification of late effects risks and need for surveillance tests, a barrier that is unrelated to whether the clinician is utilizing PFC. However, our survey results suggest that the upfront time required for initial PFC data entry is perceived by many respondents as a barrier to implementation, especially if this responsibility falls to the physician or APP without support from other clinic personnel. Recognizing that this has been a consistent identified barrier to use of this tool over time, efforts are under way to mitigate the burden of data entry through partnerships with the electronic medical record, as well as integration of modifiable chemotherapy auto-population by treatment protocol.

Strengths of this study include our survey design, which encouraged feedback from a broad spectrum of former and current users in order to gain a better understanding of potential barriers to PFC implementation and usage. We recognize that the majority of respondents to this survey were active users of the PFC, thereby potentially introducing bias to our results. However, we note efforts to invite feedback from both Active and Less Active/Inactive user sites, and that 30% of respondents elected to preserve their anonymity, which may have mitigated some response bias. Permitting respondent anonymity may have contributed to the few number of responses that we were able to identify as from less active clinics, precluding meaningful comparisons of responses between active and less active clinics. We observed no differences in user satisfaction between respondents from Active clinics and those from clinics with unknown activity status. Aside from the limited response data from Less Active/Inactive clinics, other limitations to this study include a clinic response rate that was just under 50%, so that we also cannot exclude a contribution of selection bias to our findings.

Passport for Care is a key contributor to the accurate and comprehensive dissemination of the COG Long-Term Follow-Up Guidelines, and the only CDSS that has been developed in partnership with the Guidelines leadership team. Passport for Care is available without associated cost to clinics or hospitals worldwide that provide survivorship care services to survivors of cancer in childhood and adolescence. The number of clinics with a PFC agreement at present represents 54% of clinics listed on the COG Guidelines website as providing late effects services to survivors of childhood cancer (85 out of 158, https://cogmembers.org/public/lateeffects, data obtained on May 30, 2022). The level of uptake of the PFC CDSS is encouraging (54% of clinics listed on the COG Guidelines website as providing late effects services to survivors of childhood cancer, [i.e., 85 out of 158, https://cogmembers.org/public/lateeffects], data obtained on May 30, 2022). This level of uptake may reflect the attention to stakeholder-identified contextual and implementation issues (including workflow and training) in PFC development, as contextual and implementation-related factors are more strongly associated with CDSS uptake vs. factors related to the technological, system, or content elements.¹¹ Importantly, 50 of the 135 U.S.-based PFC clinic user sites are not listed on the COG Guidelines website as having a dedicated Long-Term Survivor Clinic, suggesting that PFC may serve an important niche, providing support to clinicians practicing in smaller clinics without a formal Survivorship Program. However, reliance on clinician-user data entry remains a widely recognized barrier to third party CDSS implementation.¹¹ Current efforts to further improve PFC uptake include reducing implementation barriers by 1) improving electronic data exchange to minimize data entry and 2) expanding educational content delivery and technological innovation, e.g. development of the PFC mobile health application, to augment survivor engagement. Future studies are needed to more broadly assess determinants of PFC uptake, such as clinic organization and staffing infrastructure, clinic survivor population characteristics, and professional beliefs/attitudes regarding delivery of survivorship services.

Abbreviations:

CDSS

Clinical Decision Support System

COG

Children’s Oncology Group

LTS

Long term survivor

PFC

Passport for Care

SCP

Survivorship Care Plan