Factors Affecting the Adoption of Electronic Data Reporting and Outcomes Among Selected Central Cancer Registries of the National Program of Cancer Registries

Abstract

PURPOSE

The CDC’s National Program of Cancer Registries has expanded the use of electronic reporting to collect more timely information on newly diagnosed cancers. The adoption, implementation, and use of electronic reporting vary significantly among central cancer registries. We identify factors affecting the adoption of electronic reporting among these registries.

METHODS

Directors and data managers of nine National Program of Cancer Registries took part in separate 1-hour telephone interviews in early 2019. Directors were asked about their registry’s key data quality goals; staffing, resources, and tools used to aid processes; their definition and self-perception of electronic reporting adoption; key helpers and challenges; and cost and sustainability implications for adoption of electronic reporting. Data managers were asked about specific data collection processes, software applications, electronic reporting adoption and self-perception, information technology infrastructure, and helpers and challenges to data collection and processing, data quality, and sustainability of approach.

RESULTS

Larger registries identified organizational capacity and technical expertise as key aides. Other help for implementing electronic reporting processes came from partnerships, funding availability, management support, legislation, and access to an interstate data exchange. Common challenges among lower adopters included lack of capacity at both registry and data source levels, insufficient staffing, and a lack of information technology or technical support. Other challenges consisted of automation and interoperability of software, volume of cases received, state political environment, and quality of data received.

CONCLUSION

Feedback from the formative evaluation yielded several useful solutions that can guide implementation of electronic reporting and help refine the technical assistance provided to registries. Our findings may help guide future process and economic evaluations of electronic reporting and identify best practices to strengthen registry operations.

INTRODUCTION

In 2017, cancer was the second leading cause of death in the United States, with more than 1.7 million new diagnoses and about 600,000 deaths.¹ Cancer surveillance provides critical information on cancer incidence and trends, which decision makers can use to monitor the disease burden, as well as develop and evaluate targeted cancer prevention and control interventions at local and national levels.²

For development, implementation, and evaluation of cancer prevention and control policies and strategies to be successful, high-quality central cancer registry data (registries) must be available.^3,4 Cancer surveillance faces many challenges: delays in data availability; incomplete data sets; difficulties in standardizing data elements across disparate data sources, including hospitals, laboratories, and physician offices; and nonelectronic reporting methods for some data sources. CDC’s National Program of Cancer Registries (NPCR) has led the advancement of electronic reporting to improve registry data collection.⁵ Electronic reporting is defined as “the collection and transfer of data from source documents by hospitals, physician offices, clinics, or pathology laboratories in a standardized, coded format that does not require manual data entry at the registry level to create an abstracted record.”⁶ Electronic pathology reporting was launched by NPCR in 2006 among 18 state registries and has since been implemented nationwide. Electronic pathology reporting has included many national and regional laboratories that report information on new cancer diagnoses to registries largely through a CDC-secured system called the Public Health Information Network Messaging System. Although some registries use a secure file transfer protocol or other methods of electronic transfer,⁵ CDC helps registries adopt electronic data reporting and processing through training, technical support, and software applications.²

To date, the experiences of registries adopting electronic reporting have not been systematically evaluated. These evaluations are needed to identify best practices to strengthen registry operations and optimize electronic reporting procedures. We identify factors that affect the adoption of electronic reporting among NPCR registries by synthesizing key helpers and challenges. Findings from this study can guide efforts to help registries implement and enhance electronic reporting.

METHODS

A mixed methods study was conducted to perform a comprehensive assessment of factors affecting adoption of electronic reporting. The study used qualitative data from key informant interviews and quantitative data from NPCR’s Program Evaluation Instrument.

Registry Selection and Categorization by Electronic Reporting Adoption

NPCR registries were purposively sampled, ensuring adequate representation of registries by the following: volume of cases; size of the registry coverage area; presence of rural areas; geographic region (South, West, Northeast, Midwest); funding sources (sources in addition to NPCR); CDC expert-predicted level of electronic reporting adoption separately for hospital, physician, and pathology laboratory reporting sources (high, medium, or low); and whether 12-month data quality standards were met. Generally, a high level of electronic reporting indicates that 80%–90% of data for a particular data source was received electronically and 30% or lower was considered low level of electronic reporting. The range in-between was classified as a medium level of electronic reporting. These categorizations were reviewed and supplemented, as needed, through the registry interviews.

CDC data quality standards include key criteria on data completeness on the basis of observed-to-expected cases, unresolved duplicate rate, percentage missing critical data elements, and percentage passing a CDC-prescribed set of standard edits.⁷ For the selection process, we used program-level data maintained by CDC to monitor all 46 funded registries, including limited available information on electronic reporting adoption from 27 registries. Our goal was to select nine registries of diverse characteristics prioritizing a balanced representation of low, medium, and high electronic reporting registries to allow us to study factors that affected adoption of electronic submissions.

Table 1 presents the characteristics of the nine registries selected. Designated registries had representation from the bottom, middle, and top one third of registries nationwide in registry volume, size of the geographic area served, and the presence of rural areas. Four participating registries were in the South, one was in the West, three in the North, and one in the Midwest of the United States. In addition to NPCR funding, seven registries were financially supported by their state and two registries were financially supported by their state and through the National Cancer Institute’s SEER program.⁸ Three registries met the 12-month standards for the cases diagnosed in 2016, whereas the other six did not. Two registries were categorized as low adopters, four as medium adopters, and three as high adopters of electronic reporting.

TABLE 1.

Characteristics of Central Cancer Registries Selected for the Study

	Volume of Cases			Size of Area			Rural Areas			Region				Funding Sources (Non-NPCR)		Electronic Reporting Adoption			Meet 12-Month Standard, 2017
Registry	L	M	H	L	M	H	L	M	H	S	W	N	M	State	SEER/State	L	M	H	Y	N
Kansas		X				X			X				X	X		X				X
New Hampshire	X			X				X				X		X			X		X
Louisiana		X			X			X		X					X			X		X
Massachusetts			X	X			X					X			X	X				X
South Carolina		X		X				X		X				X			X			X
North Carolina			X		X		X			X				X				X	X
Pennsylvania			X		X		X					X		X				X	X
Texas			X			X		X		X				X			X			X
Nevada	X					X			X		X			X			X			X
Totals	2	3	4	3	3	3	3	4	2	4	1	3	1	7	2	2	4	3	3	6

NOTE. An X indicates the specific characteristics that correspond to each registry. The variables volume of cases, size of area, and rural areas are divided into high, middle, and low categories–registries representing the top, middle, and bottom thirds of NPCR registries in this attribute, respectively. Electronic reporting adoption is based on CDC expert prediction during the registry selection process. NPCR 12-month standards include key criteria related to data quality, completeness, and timeliness.

Abbreviation: NPCR, National Program of Cancer Registries.

Quantitative Data Compilation

We derived key data elements from the NPCR’s Program Evaluation Instrument, which is generally conducted every 2 years and collects information via a web-based survey instrument on various registry operational attributes. Data from the NPCR 2017 Program Evaluation Instrument were extracted to obtain information on software used and full-time equivalent (FTE) staffing by job category, as these aspects are generally stable over time. We obtained staffing details, including information on certified tumor registrars, as these individuals have specific training to code and classify cancer cases, and computer and information technology (IT) specialists, who have the technical expertise to support electronic reporting.

Qualitative Data Collection and Analysis

The directors and data managers of nine diverse registries (by characteristics) participated in 18 separate 1-hour telephone interviews from late January through early March 2019 to gather information on data processes and electronic reporting. Separate interview guides with different sets of questions were used for directors and data managers to capture their perspectives independently (Appendix 1 and Appendix 2). The list of questions were shared with participating registries ahead of the interviews so that they had time to prepare detailed answers. Directors were asked about their registry’s key current goals regarding data quality standards, staffing, resources, and tools used by the registry to facilitate processes, their definition and self-perception of electronic reporting adoption, key helpers and challenges, and cost and sustainability implications for adoption of electronic reporting. The study team asked data managers about specific data collection processes, software applications, electronic reporting adoption, IT infrastructure, challenges and helpers to data collection and processing (including electronically reported information), data quality, and sustainability of approach. Interviewers asked all participants how they defined electronic reporting and level of electronic adoption at their registry. Two researchers kept detailed notes of each interview to ensure complete and thorough representation of the responses. Both sets of notes were compiled into a single complete version for each interview and were reviewed to identify factors that affect electronic reporting adoption. Because the number of interview notes was limited, no other software was used for the analysis. Two researchers independently reviewed the notes to ensure systematic and thorough review of the information gathered through the interviews. To protect confidentiality, we did not link findings to specific registries or staff members. We identified key reoccurring themes across the interviews and categorized the findings into three thematic areas: registry staff–related factors, software or IT factors, and external contextual factors. The study team summarized the key process modifications and outcomes of electronic reporting by using two categories: (1) data quality and (2) costs, resource requirements, and efficiency.

Comparison by Adoption of Electronic Reporting

We classify registries into two general groups for comparison by using the level of adoption across all data sources: overall higher adopters (medium or high level of electronic reporting across all three sources) and overall lower adopters (do not have at least medium on all three sources). We present descriptive statistics for the two categories to highlight potential differences related to volume of cases, funding sources, software used, staffing availability and distribution across key roles, and ability to meet registry data quality standards. In addition, we summarize factors and highlight differences related to staffing, software or IT, and external contextual issues. Finally, we report on the potential impact of adopting electronic reporting on cancer data quality and resource requirements.

RESULTS

The registries’ levels of electronic reporting adoption by data sources—hospitals, physician offices, and pathology laboratories—are presented in Table 2. All nine registries were high adopters of electronic reporting from hospital sources, as most of the hospitals (about 90%) reported data to the registries electronically, largely through a CDC Registry Plus software system called WebPlus. Registries generally had the lowest level of electronic reporting from physician offices: four registries classified as low adopters (30% or lower proportion reporting electronically), three registries categorized as medium adopters (between 30% and 80% reporting electronically), and the remaining two registries categorized as high adopters (more than 80% reporting electronically), among physician offices. Registries interviewed received a majority of physician office records in a paper-based format and through fax, e-mailed PDF, or mail, which required more timely and manual data entry. However, among the instances where registries did receive physician office data electronically, the HL7 (Health Level Seven) CDA (Clinical Data Architecture) format was used most. Registries varied in their use of electronic data reporting from pathology laboratories, but most were medium or high adopters. Although some registries indicated receiving pathology records in a paper-based format, which required significant manual effort, many registries have implemented processes to receive electronic pathology data in HL7 Version 2 format, which was often received through CDC’s Public Health Information Network Messaging System (PHINMS). The five registries classified as lower adopters of electronic reporting were low adopters of at least one reporting source and at most had a medium-level adoption of electronic reporting from physician offices and pathology laboratories. Registries classified as higher adopters had a medium or high level of electronic reporting from all three data sources.

TABLE 2.

Summary of Electronic Reporting Adoption by Central Cancer Registries

	Electronic Reporting Adoption by Data Source
Overall Adoption Level	State	Hospitals	Physician Offices	Pathology Laboratories
Lower adopters of electronic reporting	Massachusetts	High	Low	None
	Kansas	High	Medium	Low
	New Hampshire	High	Low	Medium
	Nevada	High	Low	Medium
	South Carolina	High	Low	Medium
Higher adopters of electronic reporting	Louisiana	High	High	High
	Pennsylvania	High	Medium	High
	Texas	High	Medium	Medium
	North Carolina	High	High	High

NOTE. The above classifications are self-reported on the basis of interviews conducted in 2019. As a general guidance, a high level of electronic reporting indicates that 80%–90% of data for a particular data source was received electronically and 30% or lower was considered low level of electronic reporting. The range between was classified as a medium level of electronic reporting adoption.

Table 3 summarizes the characteristics of the lower and higher adopters of electronic reporting. Only one of the five lower-adopter registries (20%) was a high case volume registry, compared with three of the four (75%) higher-adopter registries. Most registries in both groups received NPCR and state funding; one registry in each group received SEER funding, as well. A mix of software was used, but across both groups, the most used software applications were those provided by NPCR. Lower adopters reported an average of 8.8 total FTE, including 5.6 FTE-certified tumor registrars and 0.35 FTE computer or IT specialist positions. Higher adopters had a significantly higher number of total staff: 29.3 total FTE, which included 17.3 FTE-certified tumor registrars and 1.6 FTE computer or IT specialist positions. Four lower-adopter registries met the 24-month data quality standards, but only one met the 12-month standards. Conversely, during 2017 submission, all higher-level adopters met the 24-month data quality standards and half also met the 12-month standards.

TABLE 3.

Central Cancer Registry Characteristics by Electronic Reporting Adoption

Characteristics	Lower Adopters of Electronic Reportinga	Higher Adopters of Electronic Reporting
Central Cancer Registries	Kansas, New Hampshire, Massachusetts, South Carolina, and Nevada	North Carolina, Pennsylvania, Texas, and Louisiana
Volume of cases	One high volume Two medium volumes Two low volumes	Three high volumes One medium volume
Funding	One funded by SEER, NPCR, and state Four funded by NPCR and state	One funded by SEER, NPCR, and state Three funded by NPCR and state
Software	Two used NPCR Prep/CRS Plus Two used Rocky Mountain Cancer Data System One used other software and own database	Three used NPCR Prep/CRS Plus One used SEER software
Staffing average FTE
All staff	8.8 FTE	29.3 FTE
Certified tumor registrars	5.6 FTE	17.3 FTE
Computer or IT specialistsb	0.35 FTE	1.6 FTE
Outcomes (meeting registry data quality standards)c	One met 12-month standards Four met 24-month standards One did not meet 24-month standards	Two met 12-month standards All met 24-month standards

Abbreviations: FTE, full-time equivalents; GIS, Geographic Information System; IT, information technology; NPCR, National Program of Cancer Registries.

^aElectronic reporting adoption levels were self-reported by registries.

^bThis category may also include some GIS specialists.

^cOutcomes on the basis of cases diagnosed in 2016.

Figure 1 presents facilitators and barriers of electronic reporting adoption related to staffing, software, and external contextual factors. Almost all interview participants described vacant positions or indicated a need for more staff, notably for positions in IT, programming, quality assurance, and data processing. Staffing shortages had a greater effect on lower adopters than higher adopters of electronic reporting because their operations often relied on a few key staff, whereas larger registries had the capacity to spread registry activities across many staff. Often, participating registry staff attributed low staffing levels to turnover, low salaries, and limited ability to recruit qualified staff. Small-volume registries were particularly affected by limited staffing and cited this as their primary barrier for limited or no implementation of electronic reporting. The main staffing helpers among higher adopters of electronic reporting included staff training, having a strong technical skill set, and the presence of a champion within the registry team, such as at the management level, who can advocate for the registry.

FIG 1.

The effect of staff, software, and external contextual factors on electronic reporting adoption. CTR, Certified Tumor Registrar; eMaRC, Electronic Mapping, Reporting, and Coding; IT, information technology.

Software interoperability and support for software implementation were also cited as major factors in the registries’ implementation of electronic reporting and their ability to meet data quality standards. Registries indicated that although data are often received electronically, a key challenge was the limitation of the software that processed the electronic data. The software applications often did not support a fully automated electronic reporting system. Registries described significant manual exporting and importing among software applications, which added to the time burden in consolidating a cancer case from multiple records or sources, which was often the registries’ most time-consuming process. Deduplicating cancer cases and sorting through the higher volume of cases received via electronic physician reporting often still required visual inspection and technical expertise to automate processes.

Factors external to the registry that facilitated electronic reporting adoption are as follows: state legislation supporting electronic reporting, ability to participate in the North American Association of Central Cancer Registries National Interstate Data Exchange Agreement, access to use NPCR’s National Interstate Data Exchange Application System, the quality of data received from data sources, partnerships, funding availability, access to the State Health Information Exchange (HIE) platforms, and support for software implementation.^9,10

Figure 2 describes data quality and resource effects of adopting electronic reporting. Electronic reporting had a significant positive effect on some registries’ abilities to achieve their data quality goals and helped them receive data more rapidly. Physician reporting and registry interstate data exchange agreements helped improve data completeness. Although receipt of physician office data helped improve completeness, the large volume of information received also required significant manual review and processing.

FIG 2.

Process modifications and outcomes of adopting electronic reporting. IT, information technology.

For resource effects, registries noted that the activities performed shifted because of electronic reporting, and thus, resource use changed, as well. Some registries saw a reduction in travel costs because they can receive data without going directly to the source, but IT and staff costs increased. Overall, registries experienced challenges in filling the skilled positions required for electronic processing and automating processes. With the adoption of electronic reporting, registries have a greater need for skilled staff with IT experience and a need for more automated processes to improve efficiency.

DISCUSSION

The comparison of the characteristics and experiences of higher adopters versus lower adopters of electronic reporting identified key helpers and challenges to improving the quality of cancer registration by increasing the receipt of data electronically. The exploratory research presented in this manuscript suggests that electronic reporting has the potential to improve timeliness and completeness of registry data. Benefits of adopting electronic reporting include cost savings from reduced staff travel to abstract data and training and software support for receiving and processing electronic data. Despite the advantages and enhancements of electronic reporting, there are challenges that could be addressed, both within registries and across data sources, by creating processes and infrastructure to support registries in their quest to adopt electronic reporting. Overall, high-volume registries appear to be more likely to adopt electronic reporting than low-volume registries because the former may have more resources to devote to the process required to revise workflow patterns and sustain the new infrastructure required.

Consistent with findings from a previous study conducted on cancer registry systems in Alabama, we found that a key challenge among registries is the need for support staff with adequate training in medical records processing, IT, and quality assurance to facilitate processing of data received electronically.¹¹ Registries in our study that did not use a dedicated IT staff member often relied on IT support from the state health department, which was the only assistance available because of state health or department-specific staffing restrictions. IT staff from the state health department, however, often were not familiar with cancer registry software, which limited registries’ abilities to identify issues or handle updates or software changes. In addition, IT staff were reported to be more expensive than traditional registry staff, which added to the strain on the budget. Another barrier cited was the need for more quality assurance staff because the number of records received increases with electronic reporting. Registries, particularly those with small case volumes, may not have the additional staff to assign to these activities. NPCR program managers recognize that several manual tasks need to be addressed, which are still required to process electronic data. Thus, NPCR program staff are studying new ways to automate cancer data collection and reporting through cloud-based computing platforms to standardize electronic data received from pathology laboratories and physician offices.¹²

Software, hardware, networks, and other electronic architecture are essential to the adoption of electronic reporting. A challenge cited by registries in our study was the lack of fully automated systems with a high level of interoperability. Registries described significant manual exporting and importing of data, which added to the time burden in consolidating a cancer case, given the higher number of records that registries process with electronic reporting. Overall, electronic reporting has increased the need for autocoding, which is the process of assigning standard codes to textual data to enable automated electronic processing, better use of Statistical Analysis System (SAS) software programs, and natural language processing methods. Some registries with internal capabilities had begun improving automation of the cancer registration workflow, but these actions, similar to other product development practices, are likely to be accomplished in iterative steps.¹³ In addition to the innovations at the registry level, the NPCR program continues to improve methods for electronic mapping, reporting, and coding within the Registry Plus software suite.¹⁴

Registries often benefited from external partnerships, use of other data sources, such as interstate data exchange agreements, or use of other data transfer mechanisms, such as National Interstate Data Exchange Application System,^9,10 to aid electronic reporting and data completeness. Most registries indicated that their state had legislation that specifies electronic data reporting into the registry for all reporting sources, but many states did not define electronic reporting within the existing law, nor did they actively enforce reporting through fines or other means. Furthermore, although many pathology laboratories have adopted electronic reporting and are able to transmit standardized electronic pathology (ePath) to registries in multiple states, smaller pathology laboratories are hesitant to start electronic reporting because of the expense involved and the necessary technical requirements.¹⁵ Similarly, smaller physician offices are also more likely to have very limited staffing, which, in turn, limits their capacity to report data electronically.¹⁶ These external contextual factors are important challenges and affect registries’ abilities to embrace the adoption of electronic reporting.

Our findings have several limitations. First, the comparisons in this manuscript are based on a small number of higher and lower adopters of electronic reporting. Second, the registries selected for this assessment are a subset of the central cancer registries; therefore, the findings may not be broadly generalizable. Third, the software and processes used, along with IT resources and support, vary among registries, so not all registries experienced the same types of software or IT-related challenges. However, the inclusion of registries of various characteristics, such as staffing size and IT capabilities, strengthened our ability to explore the key relationships between various factors and electronic reporting. We do acknowledge that successful adoption of electronic reporting requires support from IT and quality assurance staff at the central registry. Finally, some of the external challenges named may be related to policy decisions made by the state legislature, which may not be easily modified.

In conclusion, electronic reporting of cancer data may improve the completeness, timeliness, and quality of registry data and could reduce the labor resources required to process manually collected data and help continuity of operations during public health emergencies or natural disasters. In this study, we explored the reasons for variation among registries in their adoption of electronic reporting and identified helpers and challenges. Registries can use these findings to implement steps to address staffing, software, and external contextual challenges to support and maintain the successful adoption of electronic reporting.

CONTEXT.

Key Objective

To assess the key facilitators and barriers along with potential benefits and lessons learned from implementing electronic cancer data reporting among selected US central cancer registries.

Knowledge Generated

Registries face a myriad of internal and external factors, such as staffing, information technology capabilities, state legislation, software, and source data quality, which may affect their ability to process data electronically. Implementing and improving electronic reporting processes may result in increased efficiency of data collection and processing, which can help to improve the timeliness and completeness of cancer registry data and reduce costs and resources required to process manually collected data.

Relevance

Ensuring high-quality cancer registry data is crucial for the successful development, implementation, and evaluation of cancer prevention and control policies and strategies. These findings can be used by registries to implement steps to address staffing, software, and external contextual challenges to support and maintain the successful adoption of electronic reporting.

APPENDIX 1. IN-DEPTH INTERVIEW GUIDE FOR DIRECTORS

National Program of Cancer Registry Data Collection and Reporting Interview Guide—Registry Directors

Introduction

Thank you for agreeing to participate in an interview to help us learn more about how your registry receives and processes its data. RTI International has been contracted by the CDC to help gather information on registry data processes, facilitators, and barriers, along with costs of operating and sustaining National Program of Cancer Registry (NPCR)–funded cancer registry data reporting methods. We’ll be speaking with about nine NPCR registries for this short qualitative study. The information you provide from this qualitative study will help inform the CDC on current approaches to receiving and processing data. This interview will last about an hour, and we will also be taking notes to capture the details of this discussion. Do you have any questions for me before we begin?

Topic 1. Data Collection Overview

1. Please describe your registry’s key current goals regarding data quality standards.

   Probes:

   1. What are your current data quality standards objectives as related to timeliness, completeness, and quality of the data?
   2. As related to meeting 12-month and 24-month reporting standards?
   3. Can you describe how you include meeting data quality standards in your NPCR evaluation plan questions and activities?
   4. What types of quality control/assurance procedures do your registry have in place? In your NPCR evaluation plan? How is this applied toward 12- and 24-month completeness?
   5. What barriers do CCRs face around reporting high-quality complete 12-month data?

2. What processes are in place to facilitate how your registry receives, processes, and reports data (such as management, editing, and consolidation)?

   Probes:

   1. Walk us through the timeline of how and when your registry began performing this process and how implementation proceeded.
   2. Does the registry perform any direct data collection? If so, please describe these processes as well.
   3. How do these processes help you meet key objectives of your registry or NPCR data quality standards (timeliness, quality, and completeness) as well as data use and dissemination?
   4. Who are the key staff members who receive, process, and report the data, and please describe their primary roles and activities?
   5. Please walk us through the learning curve and training that the registry staff went through in order to implement your processes.
   6. What resources or tools do the registry provide to data staff to help facilitate these processes? Does the registry receive any external support to enable these processes?

3. Is there a framework or system in place to guide continued implementation or improvement of existing processes? Did this framework originate from within the registry or have you adopted external approaches?

   Probe: Can you please share any frameworks or process pathways that you are using or have developed for receiving and processing registry data?

4. How would you define electronic reporting? Please describe your registry’s level of adoption of electronic reporting, in terms of implementation progress and volume of data received and processed electronically. How is this different for data received from hospitals, physician offices, and laboratories?

   Probes:

   1. Would you classify your registry as a high- or low-level adopter of electronic reporting? What are your reasons or characteristics of the registry that helped you make this classification? Is this different by hospital, physician offices, and laboratory sources?
   2. If not performing electronic reporting, why not? If low or non-adopters, what are the reasons electronic reporting hasn’t been fully adopted yet for the different sources? Are there plans to adopt or increase electronic reporting in the future? Can you describe those plans?
   3. How would you see your level of adoption changing in future years?
   4. Have any partnerships or collaborations been formed to support or promote the implementation and use of electronic reporting?
   5. Does the registry have an advisory committee that has helped develop or improve use of electronic reporting processes?
   6. If your registry has electronic reporting, describe the level of integration of electronic reporting with existing cancer registry data procedures.

Topic 2. Facilitators and Barriers

1. Describe any internal facilitators (at the registry level) that help the registry implement or improve data receipt, processing, and reporting methods.

   Probes:

   1. Facilitator examples include staff training, organization and funding structure; improvements to completeness, timeliness, or quality of data.
   2. Any facilitators specific to electronic data reporting?
   3. Any facilitators from the state health department level?

2. Describe any external facilitators that help the registry implement or improve data receipt, processing, and reporting methods.

   Probes:

   1. Facilitator examples include state health information exchange (HIE) and CDC technical assistance, partner expertise, incentive for provider adoption.
   2. To what extent does state policy and technology support cancer surveillance?
   3. Is there a state law in place that promotes rapid case ascertainment or electronic reporting? Are there any consequences to not reporting electronically?
   4. Any facilitators specific to electronic data reporting?

3. Tell us about any barriers the registry faces internally (at the registry level) that restrict the implementation or continued use of the methods we’ve discussed.

   Probes:

   1. Staff turnover and training requirements, caseload, organization structure and funding levels, information technology infrastructure, software, or expertise in-house?
   2. Any issues specific to electronic data reporting?
   3. Any barriers from the state health department level (ie, State HIE infrastructure)?

4. Tell us about any barriers the registry faces externally that restrict the implementation or continued use of the methods we’ve discussed?

   Probes:

   1. Legislation, technical assistance, size of geographic coverage area, and quality of data received.
   2. Any other issues specific to electronic data reporting?

5. Are there any other facilitators or challenges faced by your registry in the start-up, implementation, and continued use of your registry’s methods to receive, process, and report data that we should know about?

   Probe: Onboarding, reviewing quality of the methods, training providers on data submission, and consolidating data from multiple sources.

Topic 3. Costs of Operations and Sustainability

1. Over the past three years, have the registry’s approaches to receiving, processing, and reporting the data impacted your registry’s overall cost of operations?

   Probes:

   1. How have the costs changed over time (achieved savings or incurred additional costs)?
   2. Are the changes in cost due to adoption of electronic data reporting?
   3. Have you had to cut costs from other registry areas to make funding available for implementing your processes or adopting electronic reporting?
   4. Acknowledging that CDC funding has been relatively stable, have you received resources or funding from other sources to support registry operations?
   5. Have you noticed any improvements in registry efficiency, such as time saved in performing the activities we’ve discussed, as a result of your processes?

2. Are there any notable successes that have been an outcome of the use of your registry’s methods? Any setbacks or lessons learned?

   Probe: Are there any improvements in achieving data quality standards?

3. What are the registry’s next steps or future plans for continued implementation of methods for data receipt, processing, and reporting? Short-term steps and long-term vision?

4. How will your registry aim to maintain sustainability of your current methods?

   Probe: Any emphasis on staff training or expertise, partnerships or funding, HIE, or other factors that may help with sustainability of your methods?

APPENDIX 2. IN-DEPTH INTERVIEW GUIDE FOR DATA MANAGERS

National Program of Cancer Registry Data Collection and Reporting Interview Guide—Data Managers

Introduction

Thank you for agreeing to participate in an interview to help us learn more about how your registry receives and processes its data. RTI International has been contracted by the CDC to help gather information on registry data processes, facilitators, and barriers, along with costs and sustainability of National Program of Cancer Registry (NPCR)–funded cancer registry data reporting methods. We’ll be speaking with about nine NPCR registries for this short qualitative study, and the information you provide will help inform the CDC on current approaches to receiving and processing data. This interview will last about an hour, and we will also be taking notes to capture the details of this discussion. Do you have any questions for me before we begin?

Topic 1. Data Collection Processes

1. Please describe how your registry receives and processes data from reporting sources. How does this differ for data reported from hospitals, physician offices, and laboratories?

   Probes:

   1. Are you the main person responsible for this activity—and what other staff are involved?
   2. What mechanisms does your registry use for receiving data from reporting sources (eg, secure Internet-based FTP, https, encrypted e-mail, PHIN MS, and other methods)? Are there any differences in mechanisms used for hospital sources, physician offices, and laboratories?
   3. What mechanisms does your registry use for processing data from these reporting sources? Are there any differences in mechanisms used for hospital sources, physician offices, and laboratories? Please describe the processes you go through when you receive electronic reports and how you incorporate the data from these reporting sources into your central registry database.
   4. Walk us through the timeline of how and when your registry began performing your current methods to receive and process data for each reporting source (hospital, physician office, and laboratories) and how each was implemented.
   5. Were there any differences in implementing this process by different data sources (hospitals, physician’s offices, and laboratories)?
   6. If you have a mapping or data flow diagram of your registry’s data processing, would you be able to share it with us to help our understanding? How often does this get updated?

2. What software applications does your registry use to receive, process, and report cancer surveillance data?

   Probes:

   1. Do these software applications enable you to receive information on all required (eg, NPCR, NAACCR, etc) data elements?
   2. Tell us about the format of the data that are being transferred, and whether you use a format specified by your registry or standardized national data transmission formats? Does the registry promote the use of these formats to data sources, such as hospitals, laboratories, and providers? Is the format different for the data sources (hospitals, laboratories, and providers)?
   3. Are there other tools or resources that the registry uses in the start-up, implementation, or ongoing facilitation to receive and process data?

3. Is there a framework or system in place to guide continued implementation or improvement of existing processes? Does this guidance come from the registry management or from outside the registry?

4. How would you define electronic reporting? What is your registry’s level of adoption of electronic reporting? Are there any differences by source (hospital, physician, and laboratory)?

   Probes:

   1. Would you classify your registry as a high- or low-level adopter of electronic reporting? Is this different by hospital, physician offices, and laboratory sources?
   2. If not performing electronic reporting, why not? If low or non-adopter, what are the reasons electronic reporting hasn’t been fully adopted yet for the different sources? Are there plans to adopt or increase electronic reporting in the future? Can you describe those plans?
   3. What were your registry’s motivations for starting electronic reporting?
   4. What are the respective percentages of electronic reporting from hospitals, facilities, pathology labs, and physicians’ offices to the registry? Are rates of electronic reporting increasing in your state?
   5. How much of your incoming data is automated, electronic, or in paper abstract format?
   6. How much of your registry processes are automated vs. manual? Which processes need to be automated?

Topic 2. Facilitators and Barriers

1. What has helped you the most as a data manager to implement or perform methods to receive, process, and report data?

   Probes:

   1. Was the training you and others received sufficient for performing your registry’s procedures? If it was sufficient, what types of training were most helpful? If it was not sufficient, what other training is needed?
   2. Where was most of your training from—from someone within the registry, did the registry bring a technical person in from outside the registry, or did you attend an outside training course?
   3. Do you feel your registry has enough resources, such as staff, equipment, and time to support the registry’s electronic data reporting activities? If not, what more do you think is needed? Do you feel the staff have a good understanding of the procedures?
   4. What type of information technology infrastructure is in place to support your registry?

2. Tell us about any challenges you face that restrict your registry’s methods to receive, process, and report data.

   Probes:

   1. How were the challenges different during the initial start-up and implementation phase versus current or future challenges of electronic data reporting?
   2. Are there any software-related issues that impact your registry’s methods?

3. Are there any other facilitators or challenges faced by your registry in the start-up, implementation, and continued use of your registry’s methods to receive, process, and report data that we should know about?

Topic 3. Data Quality and Sustainability

1. How have the registry’s processes affected your ability to:

   1. Receive high-quality data from hospitals, physicians, and laboratories
   2. Report high-quality, complete, and timely cancer surveillance data
   3. Meet NPCR program standards
   4. Disseminate and ensure usability of cancer surveillance data to researchers and stakeholders

2. Are there any notable successes that have been an outcome of the registry’s processes or use of electronic reporting? Any setbacks or lessons learned?

3. Are there any tools or resources that you or the registry are lacking that could help you to improve methods to receive and process data or to better implement electronic reporting?

   Probes:

   1. Are these tools and resources something you expect CDC to assist with? If so, in what way?
   2. Do you know if these tools and resources are available to your registry? If so, what barriers have kept your registry from gaining access or utilizing it?

4. Are there any other factors, such as more training or more resources, that could support the sustainability of the methods we’ve discussed or adoption of electronic reporting?