Developing and testing a cost data collection instrument for noncommunicable disease registry planning

Abstract

Background

This article reports on the methods and framework we have developed to guide economic evaluation of noncommunicable disease registries.

Methods

We developed a cost data collection instrument, the Centers for Disease Control and Prevention’s (CDC’s) International Registry Costing Tool (IntRegCosting Tool), based on established economics methods We performed in-depth case studies, site visit interviews, and pilot testing in 11 registries from multiple countries including India, Kenya, Uganda, Colombia, and Barbados to assess the overall quality of the data collected from cancer and cardiovascular registries.

Results

Overall, the registries were able to use the IntRegCosting Tool to assign operating expenditures to specific activities. We verified that registries were able to provide accurate estimation of labor costs, which is the largest expenditure incurred by registries. We also identified several factors that can influence the cost of registry operations, including size of the geographic area served, data collection approach, local cost of living, presence of rural areas, volume of cases, extent of consolidation of records to cases, and continuity of funding.

Conclusion

Internal and external registry factors reveal that a single estimate for the cost of registry operations is not feasible; costs will vary on the basis of factors that may be beyond the control of the registries. Some factors, such as data collection approach, can be modified to improve the efficiency of registry operations. These findings will inform both future economic data collection using a web-based tool and cost and cost-effectiveness analyses of registry operations in low- and middle-income countries (LMICs) and other locations with similar characteristics.

1. Introduction

Worldwide, noncommunicable diseases (NCDs) account for 43% of the disease burden, and this is expected to increase to 60% by 2020 [1]. Much of this increase will result from the projected growth in NCDs in low- and middle-income countries (LMICs) [2]. Large inequities exist in the coverage and quality of registration activities across the world, with limited information currently available in the lower-income settings. Although historically there have mostly been hospital-based cancer registries which collect information on admitted cases, there are very few longstanding population-based registries [2]. The U.S. Centers for Disease Control and Prevention’s (CDC) National Program of Cancer Registries funds cancer registries in Puerto Rico and Pacific Island Jurisdictions which may have similar characteristics to those funded in LMICs. The percentage of the population covered by cancer registries that meet the quality standards for inclusion in global statistics (Cancer Incidence in Five Continents, or CI5) ranges from nearly 100% in North America to less than 10% in Asia, Central and South America, and Africa, where most LMICs are located. The stark reality is that the burden from NCDs is increasing in LMICs [2,3]. Without detailed local information on the incidence, effectiveness of prevention interventions, treatment outcomes, and mortality, the evidence base to support prevention and control activities is limited. Investing in high-quality registry data is essential in LMICs to inform data-driven selection of interventions to ensure efficient use of available funds and optimize health benefits.

Economic evaluations of surveillance operations are now increasingly important as decision makers identify the funding required to initiate and sustain NCD registry operations. Additionally, cost data are required to evaluate the most-efficient approach and process for collecting surveillance data, and to quantify the resources needed for program activities. Although the generalized methodology for collecting cost data from programs has been well described [4–12], there is limited information on the optimal approaches to collecting cost and resource use data from registries [13–17]. Specifically, there is no systematic and tested approach to collecting economic data from registries operating in different regions of the world and in countries with a variety of income levels. Standardized methodology for collecting cost data is essential for performing comparative economic evaluations that can provide actionable feedback to policy makers [18,19].

To be successful and provide high-quality data, surveillance and registration operations require several components, including case ascertainment, data abstraction, database management, and dissemination of findings. In addition to these core activities, registries also need support functions such as management, training, and information systems. Therefore, to allow for comprehensive evaluations, economic assessments must collect detailed costs related to each activity; preparation of justifications for decision makers about the need for additional dollars to support activities and/or planning interventions to optimize costs are not feasible when only information on total cost is available [16,18,20]. Furthermore, comparative evaluations require consistent and transparent methods, as costs collected from a variety of perspectives, such as funder, program, and society, may not be directly comparable [14,15]. A common set of definitions must be available to ensure that all costs and resource use can be accounted for systematically.

In this study, we report on the development of a standardized instrument to collect cost data from cancer and cardiovascular registries, and we report the results from pilot testing the tool to assess the extent to which the tool is able to capture cost data and also the quality of the cost data collected. Additionally, we summarize the factors that are likely to impact the cost and quality of registry operations. The goal of this analysis is to identify a systematic process for collecting, reporting, and comparing cost data on NCD registry operations across countries. The findings from this study will establish standardized methodology for economic evaluation of registry-based NCD surveillance data collection. The lessons learned from pilot testing the tool will help to validate and improve the tool for future cost data collection.

2. Materials and methods

We selected a convenience sample of representative population-based registries from low-, low–middle-, high–middle-, and high-income countries to pilot test the tool. Our goal was to select established, early-phase, and research-focused registries in geographically dispersed locations (urban and rural areas). On the basis of these criteria, we invited 11 registries from multiple countries including India, Kenya, Uganda, Colombia, and Barbados to pilot test the tool. The Mumbai (India), Nairobi (Kenya) and Kampala (Uganda) registries are all established registries in predominantly urban locations. The Barshi (India) and Eldoret (Kenya) registries are both more research-focused, with a rural and rural-urban coverage area, respectively. In Colombia, we worked with 5 established registries across the country: the Barranquilla, Bucaramanga, Cali, Manizales, and Pasto registries. The cancer registry in Barbados is classified as an early-phase registry, as it is still in the process of establishing an optimal data collection approach for cancer case data collection. The Barbados registry is also unique in that the cancer, stroke, and heart attack registries are housed together.

2.1. Developing the IntRegCosting tool

We performed in-depth case studies and interviews during site visits to each registry to ensure that the cost data elements included in the CDC’s International Registry Costing Tool (IntRegCosting Tool) were comprehensive. Each registry completed a detailed Pre-Site Visit Questionnaire that provided information on the registry structure and operations, which allowed us to understand the optimal approach to collect and allocate cost data to registry activities. During the site visit, we clarified the details provided in the Pre-Site Visit Questionnaire and performed key informant interviews to understand the data collection process, procedures for developing analytic files, and activities involved in disseminating the registry data. This qualitative information was analyzed to identify common themes and then these themes were used to determine the list of activities related to registry operations and their definitions. Additionally, through these qualitative interviews, we gathered information on barriers and facilitators as well as factors that affect the cost and effectiveness of registry operations. In the future, when data are available from a much larger sample of cancer registries, we will use these factors to perform in-depth univariate and multivariate analysis to assess impact of the factors on cost of registry operations

The overall structure of the IntRegCosting Tool is based on the economic evaluation data collection tool developed for the Centers for Disease Control and Prevention’s National Program of Cancer Registries (NPCR) in the United States [15]. We modified and adapted the NPCR tool for use in the non-U.S. setting on the basis of input received from the NCD registries through the Pre-Site Visit Questionnaire and the qualitative staff interviews. The methods used to guide our adaptation of the NPCR data collection instrument to create the IntRegCosting Tool were based on standard methods of collecting program cost data [4,5,9]. We subdivided the IntRegCosting Tool into 10 modules to collect the activity-based resource use and cost information (Table 1). We attempted to design the IntRegCosting Tool modules to reflect budget categories familiar to management and fiscal staff at the registries so as to minimize any ambiguity in the data elements requested in each module. The tool collected data across budget categories, including labor; consultants; computers, travel, training, and other materials; software; and administrative or overhead expenses. The first nine modules include (1) registry details, such as name, organization type, and structure; (2) total expenditures from all sources of funding; (3) in-kind contributions; (4) personnel expenditures; (5) personnel activities; (6) consultant expenditures; (7) computers, travel, training, and other materials expenditures; (8) software used and licensing expenditure; and (9) administrative expenditures.

Table 1.

Overview of Modules in the IntRegCosting Tool.

Module	Description	Data Elements
Registry details	General registry information	Registry name, organization type (health department, university, research institute, private, other), primary contact person
Expenditures	Total spending for the program year from all funding sources	Total funds by source, total funding for current year, unobligated funds carried forward from previous year, amount of funds unspent for the current year, total funds expended
In-kind contributions	Nonmonetary resources, assistance, and support	Labor contributions, non-labor contributions, valuation method, primary activity
Personnel expenditures	Personnel expenditures related to registry activities	Job title, full-time equivalent percentage, number of hours per week, months employed in fiscal year, salary, percentage of total time spent on all registry activities, salary allocated to registry activities
Personnel activities	Percentage of employee time spent on specific registry activities	Employee percentages by registry activity
Consultant expenditures	Value of time spent by contracted consultants on registry activities	Job title, annual payment, proportion of time spent on up to three activities by each consultant
Computers, travel, training, and other materials expenditures	Costs associated with computers, travel, training and other materials	Hardware costs, IT support costs, travel costs, training fees, costs of other materials, primary activity associated with costs, cost calculation method (actual, estimate, other)
Software used and licensing expenditure	Costs associated with software licensing and other software packages used by registries	Name of software, total amount of contract, year contract started, length of contract, amount paid in current year, cost calculation method (actual, estimate, other)
Administrative expenditures	Expenditures for administrative and overhead costs (e.g., phone, rent)	Total administrative or overhead cost; allocation method (e.g., percentage of direct costs); sub-category totals—rent, repairs/maintenance, network connection/maintenance, phone service, office equipment, other costs (such as water and electricity bills)
Factors affecting registry operations	Supplemental information on factors that may affect the costs of registry operations and effectiveness	Incident cases, abstracts received, whether records were stored centrally versus in separate departments, methods of data reporting or collection, data collection process (proportion of data abstracted directly from hospital records by registry staff or hired contractors), non-resident cases reported

To supplement the cost data, the final module in the IntRegCosting Tool (module 10) collected information on selected factors that can affect the cost, effectiveness, and quality of registry operations. To reduce duplication of effort, we collected only details on factors that were not readily available from other reliable sources such as size of coverage area. We obtained information in the tool on total number of incident cases, total number of abstracts received, and methods of data reporting.

The IntRegCosting Tool is designed to collect data using the activity-based costing methodology. Therefore, in each of the modules, we requested that the registries allocate each expenditure to specific registry activities [15–18]. A listing of the 27 registry activities is shown in Table 2, categorized into fixed- versus variable cost-activities. Detailed activity definitions are provided in Appendix A. We defined fixed-cost activities as those that do not vary in cost as volume of cases change (at least in the short run), and variable-cost activities as those that do. The variable cost-related activities were further subdivided into core activities—those that are essential for registry operations—and other activities, such as enhanced analysis and research-related tasks. The registries were asked to allocate the time spent by each registry staff so labor costs could be estimated for each of the 27 activities. The tool, based on prior experience in collecting cost data from programs, provides up to three activity choices to allocate cost related to other non-labor expenditures.

Table 2.

Fixed- and Variable-Cost Registry Activities.

Fixed-Cost Registry Activities
Management	Reporting requirements
Administration	Outreach
Training of registry staff	Liaising with stakeholders
IT support
Variable Cost: Core Registry Activities
Case ascertainment	Data validation
Death certificate clearance	Developing analytic files
Data collection and abstraction	Database management
	Data analysis and reporting/tabulation
Data entry	Quality assurance
Coding	Sharing cases
Variable Cost: Other Registry Activities
Developing proposals for funding	Implementing an inquiry response system
Electronic case reporting and data encryption	Research studies and advanced analysis using registry data
Automatic case finding using electronic linkage	Publication of research studies using registry data
Linking records to other databases	Active follow-up
Training of others by registry staff

The IntRegCosting Tool collects cost data using a programmatic perspective. Registries often pool funds from multiple sources, so a direct correlation between registry activity and funding stream is not always possible. Therefore, the tool is designed to generate activity-based cost regardless of the funding source used to pay for registry expenditures. Furthermore, to ensure comparability across registries, the IntRegCosting Tool also collects data on in-kind labor and non-labor contributions. In-kind contributions include non-monetary resources, assistance, and support provided to the registries—for example, voluntary time spent by clinicians and researchers to develop registry policies or advice on specific cases that require manual review. Non-labor contributions include office space, phone service, water, electricity, and supplies, which are often contributed by the host institution (for example, the university, research institute, or cancer society where the registry is hosted). Distortions in program cost estimates have been shown to occur if these in-kind contributions are not included [13–15].

The IntRegCosting Tool allows a registry director or representative to self-report registry cost information for a specific fiscal or annual period. The tool was created in Microsoft Excel because the spreadsheet capabilities of Excel facilitate data checks during the data entry process and because Excel allows the insertion of formulas to calculate totals for each cost category automatically. The Excel-based IntRegCosting Tool and user’s guide has self-guided check posts to ensure the tool was completed accurately. Check posts include confirmation that (1) funding information is provided in module 2 for the reporting year; (2) in-kind contributions are only entered in module 3 for the reporting year; (3) salaries entered in module 4 are the total annual or fiscal period salaries earned by staff members and not only the amounts related to registry activities; (4) only payments made for consultant services during the reporting time period are reported in module 6; and (5) information in module 10 is based on cases diagnosed in the appropriate year.

2.1.1. Country-Specific adjustments and language translation

The IntRegCosting Tool and user’s guide were tailored when needed to ensure incorporation of that terminology specific to each country’s understanding of terms and procedures. These country-specific changes were generally related to wording of definitions. The overall activity categories remained the same to allow for comparison across registries and countries. To further facilitate data collection, the IntRegCosting Tool and user’s guide were translated into Spanish; however, we used the same broad standardized activity-based costing approach for both language versions. The English tool and user’s guide were translated into Spanish by professional translators and then reviewed by Spanish-speaking registry collaborators to ensure clarity of the language used. All other aspects of the data collection, including modules, data elements, and activities, remained consistent across the English and Spanish versions of the tool.

2.1.2. Cancer and cardiovascular registry cost data collection

To accommodate data collection from organizations that host multiple integrated registries, we embedded a few modifications in the tool. In each module, we allowed organizations to select activities specific for each registry. When there were shared expenditures across registries, we provided an option for reporting the proportion of the cost that need to be allocated to each specific registry. For example, when labor was shared, the time spent on each registry activity was used to assign proportion of cost. For rental cost of shared space, the proportional cost was based on the number of square feet used.

2.1.3. Data collection using the IntRegCosting tool

The data collection needed to identify the activities required to collect high quality cancer data and the resources needed to support these activities was incidental to the primary intent of providing technical assistance to support registries. Before initiating data collection, we held introductory conference calls with all registries, which were followed by interactive webinar-based training sessions. Additionally, we provided registries with the user’s guide, which contained detailed instructions and definitions for reporting the required data. We also provided technical assistance throughout the data collection period via telephone and email to ensure accuracy and completeness of the costing information. Each registry provided cost data retrospectively for the annual period that best matched the program budgeting period from January 2013 through March 2015.

3. Calculation

3.1. Analytic file creation

We reviewed data provided by the registries for internal consistency, using descriptive statistics to review cost by budget category and by comparing the expenditures reported in the tool to the total funding received. Using the details provided in the IntRegCosting Tool, we also generated activity-based cost estimates. For instance, labor hours allocated to specific registry activities were multiplied by hourly wage for each registry personnel to derive labor cost allocated to the activity. Labor and non-labor costs allocated to each activity were summed to derive total cost of each registry activity. In-kind contributions were also assigned in a similar manner. Any potential discrepancies identified were clarified with registry staff before finalizing the data inputs. We created both registry-specific and combined data files to assess distribution of cost by budget and activity categories. All analyses were performed using local currency units. To allow for comparability across the registries, we present percentage distribution across budget categories and cost expended by fixed and variable cost activity categories. Only cost and resource use data from cancer registries are presented.

3.2. Data quality assessment

We performed a series of data quality checks to assess the feasibility of using the IntRegCosting Tool to collect data for economic assessments. First, we determined whether the total cost of specific activities that received direct monetary contributions matched the total amount of funding received. When assessing the quality of the cost data submitted, we considered data “acceptable” for generating high-quality data if 90% of costs or more were allocated to specific activities. Second, our a priori expectation, based on past evaluation of registry activities, was that the majority of the funds would be spent on labor [15,21]. To confirm this, we generated cost estimates for the following budget groupings: labor; consultants; computers, travel, training, and materials; and indirect, administrative, and software. Although the general approach in economic assessments is to use an appropriate allocation methodology to assign indirect costs to program activities [4], we chose to present administrative or indirect costs as a separate cost center to ensure that these costs were indeed reported by all sites. This approach also allowed us to compare the magnitude of the indirect or administrative cost data across the registries. Third, we assessed whether the information reported in the IntRegCosting Tool allowed us to generate activity-based costs for all registry activities. That is, we identified whether costs were systematically omitted for specific registry operations.

4. Results

Table 3 provides an overview of the characteristics of the 11 registries that were selected to pilot test the IntRegCosting Tool. Overall, the registries varied substantially in terms of general characteristics, including size of population covered, number of reporting sources, and data collection approach (whether paper or electronic). Cancer was a reportable disease for about a third of the registries, and approximately half of the registries performed active follow-up. Additionally, nearly half of the registries collected data on nonresident cases to share with neighboring registries, and about a third performed death clearance to verify deaths. We used ‘inclusion in CI5′ as a proxy for quality of the cancer registry data; two-thirds of the registries generated data that was deemed to be of high enough quality to be included in global cancer statistics in CI5. Registry staffing also varied widely, reflecting the mix of both very small and very large registries. For instance, full-time data collection staff ranged from 1 to 17 across the 11 registries.

Table 3.

Registry General Characteristics, Activities, and Staffing.

General Characteristics	Range
Years of operation	9	–	62
Population covered	277,814	–	17,443,311
Square kilometers covered	121	–	11,820
Proportion of data reporting by paper	30%	–	100%
Number of reporting or data sources	12	–	180
Incident casesa	300	–	19,485
Registry Activities	Percentage of Registries
Reportable disease	36%
Perform active follow-up	55%
Collect and report nonresident cases	45%
Perform death clearance	36%
Included in Cancer in Five Continents (CI5)	64%
Registry Staffing (Full Time Equivalents)			Range
Management and administrative staff			1–3
Registrars and data collectors			1–17
Database management and IT support staff			0–8
Researchers, investigators, and medical personnel			0–3

^aFull year incidence estimates were used for registries with continuous ongoing data collection while ‘best-estimate’ averages were used for all others (specifically registries in Nairobi and Barbados where cases collected varied between annual periods).

Fig. 1 presents the distribution of cost among the cancer registries by budget categories. These costs include both monetary contributions and in-kind contributions; much of the nonmonetary support was provided by the host institutions where the registries were based. All registries met the threshold for high-quality economic data in that we were able to allocate more than 90% of the financial contributions received to specific registry activities. Labor was the largest cost component for all registries, ranging from 53% to 93% of the total cost. All registries were able to accurately report labor costs by reviewing salaries paid to staff, regardless of the source of the funding for the salary, whether from the host institution or external funds. Consultants were seldom used, and their payments could be tracked and accurately reported by registries except when in-kind contributions were involved. In those instances, to estimate the monetary contribution of the in-kind labor, hours were estimated, and appropriate salaries were assigned to the individual on the basis of level of training and experience. Overall, consultants represented a very small amount of the registry resources.

Fig. 1.

Registries’ Resources by Budget Categories.

Note: For the Nairobi Cancer Registry, Kenya, we show the average cost results using data collected for July 2012–June 2014. For all other registries, we show results from the first round of data collection for the following periods: Kampala Cancer Registry, Uganda – 2014 annual cost data; Mumbai Cancer Registry, India – fiscal year 2014–2015 cost data; Barranquilla, Bucaramanga, Cali, Manizales, and Pasto Cancer Registries, Colombia – 2013 annual cost data; and Barbados Cancer Registry – April 2014–March 2015.

Costs associated with computers, travel, training, and materials could be linked to specific events or purchases for which documentation was generally available to verify expenditure. Costs associated with indirect or administrative support and software were most often contributed by host institutions, and as there were no direct transfers of funds, these costs were the most difficult to accurately estimate. The registry staff reviewed internal documents when feasible and sought support from financial teams in their institutions to provide the best possible estimation of these costs.

Table 4 provides the mean proportion and range of resources allocated to the fixed-cost activities and the two variable-cost activity categories. All registries were able to report costs related to the key fixed-cost activities and the core variable-cost activities, thus providing additional validation of the accuracy of the data collected through the IntRegCosting Tool. On average, fixed-cost activities accounted for a third of the total cost, and core variable-cost activities were half of the registry operating cost. The ranges, reported in Table 4, show large variations across the registries.

Table 4.

Proportion and Range of Resources by Registry Activity.

	Fixed-Cost Activities	Variable Cost—Core Registry Activities	Variable Cost—Other Registry Activities
Mean	33.8%	51.5%	14.7%
Range	19.3%–62.0%	37.1%–66.9%	0.9%–24.3%

Note: The means and ranges represent registry cost results for the following periods: Nairobi Cancer Registry, Kenya – July 2012–June 2014, Kampala Cancer Registry, Uganda – 2014 annual cost data; Mumbai Cancer Registry, India – fiscal year 2014–2015 cost data; Barranquilla, Bucaramanga, Cali, Manizales, and Pasto Cancer Registries, Colombia – 2013 annual cost data; and Barbados Cancer Registry – April 2014–March 2015.

Table 5 summarizes the key findings from the qualitative interviews on the factors that could impact the cost and quality of the registry data. We identified a large number of factors, related to both internal registry operations and external aspects, which need to be considered when performing economic evaluations of registries. Some of the internal factors may be amenable to modification by registry staff, but not consistently. For example, registry staff can take steps to increase the amount and continuity of the funding, but the overall availability of funds can hamper their efforts. External factors are generally beyond the control of the registry and could explain the reasons for variation in cost. The geographic area covered by the registries and the number of reporting sources will affect the number of data collectors required and thereby affect the cost of the registry operations. Other, subtler aspects that can affect cost and quality of the data include the support infrastructure available in terms of trained personnel, and availability of high-quality data at the collection sites.

Table 5.

Factors That Affect Registry Cost and Data Quality.

	Affects Cost	Affects Quality	Comments
INTERNAL REGISTRY OPERATIONS
Organization and funding structure (host institution support versus external funding)	X	X	Continuity of funding and adequate level of resources are essential for planning and implementing efficient processes
Data collection procedures (paper versus electronic format)	X	X	Electronic data collection reduces data reentry time and can improve quality
Number and types of data elements collected	X	X	Collection of treatment and outcomes data (including use of active follow-up procedures) increases the cost but can improve quality of the data available
Staff turnover and training requirements	X	X	Employing staff on short-term contracts because of funding limitations can lead to high staff turnover and the need for repeat training sessions, which increases cost and reduces quality
Work mix (core data collection versus research activities)	X	X	Research activities tend to be more data intensive and can increase staff time required to collect and report registry data
Data exchanged, caseload, and reporting of nonresident cases	X	X	Collecting nonresident data increases cost, but receiving cases from other registries can improve completeness of registry data
EXTERNAL FEATURES
Total volume of cases	X		Data abstraction cost increases with volume; fixed costs do not
Number of reporting sources		X	A larger number of sources increases time required to travel and obtain records at multiple facilities
Number of abstracts versus incidence cases	X		Even when electronic processes are available to ensure unduplicated records, some manual review will be required
Size of area served and presence of rural areas	X		Large areas lead to high travel costs to facilities because of long distances and an increase in staff time related to travel
Availability of trained personnel	X	X	Shortage of personnel can increase cost, because of intensive training, and reduce quality
Cost of living in geographic location	X		Labor cost will vary on the basis of cost of living
Quality of facility reporting and presence of hospital-based registries	X	X	Ease of access (centralized versus decentralized records) and completeness of data at the facilities can improve data collection efficiency and quality
Case reporting mandated by law (reportable disease)	X	X	Mandatory case reporting reduces time required to negotiate with providers, improves access to the data, and also improves case ascertainment

5. Discussion

Overall, the registries were able to use the IntRegCosting Tool to assign operating expenditures to specific activities. We were able to verify the completeness of the costs derived from the IntRegCosting Tool for activities that were funded directly through external sources or those that were provided through transfer payments, such as salaries, from the host institutions. Although the registries were able to use the user’s guide and technical support provided to estimate the cost of non-labor activities, we are not able to systematically assess the accuracy of the cost of the indirect administrative support and other in-kind contribution provided by the host institution. These costs, such as use of office space, do not involve any direct transfer payments, and we had to use best-case estimates to derive these costs based on standard economics methodology [4,5]. On the basis of prior registry evaluations, we know that labor cost typically is the largest source of registry expenditure; for example, labor cost accounted for 79% of the total cost in European registries [21]. In this study, labor costs were derived to a high level of accuracy, as they were based on actual payments made to registry staff or consultants. The registries were able to use the IntRegCosting Tool to successfully allocate costs to specific registry activities. More than 90% of the total costs were assigned to specific activities, and on average, a third of the costs were expended on fixed cost registry activities. A detailed comparison of registry characteristics, cost per case, and cost per inhabitant is provided in Tangka et al. [22].

Findings from the pilot study show that there are many different factors that affect cost of registry operations. This is similar to conclusions from previous analyses of the economics of registry operation in both the United States and Europe [13–17,21,23]. In the recently completed economic evaluation of the NPCR, volume of cases was shown to be an important determinant of cost, and costs for low-volume versus high-volume registries were US $38 to US $135 per case collected, respectively [13]. We have reported in this manuscript and in past studies on several factors that can affect registry costs [15]. These include the size of the area served, the urban/rural makeup of the area served, and the data collection approach including extent to which active follow-up is performed.

The many factors identified most likely interact in complex ways, and therefore, their impact on cost of operation is not a straightforward association. For example, past analysis has revealed the existence of economies of scale, which exhibit a non-linear relationship. In addition, although volume is a critical factor in lowering cost per case, other cost drivers need to be considered [14]. Thus, it is not feasible to derive a single cost estimate for registry operations, but one can use the factors identified to determine the anticipated cost of running specific registries on the basis of both internal registry operations and external features. In future research, using a larger sample of cancer registries, we will determine the extent to which factors identified in this study decrease or increase registration costs to develop a cost estimation algorithm to project registry operational costs.

Another key finding is that although many of the factors identified may be beyond the immediate control of registry staff, especially factors external to the registry. Several of the internal registry operational characteristics can be modified to improve efficiency. For example, adoption of improved data collection approaches, such as electronic data collection that eliminates manual entry of information into the registry database, could help reduce staff time and hence decrease expenditure within the largest cost component of the registry operations. Additional research is needed to develop the evidence base on optimal approaches to improve efficiency and cost-effectiveness of the operations. These investigations could determine both enablers and barriers to producing actionable guidance for registries, and they could also assess the potential benefit of tailoring the design of the interventions based on existing registry infrastructure.

The IntRegCosting Tool collected cost data retrospectively from the 11 registries in this study. Although retrospective data can be subject to recall error, we believe recall error had minimal effect on our results, in part because registries maintain detailed records on employee compensation, the largest component of overall registry costs. To ensure accurate allocation of labor costs, we asked that all staff members report the proportion of time they spent on specific registry activities. In addition, most staff members worked on few activities; therefore, the potential for recall error was minimal. The accuracy of the information in terms of allocation of cost to specific activities, however, could not be verified in this study, which is a potential limitation. We did provide the sites with a detailed user’s guide that contained descriptions of all activities, and we provided further clarification via conference calls. We believe this hands-on approach facilitated accurate allocation of resources and cost to program activities. The IntRegCosting Tool is designed to be completed primarily by registry directors (or designees) with assistance from fiscal staff as required. Such reliance on a few key informants has been shown to be a reliable approach to collect cost data from programs and to significantly reduce the resources expended on data collection [24]. In future data collection efforts, the IntRegCosting Tool can be used to collect information about labor time related to specific project activities in a prospective manner. Additionally, data collection from a larger number of diverse registries will assist in further verifying the ability of the IntRegCosting Tool to collect activity-based cost data.

While pilot testing the IntRegCosting Tool, we identified a few ways in which the IntRegCosting Tool could be improved. First, more-detailed definitions and systematic approaches to calculate in-kind support, especially host registry contributions, can be provided. During the pilot testing process, technical support was provided to assist the registries in using an appropriate estimation approach based on established economics methodology. Lessons learned from the pilot testing can be used to provide detailed guidance on specific types of contributions, methodology to estimate cost, the data elements required to calculate costs, and approaches to confirm assumptions through sensitivity analysis. Furthermore, formulas can be embedded in the tool to improve ease of use and accuracy.

Second, to reduce the effort required to manually review completeness and overall accuracy of the results, an automated process can be implemented to verify information interactively so errors can be addressed during the initial data input process. This process will reduce overall time required by the registries to verify and input data after the initial data submission. Third, we received feedback from registry staff on their desire to use the IntRegCosting Tool features to inform registry planning through the use of scenario analysis. For example, staff would like to use the tool to answer questions such as, “What is the optimal approach and cost for expanding registry operations to cover a large population base?” The web-based tool currently in development will incorporate this feature and contain modules to allow registries to project future cost of changes to registry operations. Fourth, to capture a full set of potential factors that can affect cost of operations and explain differences between registries, we will include additional factors that may affect cost and effectiveness that have been identified through the pilot study.

6. Conclusion

The detailed, activity-specific cost data collected via the IntRegCosting Tool can help registries in LMICs understand the cost of their operations, design approaches to improve efficiency, and provide standardized data to compare with other registries, such as NPCR-funded cancer registries. For example, registries in Colombia have used the costing results to compare their activity-based costs to assess activities with large variation in order to perform follow up assessments to determine the drivers of these differences. Results from the pilot test indicated that the IntRegCosting Tool can be used to collect detailed, high-quality cost data with minimal burden to registries in LMICs. Although we tested the tool with cancer and cardiovascular registries, the activity categories and overall approach are applicable to other types of chronic disease registries. Cost data collected with a standardized tool will allow existing registries to improve economic efficiency of operations and provide guidance on the funding and design of new registries. The tool can also be used to create a continual learning environment where activities of the registries are assessed at regular intervals to track operation metrics in terms of both the cost and the quality of the data reported. In the future, scenario analytics will also be embedded in the IntRegCosting Tool to perform cost-effectiveness assessment of various data collection approaches and other registry operations.

Abbreviations

CDC

Centers for Disease Control and Prevention

IntRegCosting Tool

International Registry Costing Tool

CI5

Cancer Incidence in Five Continents

LMICs

Low- and Middle-Income Countries

NCDs

Noncommunicable Diseases

NPCR

National Program of Cancer Registries

Appendix A. IntRegCosting Tool: Listing of Registry Activities and Definitions

Management

Registry management, including, but not limited to, addressing personnel and staffing issues, and preparing registry applications for funding requests

Administration

Mailing, filing, logging, and other clerical tasks

Training of Registry Staff

Training of central registry staff; providing educational opportunities for staff; registry staff attending training-focused workshops and meetings, webinars, conference calls, and other state and local training opportunities

IT Support

Activities including, but not limited to, software management and support, hardware upgrades, network maintenance, and creation new systems and interfaces

Reporting Requirements

Preparing documents to disseminate registry statistics (including mandated and voluntary reporting) and providing data for inclusion in national and international databases

Outreach

Visiting schools and other locations to increase awareness about disease registration to support data collection efforts. Other activities could include providing continuing medical education to health professionals on the importance of disease registration

Liaising with Stakeholders

Serving as liaison to national partners, funders (health ministry, etc.), international agencies, cancer registries, and other organizations

Case Ascertainment

Using the CanReg4 or CanReg5 [25] Person Search function to determine if a patient is already in the database, and if so, whether the current data duplicates existing information or documents a new primary tumor. Also includes processing pathology reports and following up with physicians and other health care providers

Death Certificate Clearance

Confirming mortalities with local Registrar of Births and Deaths

Data Collection and Abstraction

Collecting information from hospitals, nursing homes, medical labs, and diagnostic and treatment centers

Data Entry

Entering abstracts into CanReg4, either manually or via upload

Coding

Coding into databases or statistical programs for analysis

Data Validation

Running checks to verify data quality including the CanReg4 functions Check Status for manual entry and Do Checks for imported data

Developing Analytic Files

Developing methods for matching cases, determining match criteria and categories of matching, consolidating data, and so on

Database Management

Managing datasets for registry operations and special studies, including backing up data and performing other functions using the Management option in CanReg4

Data Analysis and Reporting/Tabulation

Performing descriptive and statistic data analysis and generating tables using the CanReg4 Analysis sub-menu

Quality Assurance

Activities that include, but are not limited to, performing computerized and visual edits, reconciling discrepancies between cases, creating guidelines for data consolidation, and performing re-abstracting audits

Sharing Cases (with other registries)

Activities that include, but are not limited to, creating data sharing agreements, preparing and submitting cases for data sharing, and importing and editing incoming cases from data sharing sources

Developing Proposals for Funding

Writing and submitting proposals to different sources to receive funding or grants

Electronic Case Reporting and Data Encryption

Maintaining and increasing electronic case reporting from health care providers, including encrypting data

Automatic Case Finding Using Electronic Linkages

Linking to electronic data sources and finding cases automatically through such linkages

Linking Records to Other Databases (e.g., death certificate data)

Creating and submitting data files to reporting agency as specified for matching and linking, incorporating linked records into the registry’s data management system, procuring additional incoming data sources, importing cases from linked data sources, and performing visual review and updating cases from linked sources

Training of Others by Registry Staff

Training medical records officers and other health data reporters as part of continuing education and other activities where registry staff members train those not working at the central registry. This includes answering quality assurance questions and material development

Implementing an Inquiry Response System

Answering requests for data and other inquiry response system activities

Research Studies and Advanced Analysis Using Registry Data

Investigating disease clusters, performing special studies, and other research and analysis using registry data

Publication of Research Studies Using Registry Data

Preparing research studies resulting from registry data for publication

Active Follow-Up

Performing active, rather than passive, follow-up as necessary