Research output and the public health burden of cancer: is there any relationship?

Abstract

Purpose

The relative distribution of research output across cancer sites is not well described. Here, we evaluate whether the volume of published research is proportional to the public health burden of individual cancers. We also explore whether research output is proportional to research funding.

Methods

Statistics from the Canadian and American cancer societies were used to identify the top ten causes of cancer death in 2013. All journal articles and clinical trials published in 2013 by Canadian or U.S. authors for those cancers were identified. Total research funding in Canada by cancer site was obtained from the Canadian Cancer Research Alliance. Descriptive statistics and Pearson correlation coefficients were used to describe the relationship between research output, cancer mortality, and research funding.

Results

We identified 19,361 publications and 2661 clinical trials. The proportion of publications and clinical trials was substantially lower than the proportion of deaths for lung (41% deaths, 15% publications, 16% clinical trials), colorectal (14%, 7%, 6%), pancreatic (10%, 7%, 5%), and gastroesophageal (7%, 5%, 3%) cancers. Conversely, research output was substantially greater than the proportion of deaths for breast cancer (10% deaths, 29% publications, 30% clinical trials) and prostate cancer (8%, 15%, 17%). We observed a stronger correlation between research output and funding (publications r = 0.894, p < 0.001; clinical trials r = 0.923, p < 0.001) than between research output and cancer mortality (r = 0.363, p = 0.303; r = 0.340, p = 0.337).

Conclusions

Research output is not well correlated with the public health burden of individual cancers, but is correlated with the relative level of research funding.

INTRODUCTION

Up to 1 in 4 of today’s Canadians will die of cancer¹. Despite the rising burden of disease, survival for most cancers has improved over time². The improved outcomes are in part a result of advances in treatment that have emerged from clinical research. Given the current decline in research funding, it is critical to understand how the oncology community can optimally use limited resources to derive the greatest societal benefit. The term “payback” has been used to refer to the societal worth of clinical research^3,4. The first research deliverable in the payback model described by Buxton and Hanney and colleagues^3,5 is knowledge production, whose measurable element is scientific publications.

Relative to other diseases, oncology has greater relative funding and a disproportionate representation in higher-impact medical journals⁶. However, within oncology, the number of publications^4,6, clinical trials⁷, and research funding^8–11 might not be distributed across cancer sites relative to the burden of disease. To our knowledge, no published study has described the interrelationships of disease burden, research output, and research funding within oncology. For the present study, we evaluated whether the volume of published cancer research is proportional to the public health burden by cancer site. We also explored whether research output is proportional to research funding by cancer site. We hypothesized that cancer research output (total publications and clinical trials) does not reflect the relative mortality for the various disease sites, but is more reflective of available research funding.

METHODS

Identifying Cancers with the Greatest Public Health Burden

Using 2013 statistics from the Canadian Cancer Society and the American Cancer Society, we identified the cancers responsible for the greatest number of deaths in Canada and the United States^2,12. A priori, we excluded non-Hodgkin lymphoma, leukemia, and liver or intrahepatic biliary cancer from our study because those malignancies have multiple subtypes, and we did not feel that our bibliometric search would accurately identify related research output. Thus, the top 10 cancers, by mortality, considered here are lung, colorectal, breast, prostate, pancreatic, bladder, gastroesophageal, melanoma, kidney, and uterine cancers.

Identifying Research Output by Cancer Disease Site

Using the mesh term for the particular cancer (that is, “prostate neoplasm,” “bladder neoplasm,” “breast neoplasm,” and so on), the ovid medline database (1 January through 31 December 2013) was searched for all matching journal articles. The search was limited to English articles describing human studies in adults 19 years of age and older. Because the mortality figures used to identify the disease sites of interest came from Canada and the United States, we included only those publications in which the first author’s institutional affiliation was located in those countries. We identified clinical trials by filtering the publications by type, limiting the search to “all clinical trials (at all stages).”

Identifying Research Investment by Cancer Site

To quantify total cancer research investment in Canada, we used data from the Canadian Cancer Research Alliance (ccra). The ccra is an alliance of 41 organizations that collectively fund nearly all cancer research in Canada. The 2011 ccra report¹³ described how CA$548.3 million was invested in cancer research. Of the total funding, CA$282.4 million was attributable to specific cancer sites. The report did not include institution-specific funding from hospital foundations, research supported by private foundations, or industry-funded research. We were unable to find comparable data for all cancer research funding in the United States.

Statistical Analysis

Cancer mortality, research output, and research funding are described as the proportions of deaths, publications, and research investment related to each of the 10 cancer sites of interest. The relationships between research output (proportions of publications and clinical trials), cancer mortality, and research funding were assessed using the Pearson correlation coefficient and linear regression. A p value less than 0.05 was considered statistically significant. All analyses were performed using the SAS software application (version 9.3: SAS Institute, Cary, NC, U.S.A.).

RESULTS

Publications and Clinical Trials by Cancer Site

In 2013, 443,438 cancer deaths in Canada and the United States were attributed to the 10 cancers of interest; those cancers accounted for 68% all cancer deaths. We identified a total of 19,361 publications (including 2661 clinical trials) that were related to the 10 cancer sites and were published in 2013. Of those publications, 4999 (25.8%), including 841 clinical trials (31.6%), were led by authors from Canada and the United States and were therefore used in the subsequent analyses.

Table i shows the proportions of deaths, publications, and clinical trials attributable to each of the 10 cancer sites. The proportions of publications and clinical trials were substantially lower than the proportions of deaths for lung, colorectal, pancreatic, and gastroesophageal cancers. Conversely, the proportions of research output were substantially greater than the proportions of deaths for breast cancer and prostate cancer. Kidney, melanoma, and uterine cancer also had higher proportions of publications and clinical trials relative to total mortality.

TABLE I.

Proportional mortality, research publications, and published clinical trials for common cancers in Canada and the United States in 2013

Cancer site	Mortality (deaths)			Publications (articles)			Published clinical trials (trials)
	Rank	(n)	(%)	Rank	(n)	(%)	Rank	(n)	(%)
Lung	1	179,691	40.52	2	769	15.38	3	135	16.05
Colorectal	2	60,021	13.54	5	346	6.92	6	48	5.71
Breast	3	45,127	10.18	1	1461	29.23	1	255	30.32
Pancreas	4	42,793	9.65	6	339	6.78	7	42	4.99
Prostate	5	33,660	7.59	3	756	15.12	2	145	17.24
Gastroesophageal	6	29,791	6.72	8	257	5.14	9	28	3.33
Bladder	7	17,321	3.91	9	198	3.96	8	32	3.80
Kidney	8	15,433	3.48	4	389	7.78	5	57	6.78
Melanoma	9	10,508	2.37	7	278	5.56	4	71	8.44
Uterine	10	9,093	2.05	10	206	4.12	9	28	3.33
TOTAL		443,438	100		4999	100		841	100

As Figure 1 shows, the correlation between proportional mortality and total publications was not statistically significant (r = 0.363, p = 0.303). Similarly, we observed no statistically significant correlation between proportional mortality and published clinical trials (r = 0.340, p = 0.337).

FIGURE 1.

Relationship of 2013 proportional cancer mortality in Canada and United States with (A) total publications, and (B) published clinical trials.

Research Funding and Research Output by Cancer Site

Table ii shows proportional mortality and cancer research funding investment in Canada. In 2011, more than CA$177 million was invested in cancer research for the disease sites of interest. As Table ii shows, the proportional allocation of research funding by cancer site differed substantially from the proportional mortality in many instances. Combined, breast and prostate cancer mortality represents 17% of cancer deaths in Canada, but received more than 61% of the total funding allocated. The total research investment divided by total mortality in Canada (CA$/death) provides another metric for relative funding levels. As Table ii shows, the range across the cancer sites was considerable. For each Canadian breast cancer patient who died in 2013, CA$14,329.23 of research funding was allocated in 2011; for each bladder cancer death, just CA$589.91 was invested.

TABLE II.

Proportional mortality and research funding for common cancers in Canada in 2013^a

Cancer	Mortality (deaths)			Research investment
	Rank	(n)	(%)	Rank	CA$ total	(%)	CA$ per death
Lung	1	20,211	38.75	3	21,015,333	11.86	1,039.80
Colorectal	2	9,191	17.62	4	20,195,168	11.40	2,197.28
Breast	3	5,097	9.77	1	73,036,061	41.23	14,329.23
Pancreas	4	4,333	8.31	5	10,775,328	6.08	2,486.81
Prostate	5	3,940	7.55	2	36,297,058	20.49	9,212.45
Gastroesophageal	6	3,591	6.88	8	3,444,800	1.94	959.29
Bladder	7	2,111	4.05	10	1,264,303	0.71	598.91
Kidney	8	1,753	3.36	7	3,957,470	2.23	2,257.54
Melanoma	9	1,028	1.97	6	5,391,590	3.04	5,244.74
Uterine	10	903	1.73	9	1,776,151	1.00	1,966.94
TOTAL		52,158	100		177,153,261	100	3,396.47

^aMortality figures are from 2013, research data are from 2011.

Figure 2 shows the association between Canadian cancer research funding in 2011 and Canadian research output in 2013. A significant correlation of the relative level of research funding with total publications (r = 0.894, p < 0.001) and with published clinical trials (r = 0.926, p < 0.001) was observed.

FIGURE 2.

Relationship of Canadian cancer research funding in 2011 with (A) total Canadian publications in 2013, and (B) Canadian clinical trials published in 2013.

Evaluation of the associations between Canadian cancer research funding, research output, and cancer mortality produced these results: cancer mortality with research output, r = 0.289, p = 0.418; cancer mortality with research funding, r = 0.236, p = 0.510; and research funding with research output, r = 0.894, p < 0.001. In an exploratory analysis, we found a linear relationship between cancer mortality and research funding for cancer sites other than breast and prostate (r = 0.868, p = 0.005), suggesting that the latter two diseases received substantially more research funding than would be proportional to their mortality share (Figure 3).

FIGURE 3.

Relationship between Canadian cancer research funding in 2011 and mortality. Breast and prostate cancer are excluded from the linear regression.

CONCLUSIONS

Here, we evaluated whether clinical cancer research output (total publications and number of clinical trials) is proportional to the public health burden (measured by mortality) for 10 major cancers in Canada and the United States. We also explored the relationship between research funding and research output.

Several important findings emerged. First, the data demonstrated that cancer research output (measured as total publications and clinical trials) is not proportional to cancer mortality. In fact, the range in relative research output (% mortality / % publications) observed for the cancers assessed in our study is very large, with differences reaching as high as a factor of 8 in proportional output (breast cancer vs. lung cancer). Second, we observed a substantial range in the level of research funding, which was not proportional to relative mortality for the various cancers, with the largest difference in proportional funding reaching a factor of 28 (breast cancer vs. bladder cancer). Finally, we observed a strong correlation between research funding and research output (total publications and clinical trials). Although our study does not establish a causal relationship, those data suggest a significant disconnect between the public health burden from various cancers and the relative research intensity.

Two other studies have investigated whether research output within oncology is proportional to disease burden. In 2007, Glynn et al.⁶ performed a bibliometric analysis of all publications for the 26 most common cancers. Oncology was disproportionately represented in high-impact journals: 26 neoplasms accounted for 25% of the total output in the top 20 medical publications. Five cancers (breast, prostate, lung, intestinal, and leukemia) dominated the output of the top oncology journals. In their analysis of the publication:incidence ratio, those authors found that leukemia and cancers of the liver and central nervous system were overrepresented. Dear et al.⁷ evaluated recruitment to clinical trials in Australia and New Zealand in relation to the estimated disease burden of each cancer. Those authors found that the 368 clinical trials that were actively recruiting in 2009 underrepresented several cancers with a high disease burden, most notably lung, colorectal, prostate, and pancreatic cancers. Leukemia and lymphoma were found to be overrepresented relative to their disability-adjusted life-years.

The payback model described by Hanney et al.⁵ provides a framework to evaluate the potential societal benefits of clinical research. That framework encompasses 5 core research outcomes: knowledge production, research targeting and capacity, informing policy, health and health sector benefits, and economic benefits. In the present study, we measured scientific publications and clinical trials, which are one marker related to the knowledge production domain of payback. Although the volume of scientific publications is often used as a metric for research productivity, the extent to which volume correlates with improvements in human health is not known. Furthermore, the use of a journal’s impact factor as an indicator of research quality has critical limitations^4,14,15.

Several studies have described cancer research funding relative to disease burden. Early data from the ccra described 2003 cancer research funding relative to 2005 incidence and mortality figures¹⁰. That early view of the data showed that, proportionally, investment in lung, colorectal, and prostate cancers was substantially lower than investment in breast cancer; leukemia also received a disproportionate share of research funding. The U.K. National Cancer Research Institute saw similar funding patterns in a parallel report⁸. In a separate analysis, Burnet and colleagues evaluated the Institute’s research spending relative to cancer mortality and years of life lost⁹. When adjusting for years of life lost, they found that breast cancer and leukemia received a relatively higher degree of funding and that lung cancer was underfunded. They modelled disease burden using years of life lost and mortality, and concluded that both measures provide valuable insight into the population burden of cancer and should be considered in the allocation of cancer research funding. Building on that work, Carter and Nguyen evaluated funding allocations by the U.S. National Cancer Institute¹¹. They found considerable mismatch between funding levels and disease burden. Breast cancer, prostate cancer, and leukemia were funded at levels higher than their relative burden, while bladder, esophagus, liver, oral, pancreatic, and uterine cancers were relatively underfunded.

In 2012, the International Cancer Research Partnership reported cancer research funding during 2005–2008 for 7 countries (the United States, Canada, the United Kingdom, Australia, France, Japan, and the Netherlands)¹⁶. Although the research funding was not adjusted for relative disease burden, the overall level of funding reported for Canada was proportionally consistent with the 2012 ccra report. For example, data from the International Cancer Research Partnership show that, in 2008, funding for breast cancer was triple that for lung cancer and greater by a factor of 26 than that for bladder cancer. In our study, breast and prostate cancer funding were outliers, with the research funding received for those two cancer sites being disproportional to the associated mortality. Breast cancer and prostate cancer received $73,036,061 and $36,297,058 in research funding respectively—about 8 times and 4 times the amount that would be proportional to their corresponding cancer mortality.

There are important limitations to our study. First, we sampled only 1 year of data. Research output, funding, and cancer mortality probably vary over time; however, we feel that data from 2013 are representative of the current research environment. Second, given the retrospective nature of the study, we can show only correlation, not causation. Although the correlation between funding and research output is strong, it could be that the allocated funding reflects prior publication efforts. Third, our publication results are restricted to Canada and the United States, and therefore our results might not be generalizable to other parts of the world. Our analysis of research funding and scientific output was restricted to Canada because no comprehensive data source for cancer research funding in the United States could be located. However, in an exploratory analysis, we found that 2011 cancer site–specific research funding by the U.S. National Cancer Institute in 2011¹⁷ was highly correlated with Canadian ccra funding in 2011 (r = 0.952, p < 0.010). Fourth, we used mortality as a marker for burden of disease. “Potential years of life lost” is also useful marker of health burden because it gives weight to life lost among younger individuals. Finally, as described earlier, the volume of scientific publications and published clinical trials is only a single component of the payback model of clinical research and therefore does not provide a comprehensive perspective on the net benefit of health research.

Within the current biomedical research environment, resources are becoming increasingly scarce. Distributive justice is one framework for resource allocation. Within that construct, funding for cancer research might be expected to be allocated proportionally based on the burden of disease. Our data suggest that such an allocation is not in fact the case.

The objective of the present study was not to identify an optimal model for resource allocation within cancer research; rather, we sought to generate data to inform future discussions about how limited research resources might best be deployed to achieve the greatest benefit for the greatest number of patients with cancer. Priority-setting within oncology is known to be a complex process subject to external factors, including the influence of media, patient advocates, politicians, and industry¹⁸. We believe that the substantial range in research investment across cancer sites (that is, from CA$599/death in bladder cancer to CA$14,329/death in breast cancer) is important and speaks to the need for discussion within the oncology community about whether those allocations represent optimal and equitable use of resources. On an encouraging note, our data suggest that research funders can expect a proportional return on investment—that is, greater investment is associated with greater research output. To summarize, our study suggests that research output is not well correlated with overall cancer mortality, but is strongly correlated with the level of research funding.

CONFLICT OF INTEREST DISCLOSURES

We have read and understood Current Oncology’s policy on disclosing conflicts of interest, and we declare that we have none.