Evaluation of Health Research: Measuring Costs and Socioeconomic Effects

Abstract

Objectives:

The topic of this work is health research evaluation including basic and clinical medical research, as well as healthcare research. The main objects are to explore possible approaches for valuing research in economic terms and to prepare an analytical model for evaluation of health research using the Swedish context. The study also aims to identify potential effects and their significance, and to provide a basis for discussions about the effects of research investments.

Methods:

The study has reviewed ten articles indicating positive effects, in the form of improved health and economic growth. The study also developed a model applied to Swedish health research.

Results:

The review indicates that positive effects, in the form of improved health and economic growth, have a value that greatly exceeds the costs of the research investments. The tentative model applied to Swedish health research also indicates predominantly positive returns, but in a lower range than the review would imply. Methodological problems, however, entail major uncertainty in the cited results.

Conclusions:

Accurate determination of the economic value of research would require significantly better basic data and better knowledge of relationships between research, implementation of new knowledge, and health effects. Information in support of decisions about future allocation of research resources is preferably produced by a combination of general analyses and strategically selected case studies.

INTRODUCTION

The issue of increased healthcare demands is well known in connection with the demographic transitions of health problems such as from younger to older generations and from communicable to non-communicable diseases. Healthcare system is burdened by a rise in infectious, lifestyle and chronic degenerative diseases with elevated threats of environmental health hazards. New and hard-to-treat viruses and microbes such as HIV and Methicillin-Resistant Staphylococcus aureus (MRSA) have already challenged the research community and old public health threats such as tuberculosis reappeared with changes in living conditions. Diseases of which we have little knowledge at the local level are spreading throughout the world, and may require further readiness in the research community for new challenges. As a whole, the situations demand a strong academic knowledge and context dependent research readiness in relation to geographical and cultural issues. However, research funding should not be viewed solely as a means of averting health threats; it must also be considered as an intervention that has both positive and negative health effects as well as other societal effects.

The main purpose of health research, covering basic and clinical traditional medical research, is to improve the health of the general population in the form of better quality of life and increased longevity. More health research leads to better health, which eventually results in productivity gains. Such effects can be estimated in research evaluations. Evaluation of health research is getting more and more important as such evaluations should not only provide information to the sceptic policy makers, but also to the research community itself. Many questions arise in this context: Is increased research funding justified from a national perspective? Will the investment be covered by the potential future return in the form of improved health and greater economic growth? Will the healthcare provided be of significantly higher quality in those countries that proportionally invest the most in relation to their economies? How much is it reasonable for a small country to contribute to the collective global knowledge base?

There is lack of models for accurate evaluation of all significant socio-economic effects pertaining to health research. The aims of the current work are to explore possible approaches for valuing research and to prepare an analytical model for evaluation of health research using the Swedish context. The analysis is an attempted to clearly identify potential effects and estimate their significance. The paper is intended to provide a basis for discussing how the effects of research investments can be valued and which effects should be included.

REVIEW OF THE HEALTH RESEARCH EVALUATION LITERATURE

A literature search for health research evaluation studies (1985-Sept 2009) was performed using Pubmed. The following sets of keywords were used in different combinations: (in the title) clinical/medical/biomedical/health research; impact, value, return/s, benefits, gains; funding, economy, economics and (in all fields) evaluation, evaluating, assessment/s, assessing, measuring and estimating. A language restriction was made to documents in English and Swedish.

The Pubmed search yielded 491 references. The rather broad search resulted in 375 non-relevant titles. For the remaining 116 relevant titles, abstracts were read and 49 articles that had analysed or discussed effects of health related research were retrieved. Documents from an international workshop at Sigtuna, Sweden, 2007 and 2009 were also added and reference lists were screened for references relevant to the current study. Final inclusion criteria were: original studies that investigated societal effects of health research including broader societal benefits, health benefits and/or survival effects. The final screening yielded 10 articles which met these criteria and were included into the current review (Table 1).

Table 1.

Categorization of the literature

Different Approaches for Valuing Health-Related Research

The review of the selected ten articles covers different approaches for valuing medical research. Broadly, the approaches are: Disease- and technology-specific case studies, Case studies of specific research funding, General socio-economic approaches, and combinations of these three approaches. According to this categorization, the articles are presented in table 2.

Table 2.

Studies in the review divided into different categories

Disease- and technology-specific case studies

In this approach, the magnitude of the healthcare investments in the studied diseases is compared to the gains achieved in terms of quality of life, survival, and any savings in terms of treatment costs. Another related approach is to track significant medical innovations to see the impact they have had on health improvements, and what research and development costs they have entailed.

Case studies of specific research funding

It is becoming increasingly common for research financiers to want to evaluate what is achieved as a result of the allocated funding. Analyses of the health economic and socio-economic effects are also increasingly sought after. Many studies consequently consist of case studies of research projects in which attempts are made to find links between the financed projects and health gains in society. These studies are also often targeted to specific disease groups.

General socio-economic approach

Health gains in society can have many causes above and beyond the triumphs of medical research, such as improved diet and living conditions, safer work environments and healthier lifestyles. It is not the case that every health gain can be attributed to health-related research; on the other hand, though, medical research also produces non-medical social effects such as increased productivity, greater competitiveness and economic growth. Health-related research thus contributes indirectly to a country's GDP.

In absence of some vital information it is difficult to calculate the net value of the social effects.

Studies that take a completely economic approach must consequently be based on rough estimates. The authors of such studies are cautious in drawing their conclusions, and are more inclined toward discussing the reasonability of the values they derive. Still, the analyses in these types of studies consistently indicate that the socio-economic benefits broadly exceed the research costs. Those studies also admit that the estimate of increased longevity and changes in medical expenditures include contributions from other sources, but argues that this has a marginal influence on the calculated values.

A Combined Approach

One study has used a combined approach for valuing health research. In the first instance it analysed the health gain due to increases in healthcare expenditures. Next, it analysed the value of advances in treatments for specific diseases such as heart attack, stroke, type-2 diabetes, and breast cancer. Finally, it calculated the value of specific innovations using published cost-effectiveness data.

ANALYTICAL MODEL FOR SWEDISH HEALTH RESEARCH

The above review indicates that studies of individual research projects and individual innovations in a country must be complemented by studies conducted from a broad socio-economic perspective which reduces the risk of double-counting and makes it possible to include the positive effects that research has in terms of growth and innovation in a society.

The suggested analytical model identifies relevant investments and effects. We have calculated a monetary value wherever possible, and provide a qualitative description in other cases. The evaluated investment (intervention) pertains to Sweden's total current investment in funding health-related research. The benefits of this intervention can thus be compared directly to the costs in order to obtain some indication of the net effect.

In the analysis, the “research financing” intervention refers to the total of private, municipal, and state expenditures for clinical research and associated health-related research conducted in Sweden. The size of the investment is calculated as the sum of the 2005 research expenditures in Sweden in the areas of healthcare, medicine, physiology, and pharmacology. The effects of the intervention are estimated for the year, 2015, based on forecasts for a ten-year period. Costs and effects arise at different times and hence monetary sums (where the input values are expressed in current prices) are converted to 2005 levels. Further discounting of future costs and effects has not been performed.

Health-related research has an effect on every citizen but the effects impact different parts of the societal structure differently. The benefits to the citizen in general come mainly in the form of improvements in quality of life and additional life-years, while the benefits to the economy may consist of higher net exports or increased sales, which in turn can contribute to corporate profits. Locally, at the individual municipality level, it may be in the form of spin-off effects in the business community. Certain effects of Swedish research will naturally extend beyond the country's borders, while there is at the same time an influx of effects from the outside world. To study the effects of Swedish research funding on Swedish society, we must thus estimate how great a share of the effects can be attributed to investments made domestically, as we can include only that part of the effects which can be ascribed to the studied intervention.

In many studies of the societal effects of research, the choice is made to study the current effects of a resource investment that was made a number of years earlier. On the other hand, we have chosen an ex-ante approach; that is, an estimation of what we believe can be achieved in the future as a result of the research being done today. We do this partly because, historically speaking, changes in the medical field have been sweeping and revolutionary. New therapies and new health threats are emerging constantly. An ex-ante approach enables us to incorporate recent historical trends in the analysis, and to take into account current changes of which we are aware. Furthermore, our analysis is intended to shed light on decisions about investments in health research. Decision-makers may therefore have an interest in a forecast, and in the possibility of influencing a future outcome. We have consequently estimated future effects based on the effects observed in modern times, transferred to a future society that has evolved in a manner consistent with current trends.

Identification of Relevant Costs and Effects

First on the cost side are the financial resources made available by various research funders. Other costs arise as a result of these investments, such as unremunerated research work and input from the people who participate in clinical studies. Research can also lead to more expensive treatments, troublesome side effects, and environmental impacts. Costs also arise whenever new methods are integrated into healthcare; there are costs associated with breaking with ingrained behaviours, and reorganisations that temporarily lower productivity may be necessary. Finally, medical innovations are often added to pre-existing methods, leading to greater complexity that may have negative effects.

The financial benefits from an investment in health-related research consist largely in the avoidance of certain costs associated with illness. The social costs of illness include the costs of diagnosis, treatment, recovery, and disease prevention; these costs can be kept down by expanding our knowledge of various diseases, and by introducing new and more effective care methods. Disease prevention simultaneously produces social and emotional gains in terms of improved health, improved quality of life and additional life-years. Process improvements also occur in healthcare as a result of greater knowledge and better medical methods. Such improvements can appear in the form of, for instance, shorter treatment times, less discomfort, and improvements in the work environment.

Two different types of production gains occur in the economy; the avoidance of production losses as the result of a healthier population¹¹,¹² and increased production¹³,¹⁴ in the form of higher employment in various sectors of the economy, including healthcare and research. Increases in productivity can also arise as a result of the generally greater impetus to innovate that derives from new research findings, and as a result of the fostering of positive attitudes toward change that emerges from the encounter between research and healthcare.

Finally, research also has repercussions in the research world, in that the general body of knowledge grows; and this should be valued as an investment in future research and the development of innovations.¹¹,¹⁴ Salter and Martin assert that a country's research activities constitute an “admission ticket” to sharing in the global knowledge base.¹⁴ The conversion of research findings into medical innovations is becoming an increasingly international process, and a high-quality domestic research base is a prerequisite for establishing enterprises that can participate in global R & D projects intended to generate knowledge and create technologies in the medical field.

Once the relevant costs and benefits have been identified, they must be measured and valued. A large part of the effects results from research conducted by research institutions outside Sweden. According to statistics from the Global Innovation Scoreboard, Sweden incurs 1.42% of the total worldwide R & D expenditure.¹⁵ An analysis of medical publications from various countries indicates that Sweden's share of health-related research is actually larger, at 1.93%.¹⁶ Our impact on medical advancements is presumably even greater, given that Sweden have a relatively large domestic pharmaceutical industry. Furthermore, we believe that the local effects are relatively greater again, as regional R & D investments¹⁷ and industry exert a local influence that makes it more likely that research results will be implemented and converted into practical healthcare. We have consequently assumed that Swedish R & D contributes 3% of the total effects of health-related research on Swedish healthcare.

In addition, a large part of the health gains achieved is the result of a generally improved standard of living, and these effects can thus not be ascribed to improvements in medicine. The share accounted for by R & D investments has previously been estimated at 25-67%.²,⁸,¹⁰ An econometric study by Lichtenberg also indicates that the proportion falls within this range; the author concluded that medical innovations accounted for 40% of the increase in life expectancy that occurred between 1982 and 2001, calculated as an average for 52 countries in various stages of economic development.¹

Current researchers have their doubts about these earlier estimates, but some such estimate must nevertheless be made in order to gain an idea of the effects of the research. We have consequently made a rough estimate, and believe that the share accounted for by R & D investments in Sweden could be 50%; that is, somewhere in the middle of the range. This can be motivated by the fact that changes of standard of living cannot be expected to have as great an effect in the future as they have had historically, and by the fact that Sweden has a high level of education. However, it should be noted that the validity of using such a high percentage requires that lifestyle improvements can be attributed to health-related research, for example, advice concerning dietary and exercise habits and lifestyle coaching that can be expected to have effects in the future.

The lists of relevant costs and benefits include items of varying power, and some items cannot be valued or even ranked in order of size. We believe that there are nevertheless advantages to show that these effects exist. The fact that certain effects are reciprocally related is also problematic. For instance, an increase in “negative side effects” entails that the effect of “improved health, quality of life, and lifespan” will be lower. For informational purposes, we have chosen to list reciprocally related effects on both the costs and benefits sides.

Valuing Costs and Benefits

In the current study, costs and effects are viewed as either costs or benefits, measured in Swedish kronor [1 USD = 7.86 Swedish Kronor (SEK), 1 July 2005]. In reality, effects may be economic, physical, social, and/or emotional, and may manifest as both costs and benefits (Table 3 and 4). However, our review shows that researchers generally are more interested in the benefits and may ignore some negative effects.

Table 3.

Identified costs and benefits

Table 4.

Overview of the valuation of costs and benefits; the analysis is meant as a template for further development in evaluation of societal net benefit of health related research. Developments are needed in valuation methods of both costs and benefits.

1. Research funding is estimated at SEK 21 600 million (approximately USD 2750 million). This figure is based on statistics from Statistics Sweden regarding total 2005 expenditures for research in the areas of healthcare, medicine, physiology, and pharmacology reported by the following entities: municipalities, county councils, and regional R&D units (SEK 3021 million); the college and university sector (SEK 2991 million); national government agencies (SEK 164 million); companies (SEK 14 218 million); and the private not-for-profit sector (SEK1211 million). The expenditures include both operating expenses and investment costs. One can only speculate as to how great a share of these costs can be defined as clinical research. We have consequently chosen to combine several activities under the collective term health-related research, and to report the costs as well as the effects for this area.

   Unreported work and input on the part of doctors and clinical staff should also be included in this item. Research work is often performed during time that is compensated within the county council healthcare budget. It is difficult to find any data regarding such costs, but one study of input in clinical trials in the USA¹⁸ indicates that these costs are significant, and that the research work is only partly compensated.

2. Voluntary input (research subjects who participate voluntarily at free of charge) is estimated at SEK 570 million (approximately USD 73 million). The costs associated with voluntary research subjects are calculated as the number of days of participation multiplied by the value of a lost workday at 2005 prices. The number of lost workdays is calculated as half of the estimated number of days that volunteer research subjects contributed in 2005. This item also includes costs for individuals who perform unpaid research work during their spare time, but naturally no documentation regarding this is available.

3. More expensive healthcare due to new methods is estimated at SEK 60 million (approximately USD 8 million). The Swedish Association of Local Authorities and Regions (SKL) estimates that medical/technical advances are expected to increase resource demand by roughly 0.8% per year. Given the forecast that the total resource demand increases at a rate of 1.6% per year, the increase from 2014 to 2015 will be SEK 4107 million (in 2005 kronor).¹⁹,²⁰ If we presume that medical/technical advances stands for half of this increase (1.6/0.8) and that Sweden's contribution to such advances amounts to 3%, this would yield a cost of roughly SEK 60 million that can be ascribed to research conducted in Sweden. The rest of the increase in resource demand stems from (i) demographic effects, (ii) higher standard of living and an associated higher willingness to pay, and (iii) foreign R & D.

4. Negative side effects from treatments. Side effects entail suffering, death, and higher costs for the healthcare system and society. The risk of negative effects is high when innovations are initially put into use, before their long-term benefits have been assessed. However, this effect is difficult to differentiate from others. Suffering and death lead to an increased value of the item “Improved health” on the benefits side, and the costs are also included in the increased cost of healthcare.

5. Environmental impacts (emissions). A 2004 report from the Swedish Medical Products Agency²¹ states that the main pharmaceutical substances which have an extensive impact on the aquatic environment are oestradiol and ethinyl-oestradiol. Subsequent reports have also sounded the alarm regarding high emissions of psychotropic drugs, antibiotics, and blood pressure-lowering medications. Changes in natural capital are difficult to calculate, making it hard to valuate environmental impacts. It is, however, important to be aware that environmental effects can have long-term implications for both health and the economy; and an estimation of environmental impacts should be included in any forecast of research effects.

6. Implementation costs. New methods that necessitate reorganisation can temporarily lower productivity. Costs can also arise from the efforts which are necessary to break ingrained behaviours whenever innovations are to be integrated into the healthcare process. Innovations usually receive a positive reception in healthcare, but the addition of technologies entails increased on-the-job stress, heightens the risk of improper treatment, and leads to higher maintenance and administrative costs.²² A cleanout of technologies should thus occur in parallel with the implementation of innovations. The effects included in this item are expressed as a poorer result in process improvements and more expensive healthcare. All benefits represent estimated future effects thought to occur during 2015.

7. Improved health in the form of higher quality of life and increased lifespan is estimated at SEK 14 000 million (approximately USD 1780 million). Various approaches have been taken in the literature in attempts to quantify improved health and a lowered risk of illness.¹¹ We have chosen to express the value of this as the total value of the quality adjusted life-years (QALYs) gained within the population. The QALY is a composite metric for increased lifespan and perceived health. The monetary value of one QALY has been set at SEK 500,000 (approximately USD 63,600) in the analysis. This is an approximate value, but numerous studies have used values of the same order of magnitude or higher, based on the value of one statistical life.²,¹⁰

   The number of life-years gained is calculated for both men and women as the average annual increase in life expectancy multiplied by the forecast number of inhabitants in 2015. We further assume that the quality of life during the added years needs, on average, to be adjusted downward to 0.75 QALY per life-year gained,²⁴ as the life-years added during the final stages of life cannot be expected to be spent free of health problems. The value of an increase in life expectancy is calculated at SEK 7,057 million. This value has been adjusted as follows: half of the health gains are assumed to be a result of R & D, and the portion that can be ascribed to Swedish investments is estimated at 3%, as described above.

   A general improvement in health must also be assumed, given that lifespan is increased. Register data ascertained from measurements of self-perceived quality of life (1980/81 to 1996/97) indicate that morbidity has decreased in many areas, and people are staying active well into old age.²³ However, a more recent population study from Stockholm shows conversely that self-perceived quality of life has declined (1998 to 2002).²⁴ This perceived poorer health status can be explained by changes in perceptions of what constitutes sickness and health, and an increase in lifestyle- and stress-related illnesses. It may therefore still be reasonable to assume that, in parallel with increased lifespan, a general elevation of health status is occurring with regard to those diseases/conditions where medical ground is being gained, and where this ground has been gained as a result of health-related research. However, “net health” is not improving, as new health threats are making their presence felt. It is not possible for us to gauge the magnitude of this health gain at the present juncture, but we do believe that it should be at least as great as the value of extended survival.

8. Lower costs for diagnosing, treating, and recovering from disease. The costs associated with certain diseases are decreasing because of improved health. The benefits cannot be distinguished from the cost item “More expensive healthcare,” which could be changed accordingly. However, it has become possible to prevent many disease conditions that had previously cost society heavily. One method of identifying costs avoided in this way is to calculate how much specific disease conditions had formerly cost and then compare the result to the current costs, but drawing conclusions about the entire healthcare system on this basis would require a comprehensive and representative selection.

   Based on the foregoing, it may seem self-evident that research has a major impact in terms of reducing the costs of diagnosing, treating, and recovering from disease. However, the resources that are freed up are usually replaced with other costs, as society's willingness to pay for healthcare does not decrease; if a given disease condition requires fewer resources, we will instead invest more in some other area. This, in turn, presumably has positive effects such as improved health, quality of life, and lifespan, but the results are not discernable as savings in healthcare costs.

9. Productivity and process improvements. It is difficult to differentiate and value the significance of research in terms of the possibilities of doing healthcare work better and more efficiently. One conceivable method for gaining some indication of the contributions that research makes would be to track individual research investments that are directly intended to, for instance, facilitate healthcare work, reduce treatment times, improve patient comfort, or otherwise bring about improvements in the work environment.

10. Lower production losses due to a healthier workforce (reduced absenteeism due to illness). The value of this is viewed as the increase in production that occurs because people are able to work more. We are living longer and may spend more active years in working life, and we are also getting healthier and consequently do not need to stay home from work to the same extent as was formerly the case. We have chosen to incorporate this effect into a general production increase that generates economic growth (see below), but the question of whether this depiction of a healthier workforce is realistic may be open to discussion. For instance, to date we have not seen a decrease in absenteeism due to illness corresponding to the decline we have seen in mortality figures. We must also note here that absenteeism statistics are difficult to interpret, as the numbers are strongly influenced by the effects of policy factors and economic cycles.

11. Economic growth is estimated at SEK 10 000 million (approximately USD 1270 million). Research investments lead to economic growth via a two-step process.¹³,²⁵,²⁶ In the first step, growth is driven directly, as research activity increases. In the second step, growth is driven by innovations that confer competitive advantages and create goods and services, and hence increase GDP. Many authors currently accept a growth model in which the growth in a region is viewed largely as a result of research investments. Productivity and production increases depend in large measure on the force of innovation in the business community, which depends in turn on knowledge production and research breakthroughs.¹¹,¹⁴,²⁸ However, in order for increased research resources to lead to growth, a society must also have in place the structures needed to refine and apply the knowledge that is generated; that is, a well-established industry and highly educated human resources.¹³,²⁵,²⁷,²⁸

    GDP is defined as the value of all the goods and services produced in a country. During the 2000s, Sweden's GDP increased by roughly 3% each year.²⁹ Calculated conservatively, the increase in GDP from 2014 to 2015 will thus have a value of SEK 100,000 million (expressed in 2005 kronor). If we assume that half of this increase will be attributable to the effects of research, we arrive at a figure of SEK 50 000 million. We know that the cost of health-related research accounts for about 20% of Sweden's total research expenditures. This leaves SEK 10 000 million (100′000 × 50% × 20%) as the result of health-related research, which corresponds to nearly half the research investments made (cost item 1).

12. Expanded knowledge base (which can lead to future innovations). Some authors have noted that an expanded knowledge base can have significant value.¹⁴ Research always has a value in terms of the expertise that is amassed, for instance via post-graduate education, but its value can also be seen as an investment in the ability to develop future innovations. However, here we encounter yet another problem in terms of metrics. New findings are added to our “knowledge capital” at the same time as knowledge is being “taken out” and used, presumably to equivalent degrees. Still, something has been produced, and if it is administered properly it will have major value, not least in the form of entrée to the global research society and the consequent access to a global knowledge base. We have consequently chosen to list this as a separate item in our analysis, but without venturing a guess as to its value.

DISCUSSION AND CONCLUSION

Our analytical model represents an attempt to cover all the relevant effects in a way that has not been done before. In valuing the effects of research we have, however, encountered a number of metric-related problems, and there are many effects that can be neither quantified nor valued monetarily. A summary of all the costs and benefits shows that the estimated future socio-economic benefits exceed the socio-economic costs for the estimated values. The calculated return on investment is 1.08, but it is not currently possible to draw any conclusions in this regard, as certain effects cannot be valued in an acceptable way (See table 4).

The above analysis requires further refinement. As mentioned earlier, the current analysis has failed to consider the environmental effects. It would further be of interest to measure the distribution of research impacts on different parts of the societal structure and to compare the effects of current research funding and a hypothetical situation in which the research funding had been altered (reduced, increased or redistributed). Furthermore, to be able to conduct reliable analyses of societal effects of health research, it is necessary to increase the overall knowledge through combined analysis of case studies and broader socioeconomic studies.

It is easy to find examples of research based innovations that have yielded manifold returns. Many have tried to demonstrate the importance of research by calculating the value generated through the use of such innovations, while failing to take the research and development costs into account. Furthermore, a pro-innovation bias is readily evident in the studied literature. The authors have accepted in advance that research and innovation are profitable, and have then either simply described the positive effects, or described the costs and effects for a number of successful medical technologies in relation to older alternatives.

Many studies attempted to describe the value of individual research investments, while few have tried to estimate the scope of the entire research investment and consider it in relation to health improvements and economic growth. The problem with case studies is that it is difficult to make calculations for a large enough number of different disease conditions to enable us to draw conclusions about the entire healthcare system, and overlapping effects often occur, with the result that the effect is overestimated. Using case studies as a template for prioritising research is also problematic, as it only later becomes apparent which research investments yielded the greatest returns. If we consider the relationship between broad-based and cutting-edge research, it is likely that both serve a function in the “research factory,” and that one cannot exist without the other.

To estimate the value of research investments, researcher must be able to isolate the effects that would not have arisen had the investment not been made. This can be difficult, and opinions will vary as to what does or does not constitute a research effect. For instance, Siegler et al. Do not believe that altered living habits, such as decreased tobacco use, constitute an effect of research.³¹ There is, however, reason to presume that altered behaviours are, in part, an effect of applied research results, although the problem of determining just how much of the effect can be ascribed to research still remains.

Another problem is that of assessing which research investments have contributed to a given medical advance. How much has basic research contributed? Which basic research has contributed, and to what extent should it be factored in? In addition, research in a given field of knowledge is usually conducted in parallel in a number of countries. How then is one's own country's input to be valued? This is a line of reasoning that most of the studies overlook. Of the studies we have cited, only three incorporated the value of foreign research in their analysis; on the other hand, they did not include all costs of earlier research, an omission that would reasonably lead to an overestimation of the return on the research in question.⁴,⁶,⁹ Murphy and Topel factored in all medical research in the USA, but omitted research “spill-overs” from other countries.⁸ There are, however, studies that indicate that foreign research does account for a significant share. Eaton and Kortum claimed, for instance, that roughly 50% of the productivity increase in the USA was attributable to technological advances in other countries.³⁰

Finally, there is a methodological dilemma inherent in the fact that we can never draw comparisons between how healthy we feel at present and how healthy the population felt a number of years ago. Historically speaking, we can see increased lifespan and an increase in the number of years spent in good health. It is, however, impossible to determine whether this improved health is perceived as a higher quality of life, as higher expectations and psychological stresses in an increasingly hectic living environment may have “eaten up” those gains.

The purpose of studies of the value of health-related research is (1) to justify the size of research investments and (2) to optimise the return on those investments. The latter ideally entails that forecasts can be made to identify how and where money should be invested to achieve maximum benefit. This is not possible at present, and methods for valuing research need to be developed further. We must be able to establish and measure the health economic effects of research in a reliable manner if we are to be able to determine with some degree of certainty how research money should be invested. We also need to know more about the links and flows between basic, clinical, and applied research. The issue of how research results will be used has drawn attention in recent years. Considerable costs for the treatment of disease could have been avoided had we been able to quickly apply key research findings in practice.⁶,³² One classic example of this is the evidence of a link between smoking and lung cancer that was presented in a preliminary report as far back as 1950,³³ but which has still not achieved its full impact.

Meltzer has asserted that health economics methods could also be used to evaluate individual research projects in advance.³⁴ There is currently a great deal of activity in this area at the international level, and an approach in which the value of additional information is calculated is of great interest in this context. Prospective analyses of the economic value of assessment research have begun to be applied to a limited extent, and have been shown to be capable of providing usable information about the value of research.^35–37 International exchanges are valuable in developing methods of evaluation and prioritisation, but local studies at the national and regional level are also needed to achieve effective local guidance.

Increasing our knowledge of the economic effects of research in a more specific way will require us to combine general analyses at the societal level with case studies. This could apply to studies of general effects in society, as well as the effects of specific research projects financed by research councils or in some other way. Luce et al. Have shown how these effects can be elucidated to a greater extent by combining different approaches.¹⁰ The various approaches used to value research offer different advantages and disadvantages, but uncertainty is a problem that they all share. Continued research in this area should consequently be focused on improving the reliability of the input values used in the calculation models.

The paper concludes that positive effect of clinical research benefits excess costs. However, because of vast methodological problems none of the presented research evaluation approaches are sufficient to obtain confident results. The tentative model applied to Swedish health research indicates that the positive effects are predominant, but that the return is in a lower range than the studied literature would imply. Determination of the economic value of research would primarily require significantly better basic data and better knowledge of relationships between research, implementation of new knowledge, and health effects. For future work, to provide information in support of decisions about future allocation of research resources, we recommend a combination of general analyses and strategically selected case studies.