Public–private interaction in pharmaceutical research

Abstract

We empirically examine interaction between the public and private sectors in pharmaceutical research using qualitative data on the drug discovery process and quantitative data on the incidence of coauthorship between public and private institutions. We find evidence of significant reciprocal interaction, and reject a simple “linear” dichotomous model in which the public sector performs basic research and the private sector exploits it. Linkages to the public sector differ across firms, reflecting variation in internal incentives and policy choices, and the nature of these linkages correlates with their research performance.

The economic case for public funding of scientific and technological research rests on the belief that the private sector has inadequate incentives to invest in basic research (1). This belief in turn rests on the idea that research and development (R&D) can be usefully arrayed along a continuum, with “basic” work, or research that is orientated towards the discovery of fundamental scientific principles at one end, and “applied” work, or research designed to be immediately translated into products and processes at the other. Since basic research is likely to be relevant to a very broad range of fields, to have application over many years, and be useful only when combined with other research, economists have long believed that the returns to basic research may be difficult to appropriate privately.

This perspective is complemented by work in the sociology of science, which suggests that the norms and incentive structures that characterize publicly funded science combine to create a community in which it is much more likely that “good science” will be conducted. Researchers working in the public sector are rewarded as a function of their standing in the broad research community, or according to the “rank hierarchy” of the field (2). Because this standing is a function of priority, the public sector is characterized by the rapid publication of key ideas and a dense network of communication across key researchers that is particularly conducive to the rapid advance of scientific knowledge. Research undertaken in the private sector, in contrast, is believed to be shaped by the need to appropriate private returns from new knowledge, which leads firms to focus on applied research and to attempt to restrict communication of results. Faced with different constraints and incentives, private sector researchers are thus viewed as much less likely to publish their research or to generate basic advances in scientific knowledge (3–5).

In combination, these two perspectives have sustained a consensus that has supported substantial public commitment to basic research for the last 50 years. Nearly one-half of all the research undertaken in the United States, for example, is funded by the public sector, and spending by universities on research increased by over 100% in real terms between 1970 and 1990 (6). However, budgetary concerns are placing increasing pressure on government support for science, and questions about the appropriate level of public funding of research are now being raised on two fronts.

In the first place, it has proven very difficult to estimate the rate of return to publicly funded research with any precision (7). The conceptual problems underlying this exercise are well understood, and although those studies that have been conducted suggest that it may be quite high (8–10), it is still far from clear whether too much or too little public resources are devoted to science. In the second place, questions have been raised about the usefulness of dichotomies drawn between basic and applied and “open” versus “closed” as bases for public funding decisions. There is considerable evidence that private firms invest significantly in basic research (11, 12), while at the same time several observers have suggested that publicly funded researchers have become increasingly interested in the potential for private profit, placing the norms of open science under increasing threat.

In this paper we explore this second issue in the context of pharmaceutical research, as a contribution toward clarifying the nature of the relationship between the public and private sectors. The pharmaceutical industry provides a particularly interesting arena in which to study this issue: health related research is a very substantial portion of the total public research budget, yet some researchers have charged that this investment has yielded very few significant advances in treatment. Between 1970 and 1988, for example, public funding for the National Institutes of Health (NIH) increased more than 200% in real terms, whereas private spending on biomedical research increased over 700%. Yet at the same time the rate of introduction of new drugs remained approximately constant, and there has been little improvement in such critical variables as mortality and morbidity (13).

Prior research has shown that spending on privately funded research is correlated with NIH spending (14), whereas a number of case studies of individual firms have confirmed the importance of an investment in basic research to the activities of private firms (12, 15). Here we draw upon both qualitative evidence about the research process and quantitative data on publication rates and patterns of coauthorship to build a richer understanding of the interaction between public and private institutions in pharmaceutical research.

Our results suggest that public sector research plays an important role in the discovery of new drugs, but that the reality of the interaction between the public and private sectors is much more complex than a simple basic/applied dichotomy would suggest. While in general the public sector does focus more attention on the discovery of basic physiological and biochemical mechanisms, the private sector also invests heavily in such basic research, viewing it as fundamental to the maintenance of a productive research effort. Public and private sector scientists meet as scientific equals, solve problems together, and regard each other as scientific peers, which is reflected in extensive coauthoring of research papers between the public and private sectors. We also find some evidence that this coauthoring activity is correlated with private sector productivity. Publication of results makes the output of public sector research effort freely available, but the ability of the private sectors to access and use this knowledge appears to require a substantial investment in doing “basic science.” To take from the industry’s knowledge base, the private sector must also contribute to it.

Taken together, our results suggest that the conventional picture of public research as providing a straightforward “input” of basic knowledge to downstream, applied private research may be quite misleading, and that any estimation of the returns to publicly funded research must take account of this complexity.

Data and Methods

We gathered both qualitative and quantitative data to examine public–private interaction. We used two sources of data for our qualitative analysis. The first source is narrative histories of the discovery and development of 25 drugs introduced between 1970 and 1995, which were identified as having had the most significant impact on medical treatment by two leading industry experts. Each history was constructed from both primary and secondary sources, and aimed in each case to identify both the critical events and the key players in the discovery of each drug. (We are indebted to Richard Wurtman and Robert Bettiker for their help in constructing these histories.) Our second source of data is a series of detailed field interviews conducted with a number of eminent public sector researchers and with researchers employed at 10 major pharmaceutical firms.

Our primary source of quantitative data is bibliographic information on every paper published in the public literature between 1980 and 1994 by researchers listing their address as 1 of 10 major research-oriented pharmaceutical firms, or 1 of the NIH. This data base was constructed by searching address fields in Institute for Scientific Information’s Science Citation Index. It is important to note that Science Citation Index lists up to six addresses given for each paper, which may not correspond exactly to the number of authors. For these 10 sample firms alone, our working data set contains 35,813 papers, with over 160,000 instances of individual authorship, for which Science Citation Index records 69,329 different addresses. Our focus here is on coauthorship by researchers at different institutions. Clearly, much knowledge is exchanged at arm’s length through reading of the open literature, and in some instances coauthorship may simply be offered as a quid pro quo for supplying reagents or resources, or as a means of settling disputes about priority. Nonetheless, we believe that coauthorship of papers primarily represents evidence of a significant, sustained, and productive interaction between researchers. There are also very substantial practical problems in analyzing citation patterns. We define a “coauthorship” as a listing of more than one address for a paper: a paper with six authors listing Pharmacorp, Pharmacorp, NIH, and Massachusetts Institute of Technology as addresses would generate three such coauthorships. We classified each address according to its type: SELF, university, NIH, public, private, nonprofit, hospital, and a residual category of miscellaneous, so that we were able to develop a complete picture of the coauthoring activity of each firm. Table 1 gives a brief definition of each type.

Table 1.

Definitions of institutional type

Type	Definition
SELF	“COMPANY X” in file obtained by searching SCI for “COMPANY X”
Hospital	Hospitals, clinics, treatment centers
NIH	Any of National Institutes of Health
Public	Government-affiliated organizations, excluding NIH; e.g., National Labs, European Molecular Biology Lab
University	Universities and medical schools
Private	For profit organizations, principally pharmaceutical and biomedical firms
Nonprofit	Nonprofit nongovernment organizations, e.g., Imperial Cancer Research Fund
Miscellaneous	Unclassified

SCI, Science Citation Index. 

These data on publications and coauthorship are supplemented by an extensive data set collected on R&D activity from the internal records of these 10 firms. This data set extends from 1965 to 1990 and includes discovery and development expenditures matched to a variety of measures of output including important patents, Investigational New Drugs, New Drug Approvals, sales, and market share. These data are described in more detail in previous work (16–18). Although for reasons of confidentiality we cannot describe the overall size or nature of the firms, we can say that they cover the range of major R&D-performing pharmaceutical manufacturers and that they include both American and European manufacturers. In aggregate, the firms in our sample account for approximately 28% of United States R&D and sales, and we believe that they are not markedly unrepresentative of the industry in terms of size or of technical and commercial performance.

Qualitative Evidence: Field Interviews and Case Studies

Case Studies.

Table 2 presents a preliminary summary of 15 of our 25 case histories of drug discovery. It should be noted immediately that this is a highly selective and not necessarily representative sample of new drugs introduced since 1970. There is also significant selection induced by the fact that many potentially important drugs arising from more recent discoveries are still in development. Bearing in mind these caveats, a number of conclusions can be drawn from Table 2. First, there is some support for the “linear” model. Publicly funded research appears to have been a critical contributor to the discovery of nearly all of these drugs, in the sense that publicly funded researchers made a majority of the upstream “enabling” breakthroughs, such as identifying the biological activity of new classes of compounds or elucidating fundamental metabolic processes that laid the foundation for the discovery of the new drug. On the other hand, publicly funded research appears to be directly responsible—in the sense that publicly funded researchers isolated, synthesized, or formulated the clinically effective compound, and obtained a patent on it—for the introduction into the marketplace of only 2 of these 15 drugs.

Table 2.

Lags in drug discovery and development

Drug	Date of key enabling scientific discovery	Public?	Date of synthesis of major compound	Public?	Date of market introduction	Lag from enabling discovery to market introduction, yr.
Captopril	1965	Y	1977	N	1981	16
Cimetidine	1948	Y	1975	N	1977	29
Cisplatin	1965	Y	1967	Y	1978	13
Cyclosporin			1972	N	1983
EPO	1950	Y	1985	N	1989	39
Finasteride	1974	Y	1986	N	1992	18
Fluoxetine	1957	Y	1970	N	1987	30
Foscarnet	1924	Y	1978	Y	1991	67
Gemfibrozil		N	1968	N	1981
Lovastatin	1959	Y	1980	N	1987	28
Nifedipine		N	1971	N	1981
Omeprazole	1978	N			1989	11
Ondansetron	1957	Y	1983	N	1991	34
Propranolol	1948	Y	1964	N	1967	19
Sumatriptan	1957	Y	1988	N	1992	35
Basic discoveries:		11 public		2 public
		3 private		12 private

EPO, erythropoeitin; Y, yes; N, no. 

Second, there are very long lags between upstream “enabling discoveries” and downstream applied research. At least for these drugs, the average lag between the discovery of a specific piece of knowledge discovered by the public sector and the identification and clinical development of a new drug appears to be quite long—in the neighborhood of 10–15 years. It seems clear that the returns to public sector research may only be realized after considerable delay, and that much modern publicly funded research has yet to have an impact in the form of new therapeutic agents.

Note also that though this very stark presentation of these case histories lends some support to a linear dichotomized view of the relationship between the public and private sectors, it was also very clear from the (unreported) details of these case histories that the private sector does a considerable amount of basic science and that applied clinical research conducted by the public sector appears to have been at least as important as basic research in the discovery of some new agents.

Field Interviews.

The picture of the linear model in Table 2 was not supported by the findings of our field interviews. The notion that pharmaceutical research is a process in which the public sector funds basic research that is then transferred to a private sector that conducts the necessary applied research to translate it into products was rejected by most of our respondents. These industry experts painted a much more complex picture.

On the one hand, all interviewees reinforced conventional wisdom in stressing how critical publicly funded research was to the success of private research. They gave many examples of historical discoveries that could not have been made without knowledge of publicly funded research results and, although there have as yet been few major breakthroughs in medical treatment as a result of the revolution in molecular biology, contact with the public sector to stay current with the latest advances in cell biology and molecular physiology was viewed as a prerequisite of modern pharmaceutical research.

On the other hand, our respondents stressed the bi-directional, interactive nature of problem solving across the public and private sectors. They described a process in which key individuals, novel ideas, and novel compounds were continually exchanged in a continual process of reciprocal interaction characterized by very high levels of mutual trust. They suggested that the reciprocal nature of this process is partially a function what Cohen and Levinthal (11) have called “investment in absorptive capacity.” Major pharmaceutical firms conduct basic research both so that they can take advantage of work conducted in the public sector and so that they will have something to “trade” with leading edge researchers. Investment in hard-to-appropriate basic research is probably also a function of the need to hire research scientists of the highest possible calibre. Such key, or “star” scientists are critical to modern research both because they are capable of very good research and because they greatly facilitate the process of keeping in touch with the rest of the biomedical community (19). However, it is very difficult to attract them to a private company unless they are permitted—even actively encouraged—to publish in the leading edge journals and to stay current in their fields.

Several interviewees also raised another, deeply intriguing possibility. They suggested that contact with the public sector might also improve the nature of the problem solving process within the firm, since contact with the public sector continually reinforced in private sector researchers the habits of intellectual curiosity and open exchange that may be fundamental to major advances in science.

Taken together, our interviews suggested that the public sector may play as important a role in improving the quality of the research process in the private sector as it does in generating specific pieces of useful basic knowledge.

Quantitative Analysis

Patterns of Coauthorship.

The descriptive statistics for our data on publication and coauthoring activity provide some preliminary results consistent with this more complex picture. Private sector scientists publish extensively—roughly three papers for every million dollars of R&D spending. Leading private sector researchers publish very heavily, indeed, with the most productive researchers in our sample firms publishing more than 20 papers per year. These firms also exhibit the heavily skewed distribution of publications per researcher and disproportionate share of “star” researchers characteristic of publicly funded research communities (20).

Researchers in these firms also coauthor extensively with researchers in the public sector both in the United States and abroad. Tables 3 and 4 break down instances of coauthorship by each of the 10 firms in our sample, as well as for the NIH. After SELF (private sector researcher coauthoring with other researchers working within the same firm), universities are by far the largest type of coauthoring institution, followed by hospitals. One curious result is the remarkably small number of coauthorships with the NIH. As the last row of Table 3 indicates, this appears not to be a sample selection problem: the breakdown of over 170,000 coauthorships by the NIH is not markedly different from the firms in our sample, with the great majority of coauthorships being with SELF and universities, and relatively few with private sector institutions. While many university researchers are supported by NIH grants and thus should perhaps be re-classified as NIH, it is still interesting that linkages between the private sector and the NIH are via this indirect channel.

Table 3.

Patterns of coauthorship by type of coauthor and firm

Firm	SELF	Public	NIH	Hospital	University	Nonprofit	Private	Miscellaneous	Total
A	0.55	0.03	0.01	0.07	0.27	0.03	0.03	0.01	6,583
B	0.48	0.03	0.01	0.08	0.34	0.03	0.02	0.01	15,628
C	0.64	0.02	0.01	0.05	0.23	0.02	0.03	0.00	17,292
D	0.53	0.02	0.01	0.04	0.35	0.02	0.04	0.00	2,053
E	0.54	0.03	0.03	0.06	0.29	0.02	0.03	0.01	8,971
F	0.70	0.01	0.00	0.04	0.19	0.00	0.05	0.02	327
G	0.54	0.02	0.01	0.06	0.29	0.02	0.04	0.01	8,451
H	0.68	0.00	0.00	0.08	0.20	0.00	0.02	0.01	1,414
I	0.62	0.02	0.01	0.05	0.23	0.02	0.03	0.01	7,874
J	0.50	0.06	0.01	0.06	0.25	0.04	0.06	0.02	736
NIH	0.60	0.04	NA	0.04	0.25	0.03	0.02	0.01	170,014

Table entries are the fraction of instances each type of institution appears as an address of a coauthor on a paper published by each of the firms in the data set. The last column gives the number of instances of coauthorship for each firm. NA, not available. 

Table 4.

Patterns of coauthorship by type of coauthor and year

Year	SELF	Public	NIH	University	Hospital	Nonprofit	Private	Miscellaneous	Total
80	0.69	0.02	0.02	0.20	0.04	0.01	0.01	0.01	2,050
81	0.67	0.02	0.01	0.21	0.05	0.02	0.02	0.00	2,200
82	0.62	0.02	0.02	0.26	0.05	0.02	0.02	0.01	2,702
83	0.62	0.02	0.02	0.23	0.07	0.02	0.02	0.01	2,992
84	0.58	0.03	0.01	0.25	0.08	0.03	0.02	0.01	3,023
85	0.60	0.02	0.01	0.25	0.07	0.02	0.02	0.00	3,834
86	0.57	0.02	0.01	0.26	0.08	0.02	0.03	0.01	3,928
87	0.57	0.02	0.01	0.28	0.07	0.02	0.02	0.01	4,535
88	0.58	0.02	0.02	0.27	0.06	0.02	0.02	0.01	4,312
89	0.55	0.02	0.01	0.30	0.07	0.02	0.02	0.01	4,032
90	0.56	0.02	0.02	0.28	0.07	0.02	0.03	0.01	5,147
91	0.53	0.03	0.01	0.30	0.06	0.02	0.04	0.01	6,260
92	0.54	0.03	0.01	0.29	0.06	0.03	0.04	0.01	7,611
93	0.53	0.03	0.01	0.29	0.06	0.02	0.04	0.01	8,293
94	0.53	0.03	0.01	0.30	0.06	0.03	0.03	0.01	8,410
Total	39,175	1,780	922	19,074	4,345	1,541	2,031	483	69,239

Table entries are the fraction of instances each type of institution appears that year as an address of a coauthor on a paper published by one of the firms in the data set. The last column gives the number of instances of coauthorship that year. The last row gives totals by type of coauthor over all years. 

Some interesting trends over time are apparent, both in the number of instances of coauthorship and in the mix across different types of institutions. While the numbers of papers published by the 10 firms in the sample tripled over the 15-year period, instances of coauthorship grew more than 4-fold. Over time the fraction of coauthorships with universities rose steadily, mostly at the expense of SELF. No significant trends in the aggregate share of the other types of coauthorships are apparent.

Links to Public Sector Research and Own Research Productivity.

These data on coauthoring document significant linkages between private sector research and “upstream” public sector activity. But the impact of such linkages is unclear. Does more participation in the wider scientific community through publication or coauthoring give a private sector firm a relative advantage in conducting research? As Table 3 indicates, firms show marked differences in both the number of coauthorships and the types of institutions they collaborate with. Formal tests strongly reject homogeneity across firms in the distribution of their coauthorships over TYPE, even after controlling for a time trend.

In prior work we found substantial and sustained variation across firms in research productivity, which we believe are driven to a great extent by differences in the ability of firms to access and use knowledge spillovers. We hypothesize that this ability is a function of both the effort expended on building such linkages and their “quality.” Table 5 presents multinomial logit results from modelling firms’ choice of TYPE of coauthor as a function of some of the characteristics, which we have identified in previous work as being important determinants of research performance: the size of the firm’s research effort and two variables, which capture aspects of the firm’s internal incentives and decision-making system. Compared with the reference category (coauthoring with a private sector firm) firms which are “pro-publication” in the sense of rewarding and promoting individuals based on the standing in the wider scientific community are more likely to coauthor with public institutions, nonprofits, and universities, whereas those that allocate R&D resources through “dictatorship” rather than peer review are slightly more likely to coauthor internally. Because our prior work suggests that those firms that are pro-publication and that do not use dictatorships to allocate research resources are more productive than their competitors, these results are consistent with the hypothesis that coauthoring behavior is significantly linked to important differences in the ways in which research is managed within the firm.

Table 5.

Multinomial logit coefficients

Category: Type of coauthor institution	Explanatory variables
	Time trend	Degree to which firm is pro-publication	Degree to which R&D decisions are made by a single individual	Size of firm’s drug discovery effort in $m	Constant
Hospital	0.046*	0.071	0.029	−0.008*	−2.734
	(0.021)	(0.045)	(0.038)	(0.002)	(1.783)
Nonprofit	0.367	0.207*	0.007	−0.008*	−3.640
	(0.027)	(0.059)	(0.048)	(0.002)	(2.252)
Public, including	−0.058*	0.363*	−0.077**	−0.005*	4.494*
NIH	(0.023)	(0.055)	(0.043)	(0.002)	(1.922)
SELF	−0.041*	−0.018	0.061**	−0.001	6.761
	(0.018)	(0.039)	(0.034)	(0.001)	(1.557)
University	0.021	0.104*	0.039	−0.007*	0.489
	(0.019)	(0.041)	(0.034)	(0.001)	(1.598)

Dependent variable: Type of coauthor institution 1980–1988 data: 26,501 observations. 

Reference category: Private. 

Standard errors are in parentheses. 

^*, Significant at 5% level. 

^**, Significant at 10% level. 

Table 6 presents results from regressing a crude measure of research productivity (important patents per research dollar, where “importance” is defined by the fact that a patent was granted in two of three major world markets—Japan, the United States, and Europe) on two variables derived from the bibliographic data: the fraction of coauthorships with universities, which can be thought of as a proxy for the degree to which the firm is linked to the public sector, and the fraction of the firm’s publications attributable to the top 10% of its scientists ranked by number of publications, which proxies for the presence of a “star” system within the firm. Firm dummies, a time trend, and total publications per research dollar are also included as control variables. The fraction of coauthorships with universities is positive and significant in all of these regressions, even controlling for firm fixed effects and “propensity to publish.” The presence of a star system also correlates positively and significantly with research productivity.

Table 6.

Determinants of patent output at the firm level

	Model
	1	2	3	4
Intercept	5.159*	5.292*	4.252*	4.037*
	(1.032)	(1.042)	(0.859)	(0.839)
Percent of coauthorships	7.340*	6.897*	5.137*	4.493*
with universities	(1.611)	(1.680)	(1.789)	(1.759)
Papers per research dollar		0.005		0.061*
		(0.006)		(0.026)
Firm dummies			Yes	Yes
Time trend	−0.227*	−0.231*	−0.203**	−0.211*
	(0.045)	(0.045)	(0.038)	(0.037)
RMSE	0.987	0.987	0.777	0.754
R-squared	0.293	0.301	0.611	0.638
Intercept	4.043*	2.380*	2.551*	2.515*
	(1.358)	(1.198)	(1.146)	(1.118)
Percent of publications by	2.052	3.897*	3.358*	3.236*
top 10 authors	(1.489)	(1.717)	(1.646)	(1.613)
Percent of coauthorships			4.870*	4.305*
with universities			(1.749)	(1.726)
Papers per research dollar				0.056*
				(0.002)
Firm dummies		Yes	Yes	Yes
Time trend	−0.142*	−0.129*	−0.179*	−0.187*
	(0.048)	(0.036)	(0.039)	(0.038)
RMSE	1.093	0.792	0.757	0.738
R-squared	0.132	0.595	0.635	0.658

Ordinary least-squares regression. Dependent variable: Important patents per research dollar. 1980–1988 data, 84 observations. Standard errors are in parentheses. RMSE, root mean squared error. 

^*, Significant at 5% level. 

^**, Significant at 10% level. 

We hesitate to over-interpret these results: confounding with aggregate time trends, the small sample imposed by incomplete data, difficulties with lags, causality, and a variety of other measurement problems discussed in previous papers mean that they are not as statistically robust as we would prefer. Furthermore, they are offered as descriptive results rather than tests of an underlying behavioral model. Nonetheless, they offer support for the hypothesis that the ability to access and interact with public sector basic research activity is an important determinant of the productivity of downstream private sector research.

Conclusions and Implications for Further Research

The simple linear model of the relationship between public and private research may be misleading. Information exchange between the two sectors appears to be very much bi-directional, with extensive coauthoring between researchers in pharmaceutical firms and researchers in the public sector across a wide range of both institutions and nationalities. Our preliminary results suggest that participating in this exchange may be an important determinant of private sector research productivity: The relationship between public and private sectors appears to involve much more than the simple, costless, transfer of basic knowledge from publicly funded institutions to profit-oriented firms.

Without further work exploring the social rate of return to research it is, of course, difficult to draw conclusions for public policy from these results. However they do suggest that any estimate of the rate of return to public research, at least in this industry, must take account of this complex structure. They are also consistent with the hypothesis that public policy proposals that curtail the flow of knowledge between public and private firms in the name of preserving the appropriability of public research may be counterproductive.