The Structural Genomics Consortium

Abstract

The Structural Genomics Consortium (SGC) supports drug discovery efforts through a unique, open access model of public-private collaboration. This study presents the results of an independent evaluation of the Structural Genomics Consortium, conducted by RAND Europe with the Institute on Governance. The evaluation aimed to establish the role of the SGC within the wider drug discovery and PPP landscape, assessing the merits of the SGC open access model relative to alternative models of funding R&D in this space, as well as the key trends and opportunities in the external environment that may impact on the future of the SGC. It also established the incentives and disincentives for investment, strengths and weaknesses of the SGC's model, and the opportunities and threats the SGC will face in the future. This enabled us to assess the most convincing arguments for funding the SGC at present; important trade-offs or limitations that should be addressed in moving towards the next funding phase; and whether funders are anticipating changes either to the SGC or the wider PPP landscape. Finally, we undertook a quantitative analysis to ascertain what judgements can be made about the SGC's past and current performance track record, before unpacking the role of the external environment and particular actors within the SGC in developing scenarios for the future.

Background and Context

This study summarises the results of an independent evaluation of the Structural Genomics Consortium (SGC), conducted by RAND Europe with the Institute on Governance. The SGC is an open access public-private partnership (PPP) comprised of 20 research groups with a primary focus on pre-competitive structural biology research (namely determining 3D protein structures) and an emerging secondary focus on chemical probes and antibodies, and epigenetics research. The SGC focuses explicitly on less well-studied areas of the human genome. It makes all research outputs openly available to the scientific community and creates an open collaborative network of scientists in hundreds of universities around the world as well as nine global pharmaceutical companies. Public, private and charitable funders contribute mainly by means of a fixed annual sum over the phase of research activity in return for membership of the SGC board and a voice in determining the focus of research efforts. Current public funders are situated in the UK and Canada, and the SGC has sites at both Oxford and Toronto, in affiliation with the universities. The SGC's current funding phase ends in June 2015 and this evaluation was commissioned to feed into discussions regarding the next stage of funding.

The evaluation had three overarching objectives. Firstly, we aimed to establish the role of the SGC within the wider drug discovery and PPP landscape. To meet this aim, a thorough literature review was carried out. This enabled us to assess the merits of the SGC open access model relative to alternative models of funding R&D, and also to identify the key trends and opportunities in the external environment that may impact on the future of the SGC. Secondly, the evaluation sought to establish the incentives and disincentives for investment, strengths and weaknesses of the SGC's model and the opportunities and threats the SGC will face in the future. To do so, the team conducted key informant interviews with SGC researchers, past and present funders and external stakeholders and carried out a survey of SGC researchers. This process enabled us to assess the most convincing arguments for funding the SGC at present; important trade-offs or limitations that should be addressed in moving towards the next funding phase; and whether funders are anticipating changes either to the SGC or the wider PPP landscape. Thirdly, we aimed to ascertain what judgements can be made about the SGC's past and current performance track record, and to present ways for stakeholders in the SGC to think about its future development. In order to do this, we undertook a quantitative analysis before unpacking the role of the external environment and particular actors within the SGC. We then created four potential scenarios as ways of thinking about the SGC's priorities in its next funding phase and beyond.

Results of the Literature Review

The literature review covered the conceptual background to open innovation, intellectual property and PPPs. In reviewing academic and grey literature a number of key findings were identified. Firstly, the review found a vibrant critique from a number of angles on the possible dangers of the “anti-commons” and the potential benefits of a more collaborative approach to research and innovation. It is clear from the review that the SGC is unique but also that it shares characteristics with a wide range of other “open innovation” partnerships and collaborations. There are, then, initiatives that are comparable: whilst the SGC has singular characteristics, it is a part of a broader trend. Much of the grey and peer-reviewed literature suggests that this trend exists because there is a widely acknowledged crisis in pharmaceutical R&D. In addition to this trend towards open innovation and collaboration at the pre-competitive stage, there is a second trend that is simultaneously developing based on biotechnology, venture capital and intellectual property rights. There are grey areas where these two trends converge but the logic behind each of them is distinct. Finally, the literature review identified broad system level issues to do with the nature of the way science is funded, incentives in public and private sectors and different perspectives on what works.

To contextualise the conceptual arguments, the literature review included analysis of PPPs in the health sector and in other sectors. Here, there are some initiatives that have similarities to the SGC both in terms of their overall goals associated with contributing to drug development and also in their aims of fostering more openness and collaboration in pre-competitive research. There is also a set of initiatives that have a focus on structural genomics, such as Japan's RIKEN research institution, the USA's Protein Structures Initiative (PSI), and Europe's Structural Proteomics in Europe (SPINE) initiative. Each of these groups is organised to deliver structural genomics information in different ways. We reviewed aspects of each with a particular aim of identifying any evaluations of these organisations that might help to inform both the evaluation and the nature of our findings and insights.

In considering other sectors, we found a variety of comparable PPPs. These included formal organisational models, informal networking mechanisms and different platforms geared towards facilitating innovation and knowledge production to the benefit of different sectors. In the full report, a large number of initiatives are reviewed and summarised before a more detailed analysis of three selected “case studies” are given for the Linux, Sematech and EU Technology Platforms initiatives. Our review found that PPPs from other sectors are mobilised in multiple ways and that their characteristics differ according to their geographical coverage, funding, sector, position in the value chain, innovation model and organisational focus. Most PPPs are evolving and transform over time; their characteristics depend on the maturity of the sector, the nature of industry and firms therein and wider political, economic, technological and scientific factors influencing innovation.

Incentives for Investing in the SGC

Open Access

Open access makes the SGC unique as a PPP in this field and creates a number of desirable knock-on effects. These effects include wider societal benefits, maximising the opportunities for and efficiencies of further research and improving the competitiveness of the field. Moreover, open access enables funding to be secured and fosters the efficient establishment of diverse collaborations. This last quality is particularly aided by the SGC's ability to overcome institutional regulations and restrictions about intellectual property due to its open access nature. Several examples of efficiency in the research process, improved research outputs and new areas for drug discovery were highlighted across the collaborators, funders and researchers we spoke with and all attributed this in part to the open access philosophy of the SGC.

Collaborative Research Opportunities and Access to a Global Network

Most researchers cited the collaborative research opportunities and access to a global network in core areas of structural biology expertise that the SGC provides as key reasons for investment in the consortium. This view was shared by the majority of the funders and some external stakeholders. One reason that the SGC's collaborative network is particularly appealing and, therefore, is an incentive for investment, is that one can easily make the most of the SGC's collaborative network due to the open access format. Several interviewees commented that this format means that it is very easy to set up collaborations without worrying about contracts and legal issues. In particular, the majority of private sector funders stated that links to a global network of expertise in the area of epigenetics was especially important.

De-Risking Emerging Areas of Science

Many private sector funders highlighted the importance of the SGC model in helping to “de-risk” new areas of science as a reason for investment. In particular, the majority of pharmaceutical funders used the epigenetics programme as an example of this “de-risking” effort, and it was clear that the SGC's decision to conduct epigenetics research was a significant factor in the funders' decision to invest in the SGC. Epigenetics is a new and developing area of biology and joining a consortium offered gains in this area at relatively little cost.

Linking with Strategic Initiatives

Closely linked to the incentive of de-risking new areas of science is an incentive around the alignment with ongoing strategic initiatives within a company, public funder or collaborating organisation. Some funders commented that open innovation was part of a new strategic initiative within their companies and the SGC provided a good opportunity to put it into practice. Others commented that the SGC, particularly at the beginning, was directly aligned with major initiatives in the wake of the human genome project. In particular, the SNP Consortium and the Human Genome Project itself were seen as immediate precursors to the SGC.

Rapid and Efficient Research Processes

The majority of interviewees felt that research happened more quickly in the SGC than in either academia or industry, and this was a significant strength of the SGC. The speed and volume of SGC research is enabled at least in part through open access, the collaborative nature of the model and the ability to collectively de-risk new areas. Related, several funders reported the SGC's approach to using an 'industrial model' for research was an important factor in their decision to invest. The SGC possesses several characteristics of an industrial research model, with milestones and targets determining the scientific outputs and a commitment to ensuring that findings can be reproduced by others. Not only this, it operates on a large scale, accessing a wide range of expertise and resources that would not be available to a small laboratory. These elements of quality, scale and reproducibility are perceived to have a considerable impact on the efficiency and volume of SGC research.

Disincentives for Investing in the SGC

SGC researchers, funders and external stakeholders also identified some disincentives for investment. These included unprotected intellectual property of work conducted by the SGC and a perception of limited spillover effects for the wider community. Such regional and national spillover effects of the SGC are important to public sector funders in particular, who see these as linked to the SGC's physical location. However, it is important to note that a lack of economic and societal spillover effects was not cited as a weakness or a disincentive by all public sector funders. This difference in views demonstrates the difficulty the SGC has in meeting the needs of each individual funder.

Strengths and opportunities for the SGC

Extensive Collaborations Between Industry and Academia

Along with its world-class scientific expertise, the extensive collaborations between academia and industry were the most frequently mentioned strength of the SGC. Indeed, it was a specific aim of the founders of the SGC to ensure that the benefits of public and private sector research were brought together in the most productive ways. In order to achieve this, the SGC considered how the private sector could benefit from the consortium without involving intellectual property, as well as where the private sector could add value to the consortium aside from the provision of funding. In relation to the latter, the SGC has used the private sector's expertise in designing molecules and assays for target validation and the private sector is responsible for the commercial focus of the SGC. In terms of the former, the private sector benefits from the fact that close collaboration and networks help to prevent the duplication of effort among pharmaceutical companies. If the SGC did not exist, these organisations might be more likely to pursue the same lines of discovery independently, and in a less efficient way than that afforded by the SGC. Though we were not able to quantify this benefit in this evaluation, we can conclude that at least in some instances, overall costs to both public and private innovation efforts would likely be higher in the absence of the SGC with a potentially negative impact on drug discovery.

Rapid and Efficient Research Outputs

The SGC has a recognised strength in providing rapid and efficient research outputs. Eighty-two per cent of surveyed SGC researchers (N=17) believed their research had come to fruition more quickly through the SGC than it would have done had it been supported by traditional academic approaches. The most frequently cited reasons for this accelerated research realisation were high quality collaborations, an integrated approach, the lack of a need to spend time writing grant proposals and the efficiency of SGC processes. Other stakeholders attributed this efficiency to the lack of intellectual property, the importance of a highly interactive research process which is accelerated through open access, and the fact that the SGC is streamlined and narrowly focused, with a strong “company ethos” and industrialised research processes.

Industrially Oriented, Flexible Model of Research with Effective Management

One of the reasons that many stakeholders felt the SGC is able to operate more efficiently than other research models is because of the way it conducts its research. This model is industrially oriented and flexible, well managed, and operates with a clear focus and set of targets (as opposed to some academic research which may be more curiosity driven). In addition, the SGC's flexible approach to collaborators enables a large range of diverse networks and collaborations, which in turn affords the SGC the chance to be flexible in approaching new scientific areas. The flexibility coupled with the focused nature of the science allows the SGC to exploit economies of scale and networks in exploring new scientific areas.

Also crucial is the fact that the SGC is able to conduct “reproducible science”—that is, the SGC can be relied upon to produce results which can be reproduced by others. Although this may not be highly valued by academia (given that experiments in academia are rarely replicated), it is of particular importance to industry given that they cannot build technologies that work without it.

The SGC Has the Opportunity to Forge the Way in Pre-Competitive Research in the Drug Discovery Landscape

Our research shows that there are broad changes in the field that mean that pre-competitive research is likely to be more prominent in future. There is, moreover, a view that the SGC's open innovation model is particularly appropriate given the structural biology focus of the SGC. These factors point towards a genuine opportunity for the SGC to expand the boundaries of its field. Several interviewees expressed the view that it was the role of the public sector to help ensure the future of open innovation and to drive pre-competitive research boundaries. These interviewees felt that the private sector itself was not likely to provide a catalyst alone. This is clearly a complex issue which will be shaped by a series of external factors, including price pressures, trends towards outsourcing innovation, openness to flexible approaches, the intellectual property regime and future downsizing in the economic climate.

The SGC Has the Opportunity to Revisit its Scientific Direction

The research model of the SGC and its unique position in the burgeoning trends of PPPs and pre-competitive research mean that it has a wide range of opportunities in terms of scientific areas of focus. This wide range led to a degree of divergence among interviewees regarding how the SGC should exploit these opportunities in the future. Some wanted the SGC to narrow their focus back onto structural biology, whilst others were of the view the SGC should continue to push into areas such as epigenetics. Ultimately the scientific direction of the SGC will be determined to a certain extent by available funding. However one of the most frequently cited weaknesses of the SGC was the fact that its mission had become much more diffuse in recent years.

Weaknesses and Challenges for the SGC

There May Be Too Many Collaborators

Several stakeholders expressed concern about the volume of collaborations, and suggested that the optimum number of collaborations for the SGC may have already been surpassed. These concerns primarily centred on the lack of support staff and resources to manage these collaborations. The vast majority of those concerned about the number of collaborations felt that this could be addressed if the SGC employed more staff and acquired the facilities necessary to support them. In these conditions, continued collaborations would be welcomed. However, it is worth noting that a minority of researchers had more fundamental concerns about the SGC's growth, including that it could contribute to a lack of focus.

There Are Limited Opportunities for SCG Researchers

Given the unique nature of the SGC, there was concern among some interviewees that SGC researchers had limited opportunities for career progression within the consortium. These interviewees felt that the researchers' narrow research focus coupled with a lack of experience in writing grant proposals also meant that they were less competitive in the academic job market upon leaving the SGC. This may present a challenge for the SGC in the longer term, particularly if it results in an inability to attract world-leading scientists or a high staff turnover.

The SGC Needs to Maintain a Substantial Level of Funding for the Future

Public sector funding has diminished significantly since the SGC's inception and has been replaced with private sector funding, leaving a one to five ratio of public to private funding. All funders shared the view that the public sector/private sector mix was important. However, a public sector or “non-industrial” presence was considered to be important by private sector funders for two main reasons. First, the presence of non-private funds would keep the SGC research open and in the public domain. Second, it would keep SGC research innovative and safeguard against the SGC becoming too closely aligned solely with the needs and interests of the private sector. Non-private funds are seen as protecting the SGC from becoming more like a contract research organisation, which would result in it losing its competitive and innovative edge. Although the role of the public and private sectors were generally considered to be important, the role the two different funding types might play was debated.

Dilution of the SGC Mission

Many interviewees noted that the current SGC model is vulnerable to continued diversification, given that it offers funders the opportunity to shape its direction. One interviewee noted that the “main weakness” of the SGC is “too many offshoot activities” which prevent the SGC from focusing on its core strengths. Indeed, the research and business model may mean that the SGC itself cannot decide its own area of focus and together with discussions about its funding and scientific direction, these issues will continue to be a challenge and tension for the SGC.

Quantitative Analysis of the SGC's Outputs and Impacts

The Context for SGC Knowledge

When considering SGC quantitative outputs and outcomes, there are a number of contextual factors to take into account. Such factors are important, since the SGC does not operate in a “sterile environment” in which only the SGC activities relate to the value of impacts arising from SGC work. The SGC have set a specific task to deliver protein structures that go beyond those already developed in the scientific literature. This means that the SGC specifically targets proteins that are considered more difficult to work with, and therefore any consideration of the outcomes of SGC research should take into account the relative difficulty of the task the consortium has set itself. The green bars in Figure 1 show where the SGC has identified protein structures for the kinases shown along the x-axis. This shows how the SGC has developed structures for human protein kinases, regardless of their existing scientific body of evidence (illustrated by number of citations and publications along the y-axis).

Figure 1.

Human Protein Kinase Research and Structure Analysis by the SGC

Quantitative Outputs

The quantitative outputs of the SGC range from the main outputs of SGC work (such as publications, structures and sequences), through to the broader economic outcomes (including monetised outcomes) that are the result of SGC involvement in research. We identify a number of the SGC's scientific knowledge outputs. Firstly, since 2004, the SGC has developed and deposited the structures of 1195 proteins in the Protein Data Bank (PDB). Secondly, from 2004 to 2011, this has led to 83 new sequences deposited in Uniprot—the protein sequence database. Thirdly, the SGC has produced 452 peer-reviewed journal publications (and eight books) up to August 2013. Finally, in terms of dissemination SGC scientists attended and presented at over 250 conferences from 2007 to 2011, including 38 poster presentations and 87 invited talks as a direct result of scientist involvement in the SGC.

Our research also provides information on the productivity of the SGC compared to other structural genomics groups. Figure 2 shows the number of structures deposited in the PDB over the lifespan of the SGC. It clearly shows that the Japanese group, RIKEN, were immensely productive in the early years of the SGC, although this should be tempered by the fact that RIKEN deposited all structures identified, including homologs of existing proteins in the database. The SGC does, however, produce roughly the same number of protein structures annually as the different groups associated with the PSI (the other organisations in Figure 2). This suggests that SGC outputs are roughly comparable to those of other structural genomics organisations in terms of structures delivered. However, it is worth noting that the SGC also works on other aspects of structural genomics, including chemical and biological probes, and, to some extent, new methods and tools. These areas of the SGC's work are not captured in the analysis of structures alone.

Figure 2.

Publicly Deposited Protein Structures in the PDB from Major Structural Genomics Groups 2004–2012 (data taken from the PDB)

Possible Futures for the SGC

To inform how the SGC might look in the future we developed a set of potential scenarios for the SGC to reflect on. Each of these scenarios sees the future SGC as being motivated by a particular focus, or driver. For our scenarios analysis exercise we used a simple scenario development process rather than a more formal approach. This meant that we considered the SGC in light of future decisions it may need to take about its funding strategy, scientific direction and the external context in which these decisions would need to be made. In order to do this, we first identified contextual certainties and uncertainties which would play a role in the future. These included the nature of R&D in the pharmaceutical sector, the direction of drug discovery science and wider political economic conditions. In addition to these contextual conditions, we identified a list of critical success factors for the SGC, permutations of which are likely to be particularly important to the SGC's future. These factors included the SGC's scientific vision, business model, funders, collaborative networks, spillover effects, and location, as well as the future role of open access. Each scenario was underpinned by the assumption that the SGC continues to function as a knowledge platform in the future, with different drivers for generating knowledge, investing in knowledge and extracting value from knowledge.

Figure 3.

Scenarios

Each of these scenarios has its own merits, challenges and opportunities. They are presented as distinct, but in reality there are many overlaps between them, and these should be further explored and examined. What will be crucial for the SGC going forward is the balance between the different elements in each model, and the extent to which different drivers serve as the motivating element. It is important to note that in developing scenarios we sought to create a narrative and in many cases often exaggerated what the future might look like for the purposes of illustration. We fully recognise, and in fact argue, that SGC's future strategy is going to encompass a mix of these scenarios, but it is the process of determining that mix which is important. In order to do this, we must understand what each scenario looks like independently.

Conclusion and Recommendations

The SGC is successful and should be maintained in something akin to its current form. Our evaluation suggests that in order to understand the added value of the SGC, it is important not only to appreciate what the role of open access is, but also how both public sector and private sector actors within the SGC benefit and help to maintain it. We have found that there is a finely nuanced role of the public and private sector presence in the SGC in relation to the added value it brings. The public sector plays a fundamental role in maintaining open access, while the private sector helps to maintain the SGC's industrial quality and reproducible science. Both contribute to a form of innovation and related benefits that come out of the SGC and spill over to the wider field. Therefore, without each element, the SGC ceases to exist in its current form, and its added value to the field is reduced. We believe this goes to the core of some of the current tensions in the SGC model and its future.

All of this serves to demonstrate that each set of funders, and the public sector in particular in relation to open innovation, needs to be fully aware of the role it is currently fulfilling and the losses that would be incurred if one were to withdraw from the SGC. These losses should be considered against the backdrop of the changing nature of drug development and innovation which poses wider challenges to the field.

The study ends with a list of recommendations for the SGC to consider in moving forward. These are to:

• Maintain the SGC in something akin to its current form

• Develop a high level strategy that provides a broad plan for operations over the next five to ten years

• Incentivise the public sector to (re)invest

• Develop a strategic approach for identifying potential philanthropic and charitable funders who may be interested in investing in the SGC as a platform for knowledge

• Consider ways to enhance the sustainability of the SGC's leadership, potentially through recruiting deputy leaders

• Provide more support for scientists to aid career progression and develop transferable research skills

• Improve monitoring and evaluation processes to more effectively capture knowledge and disseminate positive impacts where they arise

• Build on the successful examples of the few small biotechnology firms which have arisen out of partnering with the SGC to focus more on possibilities for engaging small firms in its generating knowledge and extracting value models

• Undertake a more comprehensive assessment of the comparative costs and merits of the different trajectories to drug development.

We are conscious that some of the recommendations presented may not be in line with the SGC vision, but they arise from our understanding of the evidence gathered in our evaluation, the different challenges and opportunities facing the SGC and how it may need to respond. These are important considerations in understanding how the SGC can attract more funding, generate knowledge more efficiently, and extract more value from the knowledge it creates in both the scientific and wider socioeconomic sense. Moreover, these concerns are vital in addressing the pressing issue of how the consortium can be sustainable in the future.