Standardization and Omics Science: Technical and Social Dimensions Are Inseparable and Demand Symmetrical Study

Abstract

Standardization is critical to scientists and regulators to ensure the quality and interoperability of research processes, as well as the safety and efficacy of the attendant research products. This is perhaps most evident in the case of “omics science,” which is enabled by a host of diverse high-throughput technologies such as genomics, proteomics, and metabolomics. But standards are of interest to (and shaped by) others far beyond the immediate realm of individual scientists, laboratories, scientific consortia, or governments that develop, apply, and regulate them. Indeed, scientific standards have consequences for the social, ethical, and legal environment in which innovative technologies are regulated, and thereby command the attention of policy makers and citizens. This article argues that standardization of omics science is both technical and social. A critical synthesis of the social science literature indicates that: (1) standardization requires a degree of flexibility to be practical at the level of scientific practice in disparate sites; (2) the manner in which standards are created, and by whom, will impact their perceived legitimacy and therefore their potential to be used; and (3) the process of standardization itself is important to establishing the legitimacy of an area of scientific research.

Introduction

Over the past 2 decades, biology has become an enormously data-rich subject. This increasingly demands ways to integrate and standardize the multiple streams of data and the attendant research processes that generate them. The drive to develop standards in genomics and related high-throughput technologies that enable omics science has thus mobilized a remarkable number of investigators, and consumed a considerable amount of their energies (Brooksbank and Quackenbush, 2006; Field and Sansone, 2006). “Communities” of researchers have worked—through initiatives such as minimum information checklists and others [e.g., Minimum Information About a Microarray Experiment (MIAME)]—to reach consensus on a variety of scientific practices, from how experiments are run to how resulting data are analyzed and shared to promote cooperation among scientists in such international and complex fields of discovery. This goal of cooperation and openness in the pursuit of scientific progress is widely desirable.

Standards are particularly relevant for omics science because it rests on a diverse array of complex high-throughput technologies. Notably, a hallmark of omics science is its focus on integrative systems biology. For example, the high-throughput omics data obtained in parallel from successive organizational hierarchies of cellular biology (from gene sequence to cellular metabolites) help discern, in real time, the “system level” predictive value of an omics biomarker test, over and above the built-in molecular redundancies preserved in biology during the course of human evolution (Kolker, 2009; Ozdemir et al., 2009). Omics science and technologies offer the promise to improve the simplistic and reductionist experimental models that hitherto presented merely a temporal snap shot of the much more complex, longitudinal, and dynamic nature of biological networks (and their fluctuations in response to social/environmental exposures) that fundamentally govern human health and disease. Standards are thus crucial for clear communication and integration of data across the “biological dogma” from genes, and proteins to their downstream representations such as cellular metabolites.

Standards in omics science are of interest to (and shaped by) others far beyond the immediate realm of individual scientists, laboratories, scientific consortia, or governments that develop, apply, and regulate them. Indeed, scientific standards have consequences for the social, ethical, and legal environment in which innovative technologies are regulated, and thereby command the attention of policy makers and citizens. This article presents a critical synthesis of the social science literature on standardization, with a view to potential applications and future knowledge transfer in the specific context of omics science. Importantly, we argue that standardization of the high-throughput technologies oriented toward integrative systems biology is both technical and social. Specifically, we consider three questions concerning scientific standardization:

1. What are the implications of standards for everyday scientific practice?

2. Who creates standards and why is this important?

3. Why does the legitimacy of standards matter?

Context for Standardization in Omics Science

The trajectories of scientific research, policy development, and regulation are codependent. The processes and practices of standardization are therefore both influenced by and have immediate consequences for scientific research communities and for the private and public bodies that sponsor research. Scientific activities are bound by policy and regulatory practices that frequently establish funding priorities, facilitate flows of resources, develop and enforce regulations about the conduct of research, and determine the parameters by which innovation can reach the market. Policy, similarly, depends on science to provide advice, guidance, and expertise within regulatory regimes.

In this context, standards-setting by scientific consortia in omics research is especially important because multiple high-throughput technologies and stakeholders contribute and drive omics science toward integrative systems biology as noted above. This type of standardization by consortia facilitates advances in basic research, by ensuring that data are replicable and information shareable. It also provides benchmarks that affect the development and regulation of scientific innovations toward applied health products once a basic science discovery is made. OMICS journal itself has published compelling arguments for the implementation of standards in omics science [see, e.g., the special issue on data standards (Field and Sansone, 2006), and reports on the Genomics Standards Consortium Workshops (e.g., Field et al., 2008a, 2008b, 2008c). Examples of omics-related standards that have been discussed and debated toward the goal of achieving a comprehensive framework for efficient scientific cooperation in omics sciences include:

• standards for the metagenomic publication record (Garrity et al., 2008),

• annotation (Angiuoli et al., 2008) and,

• data exchange formats and controlled vocabularies supported by the Human Proteome Organization (HUPO) Proteomics Standards Initiative (Taylor et al., 2006).

Although attention to the technical aspects of standardization often obscures the close relationship between science and policy, nevertheless, the interplay of social and technical issues within any standardization process has long-term consequences for technology and knowledge transfer. Ultimately, this interplay affects innovations and their uptake by downstream stakeholders at the level of clinical practice or population health (Landry et al., 2006; Paci and Ibarreta, 2009). Market conditions and political will, for example, affect the funding and direction of research.

Innovation may arise when existing products or knowledge no longer sufficiently address a particular issue or current need. Innovation can also arise when a market is matched to a future need. New knowledge may have no direct application and meet no immediately apparent demand; so too, it may be created in response to a recognized and targeted need. Although innovative knowledge may lead to the creation of new technologies that hold the potential to contribute to human well-being through economic, health, and social development, it is not without risks. The assessment and regulation of those risks are ultimately seen to be the responsibility of national governments, often under the stewardship of international regulatory authorities (e.g., World Health Organization). Scientific standards are an essential component in regulation and risk assessment. On the other hand, we suggest that scientific standards are not impartial actors in knowledge transfer, but rather actively mediate and effect the creation of knowledge (Bowker and Star, 1999; Hogle, 2009). Scientific standards-setting processes shape the practices that lead to innovation. Innovative technologies (and the standards upon which they are based) must ultimately engage the regulatory processes each nation has established to authorize the safety and quality of new products, especially those that have consequences for human well-being such as those evolving through omics sciences. The necessity of proactively connecting scientific practices with regulatory processes is well recognized by those developing standards in omics. Morrison et al. (2006) note, for example, the importance of creating standard processes to collect meta-data if omics research is to “have any applied use in regulation and policy (e.g., chemicals management)” (p. 173), and Feihn et al. (2006) note the importance of standards for regulatory bodies and funding agencies.

Implications for Scientific Practice

Standards, in the form of protocols and practices that guide data collection, compilation, assessment, analysis, and knowledge sharing are necessary to coordinate efforts between scientists working in different locations. For example, regularization of database standards and taxonomic classification should allow reliable and valid comparison of genomes, thereby facilitating interoperability (Sterk et al., 2006). Ideally, such standards enhance both the work of individuals and that of the field as a whole by facilitating transparent methods for the transfer, synthesis, and communication of complex data from multiple sources. Nonetheless, studies of standardization show tension between globally framed standards and the local implementation of those standards (Almklov, 2008; Hogle, 1995, 2008; Rothstein et al., 1999; Timmermans and Berg, 1997; Zimmerman, 2008). In fact, the theme of global and local reconciliation is a subject of immense importance in global health as well as in social studies of diffusion of innovations in the post genomics era (Pang, 2009).

Standards are often interpreted differently from one site to another, depending on the values, interests, experiences, and knowledge that the individuals involved bring to the table. Standard procedures used in organ procurement, for example, were adjusted to fit local-level variables by coordinators bringing their own experiences, networks of interaction, and moral judgement to the clinical setting (Hogle, 1995). Similarly, the International Stem Cell Forum's plan to have its participants share protocols for culturing and characterizing cells, thereby creating a common or “best practices” protocol, was hindered by concerns about “proprietary” information, but also by strong allegiances by different groups to “their own way” of doing things (Hogle, 2008, 2009). Hogle (2008) reports that this should not simply be seen as a reluctance to conform, but as an indication that different cell suppliers felt that there was something useful added by their way of doing things. Indeed, differences in genetic markers and expression were later seen as a result of cell suppliers' varying approaches. She also notes that although such tensions meant that a truly standardized method was not achieved, it did lead to a well-documented method in the field as a whole. In practice, standards can be reinterpreted, negotiated, and resisted at the local level (Graham, 2008; Graham et al., 1996; Hogle, 1995, 2008; Jordan and Lynch, 1998; Zimmerman, 2008), but contribute to added value at the macro level (Jones and Graham, 2009).

Somewhat counterintuitively, studies show that the local flexibility of standards in local settings is important to the willingness and the ability of the individuals involved to use them. Timmermans and Berg (1997), in their study of the Cardio Pulmonary Resuscitation (CPR) protocol note that:

The ongoing subordination and (re)articulation of the protocol to meet the primary goals of the actors involved is a sine qua non for the functioning of the protocol in the first place. Leaving the enrolled actors some leeway or discretion is often the preferred way to ensure their cooperation (Timmermans and Berg, 1997).

Standards and protocols that retain some flexibility—circumventing overly rigid operational definitions for every technical element or moment in a procedure—stay alive and pertinent (Timmermans and Berg, 1997). Without this flexibility, standards could reduce innovation in research at the local level (Shapiro, 1997). Indeed, Hogan et al. (2006) note that flexibility is an important component to working toward experimental standards in proteomics. The establishment of the “minimum information” (MI) convention, as pointed out by Field and Sansone (2006), is another example where institutionalized flexibility of standards—by allowing for varied standard adoption across disparate locales—encouraged general uptake of those standards.

The “living” nature of standards permits dynamic development in the omics field, while posing ongoing challenges for those faced with regulating emergent technologies. The creation and use of standards allow communities of interest (in this case omics researchers, industry, and regulators) to communicate with one another. These interactions are central to the regulatory process that determines whether novel products can reach the marketplace (Gregersen and Johnson, 1997; Hogle, 2009).

Who Creates Standards and Why Is This Important?

Mattli (2001) raises key questions surrounding the significant economic and governance implications of standard-setting processes: (1) who may define standards and how are standards created? (2) Should standards be defined only by scientific evidence, or should public and citizen input be considered also and in what circumstances? (3) What constitutes legitimacy in standard-setting processes, and how does legitimacy relate to the type of input used to shape standards? (4) How does openness and transparency relate to legitimacy? Although the latter two questions will be dealt with in the next section on standardization and legitimacy, we consider the first two questions here. Standards-setting bodies, whether they recognize it or not, play political and economic roles when they set standards (Mattli, 2001; Salter et al., 1988). This can pose significant governance challenges; standards-setting bodies must negotiate between the interests of the scientific community, other stakeholders, and the public.

So who may set or define a standard and how is that standard(s) created? Questions remain about whether, in an international standards-setting body, representation should be based on purely technocratic considerations as to which people have the best scientific credentials or whether there should be national representation (Mattli, 2001). Should that national representation be limited to countries where researchers are engaging, for example, in omics research, or should it be more broadly constituted? What about countries where the science is not currently being conducted, but might be in the near future? Or countries where the clinical applications arising from standards-setting process have an impact on the health of citizens?

To begin with, standards need to appeal to communities of scientists sufficiently that they are used. As Field and Sansone point out in their Foreword to the OMICS issue on data standards:

To be of use, standards must gain widespread or absolute acceptance within a community. This means they must be relevant and widely used. Managing this process of consensus-building from start to finish takes time, resources, and expertise. To be successful, initiatives must overcome not only technological barriers, but also any sociological barriers that may stand in the way of building a consensus view of the information essential for describing a particular type of data, as well as resistance of data generators to provide this information. Such issues should never be underestimated … (Field and Sansone, 2006).

Although all of these issues are important—the time and resources necessary for a consensual process, the need for standards to be relevant and widely used, and especially the warning against underestimating these matters—we recommend a closer examination of some other points raised in the above quote. To what extent are “sociological barriers” or “resistance of data generators” merely obstacles, rather than day-to-day realities of laboratory practice in a particular site?

An examination of some of the standardization processes within omics shows that a variety of approaches have been adopted—many of which attempt to include or engage with stakeholders beyond the immediate scientific community. For example, Leebans-Mack et al. (2006) describe the development of MIAME standards involving multiple communities of interests. The scientific community was central, participating through discussions at international meetings and open letters to scientific journals. Information technology specialists were consulted, advancing user-friendly systems to promote widespread compliance, and in turn, legitimacy of the standards. Journal editors were also enrolled, thereby contributing to compliance by researchers and endorsing legitimacy. Industry was seen as a key partner in the processes of standardization. The latter is probably not surprising given that the industry often needs to operate in a regulated context with numerous time-sensitive tasks that demand standardization and reproducibility in an ultrahigh-throughput setting.

Ball and Brazma (2006) suggest that although grassroots inclusion in standards creation and revision may have substantial barriers, benefits are also achieved through increased use of the standards and the empowerment of grassroots scientists who can participate in the changes to existing standards.

Although researchers build standards based upon the necessary and sufficient technical protocol and procedures, once those standards are taken outside of the laboratory, the scientist-only constituency of standards-setting bodies is brought into question. Should some form of consumer or public participation in standards-setting processes be included? In fact, public engagement does occur when consumers show concern about the safety or ethical implications of a particular scientific activity or product (e.g., stem cells or genetically modified organisms). Technical/scientific standards have a more indirect impact on the public and have traditionally been regarded, certainly by scientists and regulators, as occurring in what Winickoff and Bushey (2009) term “pure, scientific space.” This distinction between scientific space and other spaces may be called into question by a public increasingly sceptical of processes of self-regulation and concerned about ensuring the integrity of scientific processes (DeAngelis et al., 2001; Jones and Graham, 2009; Lewis et al., 2001; Stossel, 2005). Moreover, it is increasingly being recognized that the omics science, and especially the diffusion of omics innovations, take place both “in” and “outside” the traditional laboratory bench space (Ozdemir et al., 2009).

At present, innovations in many omics sciences are still at an early stage of development (Khoury et al., 2007), the interface between those who set standards and the regulation and governance of these new technologies may seem far off. However, what currently might be regarded as purely “technical” standards developed to facilitate scientific research, may have social, political, and economic consequences well beyond the scientific community (Hogle, 2009; Shapiro, 1997). As omics science advances to marketed health products, it is likely that its standard-setting processes will be subjected to intensified regulatory scrutiny. Ultimately, standards designed for the laboratory have to work in a wider arena to “recreate essential aspects (such as constant physical units) of the enclosed and controlled environment of the research laboratory in the outside world so that what works within the confines of the lab will continue to work outside of it” (Shapiro, 1997, p. 291).

Standardization and Legitimacy

Standards-setting processes serve as a powerful legitimising function for scientific practices. The legitimacy of research is derived from its processes and products meeting standards of efficacy, safety, and quality. Producers and end users alike require standards be met, although the tools needed to effectively and legitimately ensure and communicate this may be different. We therefore look at how legitimacy contributes to reliable scientific and public knowledge from two points of view: (1) the scientists who use the standards; and (2) nonscientist potential end-users of the products.

It is important that standards-setting processes are perceived as legitimate, both for scientific/technical communities and the wider public, most especially end-users. But notions of what constitutes legitimacy may be interpreted differently between groups. If a standards-setting body is perceived as legitimate by those whom the standards affect, this perception may assure the speedy and wide acceptance of the standard (Mattli, 2001). If a standards-setting body is widely perceived as legitimate, other bodies—in this context funders and regulators—will lend explicit or implicit support to that process; for example, the Canadian government provides some support to the HUPO Secretariat, based in Montreal, which is engaged in standard-setting activities in the field of proteomics.

So what constitutes legitimacy? The legitimacy of the institution setting standards can be based on a broad acceptance of that body by those using the standard, a recognition that the standard setting process was inclusive and just, or on perceptions of scientific and technocratic efficiency (Winickoff and Bushey, 2009). Although it is a matter of debate which of these factors grants legitimacy to standards-setting bodies, the legitimacy of standards-setting bodies is increasingly built on a combination of these three factors. This can be seen particularly in the Omics knowledge domain. Strategies to ensure broad acceptance have adopted what has been termed “carrot-and-stick” approaches (Ball and Brazma, 2006). In terms of carrots, or inducements, many members of the Omics community and many Omics organizations, such as the HUPO and journals engaged in high-throughput large-scale biology have been actively engaged in advancing the values of standardization to the community at large, and have been mindful of the operational imperatives of making such processes as user friendly and as helpful as possible, for example, by providing the required software free of charge. In terms of the stick, the engagement of journal editors with standardization processes may result in increased compliance for pragmatic reasons. As for inclusivity, Leebans-Mack et al. (2006) note the MIAME process involved the engagement of what they described as representative stakeholders in conjunction with the scientific community. Fiehn et al. (2006) also noted the importance of engaging a number of people and organizations from a variety of backgrounds of expertise and research objectives, including—in the metabolomics context—pharmacology, toxicology, and industry.

Openness and transparency are principles that create the basis for acceptability of standards, at least when they intersect with a more overtly political regulatory process (Salter et al., 1988). Therefore, openness and transparency require clarity around: processes of appointment to standards-setting bodies; the criteria for such appointments, disclosure, and management of, or minimizing conflict of, interests; consideration of whether decisions are reached by consensus or voting; and that these process-related issues are public (Mattli, 2001; Winickoff and Bushey, 2009). In the Omics context, those involved in standardization processes indeed appear cognisant of these issues. Leebans-Mack et al. (2006), for example, note that the process for the development of MIAME was “deliberative,” and Fiehn et al. (2006) that community consensus must be reached. In respect of the HUPO PSI, the representatives in the working committees were made visible and the publication of reports served to disseminate and provide some degree of transparency about their progress (Taylor et al., 2006).

For nonscientists, standardization also provides another kind of legitimacy. It lends legitimacy and value to the scientific work done by the field as a whole. The collective backing of a set of standards by an independent scientific community increases the perception of the reliability of the end product. It therefore lifts research results to the level of facts (Latour and Woolgar, 1986).

“Collectively produced evidence produces a form of credibility that individual labs cannot (no matter how trusted the lab or how much star quality the lab leader possesses)” (Hogle, 2008). Hogle (2008) further notes that this is increasingly important when there are doubts or questions about the field, such as in the case of stem cell research.

Although omics standardization activities are currently focused on assisting scientific advancement, the potential for future clinical application of these standards lies in whether legitimacy is achieved. Currently, standardization processes play a role in legitimizing the omics science to funders and regulators, and most significantly for the present moment, in promoting and advancing the omics science itself. Although the standards being developed in proteomics by the HUPO, for example, are not mandatory in the legal sense, they will become so if adopted by national regulators as a necessary precondition for entry into regulatory processes (Shapiro, 1997). Ye et al. (2009) report that the proteins approved as clinical biomarkers by the U.S. Food and Drug Administration (FDA) are small in number compared to the large potential of proteomics research; validation methods are needed to confirm clinical utility. In the future, not complying with the HUPO's technical standards may have profound economic and social consequences for the ability to market innovative technologies emerging from proteomics research. In effect, although standards may not be legally binding at an early stage, they tend to “drift” as a greater number of stakeholders (nationally or internationally) begin to endorse or uptake such standards. Again, this attests to the inherently “living” and dynamic nature of standards in postgenomics medicine, including omics science (Ozdemir et al., 2009).

Although governments have a responsibility to support the pursuit of knowledge about health and well being and an economic interest in supporting innovation, they are also responsible for ensuring that new technologies are safe for public use. Thus, for regulators, harmonization of scientific standards creates auditable consistencies of quality and efficiencies across and within nations and further integrates the processes and practices of international regulatory goals. This consistency is not just in the manner in which standards for scientific practices are developed, but also in the way in which individual research teams as well as national regulatory authorities negotiate such standards in practice. Scientific standards function as a guarantor of the legitimacy and authority of such research, an essential position that may be undermined if there is a lack of consistency in their interpretation and implementation.

Conclusions

In this article we sought to draw attention to three key issues that arise when considering the long-term implications of standardization processes within the omics science: (1) there is always a tension between the creation of universal standards and how these standards are flexibly applied within local practices (and vice versa). This flexibility may be key to making standards work and advancing the scientific inquiries, but nonetheless, may prove problematic when it comes to regulate an innovative technology resulting from such research. (2) Who creates standards, and how they are created may have an impact on the extent to which they are accepted within a given scientific field and who may or may not participate actively in that area of science in the future. It may also impact the regulation of future innovations. As innovations increasingly result from omics sciences, additional scrutiny by regulators, the public, and other interested parties of the legitimacy of the processes of standardization, or how standards are created, is likely to increase. (3) Working toward “interoperability” or standardization may lead to an increase in the legitimacy and hence the trustworthiness of the omics field as a whole.

Understanding the factors that underlie standardization may help scientists and scientific organizations when devising and implementing standardization processes and when those standardization processes engage with frameworks for the regulation of innovation. Despite the challenges identified in this article, standardization offers the powerful advantage of not only advancing omics science, by making communication of results between scientists easier, for instance, but it also legitimizes these new areas of science. The standardization process provides an important intersection, building a relationship between scientists, those who regulate scientific innovations, and the end-user marketplace.

Acknowledgments

Drs. Holmes, Jones, and Graham acknowledge the Canadian Institutes for Health Research funding of the project “Risk and Regulation of Novel Therapeutic Products: A Case Study of Biologics and Emerging Genetic Technologies” (CIHR MOP 74473) Dr. Ozdemir is funded by a career investigator salary for science and society research in personalized medicine from the Fonds de la recherche en sante du Quebec (FRSQ) and an ethics operating grant from the Canadian Institutes of Health Research (“Unpacking the public–private gap in the application of ‘omics’ technologies: A socio-ethical analysis of the pharmacogenomics research funding;” R0016158). The authors wish to thank the anonymous reviewers for constructive critique and helpful comments on the manuscript.

Author Disclosure Statement

The authors declare that no conflicting financial interests exist.