Analysis of Time-Series Gene Expression Data: Methods, Challenges, and Opportunities

Abstract

Monitoring the change in expression patterns over time provides the distinct possibility of unraveling the mechanistic drivers characterizing cellular responses. Gene arrays measuring the level of mRNA expression of thousands of genes simultaneously provide a method of high-throughput data collection necessary for obtaining the scope of data required for understanding the complexities of living organisms. Unraveling the coherent complex structures of transcriptional dynamics is the goal of a large family of computational methods aiming at upgrading the information content of time-course gene expression data. In this review, we summarize the qualitative characteristics of these approaches, discuss the main challenges that this type of complex data present, and, finally, explore the opportunities in the context of developing mechanistic models of cellular response.

TEMPORAL GENE EXPRESSION ANALYSIS

At any given time a cell will only express a small fraction of the thousands of genes in the organism's genome. Expressed genes reflect the structure and functional capacities of the cell as well as the ability of the cell to respond to external stimuli. In a complex organism, external stimuli to a great extent take the form of chemical messages whose purpose is to coordinate the function of the complex society of cells (1). Gene arrays, which measure the level of mRNA expression of thousands of genes simultaneously, provide a method of high-throughput data collection necessary for obtaining the scope of data required for understanding the complexities of living organisms. Monitoring the change in expression patterns over time using gene arrays provides an approach for capturing the multidimensional dynamics of complex biological systems. By using gene arrays in a time series paradigm, we are able to observe the emergence of coherent temporal responses of many interacting components. The data should provide the basis for understanding evolving but complex biological processes, such as disease progression, growth, development, and drug responses.

Global gene expression analysis has been celebrated as a major revolution in modern biology (2). The ability to monitor simultaneously the expression of the genes composing the entire genome has generated unimaginable possibilities (3–5). Despite some criticism regarding the cross-platform reproducibility of expression experiments (6, 7), more recent evidence (8, 9) supports the informative nature of the experiment and the importance of the approach (10). Microarray analysis has found widespread applications from characterizing terminal states, i.e., benign versus malignant tumors (11), to attempts to decipher the evolution of complex diseases and cell fates (12– 16). Hence, the nature of the data broadly defines the nature of the problems to be addressed.

Boundary problems use the expression measurements as feature vectors that characterize static points in multidimensional spaces. Therefore, multiple samples, for example, from the same tissue of different patients (diseased/nondiseased) would define a database of multidimensional feature objects with as many dimensions as genes whose mRNA has been quantified and as many objects as the number of patients monitored. Critical questions then arise, such as how to identify coherent patterns, i.e., combinations of up- or downregulated genes that distinctly characterize the two or more classes of patients (17, 18).

Monitoring the change in expression patterns over time provides a profoundly different type of information. Instead of concentrating at terminal points in time of binary nature (benign versus malignant, type A versus type B, etc.), we now have the opportunity to observe the emergence of coherent temporal responses of many interacting components. The orchestrated response of an organism to an external stimulus and the monitoring of the temporal progression offer numerous opportunities for reverse-engineering the mechanisms that regulate the host responses (19). The latter, in turn, will define the rational foundation for the generation of testable hypotheses. Thus, the challenge becomes how to upgrade the information content of such multidimensional trajectories to address critical questions such as the characterization of the state of evolution of a system; the identification of activated pathways, their relation, and the rate limiting steps; and the synthesis of interaction networks and the characterization of points of control.

A number of questions could potentially be addressed that fall, broadly, under the following categories (20):

1. Biological systems analysis: Specific systems are monitored over time and information is assembled to understand the driving dynamics. Prototypical examples include cell cycles and circadian clocks (21, 22).

2. Response dynamics: Systems are subjected to controlled perturbations and the broad gene expression response of the system is monitored over time. Examples include drug dosing and defined trauma (39, 50).

3. Development: Morphing of organisms during development involves complex sequences of cell proliferation and differentiation. Many models have been used over the years (23) to address the process of development. Particularly exciting are the opportunities offered by recent advances in stem cell differentiation (24).

4. Disease progression: Genome-wide temporal profiling offers the possibility of elucidating the underlying pathophysiologies of human diseases (25). Instead of focusing on predefined hypotheses, global expression offers the possibility of unraveling the systemic evolution of pathological conditions.

The purpose of this review is not to provide a detailed account of the enormous complexities and uncertainties surrounding the collection of the required data (14, 26). The following sections will highlight conceptually the basic foundations of the computational approaches that have been recently proposed for the analysis of temporal gene expression data, the opportunities that exist, and, more importantly, the challenges that need to be addressed.

METHODS

The goal of temporal gene expression analysis is to identify broad sequences of molecular events in time. Such sequences can be associated with an ongoing biological process, such as the cell cycle, circadian rhythms, or development, or can be initiated by some input perturbation, such as the administration of a drug or a defined trauma. The host state is defined as the ensemble of all possible metabolites, proteins, small molecules, etc., which define the observed phenotype of the organism. For some purposes, such as the consideration of circadian rhythms, the host state may be dynamic. Any perturbation sets in motion the information transfer that defines the blueprint for the production of the relevant components of the response by activating appropriate genes whose transcription to mRNA and subsequent translation to proteins catalyzes critical functions. Therefore, the implicit underlying assumption of transcription profiling is that gene expression is causal to phenotypic responses through production of specific proteins coded by the expressed mRNAs.

One of the major limitations of monitoring exclusively mRNA transcripts is the role of posttranslation modifications, mRNA stability, and other destabilizing and complicating factors that render the products of transcription (mRNA) an inaccurate proxy for the abundance of active products of translation (27, 28). Nevertheless, analysis of the products of transcription has already provided significant insight and is undoubtedly a critical source of information. Associated with expression profiling is the implicit assumption that gene expression is tightly controlled by a fine-tuned, intricate, and robust regulatory mechanism that appropriately activates and deactivates the machinery guiding the expression of genes. By now it is almost taken for granted that genes exhibiting similar responses to signals ought to be controlled by similar regulatory mechanisms. This is often referred to as the guilt by association principle (29). Therefore, identifying coherent expression responses is important in the sense that if coexpression can be linked to coregulation then the underlying machinery driving expression can be isolated to smaller groups, deciphered, and quantified. One of the most critical problems is to verify the common regulatory mechanism (30). Hence, the first and most critical step in this endeavor it to identify those measured transcripts that appear to be somehow correlated to each other. From a computational point of view, this problem belongs to a more general class, namely, the characterization (indexing and clustering) of multidimensional trajectories (31).

Numerous methods have been developed and applied to this very challenging problem in the context of analyzing gene expression data (20, 32, 33). At the core of all the methods is the concept of similarity and we will segregate the approaches based on the relative use of this term. Given that we treat expression measurements as multidimensional trajectories, sampled at discrete points, the first definition of similarity ought to be based on some kind of point-wise metric measuring the distance among the various objects, using the relative mRNA abundance as the multidimensional feature set. Various metrics have been utilized, most notably Euclidean distances. Methods such as k-means and hierarchical clustering by and large fall under this family of approaches. Pair-wise comparisons are made and then combined to assess the relative degree of similarity. A second family of methods assumes the existence of a finite set of undetermined processes that generate the observations. Two trajectories are thus declared similar if they are the product of similar processes. Therefore, the comparison is made not in the finite-dimensional space of raw data, but in the infinite-dimensional space of the functions that generate this data. Finally, a third general family of methods defines the similarity in terms of global features and characteristics of the trajectories. Instead of focusing on point-wise differences or specific functional forms giving rise to data, these methods aim at identifying structural characteristics of the responses and define similarity based on pattern recognition methods that aim at finding salient changes and similarities in the responses.

A comprehensive review on clustering methods was recently presented in Reference 34. In the following section we attempt to partition qualitatively the methods by describing the essential characteristics of each approach.

Point-Wise Distance-Based Clustering Methods

Recently, a very nice and concise review of distance-based clustering methods was presented (35). Without loss of generality, we assume that the data are given in a matrix form:

$$\mathrm{E}=\left\{\mathrm{E}(\mathrm{i},\mathrm{t}\right\},\mathrm{i}=1,\dots ,{\mathrm{N}}_{\mathrm{g}},\mathrm{t}=1,\dots ,{\mathrm{N}}_{\mathrm{t}},$$

assuming the Ng genes are measured at Nt discrete time points. The goal of point-wise distance-based clustering methods (PwDbM) is to quantify the distance between any two samples and agglomerate samples that fall within a predefined threshold. Usual metrics include various definitions of norm-based distances and combinations of known correlation expressions. Indicative definitions are provided in Table 1.

Table 1.

Similarity metrics for PwDbM

L1-norm	${\mathrm{d}}_{\mathrm{ij}}={\sum }_{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}\mid \mathrm{E}(\mathrm{i},\mathrm{t})-\mathrm{E}(\mathrm{j},\mathrm{t})\mid$
2-norm	${\mathrm{d}}_{\mathrm{ij}}=\sqrt{{\sum }_{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}{(\mathrm{E}(\mathrm{i},\mathrm{t})-\mathrm{E}(\mathrm{j},\mathrm{t}))}^2}$
L∞-norm	${\mathrm{d}}_{\mathrm{ij}}=\underset{\mathrm{t}}{\max }\mid \mathrm{E}(\mathrm{i},\mathrm{t})-\mathrm{E}(\mathrm{j},\mathrm{t})\mid$
Mahalanoblis	${\mathrm{d}}_{\mathrm{ij}}={({\mathrm{E}}_{\mathrm{i}}-{\mathrm{E}}_{\mathrm{j}})}^{\mathrm{T}}{\Sigma }^{-1}({\mathrm{E}}_{\mathrm{i}}-{\mathrm{E}}_{\mathrm{j}})$
Correlation metric	$\begin{array}{c}\hfill {\mathrm{d}}_{\mathrm{ij}}=1-{\mathrm{r}}_{\mathrm{ij}}\hfill \\ \hfill {\mathrm{r}}_{\mathrm{ij}}=\frac{{\sum }_{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}(\mathrm{E}(\mathrm{i},\mathrm{t})-\stackrel{‒}{\mathrm{E}}\left(\mathrm{i}\right))(\mathrm{E}(\mathrm{j},\mathrm{t})-\stackrel{‒}{\mathrm{E}}\left(\mathrm{j}\right))}{\sqrt{{\sum }_{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}{(\mathrm{E}(\mathrm{i},\mathrm{t})-\stackrel{‒}{\mathrm{E}}\left(\mathrm{i}\right))}^2{\sum }_{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}{(\mathrm{E}(\mathrm{j},\mathrm{t})-\stackrel{‒}{\mathrm{E}}\left(\mathrm{j}\right))}^2}}\hfill \end{array}$

The fundamental difference between the various distance-based methods is in the way these distances are being combined to identify the proper partitioning of the data. Two major classes of methods exist: (a) partitioning and (b) hierarchical. Among partitioning methods, the prototypical example is k-means clustering, although self-organizing maps (SOM) also follow the same basic principles (15). Using these methods, a predetermined number of partitions of the feature space that are defined by a nominal center are constructed. Points are subsequently assigned to each partition based on their relative proximity to the center with the overall objective of minimizing the distance of each point from its respective center:

$$\text{Error}=\sum _{\mathrm{k}=1}^{{\mathrm{N}}_{\mathrm{k}}}\sum _{\mathrm{i}\in {\mathrm{C}}_{\mathrm{k}}}\sum _{\mathrm{t}=1}^{{\mathrm{N}}_{\mathrm{t}}}{\mid \mathrm{E}(\mathrm{i},\mathrm{t})-\mathrm{M}(\mathrm{k},\mathrm{t})\mid }^2.$$

In this general definition of the objective (Error) the objects (temporal gene expression of Ng genes) have been partitioned to Nk clusters. The purpose of the optimization search is to assign each profile to one of these clusters, such that the sum of the distances of each profile from the center of the cluster it has been assigned to, M(k, t), is minimized.

Recently, interesting combinations of k-means and kernel methods have emerged (36, 37). Essentially, kernel methods aim at identifying appropriate nonlinear transformations of the original data through the use of kernel functions that render the data linearly separable (38). The distances are then defined on the transformed data rather than the original. Even though kernel-based methods have a number of advantages, such as creating separability through transformations or relative robustness to noise, they do require the identification of a number of input parameters that would render the estimation problem user-specific, and the appropriate estimation of the necessary parameters is not trivial.

Hierarchical methods create a hierarchy of relative distances (hence the name) and place multidimensional points along a one-dimensional axis based on the relative distance between points. The result of the analysis is presented in the form of a dendogram in which the relative positions of points defines their relative distance as well. The dendogram is essentially a binary tree with the root representing the entire data set and each leaf node representing a data object. Intermediate nodes represent the extent to which objects are close to each other. Among the strongest criticisms raised for classical hierarchical clustering is the fact that these algorithms are lacking robustness to noise and are therefore sensitive to outliers.

The literature in terms of applications of distance-based clustering in the analysis of microarray data is really abundant. Typical examples of the applications of such methods include the work of Eisen and collegues and Gash and collegues (39, 40) for hierarchical clustering and the work of Tavazoie and collegues (41) using k-means clustering. Eisen et al. (39) clustered expression data in the budding yeast Saccharomyces cerevisiae to deduce that clustering gene expression data grouped together genes of known similar function, interpreting this observation as an indication of the status of cellular processes. They applied hierarchical clustering where the linkage was determined using a similarity score based on a correlation coefficient. Gash et al. (40) evaluated the response of yeast to numerous environmental perturbations to unravel the effects of environmental stresses on the cell. Tavazoie et al. (41) measured 15 time points across two cell cycles of Saccharomyces cerevisiae and analyzed the results using variance normalized profiles and k-means clustering. Clusters were subsequently characterized based on their relative functional enrichment to demonstrate the concentration of similar functions within each cluster.

The use of correlation-based distance metrics, such as Pearson's coefficient, represents a variation upon Euclidean distance by providing a scale-free distance metric between two feature vectors. This metric has been widely used as well and is particularly useful when the baseline magnitude of different mRNA messages differ greatly.

Model-Based Clustering Methods

Model-based clustering methods (MbCMs) (42–46) shift the similarity emphasis from the data to an unknown model that describes the data. These are methods based on variants of mixture-models (44). The general idea is that finite mixtures of distributions provide a flexible approach to modeling. Therefore, each point (i.e., expression profile) is taken to be the outcome of the superposition of a finite number of processes, much like expansion over a basis set, with a number of unknown parameters to be determined based on the available experimental data. Therefore, the objective is to identify this underlying set of functions (models) whose appropriate combination (mixture) assigns the data properly. The existence of such a finite and coherent set of basis functions indicates the existence of an underlying set of limited common processes that give rise to the observed behavior. Without loss of generality we will use the formalism of Pan et al. (42) to illustrate the approach. Let y denote any measurement, then each such data point is assumed to be the superposition of distributions given by

$$\mathrm{f}(\mathrm{y};{\mathrm{f}}_{\mathrm{g}})=\sum _{\mathrm{i}=1}^{\mathrm{g}}{\pi }_{\mathrm{i}}\phi (\mathrm{y};{\mu }_{\mathrm{i}},{\mathrm{V}}_{\mathrm{i}}).$$

The density function, ϕ, depends on appropriate parameters, μ and V, which, for example, could correspond to the mean and covariance matrix. These mixtures are appropriately weighted by means of the mixing proportion π. Thus the model to be estimated based on the data is composed of the triplet (π, μ, V), with parameters determined through the use of appropriate expectation-maximization algorithms.

It is important to realize the fundamental difference between the distance-based and the model-based approaches. The emphasis is now placed on the speculated underlying model, thus making the approach more robust in the presence of noisy data. However, the assumption is that such underlying processes do exist, requiring that the data follow a set of predetermined distributions. A slight variation was recently proposed in Reference 45, whereby an autoregressive model able to account for time delays was assumed to exist and was subsequently estimated based on the data. Similar in spirit are methods based on hidden Markov models (47–49), which assume an underlying HMM describing the sequence of events corresponding to the transformed temporal gene expression profiles. An interesting method was proposed (50) in which a linear dynamic model is invoked to simulate the level of mRNA that gives rise to time-dependent profiles, which are considered to be sums of exponentials. The associated parameters of the model are estimated through nonlinear regression. The number of exponentials is also minimized by making use of the concept of information theoretic arguments quantifying Occam's Razor, such as minimum description length (51) and Akaike information criterion (52). Model-based approaches have been proposed (53, 54) that consider the cell to be a system where the behaviors (responses) of the cell depend completely on the current internal state plus any external inputs, and the proposed method regards a time-course gene expression dataset as a set of time series generated by a number of stochastic processes. Each stochastic process defines a cluster and is described by an autoregressive model. Along those lines, significance analysis of time-course microarray experiments was also recently proposed as a competitive alternative (55). The method is applicable to detecting changes in expression over time within a single biological group and to detecting differences in the behavior of expression over time between two or more groups.

In summary, the fundamental assumption of model-based approaches is that the expression profiles are clusters in the space of the functionals that characterize them. The question thus becomes how to identify this functional decomposition of the data, as opposed to decomposing the raw data. One of the key drivers for such methods is the speculation that gene expression profiles are generated by time-dependent models, in the sense that the current state is a function of the cellular state at previous times (45). Therefore, these methods attempt to quantify this assumption.

Feature-Based Clustering Methods

Feature-based clustering methods (FbCMs) aim at detecting salient features and local or global shapes characteristic of the expression profiles. One of the key motivating arguments for such methods is the realization that in the presence of noise and uncertainties associated with measuring mRNA abundance, looking for specific quantifiable metrics may not necessarily yield the most informative interpretation. Instead, robust, coherent, and dominating qualitative features and similarities could be a more informative proxy for the information content of the expression experiment. The raw data are transformed to sequences of events or symbols, and these are further analyzed for consistencies, either local or global (56). Looking for general shapes as opposed to quantifying distances allows for, among other things, a more flexible representation, which uncovers more intricate relations among expression profiles, such as time shifts and inversion in expression profiles (57). Syeda-Mahmood (58) has proposed a pattern recognition approach aimed at capturing salient features of the time-varying gene expression patterns, such as inflection points based on the idea that dissimilar curves, when represented as two-dimensional curves, show a significant number of twists and turns. A new framework was recently proposed (59, 60). Both approaches share a critical similarity: The transformation of the raw expression data to a sequence of symbols and the subsequent analysis of the symbolic representation of the time series. This type of approach, motivated by recent advances in the symbolic representation of streaming data (61), effectively reduces the dimensionality of the time series from an infinite-dimensional space (continuous representation of expression level) to a finite, quantized representation where each profile is represented by a sequence of symbols. In effect, the most significant variation introduced by these methods is a fine-grained clustering, with a potentially enormous number of clusters defined.

There have been subsequent significant variations in both methods. One is based on the relative probabilities of each symbolic sequence (59) and the other is based on the ability of selected subsets to reproduce the overall dynamic response (60), with selection criteria ranking the importance of the respective clusters. Because the method proposed by Ernst & Bar-Joseph (59) needs to postulate a priori the putative sequence of events, the method is best suited for short time series, whereas the method proposed Yang et al. (60) has complexity that is effectively linear with respect to the number of genes.

Interesting algorithms for clustering (expression) data are emerging, exploring graph-theoretic properties. We discuss them in Feature-Based Clustering Methods because essentially the structure of the graph created from the original data is analyzed. In other words, in graph-based methods, the nature, structure, properties, and characteristics of a graph whose edges represent data points and the arcs relative distances between those points are treated as the features to be further analyzed. Thus, subgraphs are formed and identified containing enough nodes for effective similarity computations. Effectively, a tree representation converts the multidimensional problem to a tree partitioning problem.

Among the most popular methods are those that explore the concept of the minimum spanning tree (62) and effectively attempt to identify cliques within the data set (63, 64). An interesting extension of the MST concept is discussed in Reference 65 where a metric for assessing the clustering potential based on geometric arguments is presented. Assessing the “clusterabolity” potential of a dataset a priori will greatly enable further analyses. In the case of temporal data, this remains an open question.

FbCMs offer a higher degree of flexibility. Appropriate selection of characteristic features offers the possibility of defining a time-course representation using variables that potentially capture intrinsic and implicit characteristics of the responses. Undoubtedly, the definition of such features results in some kind of lumping of the response, which can potentially result in loss of fine-grain detail.

Clustering Across Conditions

Each gene expression experiment is essentially a set of observations generated from a single perturbation of the system, whether it is a particular growth condition, an injury, or the administration of a drug. It can be argued that extracting information from a single perturbation contains little information. Therefore, increased methods that attempt to simultaneously analyze multiple conditions are continually attracting increased attention (66). Bi-clustering in the context of gene expression analysis was first suggested by Church (67) and refers to simultaneous clustering across “columns” and “rows” in expression data by expanding the concept of similarity so that it does not become a function of pairs of genes or pairs of conditions, as is normally the case, but rather it becomes a measure of coherence of the genes and conditions. Heard and coworkers (68) expanded their original Bayesian model-based agglomerative clustering scheme (69) for time-course data. Their approach uses a spline approximation to capture the temporal variation within each cluster. The approach is particularly intriguing in that the time courses are explicitly treated. Undoubtedly, biclustering (or coclustering) methods hold tremendous promise as more systemic perturbations are becoming available and the need to develop consistent representations across multiple conditions is required. The underlying assumption, as argued below, is that biological systems are treated as systems in which external perturbations are applied. Therefore, the underlying dynamics should be consistent across conditions independent of the type of the perturbation to assess the biologically informative nature of conclusions drawn from any kind of computational analysis of transcriptional responses.

Summary

Clustering can be characterized as the process of establishing associations. At a conceptual level, the nature of the associations becomes more abstract as the methods evolve from hierarchical, to partition, to model, to feature based. Figure 1 depicts this increased level of abstraction. Hierarchical clustering would basically associate the two convex and the concave hypothetical patterns of expression by quantifying the relative differences among all members. K-means or SOM will draw an association between the raw profiles and the putative centers of the domains within which each profile lies (centers indicated by the dashed lines). Model-based methods will establish the association between individual profiles and functional representation according to the values of the model parameters. Objects therefore are associated with sets of parameters. Finally, feature-based methods will associate each profile with macroscopic features characteristic of the overall shape of the response. The relative distance between transformed individual members define the proximity. One could argue that even feature-based methods are essentially distance-based methods. However, the transformation from raw data to features relaxes the proximity restrictions and allows for the introduction of soft comparisons.

Figure 1.

Notional comparison of clustering methods. Given the four hypothetical trajectories, a distance-based method will compare the similarities pointwise, potentially creating an appropriate dendogram quantifying such distances. A model-based approach will attempt to quantify a functional description of the data in the form of a generalized model “f,” whereas a feature-based method will attempt to identify critical features, such as a sequence of events or trends, shared by various elements.

CHALLENGES

Clustering is by definition an unsupervised task. We can loosely define clustering as the process of organizing objects (expression profiles) into groups whose members are similar in some way. In evaluating the effectiveness of clustering, one could argue that if the groups are similar given some metric, then the clustering was successful. However, the fascinating thing in biology is that similarity in the input space is not the final arbitrator. If the genotype is the input, the actual observable is the phenotype. The information encoded by the objects of the clusters, the mechanisms that brought the objects together, the implications of bringing the objects together, in a nutshell, the biological insight gained by analyzing the objects that were brought together is what will decide the effectiveness of the computational analysis. Therefore, defining the quality of the clustering algorithms is not as straightforward as it may appear. A major challenge in the clustering of microarray data lies in the fact that the metric for evaluating the overall quality of a result is still an open area of research (70). Without a well-defined metric, it becomes difficult to ascertain which method outperforms the others.

Various evaluations have been proposed to quantify the relative advantages of clustering methods for microarray expression data (71, 72), and the metrics for comparison quantified the ability of various methods to generate well-separated clusters. By and large any such comparison is biased and the results to a great extent depend on the specific use of the method as well as the nature and type of data. We believe that a head on comparison between clustering methods based exclusively on some optimality criterion will probably be misleading. The complexities of the underlying biological system will probably render such analysis mute. Methods should be evaluated, and not compared, based on their ability to generate insight information and it is quite possible that the evaluation could be problem dependent. It is critical to realize that the computational steps, in the context of transcriptional analysis, should be an integrated component of the overall effort and a separate independent activity. Therefore, the effectiveness of a computational approach should be evaluated in the grand scheme of the biological content of a specific analysis.

In the sections that follow, we identify three elements of the computational analysis of time-course gene expression data that we believe could potentially impact the conclusions drawn. Thorough and detailed analyses of the challenges associated with high-dimension clustering in general have been nicely presented elsewhere (73).

Small Sample Size: Information or Noise?

The term “data deluge” is often used in conjunction with microarray data (74). However, this could not be a more misleading characterization. There is no doubt that the observables in a microarray experiment are in the thousands, particularly in temporal experiments. A typical animal study with m replicates (animals) at n time points recording k genes would produce m × n × k data points. However, the number of objects, in terms of the machine-learning problem, is quite minimal, and definitely not up to par with the number of features. Examples of the types of objects we refer to include number of patients in a cancer study or number of system perturbations (types/severity of trauma, or drug dosing).

Technological and other practical limitations severely restrict either the number of time points that can be measured or, more importantly, the number of biological and technical replicates that can be used. In the machine-learning community, this is an age-old problem known as learning in almost empty spaces (75). In such cases, it is quite difficult to distinguish noise from structure unless something is known about the underlying concept generating the data. A simple, yet informative example, of errors introduced by subsampling is presented in Reference 76. New technologies that are emerging, such as the living cell array (77), which will provide extensive data at least for model systems, will expose the host system to a wide range of insults and will create a more integrated list of cause-effect relationships. Currently, the only way to condition the data to overcome the lack of a critical mass of observations is to couple the expression data with available prior biological information and analyze simultaneously multiple perturbations. The inability of sparse data to properly capture the complexity of a classification problem is also discussed in Reference 76; however, recent advances in theoretical work on clustering sparse data (78–80) will significantly help.

As noted above, a key complexity of microarray experiments is the essential lack of observables (cell lines or tissue samples) to support the large number of probes monitored. The consequences of the small ratio of features to samples in microar-rays was discussed in Reference 81 and a nice discussion of the impact of the small sample size problem in array expression data is presented in References 82 and 83, which comment on the required optimal number of samples required for robust estimation under certain assumptions regarding the distribution of the measurements. The implication of the ratio of features to samples is critical, as sparsely populated datasets can very easily lead to random features appearing to be informative. It should be expected that simple minimization of the number of features (genes) in a model need not necessarily provide the best answer. Additional complexity restrictions will have to be imposed to balance the lack of available data, although no definite answer can be provided as no analysis can replace accurate and adequate data. Recently, a novel method for characterizing the information content of short time-course gene expression data was presented by effectively quantifying the random nature of the signal encoded in the expression time series (84).

Knowledge-Based Clustering

Although genome-wide mRNA expression analysis is slowly becoming a routine tool, translating computational results to biological information remains a major challenge. As previously mentioned, one of the key challenges is the improper conditioning of the data. Approaches are being developed that attempt to integrate prior knowledge into the analysis of expression data. In a report by Pan (85), the mixture model for clustering expression data is extended to incorporate gene ontology information as prior knowledge to increase the specificity of the method (85–87). To take advantage of accumulating gene functional annotations, Huang & Pan (88) proposed incorporating known gene functions into a new distance metric that shrinks a gene expression-based distance toward 0 if and only if the two genes share a common gene function.

The incorporation of biological (or any type of prior knowledge) into clustering algorithms will be greatly enabled by recent advances in the area of constraint-based clustering (89, 90) which aims at developing consistent methodologies that incorporate prior knowledge during the analysis, as opposed to postprocessing the results to validate the consistency of the conclusions given what is known about the system. However, one needs to be aware of the constraints that explicit, hard modeling of prior knowledge imposes in terms of discovering new knowledge about the system: Over-restricting and constraining the analysis goes against the very essence of integrative -omics approaches and data-driven (systems) approaches, as opposed to hypothesis-driven research.

Judging the Quality of Gene-Expression Clustering

Clearly there are a number of analytical rationales used to parse genes into groups. However, the quality of all grouping must be judged based on their ability to provide insight into the underlying mechanistic biology. A general concern regarding the validity of existing algorithms stems from the observation that classification algorithms can lead to conflicting results, which are often method dependent (91). The current practice is to evaluate methods based on their ability to generate results consistent with biological reality in terms of functional ontologies and putative transcription factors of coexpressed genes (92–95). Although it is not surprising that different methods yield different results, the fact is, there is a correct answer. Living organisms exist as a fine balance between entropy and enthalpy. Maintaining such a balance requires that the expression of the thousands of genes in the organism's genome be highly coordinated. At any point in time, the amount of mRNA for a particular gene is the balance between its synthesis and degradation. Processes such as circadian rhythms or input perturbations such as drugs can change the amount of an mRNA by increasing or decreasing synthesis, degradation, or some combination of both. The power of the gene array time series is that it allows the observer to broadly “watch” the dynamics of the system. The objective in conducting time-series experiments is to understand the complex sequence of regulatory events that drives the system. Clustering, regardless of method, attempts to parse genes into groups with certain defined commonalities.

These groups are useful to the biologist to the degree that they represent genes with common mechanisms of regulation. In essence, each proffered group represents a testable hypothesis. If the hypothesis is correct, then certain biological requirements follow. For example, if a group of genes is regulated by a common mechanism, then their response to a different input perturbation should be the same. On the one hand, if the process being examined is natural, such as development, cell cycle, or circadian rhythm, then a perturbation that disrupts the natural process should change the profile over time of all genes in a cluster. To the degree that it does not, then it suggests that the cluster is not entirely valid. On the other hand, if the process being examined is an input to a biological system, such as a drug treatment, then genes that belong in the same cluster should have the same response profile regardless of dosing regimen. In reality, a single temporal response profile probably does not provide sufficient constraint to accomplish biologically valid mechanistic clustering. A second test of the validity of clusters involves the mechanism of control of gene expression. The expression of genes is controlled by transcription factors (TFs). TFs are gene products, proteins that bind to specific sites in the DNA and either promote or inhibit the expression of a gene. Some TFs act on their own or in combination with other TFs, whereas some, such as the glucocorticoid receptor and the estrogen receptor, require the binding of an external ligand for activation. If a group of genes is regulated by a common mechanism, then they should contain common features in their regulatory regions. However, because transcription binding site (TFBS) motifs are short (5–9 base pairs) and fairly degenerate, most putative TFBS matches occur by chance alone and are not functional. One method that has been proposed to identify which TFBS are bona fide functional sites is excluding those that are not in evolutionarily conserved regions. Indeed, the upstream noncoding regions do not evolve in a uniform fashion among sites, but rather show blocks of fairly conserved areas interspersed with fast-evolving stretches. These fast-evolving stretches quickly lose homology with evolutionary time and are subject to insertions and deletions. But nonetheless, even among comparatively distant taxa (e.g., rodents and humans), conserved, alignable segments are preserved (96, 97). Identifying common features in the regulatory region of genes in putative clusters not only provides a degree of validation but also should provide insight into the mechanism of regulation.

OPPORTUNITIES

The traditional way of interpreting time-course expression data is to evaluate the biological similarities implied as a result of expression profile similarities, and currently an enormous number of publications have been presented advocating the potential for such analyses (12, 13, 15, 57, 98).

A healthy body of literature utilizes time-course expression data to reverse-engineer primarily regulatory networks, given that interference at the level of regulation holds significant promise for drug discovery. Targeting expression by controlling the regulatory process through the corresponding transcription factors is emerging as a viable option for the identification of drug targets (99, 100) and controlling disease progression (101). In recent years, significant efforts have been made experimentally, and computationally, to identify transcription factors, their target genes, and the interaction mechanism that control (regulate) gene expression (102, 103). Prominent examples are the decomposition-based methods, which combine ChiP and microarray data, and inversion of regression techniques to estimate transcription factor activities (TFAs) (104–107). Singular value decomposition and regression methods were combined (19) to reverse-engineer regulatory networks, and in a report by Bussemaker et al. (108), promoter elements were linearly combined to quantify the contribution of the promoter architecture on a gene's expression. Network component analysis (NCA) (109–113) was introduced as an alternative for quantifying the strength of the regulatory interactions and for elucidating true TFAs. Similarly, others (114) explored a linear superposition of expression profiles and TFA combined appropriately using binding affinities in lieu of stoichiometric coefficients, and a Bayesian error analysis of an, effectively, linear method was presented in Reference 114. The main goal of this reverse-engineering is to identify the activation program of transcription modules under particular conditions (115) so as to hypothesize how activation/deactivation of expression can be induced/suppressed (116). A fundamental difference among the methods is whether the weights of the approximation should be estimated through regression (109–113) or associated with binding affinities (114). To gain more mechanistic insight, recent approaches aim at combining time-course expression data and semimechanistic models of gene expression in an effort to evaluate the kinetics of gene expression. In a report by Yugi et al. (117), a microarray data-based kinetic method (MASK) was proposed that combined expression data with RNA synthesis kinetic models to evaluate kinetics parameters of the expression activation and repression processes, whereas in a report by Thomas et al. (118), the so-called S-system framework was explored (119).

Truly fascinating, however, are the opportunities offered by combining time-course expression data with mechanistic-models of expression in the form of pharmacokinetic/pharmacodynamic expressions. Using gene arrays in the time-series paradigm can provide the scope of data necessary for analyzing dynamic complex biological phenomena. The time series can capture the dynamic nature of processes such as disease progression or drug responses, whereas the gene arrays provide a method of high-throughput data collection necessary to address the complexity. For years, complex pathologies such as diabetes, hypertension, and obesity have, for the most part, been addressed one or two genes at a time. Although such pathologies may be instigated by a single gene defect induced through gene knockout, in general, this is not the case. For example, a major animal model for obesity and related pathologies is the ZDF rat (120). This rat contains a single gene defect, the leptin receptor. In reality, this is not the human condition. Obesity and metabolic syndrome results from a complex interplay of many genes in multiple tissues (121). However, taking advantage of the opportunity to analyze such dynamic complex biological phenomena requires quantitative approaches that are able to accommodate both the dynamics and the scope of data. Indirect effect mathematical modeling provides an approach to addressing this problem. Although developed for pharmacokinetics and pharmaco-dynamics, the basic approach can be applied to any dynamic biological system (122). The basic premise of such modeling is that a measured response (R) to an input perturbation may be produced by indirect mechanisms; for example, factors controlling the input or production of the response variable (k_in) may be either inhibited or stimulated, or the determinants of loss of the response variable (k_out) may be inhibited or stimulated. The rate of change of the response over time with no input perturbation present can be described by

$$\frac{\mathrm{dR}}{\mathrm{dt}}={\mathrm{k}}_{\mathrm{in}}-{\mathrm{k}}_{\mathrm{out}}\cdot \mathrm{R}\mathrm{R}\left(0\right)={\mathrm{R}}_0,$$

where k_in represents the zero-order constant for production of the response and k_out defines the first-order rate constant for loss of the response. It is assumed that k_in and k_out fully account for production and loss of the response. The response variable R may be a directly measured entity or an observed response, which is immediately proportional to the concentration of R. The basic assumption that both production and loss can be stimulated or inhibited leads to the four basic equations shown in Figure 2. The initial input perturbation is used as the driving force for the primary response or set of responses. However, the primary response(s) can then be employed as the driving force for a set of secondary responses. Using this approach, dynamic models for ever-more complicated converging and diverging sequences of molecular events in time can be constructed. For example, suppose a drug enhances the expression of two different transcription factors. These represent primary responses. If these transcription factors change the expression of other genes, then these become secondary responses. Changes brought by these genes become tertiary responses. In this way, the use of the four basic models can be used to construct experimentally testable models for quite complex response cascades. However, clustering of gene array time series data not only provides the foundation of such dynamic models but also determines their validity.

Figure 2.

Four basic mechanism-based indirect effect models for response dynamics indicating production and consumption inhibition, and production and consumption stimulation.