Skip to main content
NCBI home page
Journal of Research of the National Bureau of Standards logoLink to Journal of Research of the National Bureau of Standards
. 1985 Nov-Dec;90(6):479–483. doi: 10.6028/jres.090.045

Optimization

Stanley N Deming 1
PMCID: PMC6644965  PMID: 34566184

Abstract

Most research and development projects require the optimization of a system response as a function of several experimental factors. Familiar chemical examples are the maximization of product yield as a function of reaction time and temperature; the maximization of analytical sensitivity of a wet chemical method as a function of reactant concentration, pH, and detector wavelength; and the minimization of undesirable impurities in a pharamaceutical preparation as a function of numerous process variables. The “classical” approach to research and development involves answering the following three questions in sequence:

  1. What are the important factors? (Screening)

  2. In what way do these important factors affect the system? (Modeling)

  3. What are the optimum levels of the important factors?

As R. M. Driver has pointed out, when the goal of research and development is optimization, an alternative strategy is often more efficient:

  1. What is the optimum combination of all factor levels? (Optimization)

  2. In what way do these factors affect the system? (Modeling in the region of the optimum)

  3. What are the important factors?

The key to this alternative approach is the use of an efficient experimental design strategy that can optimize a relatively large number of factors in a small number of experiments. For many chemical systems involving continuously variable factors, the sequential simplex method has been found to be a highly efficient experimental design strategy that gives improved response after only a few experiments. It does not involve detailed mathematical or statistical analysis of experimental results. Sequential simplex optimization is an alternative evolutionary operation (EVOP) technique that is not based on traditional factorial designs. It can be used to optimize several factors (not just one or two) in a single study. Some research and development projects exhibit multiple optima. A familiar analytical chemical example is column chromatography which often possesses several sets of locally optimal conditions. EVOP strategies such as the sequential simplex method will operate well in the region of one of these local optima, but they are generally incapable of finding the global or overall optimum. In such situations, the “classical” approach can be used to estimate the general region of the global optimum, after which EVOP methods can be used to “fine tune” the system. For example, in chromatography the Laub and Purnell “window diagram” technique can often be applied to discover the general region of the global optimum, after which the sequential simplex method can be used to “fine tune” the system, if necessary. The theory of these techniques and applications to real situations will be discussed.

Keywords: optimization, screening, simplex

1. Introduction

Most research and development projects require the optimization of a system response (dependent variable) as a function of several experimental factors (independent variables). Familiar chemical examples are:

  1. re-establishing acceptable product yield as a function of reaction time and reaction temperature after a design change in a chemical process;

  2. maximizing the analytical sensitivity of a wet chemical method as a function of reactant concentration, pH, and detector wavelength;

  3. tuning-up a nuclear magnetic resonance spectrometer by adjusting eleven highly interacting shim coil controls to produce optimum peak shape.

  4. finding a combination of values for eluent variables that will give adequate separation in high performance liquid chromatography.

Although “optimization” is often taken literally to mean making something “as perfect, effective, or functional as possible” [1]1, in chemical practice it usually means making something “acceptable” or “adequate,” as in examples one and four above. Optimization in chemistry usually involves adjusting a system until it is brought to some desired threshold of performance.

The dual purposes of this short paper are to discuss several strategies for the optimization of chemical systems and to discuss strengths, weaknesses, and appropriate settings for each approach. The intent of these comments is not to suggest rigid guidelines for the proper uses of optimization methods, but rather to stimulate discussion directed toward a better understanding of how these methods can be used in practice.

2. Classical Experimental Designs

The “classical” approach to optimization in research and development involves answering the following three questions in sequence:

  1. What are the important factors? (SCREENING)

  2. In what way do these important factors affect the system? (MODELING)

  3. What are the optimum levels of the important factors? (OPTIMIZATION)

Classical experimental designs (e.g., fractional factorial designs and central-composite designs [2,3]) can be used to screen factors and to acquire data for modeling the system as a function of the most important variables. The resulting model can then be used to predict the treatment combination (experimental conditions) giving the optimum response [4–6]. The statistical literature is rich in examples showing how statistically designed experiments have been used in this way to solve significant chemical problems (e.g., [7]).

A. Modeling

The critical part of the classical approach is the second step, modeling. At the very least, a model that fits reasonably well over a limited region of the factor space can be used to predict a direction to move to obtain improved response (as in evolution operation, or EVOP [8]). A model that fits well over a larger region of factor space is, of course, even more useful. However, if the (usually empirical) model contains more than a few factors, then the number of experiments required to fit the model will be impractically large. For example, if a full second-order polynomial model containing k factors is used to model the system, the number of model parameters will be equal to (k + l)(k +2)/2; for five, six, seven, and eight factors the numbers of model parameters are 21, 28, 36, and 45, respectively. At least this many experiments must be carried out to provide data for the estimation of the parameter values; typically, central composite designs are used which require 2k+3k + 1 experiments (plus three replicates to estimate “pure error”) for a total of 46, 80, 146, and 276 experiments for five, six, seven, and eight factors, respectively.

Thus, a desire to avoid extraordinarly large numbers of experiments becomes a strong driving force for limiting (typically to only thee or four) the number of factors to be investigated by classical experimental designs. Hence, the need for the initial screening of factors to choose only the most important ones.

B. Screening

There are problems with screening experiments. For example, most screening experiments are based on first-order models which assume no interactions. If interactions do exist, then factors which truly have a significant affect on the system might not appear to be statistically significant and would be discarded by the screening process.

A second problem with screening experiments can occur if the effect of a factor depends upon its own level “self interaction”). If screening experiments are carried out in a region where the response is “flat” with respect to the factor of interest (a stationary region), that factor will not appear to a be very significant when, in fact, at different levels of that factor, the effect on response might be considerable.

As a final example of difficulties in screening for significant factors, the “wrong” statistical test is usually used when screening factors for their significance. It is true that if a factor is “significant at the 95% level of confidence,” then it is probably an important factor and should be retained for further investigation. However, if a factor is “not significant at the 95% level of confidence,” it does not mean that it is an unimportant factor and can be neglected. It might, for example, be significant at the 94.73% level of confidence, not enough to exceed the common threshold of 95% confidence, but still highly significant nonetheless. Ideally, the question that should be asked while carrying out screening experiments is not “which factors are significant at some high level of confidence,” but rather, “which factors are insignificant at some equally high level of confidence.” Unfortunately, the type of experimentation required to answer this second question is extensive and expensive. An alternative approach is to increase the risk (alpha) of stating that a factor is significant when in fact it is not, so that fewer truly significant factors are rejected [9].

C. Comments on the use of classical experimental design for optimization

Classical experimental designs appear to have been successful in the past for “optimizing” many existing chemical systems largely because these systems are usually run not at the true optimum but rather are operated at some “threshold of acceptability.” A response surface view of this would be that the system is being run at a point on the side of a hill; not at the top of the hill, but far enough up on the side that the system gives acceptable performance. As long as the response surface maintains its shape and position, and as long as the factor levels are kept in statistical control, the system will perform acceptably.

However, if the response surface changes its shape or “moves” slightly (as a result, for example, of scale buildup in heat exchangers, or different suppliers of feed stocks), then the previous set of factor levels might no longer produce adequate performance from the system: the same set-point will now correspond to some slightly lower position on a changed response surface. In situations like this, small screening experiments (such as saturated fractional factorial designs [2] or Plackett-Burmann designs [10]) are not too much affected by factor interactions and are likely to give nearly true estimates of the first-order factor effects (e.g., the effect of temperature, or the effect of increased amounts of feed stocks). Similarly, a first-order model offers a good approximation to the true shape of the response surface over a limited region. Thus, the application of screening experiments to choose the most significant factors is usually successful in this application. When these most significant factors are used in a model of the system that is first-order with interactions (fitted, say, to the results of a two-level factorial design), then the fitted model will usually suggest an appropriate direction to move. Changing the factor levels in this direction will usually move “up” the hill to a point lying above the threshold of performance and once again achieve adequate (“optimum”) response from the system.

3. Sequential Simplex Optimization

As R.M. Driver has pointed out [11], when the goal of research and development is optimization, an alternative strategy is often more efficient. This alternative strategy asks essentially the same questions as the classical approach to optimization, but it asks the questions in reverse order:

  1. What is the optimum combination of all factor levels? (OPTIMIZATION)

  2. In what way do these factors affect the system? (MODELING in the region of the optimum)

  3. What are the important factors? (SCREENING for effects in the region of the optimum)

The key to this alternative approach is the use of an efficient experimental design strategy that can optimize a relatively large number of factors in a small number of experiments. Once in the region of the optimum, classical experimental designs can be used to full advantage to model the system and determine factor importance in a limited region of the total factor space.

A. Ignoring initial Screening Experiments and Avoiding Models

For many chemical systems involving continuously variable factors and relative short times for each experiment, the sequential simplex method [12–38] has been found to be a highly efficient experimental design strategy that gives improved response after only a few experiments. It is a logically-driven algorithm that does not involve detailed mathematical or statistical analysis of experimental results. Sequential simplex optimization is an alternative evolutionary operation (EVOP) technique that is not based on traditional factorial designs.

There are two reasons for the efficiency of the sequential simplex method. The first reason is the number of experiments required in the experimental design itself. A simplex is a geometric figure containing a number of vertexes equal to one more than the number of dimensions of the factor space. Each vertex locates a treatment combination in factor space. Thus, the number of experiments required for a simplex is k + 1 where, again, k is the number of factors. Thus, a five, six, seven, or eight factor system would require only 6, 7, 8, or 9 experiments to define a simplex.

The second reason for the efficiency of the sequential simplex method is that it takes only one or two additional experiments to move the experimental design into an adjacent region of factor space. This is independent of the number of factors involved. When classical experimental designs are used in this type of “evolutionary operation” mode, a larger number of experiments (at least half of the factor combinations in the pattern) is usually required to move the experimental design into an adjacent region of factor space.

In our experience with the simplex, systems of up to 11 factors can be brought into the region of the optimum in only 15 or 20 experiments after the initial simplex has been constructed.

B. Limitations

The simplex does have its limitations, however. The system must be in “statistical control” if the simplex is to be used—that is, the system should have only a small amount of purely experimental uncertainty (“pure error”), It is recommended that after the initial simplex has been evaluated and before the first simplex move is begun, the vertex giving the worst response and the vertex giving the best response be repeated two more times each to evaluate the reproducibility of the system. If the reproducibility is good, then the simplex will progress well; if the reproducibility is poor, then the simplex will tend to wander. In this latter case, steps should be taken to improve the purely experimental uncertainty of the system; if this is not possible, then classical experimental designs offer advantages because of their noise-reducing capabilities [39].

The system should not drift with time. However, changes with time can often be detected and corrected for by running periodic experiments at a standard treatment combination.

The time of any one experiment must be relatively short. It has been suggested that the reason factorial experiments were developed before the sequential simplex was because of the experimental environment, specifically the improvement of agricultural crop yields. In this context, factorial experiments offer a great advantage in that several experiments can be carried out simultaneously and many results can be obtained after only one growing season. If the sequential simplex were to be used to improve agricultural production, only one move could be carried out each year and it could take several decades to optimize production.

Finally, the simplex is most powerful for continuous (“quantitative”) variables. It can be used for discrete variables where there are several levels—perhaps at least five or six—and the levels can be logically ranked. It can not be used for unranked discrete (“qualitative”) variables.

4. Systems Possessing Multiple Optima

Some research and development projects exhibit multiple optima. A familiar analytical chemical example is column chromatography which often possesses several sets of locally optimal conditions [40]. The reason for the existence of multiple optima is related to the phenomenon of changes in the order of elution with changing chromatographic conditions. EVOP strategies such as the sequential simplex method will operate well in the region of one of these local optima, but they are generally incapable of finding the global or overall optimum [23]. In such situations, classical factorial-type experiments can be used to fit models which in turn can be used to estimate the general region of the global optimum, after which EVOP methods can be used to “fine tune” the system. For example, in chromatography the Laub and Purnell “window diagram” technique [40] can often be applied to discover the general region of the global optimum, after which the sequential simplex method can be used to “fine tune” the system, if necessary [41–49].

Biography

About the Author: Stanley N. Deming is with the Department of Chemistry at the University of Houston-University Park.

Footnotes

1

Figures in brackets indicate literature references.

References

  • [1].Webster’s New Collegiate Dictionary, G. & C. Merriam Company, Springfield. MA: (1977), p. 806. [Google Scholar]
  • [2].Box G.E.P.; Hunter W.G. and Hunter J.S., Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, Wiley, New York, NY: (197B). [Google Scholar]
  • [3].Deming S.N. and Morgan S.L., The Use of Linear Models and Matrix Least Squares in Clinical Chemistry, Clin. Chem, 25, 840 (1979). [PubMed] [Google Scholar]
  • [4].Box G.E.P., and Wilson K.B., On the Experimental Attainment of Optimum Conditions, J. Roy, Stat. Soc., Ser. B, 13, 1 (1951). [Google Scholar]
  • [5].Box G.E.P., The Exploration and Exploitation of Response Surfaces: Some General Considerations and Examples, Biometrics, 10, 16 (1954). [Google Scholar]
  • [6].Box G.E.P., and Youle P.V., The Exploration and Exploitation of Response Surfaces: An Example of the Link Between the Fitted Surface and the Basic Mechanism of the System. Biometrics, 11, 287 (1955). [Google Scholar]
  • [7].Davies O.L., Ed., Design and Analysis of Industrial Experiments, 2nd ed., Hafner, New York, NY: (1971). [Google Scholar]
  • [8].Box G.E.P., and Draper N.R., Evolution Operation: A Statistical Method for Process Improvement, Wiley, New York, NY: (1969). [Google Scholar]
  • [9].Wilson E.B. Jr., An Introduction to Scientific Research, McGraw-Hill, New York, NY: (1952), p. 59. [Google Scholar]
  • [10].Plackett R.L., and Burman J.P., “The Design of Optimum Multifactorial Experiments,” Biometrika, 33, 305 (1946). [Google Scholar]
  • [11].Driver R.M., Chem. Brit., 6, 154 (1970). [Google Scholar]
  • [12].Spendley W.; Hext G.R., and Himsworth F. R., Sequential Application of Simplex Designs in Optimization and Evolutionary Operation, Technometrics, 4, 441 (1962). [Google Scholar]
  • [13].Nelder J.A., and Mead R., A Simplex Method for Function Minimization, Computer J., 7, 308 (1965). [Google Scholar]
  • [14].Deming S.N., and Morgan S.L., Simplex Optimization of Variables in Analytical Chemistry, Anal. Chem., 45, 278A (1973). [Google Scholar]
  • [15].Deming S.N. and King P.G., Computers and Experimental Optimization, Research/Development, 25(5), 22 (1974). [Google Scholar]
  • [16].King P.G., Automated Development of Analytical Methods, Ph.D. Dissertation, Emory University, Atlanta, GA: (1974). [Google Scholar]
  • [17].Morgan S.L., and Deming S.N., Simplex Optimization of Analytical Methods, Anal. Chem., 46, 1170 (1974). [Google Scholar]
  • [18].King P.G., and Deming S. N., UNIPLEX: Single-Factor Optimization of Response in the Presence of Error, Anal. Chem., 46, 1476 (1974). [Google Scholar]
  • [19].Yarbro L.A., and Deming S.N., Selection and Preprocessing of Factors for Simplex Optimization, Anal. Chim. Acta, 73, 391 (1974). [Google Scholar]
  • [20].King P.G.; Deming S.N., and Morgan S.L., Difficulties in the Application of Simplex Optimization to Analytical Chemistry, Anal. Lett., 8, 369 (1975). [Google Scholar]
  • [21].Parker L.R. Jr., Morgan S.L., and Deming S.N., Simplex Optimization of Experimental Factors in Atomic Absorption Spectrometry, App. Spectrosc., 29, 429 (1975). [Google Scholar]
  • [22].Dean W.K.; Heald K.J., and Deming S.N., Simplex Optimization of Reaction Yields, Science, 189, 805 (1975). [DOI] [PubMed] [Google Scholar]
  • [23].Morgan S.L., and Deming S.N., Optimizaktion Strategies for the Development of Gas-Liquid Chromatographic Methods, J. Chromatogr., 112, 267 (1975). [Google Scholar]
  • [24].Olansky A.S., and Deming S.N., Optimization and Interpretation of Absorbance Response in the Determination of Formaldehyde with Chromotropic Acid, Anal. Chim. Acta, 83, 241 (1976). [Google Scholar]
  • [25].Deming S.N., Morgan S.L., and Willcott M.R., Sequential Simplex Optimization, Amer. Lab., 8(10), 13 (1976). [Google Scholar]
  • [26].Turoff M.L.H. and Deming S.N., Optimization of the Extraction of Iron (if) from Water into Cyclohexane with Hexafluorcacetylacetone and Tri-n-Butyl Phosphate,” Talanta, 24, 567 (1977). [DOI] [PubMed] [Google Scholar]
  • [27].Deming S.N. and Morgan S.L., Advances in die Application of Optimization Methodology in Chemistry, Chapter I in Kowalski B.R., Ed., Chemometrics: Theory and Application, ACS Symposium Series 52, American Chemical Society, 1977, p. 1. [Google Scholar]
  • [28].Deming S.N., Optimization of Experimental Parameters in Chemical Analysis, Chapter 5 in DeVoe J.R., Ed., Validation of the Measurement Process, ACS Symposium Series 63, American Chemical Society, 1977, p. 162. [Google Scholar]
  • [29].Olansky A.S., Parker L.R. Jr., Morgan S.L., and Deming S.N., Automated Development of Analytical Chemical Methods. The Determination of Serum Calcium by the Cresolphthalein Complexone Method, Anal. Chim. Acta, 95, 107 (1977). [DOI] [PubMed] [Google Scholar]
  • [30].Deming S.N. and Parker L.R. Jr., A Review of Simplex Optimization in Analytical Chemistry, CRC Crit. Rev. Anal. Chem., 7, 187 (1978). [Google Scholar]
  • [31].Deming S.N., Optimization of Methods, Chapter 2 in Hirsch R.F., Ed., Proceedings of the Eastern Analytical Symposium on Principles of Experimentation and Data Analysis, Franklin Institute Press, 1978, p. 31. [Google Scholar]
  • [32].Olansky A.S. and Deming S.N., Automated Development of a Kinetic Method for the Continuous-Flow Determination of Creatinine, Clin. Chem., 24, 2115 (1978). [PubMed] [Google Scholar]
  • [33].Shavers C.L., Parsons M.L., and Deming S.N., Simplex Optimization of Chemical Systems, J. Chem. Educ., 56, 307 (1979). [Google Scholar]
  • [34].Deming S.N., The Role of Optimization Strategies in the Development of Analytical Chemical Methods, American Laboratory, 13(6), 42 (1981). [Google Scholar]
  • [35].Deming S.N. and Morgan S.L., Teaching the Fundamentals of Experimental Design, Anal. Chim. Acta., 150, 183 (1983). [Google Scholar]
  • [36].Nickel J.H. and Deming S.N., Use of the Sequential Simplex Optimization Algorithm in Automated Liquid Chromatographic Methods Development, LC, 414 (1983). [Google Scholar]
  • [37].Golden P.J. and Deming S.N., Sequential Simplex Optimization with Laboratory Microcomputers, Laboratory Microcomputers, 3(2), 44 (1984). [Google Scholar]
  • [38].Walters F.H. and Deming S.N., A Two-Factor Simplex Optimization of a Programmed Temperature Gas Chromatographic Separation, Anal, Lett., 17, 2197 (1984). [Google Scholar]
  • [39].Mendenhall W. Introduction to Linear Models and the Design and Analysis of Experiments, Duxbury, Belmont, CA: (1968). [Google Scholar]
  • [40].Laub R.J. and Purnell J.H., J. Chromatogr, 112, 71 (1975). [Google Scholar]
  • [41].Morgan S.L, and Deming S.N., Experimental Optimization of Chromatographic Systems, Sep. Purif. Methods, 5, 330 (1976). [Google Scholar]
  • [42].Deming S.N. and Turoff M.L.H., Optimization of Reverse-Phase Liquid Chromatographic Separation of Weak Organic Acids, Anal. Chem., 50, 546 (1978). [Google Scholar]
  • [43].Price W.P. Jr.; Edens R., Hendrix D.L., and Deming S.N., Optimized Reverse-Phase High-Performance Liquid Chromatographic Separation of Cinnamic Acids and Related Compounds, Anal. Biochem., 93, 233 (1979). [DOI] [PubMed] [Google Scholar]
  • [44].Price W.P. Jr., and Deming S.N., Optimized Separation of Scopoletin and Umbelliferone and cis-trans Isomers of Ferulic and p-Coumaric Acids by Reverse-Phase High-Performance Liquid Chromatography, Anal. Chim. Acta, 108, 227 (1979). [Google Scholar]
  • [45].Kong R.C., Sachok B., and Deming S.N., Combined Effects of pH and Surface-Active-Ion Concentration in Reversed-Phase Liquid Chromatography, J. Chromatogr., 199, 307 (1980). [Google Scholar]
  • [46].Sachok B., Kong R.C., and Deming S.N., Multifactor Optimization of Reversed-Phase Liquid Chromatographic Separations, J. Chromatogr., 199, 317 (1980). [Google Scholar]
  • [47].Sachok B., Stranahan J.J., and Deming S.N., Two-Factor Minimum Alpha plots for the Liquid Chromatographic Separation of 2,6-Disubstituted Anilines, Anal. Chem., 53, 70 (1981). [Google Scholar]
  • [48].Nickel J.H., and Deming S.N., Use of Window Diagram Techniques in Automated LC Methods Development, Amer. Lab., 16(4), 69 (1984). [Google Scholar]
  • [49].Deming S.N., Bower J.G., and Bower K.D., Multifactor Optimization of HPLC Conditions, Chapter 2 in Giddings J.C., Ed., Advances in Chromatography, 24, 35 (1984). [Google Scholar]

Articles from Journal of Research of the National Bureau of Standards are provided here courtesy of National Institute of Standards and Technology

RESOURCES