Design of Experiments (DOE): Factorial Designs, Response Surface Methods, and Taguchi

Why OFAT fails and DOE works

The one-factor-at-a-time (OFAT) approach to experimentation has an intuitive appeal: change one thing, observe what happens, then change the next thing. But OFAT has two serious limitations that make it an unreliable guide to process optimization.

First, OFAT cannot detect interactions. An interaction occurs when the effect of one factor on the output depends on the level of another factor. For example, the effect of temperature on a chemical yield might be positive at low pressure but negative at high pressure. OFAT, which holds pressure constant while varying temperature, will miss this interaction entirely and draw the wrong conclusion about the optimal temperature setting. Interactions are extremely common in manufacturing and process development, and a method that cannot detect them will systematically produce suboptimal process settings.

Second, OFAT is statistically inefficient. To estimate the effect of five factors using OFAT, a separate experiment is needed for each factor; interactions can only be estimated if additional experiments are run at combinations of factor levels that OFAT would not normally include. A two-level full factorial design for five factors (a 2^5 design with 32 runs) estimates all five main effects, all ten two-factor interactions, and all higher-order interactions simultaneously — something that would require hundreds of OFAT runs to approximate. Fractional factorial designs reduce the number of runs further, at the cost of confounding some higher-order interactions that are usually negligible in practice.

Two-level factorial designs

The most widely used DOE designs in industrial practice are two-level factorial designs, in which each factor is tested at two levels: a low level and a high level. The key design types are:

Full factorial designs (2^k): every possible combination of factor levels is included. A two-level full factorial with k factors requires 2^k experimental runs. A 2^2 design (two factors, two levels) requires 4 runs; a 2^3 design requires 8 runs; a 2^4 design requires 16 runs; and so on. Full factorial designs provide unconfounded estimates of all main effects and all interactions. They are the appropriate choice when the number of factors is small (typically four or fewer) and there is sufficient experimental material to run all combinations.

Fractional factorial designs (2^(k-p)): a carefully chosen fraction of the full factorial. A 2^(k-p) design runs 2^(k-p) experiments instead of the full 2^k. For example, a 2^(5-2) design runs 8 experiments instead of the full 32 required for a 2^5 design. The price of this reduction is that some effects are confounded (aliased) with each other: the estimate of a main effect includes the effect of certain high-order interactions that cannot be separately estimated. Fractional factorial designs are constructed using the concept of resolution: Resolution III designs confound main effects with two-factor interactions (generally not acceptable for screening); Resolution IV designs confound two-factor interactions with other two-factor interactions; Resolution V designs confound two-factor interactions only with three-factor interactions (generally acceptable because three-factor interactions are usually negligible).

Plackett-Burman designs: screening designs that efficiently identify the most important factors from a large set of candidates using the minimum number of runs. Plackett-Burman designs can screen up to k-1 factors in k runs (where k is a multiple of 4). They are used in the early stages of process development to reduce a large list of potential factors to the vital few that warrant a more detailed factorial investigation.

Main effects, interactions, and effect plots

The output of a factorial design analysis is a set of effect estimates that quantify how much each factor and each interaction changes the output response. The main effect of a factor is the average change in the response when the factor is moved from its low level to its high level, averaged across all levels of all other factors. An interaction effect is the additional change in the response caused by the combined effect of two or more factors that cannot be explained by the sum of their individual main effects.

Main effects plots display the average response at each level of each factor, showing the direction and magnitude of each main effect. Interaction plots display the response at each combination of two factor levels as two crossing or parallel lines: parallel lines indicate no interaction; crossing or nearly crossing lines indicate a significant interaction that means the effect of one factor depends on the level of the other. When a significant interaction is present, the main effects cannot be interpreted in isolation — the settings for both interacting factors must be optimized jointly.

The statistical significance of effects is assessed using analysis of variance (ANOVA), which partitions the total variation in the response into components attributable to each factor and interaction, then compares each component to the residual error variation using an F-test. Effects with p-values below the significance level (typically 0.05) are considered statistically significant. Normal probability plots of effects are also commonly used as a graphical screening tool: significant effects appear as outliers from the line representing random (non-significant) effects.

Response surface methodology

Two-level factorial designs assume that the relationship between each factor and the response is approximately linear within the experimental region. When the objective is to find the optimal (maximum, minimum, or target) value of a response rather than simply to identify which factors matter, a curved response surface is often needed and two-level designs are insufficient.

Response surface methodology (RSM) uses designs that include at least three levels per factor to fit a second-order (quadratic) model that can describe curvature and locate optima. The most commonly used RSM designs are:

Central composite designs (CCD): the most widely used RSM design, combining a two-level factorial design with center points and additional axial (star) points. The axial points extend beyond the factorial cube to allow estimation of the quadratic terms. CCD can be run sequentially: a factorial screening experiment can be augmented with axial points if curvature is detected.

Box-Behnken designs: a three-level design that avoids the extreme corner combinations of the factorial portion of a CCD, making them useful when extreme factor combinations are impractical or potentially damaging to equipment.

RSM analysis fits a second-order polynomial model to the data, then uses the fitted model to construct a response surface — a graphical representation of the predicted response over the experimental region. The optimal factor settings are identified by examining contour plots (two-dimensional cross-sections of the surface) or by numerical optimization. When multiple responses must be simultaneously optimized (a common situation in manufacturing), desirability functions combine the individual response models into a single composite objective that can be optimized numerically.

Taguchi methods

Genichi Taguchi developed a separate approach to experimental design that emphasizes engineering objectives over pure statistical inference. The Taguchi method introduces two concepts that are not explicitly part of classical factorial DOE: the signal-to-noise (S/N) ratio as a single metric for simultaneously optimizing the mean and minimizing the variation of a response, and the concept of robust design — designing products and processes so that their performance is insensitive to uncontrollable variation in operating conditions (noise factors).

Taguchi designs use orthogonal arrays (OA) as the experimental framework. Orthogonal arrays are balanced fractional factorial designs that can estimate main effects with a minimum number of runs. The most commonly used Taguchi arrays include L4 (three factors in four runs), L8 (seven factors in eight runs), L9 (four factors in nine runs, three-level), and L16 (fifteen factors in sixteen runs). Taguchi’s parameter design separates the experimental factors into control factors (factors whose levels can be set and held in production) and noise factors (factors that vary uncontrollably in field use), then selects control factor settings that make the response robust to variation in the noise factors.

Taguchi methods have been criticized by statisticians for inefficiencies in the S/N ratio analysis and the confounding structure of some orthogonal arrays, but their practical contributions — especially the concept of designing for robustness and the engineering accessibility of the orthogonal array framework — have been widely adopted and integrated into modern robust design practice.

DOE in Six Sigma and DFSS

In the Six Sigma DMAIC framework, DOE appears primarily in the Analyze and Improve phases. In the Analyze phase, a screening DOE is used to identify the vital few process variables (Xs) from a larger list of potential factors, confirming through objective experimentation which variables have a significant effect on the quality characteristic (Y) being improved. In the Improve phase, a response surface or optimization DOE is used to find the factor settings that move the Y to its target value and minimize its variation.

In Design for Six Sigma (DFSS), DOE is used during the design and characterization phases to optimize product designs and manufacturing process parameters before the design is released to production. DFSS DOE typically involves more complex response surface work and multi-response optimization to satisfy simultaneous requirements on multiple critical-to-quality characteristics.

Common errors in design of experiments

• Failing to replicate or randomize. Replication (running the same experimental conditions more than once) is necessary to estimate pure experimental error and provide a basis for statistical significance testing. Randomization of the run order is necessary to prevent systematic biases from confounding the factor effects — a time trend in the process, for example, would be confounded with the effect of any factor that was consistently run at one level early in the experiment and the other level late. Both principles are routinely sacrificed for convenience and both lead to unreliable conclusions.
• Setting factor levels too close together. If the low and high levels of a factor are not sufficiently different, the main effect of that factor will be small even if the factor is important over a wider range — and the experiment will incorrectly conclude that the factor is not significant. Factor levels should span a range that is practically meaningful and that the process can actually achieve.
• Analyzing a fractional factorial without considering the alias structure. In a fractional factorial design, each estimated effect is actually the sum of the main effect and any aliased higher-order interactions. Before interpreting the results, the analyst must examine the alias structure and verify that the significant effects are not confounded with plausible two-factor interactions that might explain the data equally well.
• Optimizing one response while ignoring others. Manufacturing processes almost always have multiple output responses that must simultaneously be within specification. Optimizing a single response (for example, maximizing strength) while ignoring others (for example, dimensional conformance) produces a process setting that optimizes one quality characteristic at the expense of others.

Frequently asked questions

What is the difference between a full factorial and a fractional factorial design?

A full factorial design includes every possible combination of the factor levels being studied. For two-level designs, a full factorial with k factors requires 2^k experimental runs and provides unconfounded estimates of all main effects and all interactions at every order. A 2^3 full factorial requires 8 runs and estimates 3 main effects, 3 two-factor interactions, and 1 three-factor interaction. A fractional factorial design runs only a fraction of the full factorial: a 2^(k-p) design runs 2^(k-p) experiments instead of 2^k. A 2^(5-2) design, for example, runs 8 instead of 32 experiments. The reduction comes at the cost of confounding (aliasing): certain effects cannot be separately estimated and are instead summed together in a single alias chain. The resolution of a fractional factorial describes which effects are confounded: Resolution V designs are generally acceptable for most industrial screening work because main effects and two-factor interactions are not confounded with each other (only with three-factor interactions, which are usually negligible). The choice between full and fractional factorial depends on the number of factors, the available experimental resources, and whether interactions at all orders must be estimated independently.

What is an interaction in DOE and why does it matter?

An interaction in DOE is a condition where the effect of one factor on the output response depends on the level of another factor. When no interaction is present, the effect of factor A is the same regardless of whether factor B is at its low or high level: the two factors act independently and their effects are additive. When an interaction is present, the effect of A at the low level of B is different from the effect of A at the high level of B: the factors must be considered jointly. Interactions matter because they make the individual main effects potentially misleading. If temperature and pressure interact, the “best” temperature setting depends on what pressure level is running: there is no single universally optimal temperature that can be set without specifying the pressure. The presence of a significant interaction means that process optimization must be done by jointly considering the interacting factors, not by optimizing each factor independently. Interactions are extremely common in chemical and thermal processes, in assembly processes with tolerance stack-up, and in biological systems. The OFAT approach cannot detect interactions, which is one of the primary reasons it is an unreliable guide to process optimization.

What is response surface methodology (RSM) and when should it be used?

Response surface methodology (RSM) is a collection of statistical and mathematical techniques for modeling and optimizing a process or product response that depends on several continuous variables. It is used when the objective is to find the combination of factor settings that optimizes (maximizes, minimizes, or hits a target value for) one or more response variables, and when the relationship between the factors and the response is expected to be nonlinear (curved) in the region of interest. Two-level factorial designs can only detect linear relationships (straight-line effects); RSM designs add a third level per factor to allow estimation of curvature (quadratic terms) and the identification of optima. RSM should be used after a screening experiment has identified the significant factors and the experimental region has been narrowed to the area of interest. Starting with RSM on a large list of unscreened factors is wasteful: the sequential approach — screen first with a Resolution IV or V fractional factorial, then optimize with RSM on the significant factors — is more efficient. RSM is particularly useful in chemical process optimization (reaction temperature, catalyst concentration, pH), formulation development (ingredient ratios, processing conditions), and parameter optimization in manufacturing processes with multiple interacting control parameters.

What are Taguchi methods and how do they differ from classical DOE?

Taguchi methods are a collection of experimental design and analysis techniques developed by Genichi Taguchi that emphasize engineering objectives — particularly the design of robust products and processes — over classical statistical inference. The key differences from classical DOE are: (1) Taguchi uses the signal-to-noise (S/N) ratio as the primary analysis metric, combining the mean and variance of the response into a single number to simultaneously optimize performance and minimize variability; (2) Taguchi explicitly separates factors into control factors (set in the design or process) and noise factors (sources of uncontrollable variation in field use), and uses the parameter design framework to find control factor settings that make the response insensitive (robust) to noise factor variation; (3) Taguchi uses pre-defined orthogonal arrays as the experimental structure, making the design process accessible to engineers without deep statistical training. Classical DOE provides more flexible and statistically rigorous analysis (particularly for estimating interactions, which Taguchi methods sometimes handle inadequately), but Taguchi’s concepts of robustness and parameter design have been widely adopted and represent genuine contributions to quality engineering practice. Modern robust design practice typically combines the engineering philosophy of Taguchi with the statistical rigor of classical factorial and response surface methods.

How is DOE used in the Six Sigma DMAIC process?

DOE is used at two points in the Six Sigma DMAIC process. In the Analyze phase, a screening DOE (typically a Resolution IV or V fractional factorial) is used to confirm which process input variables (Xs) have a statistically significant effect on the quality characteristic being improved (Y). This moves the analysis from the correlative (regression, graphical analysis, hypothesis testing) to the causal: a controlled experiment demonstrates the effect of each factor under conditions where other potential sources of variation are controlled or randomized. In the Improve phase, an optimization DOE (a full factorial on the significant factors, or an RSM design if curvature is expected) is used to find the factor settings that move Y to its target value and minimize its variation. The Improve phase DOE also provides the regression model that describes the quantitative relationship between the significant Xs and Y, which becomes the basis for setting the process operating window and the control limits in the Control phase. The combination of Analyze-phase screening and Improve-phase optimization DOE is one of the most powerful components of the DMAIC framework, because it provides statistically defensible evidence of causation and quantified process knowledge that supports robust control plan development.

What is a center point in a two-level factorial design and what does it test?

A center point is an experimental run at the midpoint of every factor’s range simultaneously: if factor A runs at levels of 100 and 200 in the factorial portion of the design, the center point runs factor A at 150. Adding center points to a two-level factorial design serves two purposes. First, center points provide an estimate of pure experimental error (replicating the center point multiple times gives a direct estimate of the repeatability of the measurement or process) that can be used as the error term in the ANOVA even when the factorial design has no replicated runs. Second, center points test for curvature: if the average response at the center points is significantly different from the average response predicted by the two-level model (which can only fit linear effects), this is evidence that at least one factor has a nonlinear (curved) relationship with the response, and a response surface design with three or more levels is needed to characterize the curvature. Center points are typically run at 3 to 5 replicates, interleaved throughout the experimental run order to also serve as a check on process stability during the experiment.