Statistics

DOE — design of experiments

In French: Plans d'expériences

A design of experiments organises the trials to measure the effect of several factors at once, and detect their interactions, with far fewer trials than the intuitive method.

Why not vary one factor at a time

The spontaneous approach fixes all the parameters but one, measures, then moves on to the next. It is intuitive, and it has two flaws that often make it useless.

It sees no interaction. If the effect of temperature depends on pressure, varying one at fixed pressure gives an answer valid for that pressure only. Interactions are the rule on real processes, not the exception.

It costs more for less information. A full factorial design with three factors at two levels requires eight trials and provides the three main effects plus all their interactions. The one-at-a-time method, for a comparable number of trials, provides only the main effects — and less precisely, each effect resting on only two trials.

The three principles that hold a design together

Randomisation. The order of the trials is drawn at random, so that a drift over time — warming up, wear, shift change — is not confused with the effect of a factor. Running the trials in the order of the table is the most frequent mistake, and it ruins the conclusion without leaving a trace.

Replication. Redoing trials under the same conditions gives an estimate of the natural variability, without which one cannot say whether an observed gap is an effect or noise.

Blocking. When a source of variation is known and unavoidable — two batches of material, two machines — it is organised into blocks rather than endured, which removes its effect from the analysis instead of letting it pollute the results.

Full or fractional

A full design tests every combination: with seven factors at two levels, that makes one hundred and twenty-eight trials, rarely affordable.

A fractional design tests only a chosen part of them, which sharply reduces the number of trials at the cost of confounding: some effects become indistinguishable from one another. The design says which ones, and that is what must be read before launching the trials, not after.

Common practice chains the two: a fractional design to rule out the factors with no effect, then a full design on the two or three remaining factors, possibly supplemented to explore non-linearities.

The common mistakes

Choosing levels that are too close. Two levels that differ only slightly produce an effect drowned in measurement variability, and the design wrongly concludes that an important factor has no effect. The levels must be as far apart as the process tolerates.

Launching a design without having checked the measurement system. If the measurement is noisy, no real effect will come out, and trials will have been paid for to measure the uncertainty of one’s own instrument.

On the shop floor — Speciality chemicals

An optimisation run factor by factor for six months had produced nothing. A three-factor factorial design, eight trials and two days, showed that the effect of reaction time reversed depending on catalyst concentration. The interaction alone explained the failure of the previous trials.

The other tools illustrated by this trade: plastics and chemicals.

Frequently asked questions

How many trials should be planned?

A full factorial design at two levels requires two to the power of the number of factors — eight trials for three factors, sixteen for four. A few replicates are added at the centre of the domain to estimate variability and detect curvature. A fractional design divides that number by two, four or more, depending on the confounding accepted.

What is an interaction?

A situation where the effect of one factor depends on the level of another. Graphically, two lines that are not parallel. It is what a design of experiments detects and what no series of one-at-a-time trials can see — and it is often where the explanation of a process one does not understand is hiding.

Does a design of experiments prove causation?

Yes, and that is what distinguishes it from the analysis of historical data. The levels are imposed instead of endured, the order is randomised, trials are repeated: what changes in the response can only come from what was deliberately varied. It is the only tool in the Six Sigma box that establishes a causal link.

Is specialised software needed?

To build and analyse a design with four factors or more, yes — building fractional designs and reading the confounding cannot be improvised. A full design with two or three factors can be calculated and plotted in a spreadsheet, and it is an excellent first step for understanding the method.