Design an experiment anticipating autocorrelation.

Question

Is there a way to anticipate and account for potential affects of previous runs in an experiment?

Two use cases:

* Lab - Samples are prepared using a mixer that cannot be (cost effectively) fully cleaned between runs.  The samples are later treated using a process that, again, cannot be fully cleaned between every run.
* Continuous Process - A process is not shut down between runs, rather new conditions are set and the process is allowed to stabilize before measuring results.

Of course it is best to fully clean or wait a very long time between runs to ensure independence, but sometimes this is either cost prohibitive, or it is not known how long to wait between runs.  If the investigator wants to accept the possibility that some autocorrelation will exist between runs, what steps can they take to at least measure and ideally account for this?

My ideas so far:

* Include previous run conditions (for each step if using the next bullet) as factors in the final model, often as the amount the factor changed from the previous run.  If this value is significant in the final model then autocorrelation influenced the results.
* Randomizing run order in each step when possible to determine which step(s) caused autocorrelation (not applicable to a continuous process)
* Ensure that when some conditions are tested with movements in both directions prior to the current run and that some runs are repeated with identical conditions.
* If the process can be fully cleaned sometimes then ensure the first conditions run after the DOE cover as much of the design space as possible, essentially design a DOE using just those points, even if the power is low.

Is it possible to tell JMP to design an experiment accounting for any of these ideas, or are the other tricks to use here?

SamGardner · Answer

One overall good strategy to minimize the impact of an overall trend in  designed experiment is to randomize the experimental run order.  This has the general effect of balancing the factor levels across the timeline of the experiment.  You can show, as long as the trend or autocorrelation is not a dominant effect, that the impact of that nuisance variation is often small.

Create an "artificial blocking" factor and include that in the design (i.e. Blocking factor named "Run Order Group", numbered 1, 2, ...) and build the DOE with that factor in it.   So you could do blocks of size 3 or 4 and randomize within blocks.  That has a similar effect as randomization, but it will tend to more directly balance factor levels within each block, again minimizing the effect of the autocorrelation.

P_Bartell · Answer

I'll add one thought to @Craige_Hales and @statman 's comments...for you to consider on the analysis side. Make sure you cast a skeptical eye on your residual plots, especially the residuals vs. experimental execution run order plot. Look at those plots for what I call Peter Bartell's '3 second rule.' If you see a pattern in the first 3 seconds...you've got some amount of autocorrelation. If you don't see a pattern in 3 seconds...stop looking. Whatever you did to actually run the experiment didn't result in substantial autocorrelation of the responses.

ih · Answer

Thanks @statman  and @Craige_Hales, as always you've given me some things to think about!  To your points:

* To represent the 'cleanliness' of the mixer we could in theory measure the amount of material left in the mixer and the concentration of contaminate in that material.  The concentration is easy enough (that is the previous run's result) but I don't think quantifying the amount of material is practical in this case.
* This might be the crux of my question, maybe I can but I don't know how!  All of the designs I know of or could find would give good distribution of the changes before each run if many runs were included, but that defeats the project goal of doing this with as few runs as possible. I hope to find a design that varies not only the values for each run but also the changes prior to that run.
* You could have a good point here, I think there is a trade off between starting with a process that is too clean versus one that is too dirty as compared to the expected run conditions in the future.
* Yes in this case we have full control over the conditions, so I agree this is a good approach.  As with point 2, I'm doing this today by manually modifying the DOE design, it would be nice to find an design method that would do it for me.
* Good idea!  I was planning to use replicated runs to quantify the measurement error but why go through all that trouble when we could just conduct a measurement system analysis using one of the runs.