Design an experiment anticipating autocorrelation.Is there a way to anticipate and account for potential affects of previous runs in an experiment?
Two use cases:
- Lab - Samples are prepared using a mixer that cannot be (cost effectively) fully cleaned between runs. The samples are later treated using a process that, again, cannot be fully cleaned between every run.
- Continuous Process - A process is not shut down between runs, rather new conditions are set and the process is allowed to stabilize before measuring results.
Of course it is best to fully clean or wait a very long time between runs to ensure independence, but sometimes this is either cost prohibitive, or it is not known how long to wait between runs. If the investigator wants to accept the possibility that some autocorrelation will exist between runs, what steps can they take to at least measure and ideally account for this?
My ideas so far:
- Include previous run conditions (for each step if using the next bullet) as factors in the final model, often as the amount the factor changed from the previous run. If this value is significant in the final model then autocorrelation influenced the results.
- Randomizing run order in each step when possible to determine which step(s) caused autocorrelation (not applicable to a continuous process)
- Ensure that when some conditions are tested with movements in both directions prior to the current run and that some runs are repeated with identical conditions.
- If the process can be fully cleaned sometimes then ensure the first conditions run after the DOE cover as much of the design space as possible, essentially design a DOE using just those points, even if the power is low.
Is it possible to tell JMP to design an experiment accounting for any of these ideas, or are the other tricks to use here?
DOERe: Design an experiment anticipating autocorrelation.Here are my thoughts, no particular order:
1. I'm not sure I would call your dilemma "autocorrelation". Seems to me more like you have a covariate or noise that can affect treatments. Can you measure the "condition or cleanliness" of the mixer in the lab? Or can you measure the conditions of the continuous process?
2. Can you take advantage of any split-plot designs to handle the noise and desired restrictions on randomization? Randomizing won't let you assign the variation due to noise.
3. I'm not sure cleaning the process or mixer before every treatment is actually a good idea. The design space is not really representative of reality (as they seldom or never clean or stop the continuous process in reality). This would be an inappropriate inference space and might even induce unusual noise or factor effects.
4. Replication is, of course, another good option, though it may be difficult to keep the conditions "constant" within the block. Can you purposely change the conditions? If so then you might be able to exaggerate the effect between blocks and let the random variation of the condition occur within blocks.
5. I might also suggest sampling (component of variation) studies to determine the consistency of the mixer or continuous process and possibly even get an idea of the magnitude of the "conditions" on the response variables. This would give some good clues for deciding how to manage the condition during experimentation.
Re: Design an experiment anticipating autocorrelation.#3 - yes. I've been thinking about this in terms of a well-seasoned cast iron skillet. Scrubbing it down to bright metal will use up a lot of good will, really fast. Nobody does that more than once. (Based on three data points.)
Re: Design an experiment anticipating autocorrelation.I'll add one thought to @Craige_Hales and @statman 's comments...for you to consider on the analysis side. Make sure you cast a skeptical eye on your residual plots, especially the residuals vs. experimental execution run order plot. Look at those plots for what I call Peter Bartell's '3 second rule.' If you see a pattern in the first 3 seconds...you've got some amount of autocorrelation. If you don't see a pattern in 3 seconds...stop looking. Whatever you did to actually run the experiment didn't result in substantial autocorrelation of the responses.
