nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Do we need maths proficiency for design of experiments?

gustavjung

I use SAS JMP to run multivariate testing experiments (MVT, regression), it generates a test plan and performs regression analysis, defining significant effects. The most important thing is to be able to interpret and know the pitfalls.

Mathematical calculations, as such, are not needed.

I have read statistics books such as Montgomery Design of experiments and they are usually full of formulas.

I still cannot understand how are they used in real world.

Yea I understand that I need to know concepts, different types of designs, basics of statistics as p value, standard deviation, how to calculate proper sample size, etc.

I speak about design of experiments mostly in web marketing and user experience fields.

Accepted answers

Peter_Bartell

I'm going to recommend NOT focusing on mathematics...as heretical as that may sound from a statistician. Focus on the following:

1. The process under study. Factors (control, nuisance, and noise). Responses (continuous, categorical, repeated measures, etc.). The measurement system. How does the process actually 'work'? Don't do anything else until you know this.

2. Don't even begin worrying about your design or analysis until you've formulated a specific experimental goal and objective that addresses a practical problem...not some design or analysis requirements.

3. Then, consistent with my colleague Mike Anderson's thinking, find the most efficient experimental design. Efficiency to me isn't and shouldn't be limited to exclusively mathematical criteria such as an optimality criteria...but more broadly. Efficiency to me is the design that provides the required information for the least expenditure of resources.

4. What do you know about your measurement system for your responses wrt to characteristics such as repeatability, reproducibility, linearity, etc.

5. Go to gemba when the experiment is run/conducted if possible. Don't presume the design matrix will be executed as your wrote it on a sheet of paper and handed some somebody to actually run.

6. Finally, when you get to analysis...I have three rules for successful experimental design results analysis: Rule 1: Plot the data; Rule 2: Plot the data; Rule 3: Plot the data. This is JMP's hallmark...sure JMP does the math under the hood, but with the cornucopia of graphical analysis techniques available in JMP...math should be the least of your worries.

Good luck!

All comments

shoffmeister

I agree on the point that many books are full of formulas, more than the typical user needs.

On the other hand design of experiments always aims at generating the right data for a specific kind of data analysis. In the end: What is the point of getting good data to perform an analysis that you do not really understand? The better you understand regression analysis the more you will learn from your DOE.

Therefore I think it is essential to have a good understanding of the statistics that are used when analyzing your DOEs. Knowing what a p-value is is of course a start but having a good understanding about potential problems and what they mean for your process is so valuable.

I would recommend attending a DOE-class given by someone with real world-DOE-experience or reading a praxis oriented book that does not skip the essential statistics, like the one of Peter Goose and Bradley Jones: Optimal Design of Experiments - A Case Study Approach (Wiley: Optimal Design of Experiments: A Case Study Approach - Peter Goos, Bradley Jones).

louv

You may want to check out

Testing 1,2,3 Experimental Design in Marketing and Service Operations by Johannes Ledolter and Arthur J. Swersey

http://www.sup.org/books/title/?id=4513

gustavjung

Thank you for suggestion. I have this books but have not read it yet. I liked its dialogue style.

Do you think this book is enough to understand the practical pitfalls of DOE and essential statistics?

What topics of maths I need to study to understand essential formulas in DOE?

I do understand basic formulas but something like this confuses me:

gustavjung

Thank you for this suggestion. I have already read this book, but I think I will re-read it. Also I know a book by Paul Berger Experimental Design with Applications in Management but havent read it yet.

MikeD_Anderson

In a nutshell, experimental design is about getting the most value and information out of the resources allocated for data acquisition. It also can help to act as a framework for adherence to experimental best practice (checking statistical significance, randomization, sample size, etc.). There is actually quite a lot of experimental design in the marketing and user experience areas as well as related topics like designing surveys. If you would like some real world examples, you might check the JMP Discovery Series area (https://community.jmp.com/community/discovery-summit-series). If you are curious about the topic in general, you might also check out some of the webinars that we have posted on the topic.

Best,

Peter_Bartell

I'm going to recommend NOT focusing on mathematics...as heretical as that may sound from a statistician. Focus on the following:

4. What do you know about your measurement system for your responses wrt to characteristics such as repeatability, reproducibility, linearity, etc.

5. Go to gemba when the experiment is run/conducted if possible. Don't presume the design matrix will be executed as your wrote it on a sheet of paper and handed some somebody to actually run.

Good luck!

Phil_Kay

For what it's worth, I've had plenty of success using DoE without understanding these formulae.