I'm
Phil
Kay
and
I'm
joined
by
Weronika,
and
we're
going
to
talk
about
a
fun
experiment
that
we
set
up
as
a
competition
with
the
idea
that
it's
sowing
the
seeds
of
love
for
design
of
experiments.
And
it's
all
about
growing
cress,
which
I'm
sure
many
of
you
will
have
done
when
you
were
at
school
or
at
home.
And
there's
a
problem,
I
think,
in
how
we
educate
our
young
scientists.
T
his
is
taken
from
the
British
Broadcasting
Corporation's
bite
size
for
Key
Stage
2,
so
for
young
scientists.
The
curriculum
in
the
United
Kingdom,
at
least,
tells
people
about
this
fair
test
idea.
And
that
is
when
you
are
testing
something,
you
need
to
make
sure
it
is
a
fair
test.
To
do
this,
everything
should
be
the
same
except
the
thing
you
are
testing.
So
we're
only
allowed
to
change
one
thing
at
a
time.
And
that's
not
ridiculous.
It's
not
necessarily
wrong,
but
it's
not
all
of
the
truth
as
well.
It's
not
necessarily
the
best
way
when
you
come
to
experiment
in
commercial
R&D
and
industry.
T
he
consequences
of
this,
if
we
accept
that
we're
only
going
to
test
one
thing
at
a
time,
let's
imagining
we're
experimenting
to
understand
what
affects
the
height
of
garden
cress,
and
we
want
to
understand
what's
the
effect
of
light
conditions,
sunlight
or
dark.
We're
pretty
sure
that's
going
to
have
an
effect,
but
we'd
like
to
understand
what
it
is.
What's
the
effect
of
the
growing
medium
whether
we
grow
on
soil
or
on
cotton
wool?
Again,
we
think
it's
probably
going
to
have
an
effect.
We'd
like
to
experiment
to
understand
or
quantify
the
effect.
T
he
fair
test
way
of
doing
this
would
be
to
take
control
conditions.
So
we
grow
some
cress
in
sunlight
and
on
soil.
And
then
we
do
a
fair
test.
We
just
change
one
thing.
So
for
Fair
test
1,
we
change
the
growing
medium
to
cotton
wool,
and
we
see
what
the
effect
is.
For
Fair
test
two,
to
understand
the
effect
of
light
conditions,
we
change
to
dark
and
we
keep
everything
else
the
same.
We
keep
our
other
factor
the
same.
T
his
would
be
fine,
fair
tests
would
be
fine,
except
nature
doesn't
necessarily
play
by
those
rules.
Nature
doesn't
play
fair
all
the
time.
And
what
we
should
really
be
doing
in
this
situation
is
a
designed
experiment.
And
in
this
case,
we
would
test
all
possible
combinations.
W
e
wouldn't
just
be
changing
one
thing
at
a
time,
we'd
make
sure
we
tested
all
the
possible
combinations
, we
changed
all
the
factors
according
to
a
strategy.
And
what
this
enables
us
to
do
is
gain
a
richer
understanding.
So
we
can
understand
things
like
interactions
between
factors.
And
for
this
cress
experiment,
we
can
see
the
height
at
day
five
after
five
days
of
growing,
we're
looking
at
the
effect
of
light
condition,
sunlight
or
dark.
And
we
can
see
that
the
effect
of
light,
whether
it's
sunlight
or
dark,
is
dependent
on
the
growing
medium.
F
or
soil,
we
are
seeing
a
bigger
difference
between
dark
and
sunlight
than
we
are
with
cotton
wool.
This
is
an
interaction.
We
can
only
understand
these
interactions
when
we
use
designed
experiments,
and
these
are
often
critical
in
commercial
R&D.
So
what
we
need
is
some
fun
ways
of
introducing
these
ideas
to
young
students,
to
students
of
any
age.
Now,
let
me
go
to
a
digression
where
we
got
the
name
of
this
talk
from.
It's
from
a
song
by
a
group
called
Tears
for
Fears.
It's
not
a
very
new
song,
so
if
you're
young,
you
may
not
have
heard
it.
If
you're
a
bit
older,
you'll
probably
know
it
because
it
was
nominated
for
the
best
postmodern
video
at
the
MTV
Music
Awards
30
odd
years
ago,
whatever
best
postmodern
video
means.
The
first
line
is,
high
time
we
make
a
stand
and
shook
up
the
views
of
the
common
man.
I
don't
know
if
I
like
that
first
line
very
much,
but
I
think
it's
appropriate
here.
We'd
like
to
shake
up
people's
views
about
how
we
should
do
experiments,
how
we
should
change
the
factors
in
an
experiment.
I
was
a
little
bit
concerned
that
this
is
a
British
band
that
people
may
not
have
heard
of
Tears
of
Fears,
may
not
have
heard
of
this
song.
So
I
looked
at
the
data
and
actually
I
found
that
it
was
a
worldwide
hit
and
particularly
big
in
Canada.
It
reached
number
one
there
in
1989.
The
Cress
experiment,
how
did
this
start?
Well,
my
colleague,
Michael,
in
marketing
here
at
JMP,
wondered
if
we
could
make
a
fun
experiment
out
of
growing
garden
cress.
That
hadn't
occurred
to
me.
When
I
first
heard
this,
I
thought,
that's
a
brilliant
idea.
What
we
wanted
to
do
was
create
an
experiment
that's
simple
enough
for
anyone,
for
experimenters
of
all
ages,
young
experimenters,
old
experimenters.
We
wanted
it
to
be
simple
enough
you
could
do
it
at
home.
One
of
the
challenges
with
coming
up
with
good
examples
of
design
of
experiments
is,
science
is
generally
expensive.
Measuring
the
outputs
of
your
scientific
experiments
often
requires
really
expensive
instruments.
So
we
wanted
something
that
was
simple
and
cheap
to
do.
And
we
wanted
it
to
be
an
interesting
way,
just
a
fun
way
to
introduce
the
key
concepts
of
statistical
design
of
experiments.
W
e
didn't
want
it
to
be
difficult,
we
didn't
want
you
to
have
to
do
lots
of
very
complex
analysis.
We
wanted
it
to
be
very
immediate
and
fun
way
of
introducing
these
ideas.
I
did
some
experiments
in
the
Kay
Family
Research
Kitchen
here
with
some
assistants.
I
had
my
eight-
year-
old
daughter
and
my
15-
year-
old
daughter
help
me
with
this.
My
12-
year-
old
daughter
was
too
busy
watching
DOC,
I
think.
And
it
was
very
successful.
They
had
a
good
time
doing
it,
I
think,
and
it
started
some
interesting
discussions.
W
e
did
this
experiment
and
we
set
up
the
experiment
so
that
we
were
growing
some
of
them
in
soil,
some
of
them
in
cotton,
some
of
them
in
dark,
some
of
them
in
light
conditions.
And
my
eight-
year-
old
child
said,
"Well,
Dad,
it
would
have
been
easier
if
we
just
put
all
the
soil
ones
in
the
dark
and
all
the
cotton
ones
in
the
light."
I
didn't
say
anything,
so
I
wait
for
my
15-
year-
old
to
respond.
She
said,
"Well,
but
then
we
wouldn't
know
if
it
was
the
soil
or
if
it
was
the
dark
that
had
the
effect."
This
is
a
beautifully
concise
way
of
describing
confounding.
This
was
a
very
proud
moment
for
me
as
a
parent
that
one
of
my
children
could
explain
this
concept
of
confounding
in
a
much
more
succinct
way
than
I
have
ever
managed
to
do.
And
we
got
great
data.
T
he
15-
year-
old
lost
interest
after
we'd
set
it
up,
but
my
eight-
year-
old
daughter
carried
on
with
the
experiment,
observing
it
over
a
number
of
days.
W
e
measured
the
height
of
the
tallest
plant
in
each
pot,
actually
within
each
compartment
of
an
egg
box.
W
e
measured
those
and
she
took
all
the
measurements
and
we
got
some
really
good
quality
data.
Let
me
just
show
you
first
of
all,
though,
the
actual
experiment.
T
hree
factors,
we
tested
substrate,
soil
or
cotton
wool,
the
light
conditions,
dark
or
light,
and
we
used
plain
or
curled
cress
types.
We
got
two
different
types
of
cress
seeds.
And
this
is
a
two-
to-
the
three
full
factorial
for
those
DOE
nerds
out
there.
And
we've
replicated
on
the
two
to
the
three
minus
one
half
factorial
there.
So
that
gives
us
12
runs,
12
pots,
which
works
well
because
in
the
UK
at
least,
egg
boxes
generally
come
in
sixes.
So
we
could
use
two
egg
boxes
to
do
these
12
runs.
And
as
I
said,
the
data
was
very
good.
We
can
do
some
simple
analysis.
This
is
one
of
the
things
I
like
about
it,
is
that
we
can
just
look
at
the
ones
that
were
grown
in
the
light
and
the
ones
in
the
dark
and
see
how
the
height
is
different
after
seven
days.
And
it's
very
compelling,
there's
a
very
big
difference.
It
wasn't
really
the
difference
that
I
was
necessarily
expecting,
and
it
was
an
interesting
surprise
to
all
of
the
experimenters
involved. W
e
can
do
some
simple
analysis,
just
some
simple
visuals.
Let's
just
plot
the
heights
versus
light
conditions.
And
again,
you
can
see
the
big
effect
there,
big
effect
of
substrate,
very
little
effect
of
cress
type
there.
So
introducing
these
simple
analysis,
and
then
we
can
obviously
take
it
to
a
greater
level
of
sophistication,
build
a
full
statistical
model.
And
that
brings
us
to
the
profiler,
which
I
think
is
just
such
a
great
way
of
understanding
design
of
experiments
and
statistical
models.
Very
powerful,
compelling
way
to
understand
the
effects
of
each
factor
and
interactions
between
factors
as
well.
I
f
we
look
at
day
seven,
we
can
see
there's
an
interaction
between
light
conditions
and
our
substrate.
And
we
can
take
it
to
an
even
greater
level
of
sophistication
because
this
is
actual
functional
data.
I
f
you're
interested
in
Functional
Data
Explorer,
well,
this
is
a
great
example
data
set
because
we're
collecting
the
height
data
as
a
function
of
time
for
each
of
the
runs
of
our
experiment.
W
e
can
use
Functional
Data
Explorer
and
Functional
DOE
to
understand
how
the
factors
affect
the
shape
of
this
growth
curve.
We
can
see
the
rapid
growth
with
soil
versus
cotton
wool.
We
can
see
the
rapid
growth,
increased
rate
of
growth
in
the
dark,
and
actually
the
fact
that
it's
starting
to
die
off
towards
the
end
of
the
experiment
here.
I
was
really
delighted
with
how
the
experiment
went.
It
was
very
simple
to
do,
very
compelling,
really
accurate
results.
It's
so
hard
to
find
experiments
that
people
can
do
at
home
where
they
can
get
an
accurate,
continuous
quantitative
response
out
that
they
can
measure
just
with
a
plastic
ruler
in
this
case.
We
went
ahead
and
did
this
as
a
competition.
I
wrote
a
blog
post
about
it,
which
we'll
provide
the
link
to
that
as
well.
W
e
ran
this
last
summer,
summer
of
2022.
I'm
going
to
introduce
next
our
competition
winner,
Weronika.
I've
also
done
some
visuals
of
Weronika's
results
in
JMP
Public.
We'll
share
the
link
to
that
as
well
so
you
can
actually
see
Weronika's
data,
download
the
data
yourself
if
you
log
into
JMP
Public
and
see
the
results
for
yourself.
But
now,
Weronika
is
going
to
show
you
what
she
found
in
this
cress
experiment
and
the
impressive
results
that
she
got
that
meant
she
was
the
competition
winner.
I
think
you're
on
mute,
Weronika.
Thank
you,
Phil
for
introducing
me.
I
would
like
to
share
with
you
my
experience
in
a
competition,
my
experience
regarding
design
of
experiments,
regarding
the
planting
of
the
cress.
T
he
main
aim
of
the
challenge
was
to
introduce
design
of
experiments
to
researchers,
to
engineers,
to
students,
to
any
of
the
people.
But
also
in
that
experiment,
we
have
to
check
with
design
of
experiments
what
factors
has
influence
on
the
health
of
the
garden
products.
T
hen
defined
factors
by
the
organizers,
Phil
,
there
were three
factors.
It
was
the
surface.
W
e
use
cotton
wool,
and
g
arden
soil.
Then
the
second
factor
was
light
conditions.
So
we
plant
garden
cress
in
sunlight
and
in
the
dark.
And
also
we
had
to
check
what
influence
has
been
soaking
on
the
height.
What
was
my
first
impression?
As
Phil
said,
they
wanted
experiment
to
be
simple
enough
for
everybody.
But
I
was
not
so
convinced
at
the
beginning
because
when
taking
a
look
at
my
previous
experiments
with
planting,
it
was
not
so
good.
So
I
didn't
expect
that
my
garden
cress
acted
in
different
way.
And
I
wasn't
mistake,
I
wasn't
wrong.
My
first
results
were
good.
First
of
all,
I
put
so
many
seeds
in
one
spot
that
the
pre-
soak
samples
become
a
shell.
Some
kind of shell.
They
didn't
germinate
,
so
I
don't
receive
any
plants.
Moreover,
my
egg
box
was
broken
by
the
water,
which
can
be
seen
here.
It
was
broken.
Also,
some
marker
was
destroyed
and
I
saw
no
numbers
of
spot.
And
also
the
soil
migrated
from
one
hole
to
the
adjacent
one.
I
t
was
mixed
with
the
cotton,
especially
when
I
put
the
waters
on
the
soil,
it
was
not
good.
After
my
first
failure,
I
drove
some
conclusion
why
I
received
a
failure.
First
of
all,
I
decided
to
use
plastic
espresso
cups
instead
of
the
paper
cups
because
plastic
is
better
than
water.
Use
less
number
of
seeds
in
each
hole.
Don't
put
as
many
as
I
can,
but
do
it
smartly.
And
also
in
this
moment,
I
come
to
idea
to
maybe
add
the
fourth
factor
to
my
experiment,
density
of
the
seeds.
Also,
I
wanted
to
check
not
only
how
surface,
light
condition,
and
soaking
influenced
the
height,
but
also
the
density
of
seed.
I
set
two
levels,
low
and
high.
In
low
density,
I
use
20
seeds
and
evenly
spread
them
in
a
cup.
In
high
density,
I
took
40
seeds
and
try
to
put
every
in
the
middle
of
the
cup, so it's
[inaudible 00:16:09]
.
M
y
design
had
four
factors.
Each
factor
had
two
levels.
I
used
a
full
factorial
design
as
a
design
time.
I
received
16
number
of
treatments,
2
to
the
power
of
4.
I
decided
to
replicate
eight
treatments
in
order
to
receive
variability
and
be
able
to
estimate
some
standard
deviation
and
so
on.
I
n
total,
I
received
24
test
runs.
Experiment
was
done
in
August
when
it
was
very
warm,
so
it
was
nice
weather
for
a
planting
and
being
a
gardener.
Okay,
those
are
my
results.
Here
we
can
see
design
table
with
all
factors,
test
runs
24.
Here
I
put
the
height
after
three,
five,
and
seven
days.
In
that
table,
you
can
see
the
factor
effect
estimates
after
seven
days.
We
can
end
with
the
bold
font.
I
marked
variables,
factors
which
I
found
to
be
statistically
significant,
and
it
was
surface,
light,
density.
Soak
occurred
not
to
be
important,
but
alone
as
a
main
effect
only.
It
occurred
to
be
important
in
two
factor
interactions.
T
he
interaction
between
surface
and
soak
and
the
light
occurred
to
be
significant,
so
we
cannot
assume
that
the
soaking
is
not
important.
Also,
two
three-way
interactions
were
significant,
four-
way
interaction
not
significant.
N
ow
I
would
like
to
present
you
some
pipeline
and
some
steps
which
I
used
in
my
design
experiment.
I
think
that
it's
quite
a
good
approach
which
everyone
can
use
in
the
experiment.
First
of
all,
we
have
to
generate
the
design.
As
a
first
step,
we
shall
define
what
factor
we
want
to
check
and
what
levels.
And
when
we
set
it,
we
have
to
choose
design
type,
because
usually
choosing
the
type
is
dependent
on
the
factors,
how
many
factors
we
have,
or
is
only
two
or
three,
or
maybe
we
have
no
factors,
how
many
levels.
D
efine
number
of
replicates
we
have
to
include,
and
then
we
can
generate
this
table
with
which
in
JMP
is
very
quickly
and
convenient.
When
we
have
a
table,
we
can
run
experiment,
collect
data, put
in
a
table.
When
we
have
everything,
we
can
go
to
the
next
step,
estimation
of
the
factors.
We
formulate
the
full
regression
model
and
estimate
factor
effects.
So
we
check
which
factor
is
important.
Here
you
can
see
the
main
effects
plots,
two-
way
interaction
plots,
three-
way
interaction
plots
after
seven
days.
What
it's
worth
to
mention
is
that
interaction.
This
is
what
Phil
said,
that
the
interactions
are
important.
They
happen
in
the
real
world.
And
here
it's
a
good
example.
F
or
example,
when
we
have
cotton
at
the
surface,
it's
better
to
use
no
soaking.
If
you
use
soil,
it's
better
to
pre-soak
samples.
And
this
is
when
we
would
check
only
one
factor at
a
time,
so
for
example,
take
soil.
And
with
soil,
we
would
receive
that
presoking
is
better.
With
a
cotton
,
we
would
use
also
pre-soaking,
but
in
that
case
it's
not
true.
T
his
is
the
beauty
of
the
interactions.
And
that's
why
we
have
to
take
into
consideration
their
health.
Then,
statistical
test.
C
hecking
which
effect
is
important.
In
JMP,
we
can
also
see
parameter
estimates,
the
effect
tests,
and
conclude
which
is
significant.
W
hen
we
see
which
are
not
significant,
we
should
redefine
the
model
after
dropping
the
non-
significant
effect
and
calculate
estimate
one
more
time,
linear
regression.
In
that
case,
linear
regression
.
But
we
cannot
finish
on
that,
but
we
have
to
also
check
assumption
that
our
model
is
correct,
statistically
correct.
So
we
have
to,
for
example,
check
the
residuals
for
normal
distribution.
It
can
be
done
with
the
normal
probability
graph
of
residuals
in
a
JMP.
When
we
see
the
observations,
residuals
follow
the
straight
line
and
are
in
the
border
range,
it
means
that
it's
correct,
it's
normal
distribution.
But
also
we
can
check
it
with
the
test,
with
numerical
test
like
Shapiro-
Wilk
test,
to
check
if
residuals
follow
normal
distribution,
and
then
mean
test
to
check
if
the
mean
value
is
equal
to
zero.
When
we
finished
that,
we
can
draw
the
conclusion.
In
my
conclusion,
in
my
experiment,
was
that
the
most
important
factor
was
light,
and
its
effect
was
about
eight
times
higher
than
the
effect
of
the
second
most
important.
Plants
cultivated
in
dark
grow
higher
than
those
in
the sun
.
The
other
significant
factor
was
surface,
and
I
obtain
the
result
that
the
garden
soil
is
better.
In
garden
soil,
the
plants
grow
higher.
Also,
the
fourth
factor
which
I
added,
sowing
density,
also
occurred
to
be
important,
but
its
significance
increased
over
the
time.
After
the
three
days,
sowing
density
was
not
insignificant,
but
after
five
days
it
was
significant,
and
after
seven
days
it
was
even
more
significant.
So
it
increased
with
time.
A
lso
in
general,
during
seven
days,
three
different
three-way
interaction
were
significant,
which
suggests
that
all
factors
interact
really
together
and
we
cannot
interpret
them
separately.
That
all,
sun,
soil,
water,
everything
in
nature
is
combined
and
have
some
dialog
inner
dialog.
Also,
except
of
that,
I
checked
different
physical
things,
let's
say.
And
cress
cultivated
inside
light
become
green
and
developed
big
leaves.
Whereas
in
dark,
they
were
very
yellowish,
they
were
fragile.
When
I
touched
them,
they
broke
down.
E
xcept
that
they
were
higher,
but
they
were,
I
would
say,
not
healthy. And
also
roots
for
plants
cultivated
inside
light
go
longer.
Here
you
can
see
insideâ¦
it's
very
difficult
to
see
because
roots
are
white
and
cotton
is
white.
But
you
can
see
somehow
that
they
are
here
rolling
around
the
roots,
and
here
there
is
just
plain
cotton.
With
soil,
it's
better
to
visualize
because
it's
better
to
discern the s
oil.
And
we
can
see
that
in
some
light,
we
have
longer
roots,
whereas
in
dark,
they
are
very
short.
And
to
maximize
the
height
after
seven
days,
we
shall
use
soil,
we
should
pre-soak
samples,
seeds,
we
shall
put
them
in
dark
and
use
high
density.
Those
are
the
picture
of
my
results.
We
can
see
that
throughout
the
experiments,
samples
in
dark
all
the
time
they
were
yellow,
they
were
thin,
whereas
in
the
sunlight
they
were
healthy
green and thicker.
My
conclusion
regarding
the
design
of
experiments,
my
experience.
Design
of
experiment
is
a
great
tool
which
can
be
used
to
optimize
any
process.
Even
something
like
cultivated
garden
cress
can
be
fitted
to
the
design
of
experiments.
It
helps
to
incrementally
gain
knowledge
about
the
process.
For
example,
like
me
at
the
beginning,
I
had
no
idea
how
the
density
influenced
the
height,
but
when
I
put
so
many
things,
I
decided
that
I
gain
knowledge
that
it
has
influenced
and
I
have
to
do
something
about
it
and
also
consider
it.
We
can
also
increase
our
confidence
about
our
our
results,
and
that
our
results
will
be
indeed
statistically
significant.
So
we
will
have
no
biases.
We
know
that
interactions
are
involved.
Of
course,
some
factors
can
be
alias
with
others,
for
example,
in
factor
design.
But
the
advantage
of
design
of
experiment
is
that
we
are
aware
which
one
are
confounded,
and
we
can
draw
proper
conclusion
based
on
that.
So
if,
for
example,
one
pair
of
confound
factors
appears
to
be
significant,
but
we
don't
know
exactly
which
one,
we
know
to
what
we
have
to
focus
on.
And
also,
do
not
be
afraid
and
disaffected
in
the
first
try
to
not
be
successful,
treat
it
as
a
lesson
and
draw
a
conclusion
why
it
happened.
D
on't
give
up,
but
sit,
think,
why
I
failed,
what
I
can
do
in
other
way,
what
I
can
improve,
and
do
it
and
try
one
more
time.
And
design
of
experiment
can
bring
fun
with
the
proper
attitude
because
this
experiment
really,
really
have
fun.
And
as
I
said,
it
was
August,
it
was
very
sunny,
so
it
was
nice
weather,
nice
time
to
spending
time
on
the
[inaudible 00:27:04]
.
Thank
you
for
your
attention.
Yes,
thanks
very
much
and
thanks,
Weronika.
