Hello,
I'm
Ryan
Lekivetz,
Manager
of
the
DOE
and
reliability
team
at
JMP.
Today
I'm
going
to
talk
to
you
about
a
class
of
saturated
mix-
level
main
effects
designs
for
even
number
of
runs.
That
sounds
like
a
mouthful,
but
we'll
get
an
understanding
of
what
that
is
by
the
time
we're
finished.
If
you
see
in
my
JMP
journal
here,
here's
a
rough
idea
as
to
what
the
outline
is
going
to
look
like.
Now,
at
the
beginning
of
though,
I
do
have
to
call
out,
so
I
have
three
collaborators
on
this
project.
Bradley
Jones,
who
should
be
familiar
to
many
of
you
at
JMP,
Dibyen
Majumdar
and
Chris
Nachtsheim.
Just
some
history
and
preliminaries
.
I'll
say
usually...
If
you
see
in
there,
we
talked
about
main
effects
designs.
Now,
usually,
when
we
think
of
screening
designs,
we
often
think
of
factors
being
all
at
two
levels.
Your
standard
fractional
factorial
designs
that
you
see
in
textbooks,
but
really
there's
this
big
question,
what
about
nonlinear
effects?
Of
course,
it's
great
we
want
to
find
out
those
most
important
main
effects,
but
what
happens
if
there
is
some
nonlinearity?
In
particular,
in
2011,
Definitive
Screening
Designs
or
DSDs
hit
the
scene.
The
big
thing
with
those
DSDs,
here
we
were
assuming
all
the
factors
were
continuous.
Each
factor
and
DSDs
are
going
to
be
at
three
levels.
That
gave
some
hope
of
being
able
to
detect
quadratic
effects
when
they
were
large.
I'll
say
that
was
one
of
the
big
popularity
of
DSDs.
The
designs
we're
going
to
talk
about
here.
Our
main
effect,
if
you
think
of
DSDs,
not
only
were
they
good
at
picking
up
main
effects,
we're
also
looking
at
quadratics
and
interactions.
The
designs
we're
looking
at
here
are
really
main
effects
screening
as
our
primary
goal.
By
main
effects
screening,
I'm
saying
we
have
our
list
of
factors
and
we
want
to
find
out
which
of
those
are
the
primary
drivers.
We
want
to
find
out
those
significant
main
effects.
If
we're
really
lucky,
we
may
get
some
quadratic
interactions.
But
again,
main
effects
screening
is
the
big
thing.
What
you're
going
to
find
here
is
we're
going
to
have
a
mix
of
three-
level
and
two-
level
factors.
Whereas
the
DSDs
had
everything
at
three-
levels,
here
we
have
more
of
this
mix
of
three
and
two-
levels.
One
of
the
things
to
pay
attention
to
is
just
like
definitive
screening
designs.
When
we're
talking
about
these
three-level
factors,
we're
going
to
assume
that
they're
continuous.
In
particular,
that
means
we're
not
looking
for
balance.
Often
you
see
in
that
title
where
we
talk
about
mixed
levels.
We're
talking
about
three
and
two-level
factors.
But
I'll
say
traditionally
when
we
think
of
orthogonal
rays,
the
three-level
factors
are
categorical,
and
so
we
want
to
see
each
level
an
equal
number
of
times.
What
we're
going
to
find
here
is
that
these
three-level
factors
are j
ust
going
to
have
a
few
zeros.
One
of
the
other
big
things,
though,
is
that
we're
going
to
have
almost
as
many
factors
as
we
do
runs.
That's
where
that
idea
of
saturated
and
the
title
came
from.
Saturated
just
means
effectively
that
I
have
just
as
many
factors
as
I
do
runs.
That's
where
also
that
main
effects
screening
because
now
I
have
so
many
factors,
if
you
start
considering
quadratics
interactions,
that's
an
awful
lot
of
terms.
Because
we're
looking
at
so
many
factors,
we're
just
hoping
we
can
detect
those
significant
main
effects.
If
you
think
about
other
designs,
if
you
think
about
this
mix
of
three-
level
and
two
levels,
now
immediately
what
might
come
to
mind
is
some
of
the
classical
Taguchi
designs,
so the
L18
or
the
L36.
Those
of
you
who
are
familiar
with
JMP,
you
may
know
that
you
can
create
Definitive
Screening
Design
but
added
two-level
factors.
The
designs
here
I
would
almost
look
at as
an
extension
to
some
of
those.
Here
we're
going
to
have
a
lot
more
two-level
factors
than
you
might
want
to,
though,
in
our
standard
DSDs.
I'll
say
this
is
also
an
area,
though,
that's
picked
up
in
steam
a
lot
in
these
past
few
years.
You
see
some
references
here
as
to
other
authors
that
are
thinking
about
this
same
problem,
including
a
paper
that
I'll
come
back
to
at
the
end.
You
see
that
last,
the
Jones,
Lekivetz,
and
Nachshteim
paper.
That's
actually
related
to
this
work
as
well.
Let's
see,
hopefully
we're
all
in
the
same.
If
we
go
back
to
that
title.
The
saturated
just
means
we
have
a
lot
of
factors
relative
to
the
run
size.
Our
mixed level
means
that
we
have
some
two-level
and
three-level.
Just
to
mention
the
two-
level
could
be
continuous
as
well,
where
we're
just
not
interested
in
looking
at
those
quadratic
effects.
Main
effects
design
says
that's
where
our
most
important
thing
is
finding
those
significant
main
effects
and
that
even
numbers
of
runs,
it
turns
out
the
designs,
if
you
remember
that
recall
the
outline,
all
these
designs
we're
looking
at
are
going
to
be
an
even
number
of
runs.
Now,
when
we
think
about
building
these
designs,
so
in
some
of
these
preliminary,
I
should
really
talk
about
the
building
blocks.
These
designs
are
going
to
be
built
upon
using
other
matrices
or
other
designs
in
the
literature.
Depending
on
how
familiar
you
are
with
Definitive
Screening
Designs,
there's
this
idea
of
something
called
a
conference
matrix.
A
conference
matrix
is
just
an
M
by
M
matrix
that
we
use
to
construct
Definitive
Screening
Designs.
One
of
the
nice
things
with
conference
matrices
is
that
in
general,
they
exist
for
multiples
of
two
rows
and
columns.
I
said
in
general,
there's
a
conference
matrix
for
every
multiple
of
two
from
2 to 30,
except
for
22.
Something
to
mention
here,
though,
is
that
in
the
cases
where
a
conference
matrix
does
not
exist,
or
let's
say
if
you
have
an
odd
number
of
runs,
you
can
use
something
called
a
pseudo-
conference
matrix.
A
pseudo-
conference
matrix
says,
I
want
to
make
it
look
as
close
as
I
can
to
a
conference
matrix,
which
has
this
special
property.
What
properties
a
conference
matrix
have?
Well,
if
I
take
C
transpose
T,
or
conversely,
C
C
transpose,
it's
going
to
be
M
minus
1
times
the
identity
matrix.
What
that
really
means
is
that
these
columns,
the
columns
of
the
matrix
C,
orthogonal.
If
I
take
that
cross-
product
of
any
two
columns,
we're
going
to
get
a
zero.
But
notice
we
have
that
M minus 1,
because
one
of
the
other
features
of
the
conference
matrix
is
that
each
row
and
column
has
exactly
one
zero.
If
you
have
JMP
17.2,
so
I'll
admit
that
this
JSL
first
shows
up
in
JMP
17.2,
so
we
have
this
command
for
conference
matrix.
Here
I
can
just
ask
for
the
same
6x6
conference
matrix.
Again,
what's
the
special
property?
Well,
if
we
look,
so
if
I
take
C
times
C
transpose
or
conversely,
I
get
6
minus
1,
which
is
5
times
the
identity
matrix.
Each
of
those
columns
is
going
to
be
orthogonal.
Again,
starting
a
JMP
17.2,
you
can
create
your
own
conference
matrix
just
by
giving
it
the
order.
Here
I
wanted
a
six-run
conference
matrix,
so
I
was
able
to
put
that
in.
Let's
try
it
with
eight
as you'll
see.
There's
an
8
by
8
conference
matrix.
That's
one
of
the
building
blocks
that
we
need.
Another
one
is
a
similar
type
of
structure.
It's
called
a
Hadamard
Matrix.
The
difference
is
a
conference
matrix
had
values
of
negative
one,
zero,
and
one.
A
Hadamard
matrix
just
has
values
of
plus
and
minus
one.
A
Hadamard
matrix
exists
for
most
multiples
of
four.
In
particular,
this
is
where
when
you
hear
Hadamard
matrix,
another
thing
you'll
hear
about
often
times
is
an
orthogonal
array.
In
particular
for
a
Hadamard
matrix,
it
has
that
same
property.
If
I
take
it
by
its
transpose,
in
this
case,
I
get
N
times
the
identity
matrix.
If
you
recall
the
conference
matrix,
it
was
that
M
minus
1,
the
order
because
we
had
one
zero
in
each
row
and
column.
Here
it's
going
to
be
N
times
the
identity
matrix.
The
idea
here
is
that
if
I
take
any
pair
of
columns,
we
have
that
concept
of
orthogonality
for
any
pair
of
columns.
Similar
to
the
conference
matrix,
we
actually
have
a
special
command
in
JSL
for
constructing
a
Hadamard.
Let's
take
a
look
here.
If
you
notice,
let's
just
say
Hadamard
8,
this
is
going
to
give
me
an
8-run
Hadamard
matrix.
If
we
see
about
taking
a
Hadamard,
the
strand
rows,
there
we
get
eight
times
the
identity
matrix.
Again,
so
what
that
means,
if
I
take
any
pair
of
columns,
they're
going
to
be
orthogonal,
which
you
can
actually
already
see.
Just
pretend
that
this
first
one
is
like
an
intercept.
You
see,
all
of
these
are
balanced.
I'm
going
to
get
that
idea
of
orthogonality.
We're
actually
almost
there
with
our
building
blocks.
The
last
piece
that
we
need
is
something
called
the
Kronecker
product.
What
you'll
see
is
that
throughout
we
may
not
even
really
need
to
think
of
it
in
terms
of
a
Kronecker product.
Just
often
when
we
create
these
designs,
that's
the
way
we
like
to
think
of
these.
You'll
see
a
Kronecker product
is
denoted
by
this
symbol.
It
looks
like
a
multiplication
with
a
circle
around
it.
In
JMP
or
another
definition for
Kronecker product
is
called
the
direct
product.
It
just
happened
it's
a
convenient
way
to
construct
designs.
All
of
this,
Kronecker product
or
direct
product
is,
if
I
take
a
matrix
A
and
the
Kronecker product
of
B,
I
take
each
element
of
A
and
apply
it
to
the
entire
matrix
B
over
and
over
again.
Where
this
comes
in
handy,
let's
see
where
you
may
have
seen
something
like
this
before.
Again,
we
have
a
direct
product,
a JSL
command.
Let's
say
if
I
just
wanted
to
start
with
that
6x6
conference
matrix.
Now
let's
say
if
my
matrix
A,
so
think
here
in
this
Kronecker product
A,
I
just
take
these
2
by
1,
so
two
rows
plus
one
and
a
minus
one,
and
direct
product
with
C.
What
do
I
get
here?
Well,
this
is
effectively
if
I
added
a
center
run,
this
would
be
a
13-run
DSD
and
six
factors.
By
taking
this
one
and
minus
one,
so
with
the
Kronecker product
up
here
with
this
plus
one,
I
get
that
conference
matrix
C,
and
down
below
I
get
negative
C.
This
Kronecker product
is
just
a
convenient
way
to
think
about
things
like
what
we
might
call
a f
oldover.
With
those
preliminary
is
done,
now
we
can
actually
start
talking
about
these
different
constructions.
The
first
method
is
the
most,
I'll
say
the
nicest
of
all
of
them.
What
we're
going
to
say,
this
is
where
our
run
size
is
going
to
be
a
multiple
of
eight.
What
are
we
doing
here?
Well,
here
we're
going
to
start.
We're
going
to
have
a
conference
matrix
of
order
M
equals
4K.
Again,
remember,
the
conference
matrices
tend
to
exist
as
long
as
we
have
an
even
number.
Likewise,
for
the
Hadamard
matrix,
that's
where
we're
looking
for
multiples
of
four.
Here
I'm
going
to
take
a
conference
matrix
of
order
M
and
a
Hadamard
matrix
of
the
same
order
where
we're
assuming
here
that
both
exist.
What
I'm
doing
here
is
I'm
going
to
fold
over.
You
can
actually
express
this
as
a
Kronecker product,
but
here
I
find
it
more
convenient
just
to
write
it
this
way.
You
see,
this
first
part
looks
like
a
Definitive
Screening
Design.
I'm
taking
a
conference
matrix
and
folding
it
over.
Then
on
the
other
side,
I'm
replicating
this
Hadamard
matrix.
I'm
taking
a
copy
of
that.
What
do
we
get
with
this
design?
Well,
these
first
M
columns
that
are
formed
from
the
conference
matrix,
that's
going
to
give
us
M
three-level
factors.
The
remaining
M
minus
1
are
all
going
to
be
two-level
factors.
This
C
part
is
going
to
be
for
three-level,
this
H
part
for
two-level.
What
do
we
end
up
with?
Well,
what
we
did,
we
basically
doubled
these
here.
We're
going
to
have
two
M-runs
and
two
M
minus
1
factors.
Let's
take
a
look
at
what
this
might
look
like.
Let's
see
an
example
here.
Let's
take
my
C.
I'm
going
to
just
create
an
8
by
8
conference
matrix.
First,
let's
construct
these
three-level
columns.
I'm
going
to
take
that
direct
product.
Again,
in
this
case,
I
want
that
foldover,
I
want
C
and
minus
C.
Let's
take
a
look
at
what
C
looks
like.
Again,
this
is
just
that
foldover
structure
on
that.
One
thing
here,
you
noticed
I
said
the
remaining
M-1
column
for
the
two-level
factors.
The
reason
for
that...
Let's
take
a
look
at
the
Hadamard
matrix
of
Order
8.
If
I'm
going
to
replicate
this,
if
I'm
going
to
copy
this,
well,
this
first
column
here
is
going
to
be
for
the
intercept.
I
don't
want
to
put
that
as
one
of
my
design
factors
if
it
never
changes,
if
it's
constant
throughout
the
entire
thing.
If
you
notice
here,
I'm
going
to
just
drop
the
intercept.
Now
I
have
an
8
by
7
design.
If
you
notice,
I'm
going
to
use
the
direct
product
again
here.
But
instead
of
with
the
conference
matrix
where
I
was
using
one
and
minus
one,
I
just
want
to
make
a
copy
of
that
Hadamard
matrix
H
without
the
intercept.
Let's
give
that
a
look.
We
can
take
a
look
here
on
this
matrix
where
you
see
that
the
one,
one,
one.
You
can
actually
see
where
it
gets
just
copy
it
again.
It's
just
the
same
matrix
stacked
on
top
of
itself.
If
I
concatenate
all
of
those
together,
you
can
see
I
have
a
16
by
15.
I
can
actually
just
create
that
data
table.
I
have
this
design
now,
a
16-run
design
with
15
factors.
The
first
eight
of
those
are
three
level
and
the
remaining
seven
are
at
two
levels.
Let's
just
take
a
look.
Let's
go
to
design
diagnostics
and
let's
see
what
this
looks
like
and
evaluate
design.
You
can
see
I
just
created
a
main
effects.
This
might
be
hard
to
see,
so
this
looks
pretty
messy,
but
you
can
already
see
there's
a
special
structure
with
these
designs.
One
thing
I
want
to
point
out,
let's
get
rid
of
the
alias
terms.
Let's
just
look
at
the
correlations
here
with
these
main
effects.
You
can
see
in
this
case,
actually,
all
my
main
effects
are
orthogonal
to
each
other.
One
thing
to
point
out
here,
because
I
was
using
that
Hadamard
matrix
as
a
building
block,
those
two-level
designs,
we
have
all
that
nice
orthogonality
there.
You'll
notice
this
fractional
increase
in
confidence
interval
length
is
a
little
bit
higher.
Why
is
that?
Well,
that's
because
we
have
these
three-level
factors.
Those
first
factors
are
at
three
levels.
What
that
means
is
that
if
there
is
a
quadratic
effect,
now
this
is
giving
me
some
hope
of
detecting
that.
Again,
don't
forget,
we're
already
at
16
runs
and
15
main
effects.
If
we
start
thinking
about
quadratic
effects
and
interactions,
we
really
have
to
hope
for
those
large
effects
when
it
comes
to
doing
any
model
selection.
If
you
think
of
a
traditional
design
where
we
only
had
two
levels,
in
that
case,
we
would
have
no
hope
of
being
able
to
detect
any
quadratic
effect.
That's
that
first
construction.
In
some
sense,
that's
where
everything
works
out
nicely.
I
have
a
conference
matrix
and
I
had
a
run
matrix
available
to
me.
That
gives
me
a
run
size
that's
a
multiple
of
eight.
Now
what
happens
if
I
don't
have
one
of
those?
Let's
say
my
design
is
going
to
be
a
multiple
of
four.
The
run
size
is
going
to
be
a
multiple
of
four.
Let's
assume
now
that
I
have
a
conference
matrix
available
to
me,
but
maybe
I
don't
have
a
Hadamard
matrix.
If
you
recall
before,
where
for
my
Hadamard,
I
need
it
to
be
a
multiple
of
four,
the
original
run
size,
so
that
when
I
doubled
it,
it
was
a
multiple
of
eight.
Instead,
maybe
if
I
don't
have
a
Hadamard
matrix,
I
could
use
something
like
a
D-optimal
main
effects
plan.
This
would
just
be
if
I
went
into
custom
design.
Let's
say
if
I
wanted
for
six
runs,
I
would
go
into
custom
design
and
say
I
have
five
main
effects,
five
factors,
and
I
want
six
runs
for
a
main
effects.
This
construction
actually
looks
a
lot
like
it
did
in
method
1.
The
only
real
difference
is
instead
of
that
Hadamard
matrix,
now
I'm
going
to
be
using
this
D-optimal
main
effects
plan.
But
it
turns
out
to
be
the
same
thing
here.
I'm
going
to
have
those
first
M
columns
for
the
three-level
factors,
the
remaining
M
minus 1
for
the
two-level.
I'm
really
at
that
idea
of
saturation
because
by
the
time
I
factor
in
the
intercept,
I'm
at
two
M-runs
and
2
M
minus
1
factors.
Let's
take
a
look
at
how
this
one
might
look.
In
this
case,
I'm
going
to
go
with
a
10-run
conference
matrix,
which
again,
because
that's
a
multiple
of
two,
I
can
create
a
conference
matrix.
The
one
thing
to
pay
attention
to
here,
though,
because
we're
at
10-
runs,
there's
not
going
to
be
a
Hadamard
matrix
of
order
10
available
to
us.
Let's
construct
though
first
though,
let's
just
fold
over
that
CC.
Again,
that's
just
that
DSD-like
structure.
Now
in
this
case,
because
I
don't
have
that
command,
and
so
this
is
where
you
could
decide
how
to
do
this.
You
may
actually
even
have
your
own
design
that
you
want
to
use
in
this
case.
I'll
say
in
this
case,
I'm
showing
a
D-optimal
main
effects.
You
may
want
to
use
something
l ike
a
Bayesian
D-optimal
where
two-factor
interactions
are
if
possible.
But
so
in
this
case,
all
that
I've
done
is
I've
taken
the
D-optimal,
10-run
design
for
nine
factors,
and
I've
just
happened
to
include
the
intercept
here.
If
you
go
into
Custom
Design,
this
is
what
you
get
from
the
model
matrix.
The
model
matrix
actually
includes
the
intercept
by
default.
That's
in
fact
where
I
had
this
from
here.
Again,
I'm
going
to
drop.
Let's
drop
that
intercept
column.
Again,
here
I'm
just
replicating
the
exact
same
thing
again.
I've
just
folded
over,
not
folded
over.
I've
just
replicated
the
exact
same
thing
twice.
It'd
be
hard
to
see
as
to
where
it
was
replicated
here.
But
again,
all
that
we've
done
is
just
made
a
copy.
I
made
a
copy
of
that
doptN10m9
twice,
concatenate
those
together.
Let's
create
the
data
table
again.
Again,
20-
runs
and
19
factors.
Again,
keep
in
mind
that
my
first
10
factors
are
all
at
three
levels,
the
remaining
at
two.
Let's
take
a
look
at
evaluate
design
again
here.
I'm
going
to
put
all
of
these
factors
in.
In
this
case,
before,
let's
go
directly
to
remove
those
alias
terms.
Let's
take
a
look
at
the
color
map.
This
still
looks
like
a
pretty
good
color
map
to
me.
What
do
we
notice
here
versus
the
last
one?
One
of
the
biggest
differences
is,
well,
here
our
three
levels
are
still
going
to
be
orthogonal,
and
that's
because
we
were
using
a
conference
matrix.
The
three-
levels
and
the
two-
levels
are
orthogonal.
In
particular,
my
three-level
factors
are
orthogonal
to
the
two-level
factors.
But
because
I
was
using
that
10-run
design,
it
turns
out
we
can't
get
perfect
orthogonality
for
the
10-run
design
with
nine
factors.
If
we
look
among
those
two-level
factors,
we
have
some
small
correlations
there.
One
thing
to
point
out,
there
does
exist
a
Hadamard
matrix
of
order
20.
In
some
sense,
we
are
taking
a
little
bit
of
a
hit
on
those
two-level
factors.
But
if
you
take
a
look,
the
cost
of
using
this
type
of
design
versus,
let's
say,
everything
at
two
levels,
we
have
about
a
5%
increase
in
the
confidence
interval
length.
But
the
nice
thing
here
now
is
that
now
we
have
those
factors
at
three
levels.
If
we
really
are
worried
about
quadratic
effects
appearing
in
these
first
ones,
now
we
have
a
chance
of
detecting
those
at
a
small
price
to
that
estimation
efficiency
for
those
main
effects.
That
was
great
in
the
case
that
we
still
had
a
conference
matrix
that
existed,
but
no
Hadamard.
This
last
method,
this
is
where
we
don't
really
fit
into
either
of
those
cases.
In
this
case
now,
we're
talking
about
a
run
size
is
going
to
be
a
multiple
of
two.
In
this
case,
what
we
have
is
we
don't
have
a
conference
matrix
available
to
us.
This
is
where
we're
going
to
use
that
pseudo-
conference
matrix.
I'll
say
if
you're
really
interested
in
that,
I'll
go
back
to
these
preliminary.
In
the...
Let's
see.
Actually,
it
was
the
original
DSD
design.
Sorry,
this
2011
Definitive
Screening
Design.
If
you
also
look,
I
have
it
in
the
list
of
references
at
the
end.
Let's
see,
those
who
are
particularly
interested.
This
original
class
of
three-level
design
for
Definitive
Screening
in
the
presence
of
second-
order
effects.
This
paper
was
written
before
they
were
aware
of
the
existence
of
this
idea
of
conference
matrices.
In
that
paper,
they
talk
about
a
general
purpose
algorithm
for
creating
something
that
looks
like
a
conference
matrix.
You
set
zeros
along
the
diagonal,
and
then
the
rest
of
the
values
are
going
to
be
plus
and
minus
one,
where
you're
trying
to
make
a
main
effects
D-optimal
design.
These
pseudo-
conference
matrices,
as
they
were,
they
look
like
a
conference
matrix.
You
can
use
that
algorithm
when
the
conference
matrix
doesn't
exist.
Really,
what
it's
trying
to
do
is
to
drive
it
to
look
as
close
as
it
can.
If
you
remember
a
conference
matrix,
if
you
take
C
transpose,
you
get
zeros
and
the
off
diagonal
for
perfect
orthogonality,
it's
going
to
be
trying
to
drive
it
to
look
like
that
where
I
can't
make
it
perfectly.
Similar
to
the
case
we
had
for
method
2,
our
T
is
going
to
be
a
D-optimal
main
effects
plan.
What
do
we
end
up
with
here?
Again,
our
first
M
columns
are
still
going
to
be
three-level
factors,
the
remaining
M minus 1
for
the
two-
level.
Again,
we're
still
at
that
case
of
saturation
by
the
time
we
factor
in
the
intercept.
The
cost
here
is
that
we're
not
going
to
get
the
nice
orthogonality
that
we
may
have
had
in
method
1
and
2.
Let's
take
a
look
at
how
this
one
works
here.
My
C
in
this
case,
and
I'll
do
that
CC.
What
I
started
with
here
was
a
pseudo-
conference
matrix
of
order
nine.
Nine
is
not
a
multiple
of
two.
I
have
to
do
something
that
looked
like
a
pseudo-
conference
matrix.
Let's
just
take
a
look
here.
Let's
actually
take
a
look
and
see
what
that
C
transport
C
looks
like.
You
notice
I
can't
get
that
perfect
orthogonality,
but
instead
I
have
eight
along
the
diagonal
and
then
these
plus
and
minus
ones
on
the
off
diagonals.
We're
close.
I
have
that
CC
that
was
just
doing
the
same
thing,
folding
that
matrix
over.
Likewise,
so
T,
that's
going
to
be
of
the
same
order.
I
took
the
nine-run,
eight-factor,
D-optimal
design.
In
this
case,
I've
actually
already
removed
the
intercept.
We're
just
going
to
replicate
that
design
again.
I
have
this
18
by
8.
I'm
going
to
combine
those
two
pieces
together.
Let's
take
a
look
at
the
table.
This
is
a
particularly
difficult
design
to
generate
in
general.
In
this
case,
I
don't
have
a
nice
number.
My
run
size
is
18.
Now
I
have
these
first
nine
factors
at
three
levels
and
the
remaining
eight
at
two
levels.
Not
surprisingly,
there
is
a
cost
of
this.
Again,
let's
just
remove
those
alias
terms
again
and
take
a
look
at
our
color
map.
You
see
now,
my
two
and
three
levels
are
actually
orthogonal
to
each
other,
but
the
three
levels
have
a
small
correlation
among
them.
That
was
where,
remember,
if
you
recall,
we
saw
that
plus
and
minus
one
and
that
off
diagonal.
Likewise
for
the
two
levels.
We
actually
have
one
quite
high
correlation
here,
but
in
general,
so
about
0.1.
In
this
case,
we
don't
even
have
an
orthogonal
design
to
compare
it
to.
We
actually
have
a
couple
of
factors
that
we
may
be
worried
about
a
bit,
have
a
little
bit
of
a
larger
fractional
increase.
Those
three
levels
will
say,
well,
we
can't
get
those
to
be
perfect
as
it
is.
This
fractional
increase
is
compared
to
the
hypothetical
orthogonal
design,
the
orthogonal
array,
which
doesn't
even
exist.
We're
paying
a
small
price,
but
we
still
have
some
generally
nice
properties.
The
three
levels
are
orthogonal
to
the
two
levels,
and
we've
minimized
that
correlation
in
general.
In
some
sense,
those
are
those
three
methods.
The
nice
thing
is
depending
on
the
run
size
that
you
have
and
depending
on
the
number
of
factors,
this
methodology
or
this
class
of
designs
really
gives
you
some
flexibility
in
the
number
of
runs.
Of
course,
if
you
can
afford
a
multiple
of
eight
or
a
multiple
of
four,
the
properties
are
going
to
look
a
lot
nicer.
But
I
mean,
when
runs
are
really
expensive,
we
still
have
this
method
3,
which
is
going
to
give
you
a
reasonable
design.
With
that,
I
just
want
to
give
some
final
thoughts
here.
I
have
a
link.
If
you
take
a
look
at
this
journal
afterwards,
I
have
a
link
to
a
presentation
from
JMP
Discovery
Europe
from
Bradley
Jones
that
was
on
orthogonal
mix
level
designs.
The
designs
in
this
presentation,
the
orthogonal
mix
level
designs,
will
almost
look
very
similar
to
what
was
presented
here.
The
designs
presented
in
this
presentation
are
for
when
you're
closer
to
saturation.
I
would
say
these
orthogonal
mix-level
designs
from
the
previous
discovery,
and
we
actually
have
a
paper
on
that.
That's
this
Jones
Lekivetz,
and
Nachtsheim
that
was
in
JQT,
Journal
of
Quality
Technology.
Those
designs
work
very
well
when
you
have
about
half
the
number
of
factors
relative
to
the
run
size,
which
sounds
a
lot
like
DSDs,
but
it's
when
you
have
more
two-level
factors.
In
the
design
presented
in
that
presentation,
you
can
go
up
to
about
three-quarters
of
the
run
size.
The
designs
presented
here
are
really
when
you're
closer
to
that
saturation,
where
you
say,
well,
no,
I
really
have
a
lot
of
factors
that
I'm
interested
in,
and
the
runs
are
really
expensive.
The
designs
here
to
fill
that
last
gap
when
you
want
to
get
close
to
saturation.
I
showed
you
these
designs
look
nice,
but
can
they
actually
do
anything
when
it
comes
to
model
selection?
I'll
say
here.
Again,
we
have
run
a
lot
of
simulations
on
these.
This
isn't
doing
it
properly,
this
is
just
for
a
single
realization.
Let's
say
here
my
factors,
so
call
one,
call
five,
call
13
and
14,
I've
chosen
it
as
being
significant.
In
base
JMP,
you
might
want
to
use
something
like
stepwise.
Let's
take
a
look.
Here
I've
done
a
reasonable
job.
I
may
have
ended
up
with
one
extra
factor,
but
in
some
sense,
I'd
still
be
quite
happy
that
I've
been
able
to
pick
up
those
effects
there.
Let's
try
this
same
thing
with
generalized
regression.
If
we
take
a
look,
the
same
thing.
Let's
see,
1,
5,
13,
and
14.
If
we
go
back,
1,
5,
13,
and
14.
In
this
case,
both
of
you
have
JMP
Pro
with
generalized
regression
or
Fit
stepwise,
both
were
able
to
pick
up
those
main
effects
that
we
had
in
there.
Just
to
show
you
now,
if
you
have
a
large
quadratic
effect,
I'll
say
because
compared
to
DSDs
where
we
had
that
center
point
run,
this
we
only
have
two
zeros.
We
don't
even
have
a
third
for
a
center
point
and
we
have
a
lot
more
factors.
Detecting
quadratic
effects
is
still
going
to
be
difficult.
But
let's
just
pretend
we
have
a
large
one.
Here
I
have
again,
the
1,
5,
13,
and
14.
But
now
I've
also
added
a
quite
large
quadratic
effect
for
the
column
one.
Let's
just
take
a
look
here.
Let's
try
to
fit
stepwise
again.
Now
I've
actually
just
added
those
quadratics
in
the
model.
Again,
I
have
one
extra
term,
but
if
you
see,
even
in
this
case,
it
did
actually
pick
up
that
quadratic
effect.
Not
only
did
it
pick
up
the
correct
main
effects,
it
did
detect
the
quadratic
effect
that
I
had.
Let's
try
the
same
thing.
Let's
see
the
model
launch.
It
looks
like
I
already
have
it
shown
here.
The
same
thing.
You
see
the
1, 5,
13,
14,
and
generalized
regression
was
also
able
to
pick
up
that
quadratic
effect.
I'll
say
I
would
not
expect
to
anticipate
that
you
can
detect
that
many
quadratic
effects.
But
even
if
you
look
at
your
residual
plots
or
your
main
effects
plots,
even
those
two
zeros
give
you
some
indication
that
maybe
I
do
want
to
follow
up
and
take
a
look
at
those
factors
a
little
bit
deeper
for
quadratic
effects.
Where
again,
in
your
traditional
screening,
if
you're
only
doing
things
at
two-
level,
you
wouldn't
have
the
chance
to
do
that.
With
that,
again,
I
will
post
this
journal
where
this
video
is
located,
but
I'll
also
flash
up
these
references
at
this
time.
With
that,
thank
you
for
taking
the
time
to
watch
this
video,
and
please
share
any
messages
you
have
in
the
community
below.
