Thank you
for
giving
us
this
opportunity
to
talk
about
how
we
use
JMP
in
pharma
process
validation
and
continuous
process
verification.
My
colleague,
Louis
and
I
will
give
you
a
guide
to
the
background
of
it,
how
we
do
it,
and
what
is
the
outcome.
The
agenda
is
that
first
we
will
describe
the
background
and
the
concepts.
Since
its
validation,
we
must
justify
the
assumptions
we
have
behind
our
models,
so
sanity
check
is
important.
Then
I
will
go
on
to
talk
about
the
particular
methods
we
are
using
and
which
formulas
we
have
to
make
for
things
that
are
not
there
in
JMP
upfront.
Then
Louis
will
take
over
and
show
how
the
JMP
script
is
doing
it
to
automate
the
calculations
and
implement
what
is
missing
in
JMP.
Finally,
we
will
make
some
conclusions
and
hopefully
there
will
also
be
time
for
questions.
Now
to
the
background
and
the
concept
we
are
working
with.
Louis
and
I
come
from
a
company
called
NNE,
and
we
do
consultancy
within
the
pharmaceutical
industry,
and
we
are
JMP
partners,
so
we
help
our
clients
getting
value
out
of
the
data,
of
course,
using
JMP.
A big
issue
within
the
pharmaceutical
industry
is
process
validation
when
they're
going
to
launch
a
new
product. T
raditionally
for
many,
many
years,
it
has
been
done
by
making
three
batches.
I
f
they
were
good,
the
validation
was
passed.
But
this
is
not
really
predicting
the
future,
this
is
predicting
the
past.
So, o
f
course,
you
can
make
three
batches
that
were
good,
and
then
the
problem
can
appear
later
on.
To
compensate
for
that,
that,
the
industry
has
traditionally
had
an
extensive
and
thereby
costly
QC
inspection
on
every
batch
to
ensure
it
was
okay.
However,
if
we
could
prove
that
all
future
batches
would
be
good,
then
we
could
reduce
the
QC
cost
heavily.
But
how
do
we
predict
that
the
future
batches
will
be
good
with
confidence?
Yeah,
we
could
use
prediction
intervals
or
individual
confidence
levels,
as
I
call
in
JMP,
at
least
in
some
menus.
If
we
put
batch
as
a
random
factor,
then
the
validation
batches
we
are
looking
at
will
be
seen
as
a
random
sampling
of
all
future
batches.
T
hereby,
we
can
predict
how
future
batches
are
going
to
be.
If
this
prediction
is
inside
specification,
we
have
actually
proven
that
all
future
batches
will
be
good
given
that
we
can
maintain
the
validator
state
force.
However,
this
might
take
more
than
three
batches,
but
do
they
all
have
to
be
in
state
two,
as
we
show
in
this
graph?
Maybe
not.
We
actually
should
just
still
make
three
batches
in
your
first
process
of
validation.
If
it's
not
enough,
then
make
the
rest
in
which
we
call
straight
three
A,
which
is
after
commercialization,
because
then
you
can
just
sell
them
one
by
one.
It's
not
a
big
problem
that
you
have
to
make
more.
However,
this
requires
two
things
to
be
fulfilled.
The
first
is
that
we
still
have
this
extensive
QC
test
that
we
can
find
out
if
your
batch
is
bad
because
we
are
not
yet
tooled,
it
will
not
happen.
A
lso,
of
course,
it
requires
that
the
estimated
performance
is
okay.
How
can
we
see
if
the
estimated
performance
is
okay?
We
can
do
that
by
seeing
if
control
limits
are
inside
the
specification.
Then
we
can
go
on
the
market
and
later
on,
when
also
prediction
limits
are
in
such
specification,
then
we
can
actually
remove
or
at
least
reduce
the
QC
inspection.
We
might
be
going
from
monitoring
the
product
until
monitoring
the
process.
What
is
it
all
about?
Validation
is
about
predicting
the
forms
of
future
batches
with
confidence,
not
just
predicting
the
past.
How
can
we
do
that?
We
can
do
that
by
using
prediction
or
12 intervals
in
JMP.
How
many
batches
will
we
suggest
to
make
in
stage
two?
We
still
suggest
three,
but
you
might
not
be
done.
How
can
you
then
pass
stage
two
with
few
batches
so
we
can
go
on
to
the
market?
That's,
as
I
said
before,
look
at
control
intervals
because
they
are
without
confidence.
You're
not
punished
by
only
having
three
batches.
T
hen
later
on,
look
at
prediction
or
12
intervals.
If
control
intervals
are
inside
specification,
prediction
intervals
are
not.
The
most problem
of
the
reason
for
that
is
lack
of
data.
This
shouldn't
be
limiting
you
to
go
onto
the
market.
Then
how
many
batches
to
make
in
stage
3A?
That's
fairly
simple.
Simply
until
your
prediction
limits
are
inside
specification,
then
you
have
passed
your
stage
3
A.
These
limits
you
can
also
use
as
prediction
limit
after
stage
3
A
when
you
go
into
stage
3B.
We
can
also
see
it
on
this
flow
chart
where
you
in
upper
left
corner,
start
with
analyzing
your
validation
ones,
typically
tree.
Then
you
calculate
your
prediction
limits
in
JMP
as
a
random
factor.
If
they
are
inside
spec
limits,
everything
is
fine.
You
have
passed
both
stage
2
and
3
A
and
everything
is
fine.
If
it
turns
out
that
prediction
limits
are
too
wide
compared
to
the
spec
limit,
I
would,
as
a
next
thing
l
ook
at
my
control
limits
which
are
without
confidence.
If
they
are
inside,
then
we
just
need
more
data
and
we
will
collect
more
data
and
we
calculate
and
a
point
of
time
we
will
go
out
here
and
have
passed
also
strict
TA.
Of
course,
if
it
happens
that
even
control
limits
are
outside
specification,
then
actually
the
estimated
value
is
bad
and
you
are
not
really
ready
for
validation
and
you
have
failed.
Hopefully,
this
is
not
going
to
happen.
Let's
go
on
to
the
sanity
check.
Now
I
will
go
into
into
JMP.
I
will
try
to
demonstrate
how
it
works.
Here
you
see
a
JMP
table
which
we
have
made,
or
not
we
have
made,
it
actually
comes
from
ISPE,
International
Society
of
Pharmacoepidemiology,
who
put
us
this
data
set
as
a
good
example
on
how
a
validation
set
should
look
like.
Many
companies
or
consultancy
has
tried
to
calculate
on
that
and
see
how
would
they
conclude.
It's
basically
a
data
set
from
tablet
manufacturing,
and
classically
it
contains
these
three
batches
A,
B
and
C.
Then
we
are
taking
out
powders
in
15
locations
in
the
blender,
and
then
we
are
measuring
the
API
content,
that's
the
strength
of
the
actual
[inaudible 00:06:52]
ingredient,
which
would
be
between
85
and
115
to
be
inside
specification.
If
you
look
at
the
variability
chart
of
the
data,
you
can
see
here
we
have
the
three
batches
A,
B
and
C,
and
we
also
have
the
15
locations
in
each
batch.
Then
within
each
batch
location
combination,
we
have
four
measurements.
The
first
you
can
see
is
that
the
within
location
variation
on
batch
B
is
somewhat
bigger
than
on
batch
A
and
C.
We
do
not
have
ba
tch heterogeneity.
To
make
that
more
objective,
you
can
put
S
control
limit
on
your
standard
deviation
chart
and
you
can
see
that
you
are
out
of
control
in
Batch
B.
Another
way
of
seeing
it
would
be
to
make
a
heterogeneity
of
variance
test
as
you're
seeing
down
here.
Then
it's
also
very
clear
that
there's
more
within
or
in
location
variation
on
batch
B
than
on
the
other
ones.
Strictly,
you
cannot
pool
these
variances
however
you
can,
but
we
need
then
to
do
weighted
regression
where
we
weigh
with
inverse
variance.
That's
the
next
step
we
will
do,
go
into
log
variance
modeling,
which
you
can
do
in
JMP.
We
have
simply
here
just
made
a
log
variance
model
where
you
can
see
that
the
location,
the
batch,
the
interaction
all
have
a
significant
influence
on
the
level,
but
only
the
batch
has
a
significant
influence
on
the
variance.
As
you
can
see
that
the
variance
for
batch
B
is
somewhat
bigger
than
within
variance
on
batch
A
and
C.
When
you
have
built
this
model,
you
can
from
up
here
just
save
the
columns
and
you
can
save
your
variance
formula.
I've
already
done
that.
It's
already
in
my
data
set.
If
I
unhide
here,
here
you
can
see
my
variance
column
from
my
log
variance
model.
You
have
a
formula
here
saying,
what
is
it
in
group
A
and
C
for
batch
A
and
C,
and
what
is
it
for
batch
B?
Based
on
that,
I
can
now
make
weighted
regression
where
I
simply
just
make
a
weight
column
where
I
weight
my
regression
classically
with
inverse
variance.
Then
I'm
ready
for
making
a
sanity
check
on
my
data
because
now
I
can
pool
the
variances
after
weighted
regression,
so
I
can
start
doing
sanity
check
on
my
data.
The
first
thing
I
would
do
is
to
look
at
a
systematic
model
where
I
have
my
location,
batches,
and
the
interaction. T
hen
I
can
see
on
my
[inaudible 00:09:28]
type
residual
plot,
I
have
no
outliers
and
outlaws
,
but
it
is
inside.
I
can
also
see
on
my
box
plot
transformation,
there's
no
need
to
transform
my
data.
Here
it's
reasonable
to
assume
that
I
have
these
normal
distributed
residuals,
which
is
the
assumption
behind
a
fixed
least
squares
model.
Then
next
step
is
to
put
batches
as
a
random
factor,
not
just
a
systematic
factor,
because
I
would
like
to
predict
future
batches.
This
is
what
I
have
done
here.
T
hen
suddenly
you
get
these
variance
components.
What
is
the
between
batch
variation?
What
is
the
batch
location
variation?
W
hat
is
the
residual
mean
within
batch
within
location
variation?
Since,
of
course,
I
have
weighted
it
with
inverse
variance,
this
number,
of
course,
gets
close
to
one.
However,
in
this
model,
there
are
also
some
additional
assumptions,
namely
that
the
batch
effect
and
the
batch
location
effect
is
randomly
distributed,
meaning
that
these
random
effect
predictions
up
here
should
follow
a
normal
distribution.
To
test
that,
you
can
just
put
them
into
a
table,
which
I've
done
here,
and
we
can
test
the
loops
just
making
an
empty
model
where
you
put
them
as
a
response.
You
can
see
both
the
batch
signals
or
the
batch
variances
and
the
interaction
variances
all
are
inside
the
residual
limits.
Also
here
we
can
justify
the
assumption
that
these
are
just
random
factors.
This
is
the
sanity
check
of
the
model.
Now
to
make
models
for
batch
A
and
batch
C,
we
need
to
scale
them
differently.
That's
what
I
have
done
in
these
columns
over
here.
If
I
unhide
my
columns
here,
you
can
see
I
made
an
AC
scale
weight
where
I
simply
just
multiplied
my
weight
factor
with
the
scaling.
So
I
get
the
variation
I
have
seen
for
batches
A
and
C,
and
the
same
thing
for
my batch
B.
I
can
scale
the
weight
with
the
variance
I
would
like
to
have
as
a
residual
variance,
which
is
just
coming
from
my
block
variance
modeling.
That's
also
pretty
straightforward
to
do.
N
ow
I
can
actually
have
my
two
models
from
where
I
can
predict.
One
where
I
scale
with
my
AC
scale
weight
and
one
where
I
scale
with
my
B
scale
weight.
Thereby
I
can
see,
if
I
just
run
one
of
them,
one
scale
with
the
AC
scale
weight,
then
you
can
see
that
you
have
this
model,
you
have
these
residuals,
you
have
these
balance
components.
What
is
very
easy
in
JMP
is
just
to
go
up
here
and
you
can
go
and
you
can
save
your
columns.
F
rom
up
here,
you
can
actually
save
the
prediction
formula
that
would
be
your
center
line.
But
you
can
also
save
your
prediction
interval
that
here
are
called
individual
confidence
interval.
I
have
done
that
both
for
this
model
and
also
for
the
other
model.
I
also
want
the
other
model
with
the
different
scaling
for
batch
B.
T
hen
I
save
these
columns
as
individual
confidence
intervals.
Then
they
are
just
saved
down
here
as
these
prediction
formulas.
Then
I
can
make
a
combined
one
by
just
combining
these
depending
on
which
batch
group
things
are
coming
from.
Now
I
have
limits
for
both
batch
A,
B
and
C,
which
I
can
plot
on
the
same
graph.
This
is
straight
over
to
do.
Until
now,
I
just
used
what's i
nside
JMP.
But
to
get
the
control
limits
and
to
get
the
tolerance
limit,
I
need
to
calculate
them
myself,
because in
JMP,
you
don't
have
tolerance
limits,
you
don't
have
control
limits
in
Fit Model.
Of
course,
you
have
control
limits
when
you
do
control
chart,
but
that's
also
only
for
one
mean
and
one
standard
deviation.
Of
course,
you
also
have
tolerance
limits
in
analyzed
distribution.
But
again,
that's
only
for
one
mean
and
one
standard
deviation.
Here
we
have
to
calculate
them
ourselves.
The
first
thing
we
then
do
is
just
get
the
variance
components.
This
is
what
we
have
here.
W
e
just
saved
the
variance
components
from
the
two
models,
one
where
we
scaled
to
batch
A
and
C,
and
one
where
we
scale
to
batch
B.
Because
when
we
have
the
variance
components,
it's
pretty
straightforward
to
calculate
the
control
limit
because
it's
basically
just
the
prediction
formula
minus
a
normal
quantile,
and
then
the
square
root
of
the
total
variance,
and
then
either
variance
A
and
C
or
B,
depending
on
what we're
looking
at.
I
t's
actually
pretty
straightforward
to
calculate
the
control
limits
that
way.
The
same
we
can
do
for
the
tolerance
limits.
If
you
prefer
tolerance
limits
instead
of
prediction
limits,
many
companies
do.
Again,
unfortunately,
they
are
not
in
JMP
model,
but
we
can
calculate
them.
There
we
simply
just
put
in
the
classical
formula
for
a
tolerance
limit.
Where
one
sided,
we
put
in
the
T
quantile,
we
put
in
the
degrees
of
freedom,
we
put
in
the
normal
quantile,
and
as
before,
of
the
proportion,
and
then
we
multiply
by
the
total
balances
from
the
variance
component.
The
only
little
bit
tricky
thing
here
is
that
we
need
the
degrees
of
freedom
under
total
variance.
JMP
is
not
really
giving
you
that.
It's
only
giving
you
degrees
of
freedom
in
systematic
models.
But
we
can
see
based
on
the
width
of
the
prediction
interval,
that's
what
we
have
here.
Then
we
can
actually
calculate
based
on
the
width
of
the
prediction
intervals,
how
many
degrees
of
freedom
do
we
have
there,
and
then
we
just
apply
the
same
degrees
of
freedom
on
the
tolerance
limits
and
then
we
are
ready
to
go
and
we
can
save
all
of
these
limits
down
here.
If
we
want,
we
can
plot
them.
This
is
what
we
have
done
here.
Here
you
can
see
actually
the
prediction
limits
and
the
tolerance
limits
and
the
control
limits.
The
control
limits
are
the
dotted
line,
the
prediction
limits
are
the
full
line,
and
the
rough
dotted
line
are
the
tolerance
limits.
You
see
that
for
the
15
different
locations
and
for
the
three
batches
A,
B
and
C and
of
course,
limits
are
wider
for
batch
B
because
it
has
more
within
location
variances.
You
can
actually
see
that
even
the
control
limits
at
the
dotted
line
are
outside
for
location
1
and
2
and
actually
also
four.
Even
though
this
process,
this
is
a
good
example
of
a
validation,
I
would
say,
well,
if
batch
B
is
representative,
you
have
actually
failed
the
validation.
If
you
look
at
A
and
C,
then
all
the
dotted
lines
are
inside,
meaning
that
we
have
passed
stage
two
because
the
estimate
value
is
fine.
But
on
a
few
of
them,
you
can
see
that
the
prediction
limits
are
just
outside.
But
when
you
have
this
big
gap
between
control
limits
and
prediction
limit,
just
one
batch
more
would
probably
make
them
inside.
This
is
how
it
works.
Some
companies
they
do
not
think
it's
enough
to
look
at
if
the
limits
are
inside
specification.
They
would
actually
like
to
get
a
Ppk
capability
index
on
them.
We
also
put
that
one
into
our
method
here.
If
you
look
at
the
formulas
here,
you
can
see
where
we
have
the
prediction
limits,
that's
also
put
in
a
Ppk
formula.
Basically
what
we
do,
we
just
take
the
prediction
formulas,
this
into
the
spec
limit,
and
and
divided
by
the
half
width
of
the
prediction
limits.
This
is
this
Ppk
that
corresponds
to
the
prediction
limits.
Because
they
are
with
confidence,
this
is
actually
Ppk
with
confidence.
You
can
do
exactly
the
same
with
your
control
limits.
Just
take
the
classical
Ppk
formula
and
just
put
in
control
limits
instead
of
prediction
limits.
B
ecause
control
limits
are
just
estimated
values,
this
is
Ppk
without
confidence.
Of
course,
you
can
do
exactly
the
same
thing
also
for
the
tolerance
limits,
which
we
have
down
here,
a
Ppk
based
on
the
tolerance
limits.
You
can
also
calculate
all
of
these.
If
you
want,
of
course,
you
can
plot
them
on
a
graph
as
you
see
here.
Up
here,
we
have
estimated
Ppk
without
confidence,
that's
the
red
one.
T
hen
we
have
the
blue
one
that
is
based
on
tolerance
limits
and
the
green
one
that
is
based
on
prediction
limits.
Up
here,
it's
for
the
good
group
A
and
C
with
indication
variation.
Down
here
how
it
is
with batch
B, is
a
bigger
one.
You
can
actually
see,
as
we
saw
with
the
limits
for
batch
B
variation,
even
the
estimated
value
has
a
Ppk
below
one
for
one,
two,
and
four
positions.
While
all
the
estimated
value
is
nicely
about
one
for
A
and
C. A
lso
the
Ppk
based
on
prediction,
the
green
one,
most
of
them
are
about
one,
and
they're
very
close
to
one
on
location
one
and
two.
This
would
just
require
one
additional
batch
probably,
and
then
they
would
be
inside.
However,
as
you'll
probably
see
it's
a
bit
tedious
to
do
all
these
calculations
by
hand.
This
is
why
we
have
made
a
script
for
it,
which
Louis
is
now
going
to
demonstrate.
I
should
just
Louis
take
over
now
and
shows
how
the
script
works.
I'm
sure
he
will
also
make
a
few
more
examples.
Yes,
Pierre,
thank
you
very
much.
I
will
just
try
to
share
my
screen
here
then.
Yes.
First
thing
first,
as
Pierre
has
shown
us,
we
face
a
lot
of
issues
during
the
special
validation.
Luckily
for
us,
JMP
has
a
lot
of
straight
up
box
functionalities
to
help
us
deal
with
these
issues.
We
really
like
to
use
the
functionalities
coming
out
of
the
box
with
JMP
because
this
really
lessens
our
validation
burden
when
going
forward.
Some
of
the
things
JMP
has
solved
for
us
is,
for
example,
the
issue
that
we
are
working
with
complex
processes
and
these
cannot
be
described
by
just
one
mean
and
one
standard
deviation.
Here
to
deal
with
this,
we're
leveraging
the
Fit Model
platform
in
JMP
where
we
can
have
multiple
systematic
factors
to
handle
the
many
means,
and
we
can
also
have
multiple
random
factors
to
deal
with
many
variance
components.
Then
there's
the
issue
with
data
often
needing
some
data
transformation.
Again,
JMP
has
the
Box-Cox
transformation
window
we
use
to
deal
with
this
directly
in
the
Fit Model
platform.
Then
we
usually
see
a
lot
of
outliers
we
have
to
handle
in
bigger
data
sets.
H
ere
we
again
leverage
the
Fit Model
platform,
more
specifically
the
student
sized
residuals
plot
where
we
exclude
outliers
based
on
the
Bonferroni
limits.
Here
it's
just
important
to
note
that
at
least
our
concept
is
that
we
like
to
exclude
the
actual
model,
but
we
always
report
them
separately.
It
has
to
be
visual
that
these
are
not
included
in
the
analysis.
Then
there
is
the
lack
of
homogeneity
of
variance,
which
Pierre
also
showed
how
to
deal
with
using
the
log
variance
modeling
to
get
out
the
variation
and
then
use
this
variance
to
compute
a
weight
formula.
Then
again,
there
is
the
prediction
intervals
which
we
use
quite
a
lot
here.
Here
as
well
JMP
is
ready
to
go
with
individual
confidence
intervals
directly
in
the
Fit Model
platform.
But
then
as
Pierre
also
mentioned,
we
felt
like
there
were
some
parts
where
JMP
could
not
bring
us
all
the
way
home.
This
includes,
for
example,
controlled
intervals
in
JMP,
which
can
only
handle
the
one
mean
and
one
variance
component.
How
we
overcome
this
is
that
we
calculate
controlled
intervals
from
the
Fit Model
using
the
prediction
formula
and
standard
error
on
prediction
formula
and
residuals
to
calculate
the
total
variance,
and
from
here,
the
control
intervals.
Then
there
is
the
fact
that
tolerance
intervals
is
preferred
by
many
of
our
customers
because
this
allows
you
to
separately
set
the
confidence
and
coverage.
However,
we
only
found
that
JMP
can
do
tolerance
intervals
in
the
distribution
platform,
and
it
can
only
handle
one
mean
and
one
variance
component
again.
Of
course,
in
the
script
built-
ins
that
it
can
calculate
the
tolerance
intervals
similar
to
the
control
intervals.
W
hen
we
do
the
calculation
of
the
tolerance
intervals,
we
use
the
same
degrees
of
freedom
as
used
when
JMP
calculates
its
prediction
intervals.
Then
maybe
the
primary
reason
for
doing
this
script,
or
the
initial
reason
at
least,
is
that
calculation
and
visualization
of
all
these
intervals
is
very
time
consuming
and
very
prone
to
human
errors.
Actually,
we
decided
to
make
this
script
to
automate
calculations
and
visualizations.
Then
lastly,
many
of
our
customers
require
capability
and
dialysis
such
as
Ppk.
We
have,
of
course,
also
included
this
in
our
script
such
that
it
calculates
Ppk
corresponding
to
both
prediction
limits,
what
we
call
Ppk
with
confidence,
and
also
control
limits,
what
we
call
Ppk
without
confidence.
To
talk
about
the
script
a
little
bit,
the
script
we
have
designed
here
consists
of
two
parts.
We
primarily
have
the
JMP
script
you
see
here
in
the
middle,
and
then
we
also
have
a
template
document,
as
you
see
up
here
in
the
upper
left
corner.
When
making
this
script,
there
were
a
few
things
we
really
wanted
to
achieve.
We
wanted
to
make
it
really
transparent,
meaning
that
our
customers
could
go
in
and
look
under
the
hood,
so
to
say,
and
follow
the
calculations
of
these
different
limits
and
potentially
even
edit
them
themselves,
which
takes
us
to
the
next
thing.
We
wanted
this
to
be,
at
least
to
some
extent,
customizable
by
our
customers.
If
they
don't
like
the
way
we
calculate
Ppk,
we
would
like
to
give
them
the
option
to
go
in
and
edit
the
equations.
The
template
document
in
itself
is
actually, so to say,
an
empty
data
table.
It
has
a
lot
of
defined
columns
with
column
formulas,
but
it
has
no
rows
in
there
yet.
What
our
script
does,
essentially,
is
that
it
works
from
this
template
document
as
an
input.
It
also
takes
the
original
data
file
on
which
you
base
your
analysis
as
an
input,
and
then
it
takes
the
model
window.
Here
it
is
to
be
said
that
the
model
window
is
assumed
to
be
a
complete
model.
So
you
have
done
all
the
all
your
data
manipulations,
ensured
that
this
is
a
proper
response
you
get
from
the
model. A
lso
you
have
made
these
sanity
checks
and
justified
the
assumptions
behind
the
model
ensuring
it's
valid.
What
the
script
does
itself
is
that
it
takes
the
template
document,
it
takes
the
original
data
file,
and
then
it
copies
the
data
from
the
original
data
file
to
the
template
document.
We've
done
this
such
that
we
don't
interfere
with
our
customer's
original
data
either.
It
then
also
takes
all
the
relevant
parameters
from
the
model
window
and
insert
it
into
this
template
document.
F
rom
there
on,
more
or
less,
this
template
document
will
fill
out
itself
through
the
already
defined
formulas
in
the
columns.
This
template
document
will
then
in
the
end
be
converted
to
an
output
table
where
the
first
many
columns
would
look
very
much
like
the
original
data
file,
and
then
you
will
have
the
remaining
calculation
and
input
values
as
additional
columns
further
on.
I
think
we
should
go
to
an
example
and
see
how
this
looks
in
real
life.
I
will
just
try
to
stop
sharing
this
screen
and
share
this
one.
What
we
have
here
is
in
fact,
the
same
ESBIC
case
as
[inaudible 00:25:20]
here.
What
you
would
do
is
you
would
have
your
data,
you
will
run
a
model.
In
this
case,
I
run
the
model
where
peer
scale
it
to
correspond
to
the
variation
in
batch
A
and
C,
the
batches
with
the
lower
variation.
What
you
simply
do
is
that
you
have
your
model,
ensure
it's
done
in
sense
of
sanity
checks
and
so
forth.
Then
we
run
this
script
on
this
model,
which
is
here.
We
have
the
model
and
we
run
the
script.
We'll
see
a
few
things
pop
up
here.
First,
we
have
the template
document,
output
document
which
has
now
been
populated
with
a
lot
of
different
rows,
some
table
variables,
and
so
forth.
We
have
our
Ppk
graph
here
visualizing
the
Ppk
based
on
different
limits.
The
blue
being
based
on
control
limits,
the
red
being
based
on
prediction
limits,
and
the
green
one
being
based
on
tolerance
limits.
Then
lastly,
we
have
our
graph
here
plotting
all
the
data,
specification
lines,
and
as
well
our
prediction
limit,
control
limits,
and
tolerance
limits
here.
We
get
the
image
that
Pierre
also
showed
earlier.
What
we
see
here
is,
in
fact,
that
if
we
look
at
our
data
and
assume
all
batches
varies
as
much
as
batch
A
and
C,
we
are
actually
in
an
okay
position
here.
At
least
we
would
have
passed
our
stage
two
because
control
limits
are
for
all
batches,
all
locations
within
specification.
We
do,
however,
have
a
small
problem
out
here
on
the
location
1
and
2,
where
we
see
at
least
the
prediction
of
tolerance
limits
extending
a
bit
over
the
upper
specification
limit.
However,
we
believe
this
could
be
fixed
by
just
adding
a
few
more
batches
to
know
the
variation
better.
But
what
we
also
see
here
is,
of
course,
we
have
an
observation
outside
our
limits,
and
this
is
because
what
we're
looking
at
here
is
in
fact
batch
B,
which
as
we
remember,
had
a
higher
variance.
I
would
just
try
to
run
this
model
here,
scale
according
to
batch
B,
so
you
can
see
the
difference.
Now
what
we
see
here
is
that
all
our
limits
has
moved
out
from
the
mean.
Now
they're
actually
going
beyond
specification,
not
only
on
the
first
two
locations,
but
on
multiple
locations
here,
which
would,
in
essence,
for
us,
especially
because
you
have
the
control
them
as
outside
specification,
mean
that
you
have
failed
your
validation
here.
Maybe
not
the
best
example
for
this
data
set
to
showcase
a
good
validation.
Now
I
thought
I
would
just
try
to
go
through
how
we
envisioned
the
entire
process
relating
more
to
the
flowchart,
Pierre,
showed
you
in
the
beginning
of
this
presentation.
I
have
brought
a
customer
example
from
an
actual
PPQ
validation
run.
They
ended
up
producing
a
total
of
six
batches
here.
What
we
have
done
here
is
that
we
have
made
a
model.
I
will
just
deploy
local
data
filter
to
simulate
where
it
started,
having
only
these
three
batches.
Then
what
we
have
here
is
just,
so
to
say,
our
finished
model.
What
we
do
now,
after
having
produced
the
first
batches,
analyzed
the
data,
we
just
run
the
script.
We
get
a
result
looking
like
this.
The
scenario
we
see
here
is
that
both
our
control
limits
and
prediction
limits,
the
prediction
limits
up
here,
tolerance
limits
here,
are
both
very
much
outside
the
specification.
However,
lucky
enough
for
us,
our
control
limits
are
actually
inside
specification.
This
means
in
the
sense
of
the
flowchart
that
we
now
have
passed
stage
2
and
we
can
actually
start
from
now
on
to
when
we
do
our
continued
validation,
getting
our
control
prediction
limit
and
tolerance
limits
inside
specification,
that
we
actually
can
start
sending
batches
to
market
simultaneously.
But
bear
in
mind
here,
we
still
have
a
pretty
high
QC
effort
in
the
end.
The
reason
we
believe
that,
because
many
customers
also
ask,
how
can
we
do
this
if
we
are
so
uncertain
about
the
batches
we
produce?
But
if
you
want
to
know
the
quality
of
your
already
produced
batches,
we
actually
have
to
look
at
this
in
a
different
way.
Here
in
this
model,
we
have
batch
as
a
random
effect
because
as
we
consider
it
as
a
part
of
a
larger
population.
But
if
we
want
to
look
at
the
batch
individually,
we
actually
have
to
put
it
as
a
systematic
effect
in
here.
If
I
run
the
script
on
this
one,
we
will
see
the
limits
narrowing
significantly
in
here.
I
forgot
to
deploy
data
filter,
but
I
think
this
makes
sense
anyway.
What
we
see
here
is
that
the
limits
now
are
much
closer
to
the
center
lines
of
each
batch
here,
and
they
are
way
within
our
specification
because
we
are
now
looking
at
the
batches
we
have
already
produced
and
not
trying
to
predict
the
performance
of
future
batches.
Which
is
also
why
we
believe
that
this
is
an
acceptable
point
to
go
to
market
while
still
having
the
high
QC
effort
at
the
end.
If
I
just
continue
with
our
example,
what
we
saw
in
this
example
was
we
have
passed
stage
two
because
control
limits
are inside
specification,
but
prediction
limits
are outside.
The
next
step
here
would
be
to
simply
produce
another
batch,
create
your
model,
it's
done
here,
and
then
run
the
script
again.
What
we
see
now
here
compared
to
the
first
view
is
that
our
prediction
and
tolerance
limits
are
moving
towards
spec
and
also
toward
our
control
limits,
which
is
the
behavior
we
will
assume
as
we
start
to
get
the
between
batch v
ariants or s
tarting
to
know
the
between
batch
variants
better
and
better.
However,
still,
prediction
limits
are
not
inside
specification,
so
the
routine
would
simply
be
to
do
everything
again.
Now
because
I
know
how
this
will
end,
I
will
simply
include
the
two
last batch
and
we
will
on
the
model
again.
Finally,
after
including
the
sixth
batch,
the
results
we
get
here
are
that
all
our
limits
are
now
within
our
specification,
which
means
we
have
passed
both
stage
2
after
3
batches,
and
now
we
have
also
passed
stage
3
A.
Which
means
we
now
go
into
stage
3B,
where
we
will
start
to
put
our
attention
on
reducing
the
high
amount
of
QC
effort
in
the
end
and
replacing
this
with
continued
process
verification
and
CTV.
Just
to
sum
up,
I
will
go
back
to
the
presentation.
The
purpose
of
a
validation
is,
at
least
in
our
opinion,
to
predict
the
future
of
batches
will
be
okay.
We
firmly
believe
that
if
you
can
describe
your
validation
set
with
a
model,
you
can
also
predict
the
future
with
confidence.
This
can
be
done
from
either
prediction
intervals
or
tolerance
intervals.
We
believe
that
JMP
has
unique
possibilities
to
model
your
data
and
also
to
justify
assumptions
behind
your
model. I
t
offers
the
opportunity
to
look
at
variance
heterogeneity.
If
we
do
not
have
this,
we
have
the
option
to
do
a
log
variance
model
to
find
a
weight
factor
as
Pierre
showed
you,
and
include
that
in
our
normal
regression
models.
Then
we
can
look
at
whether
or
not
the
residuals
are
normally
distributed.
If
they
are
not,
we
can
do
a
Box-Cox
transformation,
also
directly
from
the
Fit Model
platform.
Then
we
have
the
problem
of
outliers,
which
we
can
handle
through
the
student
task
residual
plot,
excluding
observations
outside
the
95
%
Bonferroni limits.
Again,
a
window
directly
accessible
through
the
Fit Model
platform.
Then
lastly,
there
is
also
that
we
request
our
random
factors
to
be
normally
distributed,
at
least
effect
fronted.
This
we
did
by
the
blue
test,
and
here
we
can
also
exclude
outlier
level.
Then
to
make
the
process
easier
for
us,
our
customers,
and
also
adding
a
bit
on
top
of
this
to
actually
achieve
the
functionalities
we
desire,
we
have
decided
to
make
a
script
that
can
automate
visualization
of
prediction
intervals.
It
calculates
and
visualizes
tolerance
intervals
using
the
same
number
of
degrees
of
freedom
as
when
JMP
calculates
prediction
interval.
It
also
calculates
control
limits
for
more
complex
processes
where
you
have
many
means
and
many
variance
components.
Lastly,
it
calculates
also
capability
values
for
where
these
might
be
needed.
