Welcome
to
the
talk
E
xplainable
AI:
Unboxing
the
Black
box.
Let's
introduce
ourselves
and
let's
start
with
Laura.
Hello,
I'm
Laura
Lancaster,
and
I'm
a S
tatistical Developer
in
JMP
and
I'm
located
in
the
Cary
office.
Thanks.
What
about
you,
Russ?
Hey, everyone. Russ
Wolfinger.
I'm the
Director
of
Life
Sciences
R&D
in
the
JMP
Group
and
a Research
fellow
as
well.
Looking
forward
to
the
talk
today.
And
Pete?
My
name's
Peter
Hersch.
I'm
part
of
the
Global
Technical
Enablement T
eam
and
I'm
located
in
Denver,
Colorado.
Great
and
my
name
is
Florian
Vogt.
I'm
Systems
Engineer
for
the
Chemical
Team
in
Europe
and
I'm
located
in
beautiful
Heidelberg,
Germany.
Welcome
to
the
talk.
AI
is
a
hot
topic
at
the
moment
and
a
lot
of
people
want
to
do
it.
But
what
does
that
mean
for
the
industries?
Does
it
mean
that
scientists
and
engineers
need
to
become
coders,
for
processes
in
the
future
run
by
data
scientists?
A
recent
publication
called
Industrial
Data
Science -
a
review
of
Machine
Learning
Applications
for
Chemical
and
Process
Industries
explains
industrial
data
science
fundamentals,
reviews
industrial
applications
using
state
of
the
art
machine
learning
techniques
and
it
points
out
some
important
aspects
of
industrial
AI.
These
are
the
accessibility
of
AI,
the
understandability
of
AI
and
the
consumability of
AI,
and
in
particular
the
output.
We'll
show
you
some
of
the
features
that
we
think
are
contributing
to
this
topic
very
well
in
JMP.
Before
we
start
into
the
program
of
today,
let's
briefly
review
what
AI
encompasses
and
what
our
focus
today
is
located
on.
I've
picked
a
source
that
actually
separates
it
into
four
groups.
Those
groups
are:
first,
supporting
AI
also
called
Reactive
Machines
and
this
aims
at
decision
support.
The
second
group
is
called
Augmenting
AI
or
Limited
Theory
and
that
focuses
on
process
optimization
and
the
third
group
is
Automating
AI,
or
Theory
of
Mind,
which,
as
the
name
suggests,
aims
on
automation,
and
the
fourth
is
called
autonomous
AI
or
self
aware
AI,
which
encompasses
autonomous
optimizations.
Today's
focus
is
really
on
the
first
and
the
second
topic.
We
had
a
brief
discussion
before.
Russ,
what
are
your
thoughts
on
these
also
with
respect
to
what
JMP
can
cover?
Well,
certainly
the
term
AI
gets
thrown
around
a
lot.
It's
used
in
many
kind
of
different
nuanced
meetings.
I
tend
to
prefer
meetings
that
are
definitely
more
tangible
and
usable
and
more
focused.
Like
we're
going
to
zoom
in
on
today
with
some
specific
examples.
The
terminology
can
get
a
little
confusing
though.
I
guess
I
just
tend
to
kind
of
keep
it
fairly
broad,
open
mind
whenever
anyone
uses
the
term
AI
and
try
to
infer
its
meaning
from
the
context.
Right.
That's in
terms
of
introduction,
now
we'll
get
a
little
bit
more
into
the
details
and
specifically
into
why
it
is
important
to
actually
understand
your
AI
models.
Over
to
you,
Pete.
Perfect.
Thanks,
Florian.
I
think
what
Russ
was
hitting
on
there
and
Florian's
introduction
is,
we
oftentimes
don't
know
what
an
AI
model
is
telling
us
and
what's
under
the
hood.
When
we're
thinking
about
how
well
a
model
performs,
we
think
about
how
well
that
fits
the
data.
If
we
look
here,
we're
looking
at
a
neural
network
diagram
and
as
you
can
see,
these
can
get
pretty
complex.
These AI
models
are
becoming
more
and
more
prevalent
and
relied
upon
for
decision
making.
Really,
understanding
why
an
AI
model
is
making
a
certain
decision,
what
criteria
it's
basing
that
decision on,
is imperative
to
taking
full
advantage
of
these
models.
When
a
model
changes
or
updates,
especially
with
that
autonomous
AI
or
the
automating
AI,
we
need
to
understand
why.
We
need
to
confirm
that
this
model
is
maybe
not
extrapolating
or
basing
it
on
a
few
points
outside
of
our
normal
operating
range.
Hold
on.
Let
me
steal
the
screen
here
from
Florian,
and
I'm
going
to
go
ahead
and
walk
through
a
case
study
here.
All
right,
so
this
case
study
is
based
on
directional
drilling
from
wells
near
Fort
Worth,
Texas.
The
idea
with
this
type
of
drilling
is
unlike
conventional
wells,
where
you
would
just
go
vertically,
you
go
down
a
certain
depth,
and
then
you
start
going
horizontal.
The
idea
is
these
are
much
more
efficient
than
the
traditional
wells.
You
have
these
areas
of
trapped
oil
and
gas
that
you
can
get
at
with
some
special
completion
parameters.
We're
looking
at
the
data
here
from
these
wells,
and
we're
trying
to
figure
out
what
are
the
most
important
factors,
including
the
geology,
the
location,
and
the
completion
factors,
and
can
we
optimize
these
factors
to
increase
or
optimize
our
well
production?
To
give
you
an
idea,
here's
a
map
of
that
basin,
so
like
I
mentioned,
this
is
Fort
Worth,
Texas.
You
can
see
we
have
wells
all
around
this.
We
have
certain
areas
where
our
yearly
production
is
higher,
others
where
it's
lower.
We
wanted
to
ask
a
few
questions
looking
at
this
data.
What
factors
have
the
biggest
influence
on
production?
If
we
know
certain
levels
for
a
new
well,
can
we
predict
what
our
production
will
be?
Is
there
a
way
to
alter
our
factors,
maybe
some
of
the
completion
parameters
and
impact
our
production?
We're
going
to
go
ahead
and
answer
some
questions
with
a
model.
But
before
we
get
into
that,
I
wanted
to
ask
Russ,
since
he's
got
lots
of
experience
with
this.
When
you're
starting
to
dig
into
data,
Russ,
what's
the
best
place
to
start
and
why?
Well,
I
guess
maybe
I'm
biased,
but
I
love
JMP
for
this
type
of
application,
Pete,
just
because
it's
so
well
suited
for
quick
exploratory
data
analysis.
You
want
to
get
a
feel
for
what
target
you're
trying
to
predict
and
the
predictors
you're
about
to
use,
looking
at
their
distributions,
checking
for
outliers
or
any
unusual
patterns
in
the
data.
You
may
even
want
to
do
some
quick
pattern
discovery
clustering
or
PCA
type
analysis
just
to
get
a
feel
for
any
structure
that's
in
the
data.
Then
also
be
thinking
carefully
about
what
performance
metric
would
make
the
most
sense
for
the
application
at
hand.
Typically
the
common
one,
obviously
for
continuous
predictors
would be
like
root
means
square
error,
but
there
could
be
cases
where
may be that's
not
quite
appropriate,
especially
if
there's
direct
cost
involved,
sometimes
absolute
error is
more
relevant
for
a
true
profit-
loss
type
decision.
These
are
all
things
that
you
want
to
start
thinking
about
as
well
as
how
you're
going
to
validate
your
model.
I'm
a
big
fan
of
k-fold cross validation.
Where
you
split
your
data
into
distinct
subsets
and
hold
one
out
and
being
very
careful
about
not
allowing
leakage
or
also
careful
about
overfitting.
These
are
all
concerns
that
tend
to
come
top
of
mind
for
me
when
I
start
out
with
a
new
problem.
Perfect.
Thanks, Russ
.
I'm
going
to
walk
through
this
in
JMP
some
of
the
tools
we
can
use
to
start
looking
at
our
problem.
Then
we're
going
to
cover
some
of
the
things
that
help
us
determine
which
of
these
factors
are
having
the
biggest
impact
on
well
production.
I'm
going
to
show
Variable
Importance
and
then
Shapley
Values
and
we'll
have
Laura
talk
to
that
and
how
we
do
that.
But first,
let's
go
ahead
and
look
at
this
data
inside
a
JMP.
Like
we
mentioned
here,
I
have
my
production
from
these
wells.
I
have
some
location
parameters
so
where
it
is
latitude
and
longitude,
I
have
some
geologic
parameters.
This
is
about
the
rock
formation
we're
drilling
through.
Then
I
have
some
completion
parameters
and
this
is
factors
that
we
can
change
while
we're
drilling
as
well,
some
of
these
we can
have
influence
on.
If
we
wanted
to
go
through
and
this
dataset
only
has
5,000
rows.
When
talking
to
Russ
in
starting
to
prep
this
talk,
he
said
just
go
ahead
and
run
some
model
screening
and
see
what
type
of
model
fits
this
data
best.
To
do
that,
we're
going
to
go
ahead
and
go
under
the
Analyze
menu,
go
to
Predictive
Model
and
hit
Model
Screening.
I'm
going
to
put
my
response,
which
is
that
production,
take
all
of
our
factors,
location,
geology
and
completion
parameters,
put
those
into
the
X
and
grab
my
Validation
and
put
it
into
Validation.
Down
here
we
have
all
sorts
of
different
options
on
types
of
models
we
can
run.
We
can
pick
and
choose
which
ones
maybe
make
sense
or
don't
make
sense
for
this
type
of
data.
We
can
pick
out
some
different
modeling
options
for
our
linear
models.
Even,
like
Russ
mentioned
here,
if
we
don't
have
enough
data
to
hold
back
and
do
our
validation.
That
way
we
can
utilize
a
k-fold cross validation in here.
Now
to
save
some
time,
I've
gone
ahead
and
run
this
already
so
you
don't
have
to
watch
JMP
create
all
these
models.
Here's
the
results.
For
this
data,
you
can
see
that
these
tree
based
methods:
Boosted
Tree,
Bootstrap f
orest
and
XGB
oost
all
did
very
well
at
fitting
the
data,
compared
to
some
of
the
other
techniques.
We
could
go
through
and
run
several
of
these,
but
for
this
one,
I'm
going
to
just
pick
the
Boosted
Tree
since
it
had
the
best
RS
quare
and
root
average
square
error
for
this
dataset.
We'll
go
ahead
and
run
that.
After
we've
run
the
screening,
we're
going
to
go
ahead
and
pick
a
model
or
a
couple
of
models
that
fit
well
and
just
run
them.
Al
right,
so
here's
the
overall
fit
in
this
case.
Depending
on
what
type
of
data
you're
looking
at,
maybe
an
RS
quare
of
.5
is
great,
maybe
an
RSquare
of
.5
is
not
so
great.
Just
depending
on
what
type
of
data
you
have,
you
can
judge
if
this
is
a
good
enough
model
or
not.
Now
that
I
have
this,
I
want
to
answer
that
first
question.
Knowing
a
few
parameters
going
in,
what
can
I
expect
my
production
level
to
be?
An
easy
way
to
do
that
inside
a
JMP
with
any
type
model
is
with
this
profiler.
Okay,
so
we
have
the
profiler
here,
we
have
all
of
the
factors
that
were
included
in
the
model,
and
we
have
what
our
expected
12
month
production
to
be.
Here
I
can
adjust
if
I
know
my
certain
location.
I
know
the
latitude
and
longitude
going in
maybe
I
know
some
of
these
geologic
parameters.
I
can
maybe
adjust
several
of
these
factors
and
figure
out
of
the
completion
parameters
and
figure
out
a
way
to
optimize
this.
But
I
think
here
with
a
lot
of
factors,
this
can
be
complex.
Let's
talk
about
the
second
question,
where
we
were
wondering
which
one
of
these
factors
was
having
the
biggest
influence.
You
can
see
based
on
which
of
these
lines
are
flatter
or
have
more
shape
to
them,
what
is
the
biggest
influence.
But
let's
JMP
do
that
for
us.
Under
Assess
Variable
Importance,
I'm
going
to
just
let
JMP
go
through
and
pick
the
factors
that
are
most
important.
Here
you
can
see
it's
outlined
the
most
important
down
to
ones
that
are
less
important.
I
like
this
feature
colorize
the
profiler.
Now
it's
highlighted
the
most
important
factors
and
gone
down
to
the
least
important
factors.
Again,
I
can
adjust
these
and
see,
oh,
it
looks
like
maybe
adjusting
the
depth
of
the
well,
adding
some
more
[inaudible 00:15:51],
it
might
improve
my
production.
That
is
the
way
we
could
do
this
but
we
have
a
new
way
of
looking
at
the
impact
of
each
one
of
these
factors
on
a
certain
well.
We
can
launch
that
under
the
red
triangle
in
JMP
17
and
Shapley
values.
I
can
set
my
options
or
save
out
the
Shapley
values.
Once
I
do
that,
it
will
create
new
columns
in
my
data
table
that
save
out
the
contributions
from
each
one
of
those
factors.
This
is
where
I'm
going
to
let
Laura
talk
to
some
Shapley
values.
I
just
wanted
going
to
talk
briefly
about
what
Shapley
values
are
and
how
we
use
them.
Shapley
values
are
a
model
agnostic
method
for
explaining
model
predictions
and
they
are
really
helpful
for
Black
box
models
that
are
really
hard
to
interpret
or
explain.
The
method
comes
from
cooperative
game
theory
and
I
don't
have
time
to
talk
about
the
background
or
the
math
behind
the
computations,
but
we
have
a
reference
at
the
bottom
of
the
slide
and
if
you
Google
it,
you
should
be
able
to
find
a
lot
of
references
to
learn
more
if
you're
interested.
What
these
Shapley
values
do,
they
tell
you
how
much
each
input
variable
is
contributing
to
an
individual
prediction
for
a
model.
That's
away
from
the
average
predicted
value
across
the
input
dataset
and
your
input
dataset
is
going
to
come
from
your
training data.
Shapley values
are
additive,
which
makes
them
really
nice
and
easy
to
interpret
and
understand.
Every
prediction
can
be
written
as
a
sum
of
your
Shapley values
plus
that
average
predicted
value,
which
we
refer
to
as
the
Shapley
intercept
in
JMP.
They
can
be
computationally
intensive
to
compute
if
you
have
a
lot
of
input
values,
input variables in
your
model
or
if
you're
trying
to
create
Shapley values
for
a
lot
of
predictions.
We
try
to
give
some
options
for
helping
to
reduce
time
in JMP
.
These
Shapley
values,
as
Peter
mentioned,
were
added
to
the
prediction
profiler
for
quite
a
few
of
the
models
in
JMP
Pro17
and
they're
also
available
in
the
graph
profiler.
They're
available
for
Fit Least S
quares
Nominal Logistic, O
rdinal
Logistics,
Neural,
Gen Reg, P
artition,
Bootstrap
Forest
and
Boosted
Tree.
They're
also
available
if
you
have
the
XB
oost
Add
-In.
Except
in
that
Add-I
n,
they're
available
from
the
model
menu
and
not
from
the
prediction
profiler.
Okay,
next
slide.
In
this
slide
I
want
to
just
look
at
some
of
the
predictions
from
Peter's
model.
This
is
from
a
model
using
five
input
variables.
These
are
stack
bar
charts
of
the
first
three
predictions
coming
from
the
first
three
rows
of
his
data.
On
the
left
you
see
a
stack
bar chart
of
the
Shapley
values
for
the
first
row.
That
first
prediction
is
11.184
production
barrels
in hundreds of thousands.
Each
color
in
that
bar
graph
is
divided
out
by
the
different
input
variables.
Inside
the
bars
are
the
Shapley values.
If
you
add
up
all
of
those
values,
plus
the
Shapley
intercept
that
I
have
in
the
middle
of
the
graph,
you
get
that
prediction
value.
This
is
showing
you
that
first
of
all,
all
of
these
are
making
positive
contributions
to
the
production
and
they
show
you
how
much,
so the
size,
I
can
see
that
longitude
and
proppant
are
contributing
the
most
for
this
particular
prediction.
Then
if
I
look
to
the
right
side
to
the
third
prediction,
which
is
2.916
production
barrels
and
hundreds
of
thousands,
I
can
see
that
two
of
my
input
variables
are
contributing
positively
to
my
production
and
three
of
them
are
having
negative
contributions,
the
bottom
three
here.
You
can
use
graphs
like
this
to
help
visualize
your
Shapley Values
.
That
helps
you
really
understand
these
individual
predictions.
Next
slide.
This
is
just
one
of
many
types
of
graphs
you
can
create.
The
Shapley
values
get
saved
into
your
data
table.
You
can
manipulate
them
and
create
all
kinds
of
graphs
in
Graph
Builder
and
JMP.
This
graph
is
just
a
graph
of
all
the
Shapley
values
from
over
5,000
rows
of
the
data
split
out
by
each
input
variable.
It just
gives
you
an
idea
of
the
contributions
of
those
variables,
both
positive
and
negatively
to
the
predictions.
Now
I'm
going
to
hand
it
back
over
to
Peter.
Great.
Thanks,
Laura.
I
think
now
we'll
go
ahead
and
transition
to
our
second
case
study
that
Florian
is
going
to
do.
Should
I
pass
the
screen
share
back
to
you?
Yeah,
that
would
be
great.
Make the transition.
Thanks
for
this
first
case
study
and
thanks
for
the
contributions.
Really
interesting.
I hope
we
can
bring
some
light
onto
a
different
kind
of
application
with
our
second
case
study.
I
have
given
it
the
subtitle,
Was
it
Torque?
Because
that's
a
question
we'll
have
hopefully
answered
by
the
end
of
the
second
case
study
presentation.
This
second
case
study
is
about
predictive
maintenance
and
the
particular
aspects
of
why
it
is
important
to
understand
your
models
in
this
scenario.
Most
likely
everybody
can
think
that
it's
very
important
to
have
a
sense
for
when
machines
require
maintenance.
Because
if
machines
fail,
then that's
a
lot
of
trouble,
a
lot
of
costs,
when
plants
have
to
shut
down
and
so
on.
It's
really
necessary
to
do
preventative
maintenance
to
keep
systems
running.
A
major
task
in
this
is
to
determine
when
the
maintenance
should
be
performed
and
not
too
early,
not
too
late,
certainly.
Therefore,
it's
a
task
to
find
a
balance
which
limits
failures
and
also
saves
costs
on
maintenance.
There's
a
case
study
that
we're
using
to
highlight
some
functions
features
and
it's
actually
a
synthetic
data
set
which
comes
from
a
published
study.
The
source is
down
there
at
the
bottom.
You
can
find
it.
It
was
published
in
the
AI
for
Industry
event
in
2020.
The
basic
content
of
this
dataset
is
that
it
has
six
different
features
of
process
settings,
which
are
product
or product type
which
denotes
for
different
quality
variants.
Then
we
have
air
temperature,
process
temperature,
rotational
speed,
torque,
and
tool wear.
We
have
one
main
response
and
that
is
whether
the
machine
fails
or
not.
When
we
think
of
questions
that
we
could
answer
with
a
model
or
models
or
generally
data.
There's
several
that
come
to
mind.
Now,
the
most
obvious
question
is
probably
how
we
can
explain
and
interpret
settings,
which
likely
lead
to
machine
failure.
This
is
something
that [inaudible 00:24:38]
to
create
and
compare
multiple
models
and
then
choose
the
one
that's
most
suitable.
Now,
in
this
particular
setting
where
we
want
to
predict
whether
a machine
fails
or
not.
We
also
have
to
account
for
misclassifications
that
is
either
a
false
positive
or
a
false
negative
prediction.
With
JMP's
decision
threshold
graphs
and
the
profit
matrix,
we
can
actually
specify
an
emphasis
or
importance
to
which
outcome
is
less
desirable.
For
example,
it
is
typically
less
desirable
to
actually
have
a
failure
when
the
model
didn't
predict
one
compared
to
the
opposite,
misclassification.
Then
besides
the
binary
classification,
of
course,
you'd
be
also
interested
in
understanding
what
drives
failure
typically.
There
are
certainly
several
ways
to
deal
with
this
question.
I
think
visualization
is
always
a
part
of
it.
But
when
we're
using
models
we
can
consider,
for
example,
this
self
explaining
models
like
decision
trees
or
we
can
use
built-
in
functionality
like
the
prediction
profiler
and
the
variable
importance
feature.
The
last
point
here
when
we
investigate and rate
which
factors
are
most
important
for
the
predictive
outcome,
we
assume
that
there
is
an
underlying
behavior.
The
most
important
factor
is
XYZ,
but
we
do
not
know
which
factor
actually
has
contributed
to
what
extent
to
an
individual
prediction.
A
gain,
Shapley
values
are
a
very
helpful
addition
that
can
allow
us
to
understand
the
contribution
for
each
of
the
factors
in
individual
prediction.
On a
general
level,
now,
let's
take
a
look
into
three
specific
questions
and
how
we
can
answer
those
with
the
software.
The
first
one
is
how
do
we
adjust
predictive
model
with
respect
to
the
high
importance
of
omitting
false
negative
predictions?
This
assumes
a
little
bit
that
we've
already
done
a
first
step
because
we've
already
seen
model
screening
and
how
we
can
get
there.
I'm
starting
one
step
ahead.
Let's
move
into
JMP
to
actually
take
a
look
at
this.
We
see
the
dataset,
we can see it's
fairly
small,
not
too
many
columns.
It
looks
very
simple.
We
only
have
these
few
predictors
and
there's
some
more
columns.
There's
also
a
validation
column
that
I've
added,
but
it's
not
shown
here.
As
for
the
first
question,
let's
assume
we
have
already
done
the
model
screening.
Again,
this
is
accessible
on the
analyzed
predictive
model
screening
where
we
don't
specify
what
we
want
to
predict
and
the
factors
that
we
want
to
investigate.
Again,
I
have
already
prepared
this.
We
have
an
outcome
that
looks
like
this.
It
looks
a
little
bit
different
than
in
the
first
use
case
because
now
we
have
this
binary
outcome
and
so
we
have
some
different
measures
that
we
can
use
to
compare.
But
again,
what's
important
is
that
we
have
an
overview
of
which
of
the
methods
are
performing
better
than
other
ones.
As
we
said,
in
order
to
now
improve
the
model
and
emphasize
on
omitting
these
false
negative
predictions.
Let's
just
pick
the
one
and
see
what
we
can
do
here.
Let's
maybe
even
pick
the
first
three
here,
so
we
can
just
do
that
by
holding
the
control
key.
Another
feature
that
will
help
us
here
is
called
decision
threshold
and
it's
located
in
the
red
triangle decision
threshold.
The
decision
threshold
gives
us
several
contents.
We
have
these
graphs
here,
these
shows
the
actual
data
points.
We
have
this
confusion
matrix
and
we
have
some
additional
graphs
and
matrix,
but
we
will
focus
on
the
upper
part
here.
Let's actually
take
a
look
at
the
test
portion
of
the
set.
When
we
take
a
look
at
this,
we
can
see
that
we
have
different
types
of
outcomes.
The
default
of
this
probabilities
threshold
is
the
middle,
which
would
be
here
at
.5.
We
have
now
several
options
to
see
and
optimize
this
model
and
how
effective
it
is
with
respect
to
the
confusion
matrix.
Confusion
matrix,
we
can
see
the
predicted
value
and
whether
that
actually
was
true
or
not.
If
we
look
at
when
o
failure
is
predicted,
we
can
see
that
here,
with
this
setting,
we
actually
have
quite
a
high
number
of
failures,
even
though
there
were
no
predicted.
Now
we
can
interactively
explore
how
adjusting
this
threshold
actually
affects
the
accuracy
of
the
model
or
the
misclassification
rates.
Or
in
some
cases,
we
can
also
put
an
emphasis
on
what's
really
worse
than
an
other
failure.
We
can
do
this
with
the
so
called
profit
matrix.
If
we
go
here,
we
can
set
a
value
on
which
of
the
misclassifications
is
actually
worse
than
the
other
one.
In
this
case,
we
really
do
not
want
to
have
a
prediction
of
no
failure
when
there
actually
is
a
failure
happening.
We
would
put
something
like
20
times.
More
importantly,
we
do
not
get
this
misclassification
and
we
set
it
and
hit
okay,
and
then
it
will
automatically
update
the
graph
and
then
we
can
see
that
the
values
for
the
misclassification
have
dropped
now
in
each
of
the
models
and
we
can
use
this
as
an
additional
tool
to
select
a
model
that's
maybe
most
appropriate.
That's
for
the
first
question
of
how
we
can
adjust
a
predictive
model
with
respect
to
the
higher
importance
of
omitting
false
negative
predictions.
Now,
another
question
here
is
also
when
we
think
of
maintenance
and
where
we
put
our
efforts
into
maintenance,
is
how
can
we
identify
and
communicate
the
overall
importance
of
predictors?
What
factors
are
driving
the
system,
the
failures?
Let's
go
back
to
the
data
table
to
say
that
first,
I
personally
like
visual
and
simplistic
ones.
One
of
them
that
I
like
to
use
is
stuff
like
the
parallel
plot.
Because
it's
really
a
nice
overview
summarizing
where
the
failures
group
and which
parameters
settings
and
so
on.
On
the
modeling
and
machine
learning
side,
there's
a
few
other
options
that
we
can
actually
use.
One
that
I
like
because
it's
very
crisp
and
clear,
is
the
predictor
screening.
Predictor
screening
gives
us
very
compact
output
about
what
is
important
and
it's
very
easy
to
do
and
it's
under
analyzed
screening,
predictor
screening.
A
ll
we
need
to
do
is
say
what
we
want
to
understand
and
then
specify
the
parameters
that
we
want
to
use
for
this.
Click
okay,
and
then
it
recalculates
and
we
have
this
output.
For
me,
it's
a
really
nice
thing
because as I said,
crisp
and
clear
and
consuming.
But
we've
talked
about
this
before
and
Russ,
when
we're
working
with
models
particularly,
do
you
have
any
other
suggestion
or
do
you
have
anything
to
add
to
my
approach
to
understanding
the
factors
of
which
predictors
are
important.
Yes,
it
is
a
good
thing
to
try.
As
I
mentioned
earlier,
you
got
to
be
really
careful
about
overfitting.
I tend to work
with
a
lot
of
these
wide
problems,
say
from
Genomics
and
other
applications,
where
you
might
even
have
many
more
predictors
than
you
have
observations.
In
such
a
case,
if
you
were
to
run
predictor
screening,
say
maybe
pick
the
top
10
best
and
then
go
turn
right
around
and
fit
a
new
model
with
those
10
only,
you've
actually
just
set
yourself
up
for
overfitting
if
you
did
the
predictor
screening
on
the
entire
data
set.
That's
the
scenario
I'm
concerned
about.
It's
an
easy
trap
to
fall
into,
because
you
think
you're
just
filtering
things
down,
but
you've
got
to
reuse
the
same
data
twice.
The
danger
would
be
if
you
were
to
plug then
apply
that
model
to
some
new
data,
it
likely
won't
do
nearly
as
well.
If
you're
in
the
game
where
you
want
to
reduce
predictors,
I
tend
to
like
to
prefer
to
do
it
within
each
fold
of
a
K-fold.
The
drawback
of
that
is
you'll
get
a
different
set
every
time,
but
you
can't
aggregate
those
things.
If
you
got
a
certain
predictor
that's just showing
up
consistently
across
folds.
It's
very
good
evidence
of that.
That's
a
very
important
one.
I
expect
that's
what
would
happen
in
this
case
with,
say,
torque.
Even
if
you
were
to
do
this
exercise,
say
10
times
with
10
different
folds,
you'd
likely
get
a
pretty
similar
ranking,
but
it's
more
of
a
subtlety
but
again,
a
danger
that
you
have
to
watch
out
for.
Job
can
make
it
a
little
bit
easier
just
because
things
are
so
quick
and
clean,
like
you
mentioned,
that
you
might
fall
into
that
trap
if
you're
not
careful.
Yeah,
that's
very
valuable
addition
to
this
approach.
Just accompanying
to
this
additional
information,
there's
also
the
other
option
that
we
have,
particularly
when
we
have
already
gone
through
the
process
of
creating
a
model
where
we
can
then
actually
again,
use
the
prediction
profiler
and
the
variable
importance.
It's
another
way
where
we
can
assess
which
of
the
variables
have
the
higher
importance.
Russ,
do
you
want
to
say
word
on
that
also
in
contrast
maybe
to
the
predictor
screening?
Yeah.
Honestly,
Vogt,
I
like
the
importance
were it a
little
better.
Just
dive
right
into
the
modeling.
Again,
I
would
prefer
with
K-fold.
Then
you
can
just
use
the
variable
importance
measures,
which
are
often
really
informative
directly.
They're
very
similar.
In
fact,
predictor
screening,
I
believe,
it's
just
calling
bootstrap
forest
in
the
background
and
collecting
the
most
important
variables.
It's
basically
the
same
thing.
Then
following
up
with
the
profiler,
which
can
be
excellent
for
seeing
exactly
how
certain
variables
are
marginally
affecting
the
response,
and
then
drilling
that
even
further
with
Shapley
to
be
able
to
break
down
individual
predictions
into
their
components.
To
me,
it's
a
very
compelling
and
interesting
way
to
dive
into
a
predictive
model
and
understand
what's
really
going
on
with
it,
kind of unpacking
the
black
box
and
letting
you
see
what's
really
happening.
Yeah,
thanks.
I
think
that's
the
whole
point,
making
it
understandable
and
making
it
consumable
besides,
of
course,
actually
getting
to
the
results,
which
is
understanding
which
factors
are
influencing
the
outcome.
Thanks.
Now,
I
have
one
more
question,
and
you've
already
mentioned
it.
When
we
score
new
data,
in
particular,
what
can
we
do
to
identify
which
predictors
have
actually
influenced
the
model
outcome?
Now,
with
what
we
have
done
so
far,
we
have
gained
a
good
understanding
of
the
system
and
know
Which
of
the
factors
are
the
most
dominant
and
we
can
even
derive
operating
ranges.
If
the
system
changes,
what
if
a
different
factor
actually
drives
a
failure?
Then
as
it
would
be
expected
in
this
case
and we have talked to
Laura
beforehand,
and
Shapley V
alues
again
are
a
great
addition
that
will
help
us
to
interpret.
we've
seen
how
we
can
generate
them,
and
you've
learned
on
which
platforms
they'll
be
available.
Now,
the
output
that
you
get
when
you
save
out Shapley Values is,
for
example,
also
a
graph
that
shows
us
per
actual
per
row.
In
this
case,
we
have
10,000
rows
in
the
data
tab,
so we have 10,000 stack bar charts,
and
then
we
can
already
see
that besides the common pattern,
there's
also
times
when
there's
actually other influencing factors
that
drive
the
decision
of
the
model.
It's
really
a
helpful
tool
to
not
only
raise an individual prediction,
but
also
add
on
to
that
what
Russ just
said,
understanding
of
the
system,
which
factors
contribute.
When
we
move
a
little
bit on
in
this
understandable
or
exploratory path,
we
can
use
these
Shapley Values
in
different
ways.
What
I
personally
liked
was the suggestion
to
actually
plot
the
Shapley
value
by
their
actual parameters setting,
because
that
allows
us
to
identify
areas
of
settings.
For
example,
if
we
take
rotational
speed
here,
we
can
see
that
there's
actually
areas
of
this
parameter
that
tend
to
contribute
a
lot
in
terms
of
the
model
outcome,
but
also
in
terms
of
the
actual
failure.
That
also
helps
us in getting
more understanding
with
respect
to
the
actual
problem
of machine failure and what's causing it,
and
also
with
respect
to
why
the
model predicts something.
Now,
finally,
I
like
to
answer
the
question.
When
we
take
these
graphs
of
Shapley values,
and
we
have
seen
it
before
in
several occasions,
stock
is
certainly
a
dominant
factor.
But
from
all
of
these,
I've
just
picked
a
few
predictions,
and
we
can
see
that
sometimes
it
stocks,
sometimes
it's
not.
With
the
Shapley
values,
we have really a great way
to
interpreting
a
specific
prediction
by
the
model.
All
right,
so
those
were
the
things
we
wanted
to
show.
I
hope
this
gives
some
great
insight
into how
we
can
make
AI
models
more
explainable,
more
understandable,
more
easy
to
digest
and
to
work
with,
because
that's
the
intention
here.
Yeah,
I'd
like
to
summarize
a
little
bit.
Pete,
maybe
you
want
to come
in
and
help
me
here.
I
think
what
we're
hoping
to
show
is
that
as
these
AI
models
become
more
and
more
prevalent
and
are
relied
upon
for
decision
making,
that
understanding,
interpreting,
and
being
able
to
communicate
those
models
is
very
important.
We
hope
that
with
these
Shapley v
alues,
with
the
variable
importance,
and with the profiler,
we've
shown
you
a
couple
of
ways
that
you can share share those results
and
have
them
easily
understandable.
That
was
the
take-
home
there
between that
and
being
able
to
utilize
model
screening, and things like that,
that hopefully,
you
found
a
few
techniques
that
will
make
this
more
understandable
and
less
of
a
black
box.
Yeah,
I
absolutely
agree.
Just
to
summarize,
I
really
like
to
thank Russ and Laura
for
contributing
here
with
your
expertise.
Thanks,
Pete.
It was
a
pleasure.
Thanks,
everybody for listening.
We're
looking
forward
to
having
discussions
and
questions
to
answer.
