Thank
you
all
for
clicking
on
this
talk
and
coming
to
watch
it.
This
is
really
about
bias
in
data.
Every
analyst
that
works
on
an
project
understands
the
importance
of
ensuring
that
the
data
that
they
collect
is
unbiased.
The
steps
are
taken
to
avoid
that
at
the
start
of
the
data
collection,
before
the
projects
even
really
begun,
there
are
numerous
checks
that points
along
the
analysis,
and
then
at
the
end,
any
conclusion
reached
is
taken
in
the
context
of
potential
additional
sources
of
bias.
But this
same
level
of
care
isn't
applied
to
online
searches
on
topics
of
interest.
Search
engines
use
algorithms
that
are
designed
to
deliver
personalized
content
that
is
relevant
for
us
as
individuals.
Now,
this
has
advantages.
It
means
that
return
search
hits
are
more
likely
to
be
relevant
and
interesting,
but
it
also
has
disadvantages.
By
definition,
these
are
not
unbiased.
We
have
an
example.
Yeah, I
heard
they
brought
up
a
great
point
there,
in
science
and
engineering,
we
take
a
great
care
to
make
sure
that
our
samples
are
unbiased.
But
let's
think
of
a
library.
We
walk
into
the
library
and
there's
two
people
interested
in
informing
themselves
on
vaccine
safety.
Let's
say
they
walk
into
a
library
and
ask
a
librarian
for
these
books
on
vaccines,
so the
first
person
receives
three
books.
These
are
actual
books.
Smallp
ox: A
Vaccine
Success
,
Anti-vaxxers: H
ow
to
Challenge
the
Misinformed
Movement,
and
Stuck:
How
Vaccine
Rumors
Start
and
Why
They
Don't
Go
Away.
Now,
let's
say
a
different
person
walks
in
and
receives
three
completely
different
books.
THE COVID
VACCINE: A
nd
silencing
of
our
doctors
and
scientists,
Jabbed: H
ow
the
Vaccine
Industry,
Medical
Establishment,
and
Government
Stick
it
to
You
and
Your
Family,
Anyone Who Tells You Vaccines Are Safe and Effective is Lying.
These
are
actual
book
titles.
Let's
say
that
looking
at
who
you
are,
so
where
you
live,
how
old
you
are,
your
gender,
maybe
even
your
browser
history
determines
which
of
these
sets
of
books
you
get.
This
is
essentially
some
of
the
problem
with
the
bias
as
you
go
in
to
search
for
things
online.
It
may
be
that
before
we
even
start
looking
at
our
browser
search,
we've
already
got
bias
in
there
and
we
want
to
understand
if
that's
the
case
or
not.
That's
what
motivated
this.
You
got
any
thoughts
on
that, H
adley?
Well,
I
think
that
the
thought
that
I'd
like
to
express
right
now
is
that
the
purpose
of
this
presentation
isn't
to
judge
or
to
opine
on
the
advantages
or
disadvantages
of
the
search
algorithms
that
may
or
may
not
be
used.
The
purpose
here
really
is
just
to
take
an
example
of complex/
unstructured
data
and complex
because
it
is
unstructured
and
this
was
search
results.
Then
to
use
some
of
the
exploratory
visual
and
analytic
capabilities
found
in
JMP
Pro
17
to
try
to
understand
what
we
were
seeing
and
to
present
it
in
such
a
way
to
help
you
to
understand.
The
purpose
of
this
presentation
is
to
inspire
you
to
try
these
techniques
for
yourselves
and
others
like
them
on
your
own
data.
Let's
go
through
briefly
the
methodology.
What
Pete
and
I
did
was
we
came
up
with
a
few
search
terms
we
thought
would
lead
to
interesting
results.
You
can
see
those
terms
here.
We
define
some
potential
input
variables
which
may
or
may
not
be
affecting
the
results
of
the
search.
We
know
that
there
are
very
likely
others
as
well
that
we
didn't
include.
This
is
true
with
any
designed
experiment.
We
can't
capture
every
variable,
but
we
took
a
few.
We'll
see
whether
these
are
significant
or
not.
We
developed
a
data
collection
procedure
whereby
we
use
the
MSA
design
in
the
DOE
menu.
This
is
a
convenient
way
to
create
these
tables
that
we
can
then
send
to
JMP
SEs
and
friends
of
SEs.
Now,
right
away
this
isn't
an
unbiased
random
assortment
of
people
we've
asked
to
fill
out
these.
They're
all
people
that
work
for
the
same
company
and
have
the
same
job
title.
As
we
said,
the
purpose
is
really
to
understand
the
techniques
and
methods
that
we
use
to
try
to
understand
the
data,
and
then
to
think
about
how
you
can
apply
it
yourself.
We
explored
the
results
which
we'll
show
you.
Then
finally
we
presented
the
findings
at
the
JMP
Discovery
Summit
America
2022,
which
is
what
you
are
watching
right
now.
Without
further
ado,
let's
jump
into
the
data.
I'll
start
out
by
talking
just
briefly
about
the
MSA
design
that
you
see
here.
What
we've
done
is
we've
added
the
factors
of
interest,
we've
added
the
terms
of
interest
that
we
were
looking
at.
then
the
nice
thing
about
this
is
that
when
we
make
the
table,
what
we
could
always
do
is
press
this
button,
send
these
out
to
everybody
that
needed
to
complete
the
results
for
us,
send
them
back,
catenate them
and
then
we're
ready
to
start
beginning
our
analysis.
But
as
any
analyst
who's
ever
collected
data
and
tried
to
analyze
knows
the
data
very
often
isn't
in
a
format
where
you
can
immediately
start
with
your
analysis,
some
cleaning
needs
to
be
done.
I'll
pass
things
over
to
Pete
to
talk
about
that.
Pete?
Yes,
great
point.
I
think
everybody
has
gone
through
this.
Even
with
a
well-
designed DOE,
you
oftentimes
have
to
make
some
adjustments
to
do
the
analysis.
Hadley
showed
those
operator
worksheets
that
came
out,
and
here
is
one
that
I
filled
out.
I'm
not
going
to
keep
myself
anonymous,
but
I
didn't
want
to
share
someone
else's
results.
But
just
to
give
you
an
idea,
we
had
folks
answer
a
few
demographic
questions
that
hopefully
weren't
too
revealing,
but
basically
where
you
were
located,
how
old
you
are,
and
then
the
search
term
you
used.
Like
Hadley
showed,
there
was
three
responses.
We
had
people
do
this
search
and
then
show
the
top
three
responses
that
that
search
engine
recommended.
To
do
the
analysis,
the
first
thing
we
had
to
do
is
just
take
these
three
and
bring
them
together.
This
is
a
nice
easy
way
to
do
this,
is
to
go
under
the
columns,
utilities
and
just
combined
columns.
Now
I
just
called
these
responses
and
made
a
little
delimiter.
I
unchecked
that
multiple
response
'cause
we're
going
to
just
do
text
analytics
on
this.
Then
you
get
this,
which
is
the
table
that
we, excuse me,
the column
we're
using
for
Text
Explorer.
Now,
I
did
this
and
then
I
brought
in
all
of
the
results
from
all
the
different
people
who
took
the
survey
and
tweaked
a
little
bit
more
like
combined
whether
you
were
in
the
US,
which
state
you
were
in,
and
then
summarize
that
into
region
between
America's
and
Europe,
'cause
we
didn't
have
enough
respondents
to
break
it
up
by
state.
But
in
the
end,
we
end
up
with
a
table
that
looks
like
this.
We
had
to
do
a
little
bit
of
recoding,
we
had
to
do
a
little
bit
of
filling
this
in
and
then
anonymize
that
search
engine.
The
folks
that
got
the
survey
knew
which
search
engine
to
use,
but
we're
not
sharing
that
here.
Hadley
is
going
to
now
talk
about
some
of
the
results
we
saw
out
of
this
once
we
had
it
in
this
form.
Let's
open
up
this
dashboard
right
here.
What
you're
seeing
are
the
most
popular
terms
in
order
of
popularity
from
left
to
right,
descending
order
for
the
first
response,
second
response,
and
third
response
for
every
one
of
the
search
terms,
for
every
gender
and
age
and
all
of
the
other
factors.
W
can
use
this
and
this
hierarchical
filtering
on
the
dashboard
to
explore
this
a
little
closer,
see
if
we
can
learn
anything.
One
thing
I
happen
to
notice
if
we
look
at
the
world
is
and
we
click
on
male,
you'll
see
that
for
most
people,
or for
many
people,
the
first
search
they
found
was
that
the
world
is
not
enough
if
you're
female,
equally
likely
is
to
find
it
your
oyster.
Interestingly,
if
you're
less
than
40
years
old,
that's
when
the
world
is
not
enough,
suddenly
becomes
the
world
is
yours.
I
think
we
could
probably
agree
with
it's
probably
true
for
people
under
40,
isn't
it?
What
else
have
we
got
here?
If
I
look
at
climate
change,
so
another of course
hot
topic
of
interest
these
days,
as
well
it
should
be.
If
I
were
to
look
at
people
over
50,
apparently
a
huge
concern
for
them
is
whether
climate
change
is
changing
babies
in
the
womb,
which,
interestingly,
isn't
a
concern
for
people
below
40.
I
wondered
whether
this
is
a
valid
concern
for
people
over
50,
whether
they're
more
likely
to
have
their
babies
change
in
their
wombs
or
not.
But
aside
from
that,
let's
take
a
step
back
and
see
how
we
can
go
about
creating
this
dashboard.
It's
quite
simple.
The
first
thing
we
need
to
do
is
to
create
our
filter
variables.
I've
done
that
here.
Here's
our
search
terms
and
our
distributions.
What
I'll
do
is
I'll
go
through
how
to
create
the
graph
builder
report,
because
that's
something
that
you
may
not
be
familiar
with,
that
you
might
be
interested
in
doing.
I'm
going
to
take
my
first
response,
put
it
here,
and
simply
choose
the
number
that
those
occur.
Then
I
can
right
click
and
order
by
count
descending.
T
hat's
it.
I've
done
the
same
thing
for
my
second
response
and
my
third
response
as
well.
Now
we
can
go
ahead
putting
together
the
pieces
of
the
dashboard.
We'll
click
on
new
dashboard,
we'll
choose
the
hierarchical
filter
plus
one.
I'll
take
my
distribution
results
put
them
there,
my
input
parameters,
put
them
there,
and
then
my
graphs.
Let's
see,
is
this
one
first?
Well, I cant tell.
We'll
just
put
them
in
order
like
this,
and
I
can
always
change
the
order
if
I
want.
All
right.
I'll
run
the
dashboard,
and
there
we
have
it.
It
really
is
as
simple
as
that.
Then
I
can
go
ahead
and
save
it
to
the
table.
That
was
one
use
of
a
dashboard.
I'll
show
you
another
use
of
a
dashboard,
which
was
to
use
it
with
a
Text
Explorer
Word Cloud.
This
is
the
most
common
words,
not
just
entire
phrases
or
entries,
but
individual
words.
You
can
see
the
word
design
seem
to
be
used
a
lot.
If
I
were
to
look
at,
for
example,
statistics,
so
it
looks
like
everybody
can
agree
that
statistics
is
a
science.
Interestingly,
if
you're
in
Europe,
apparently
you
find
it
harder
than
you
do
if
you're
in
America,
where
that
doesn't
come
up,
so
something
I
happen
to
notice
there.
To
create
this
dashboard,
it's
very
much
the
same
as
the
other
one.
We'll
add
our
distribution
items.
Here's
the
first
one,
here's
the
second
one.
We'll
add
our
Text
Explorer
Word cloud,
and
then
we'll
simply
put
this
one
together
just
as
we
did
the
previous
one.
With
that,
I'd
like
to
thank
you
for
this
part
of
the
presentation
about
the
exploratory
visual
analysis.
I've
shown
you
how
you
can
go
about
doing
this
using
the
hierarchical
dashboards.
Now
I'll
turn
things
back
over
to
Pete,
who will
take
us
through
some
more
in
depth
use
of
the
Text
Explorer.
Perfect.
Thanks,
Hadley.
Like
Hadley
mentioned
this
is
a
different
way
to
display
this,
but
this
is
the
end
result
of
using
the
Text
Explorer
and
looking
just
at
the
Word
Cloud
here.
He
had
made
this
a
dashboard
and
used
filters
that
were
graphical
of
nature,
which
is
great.
You
could
do
this
also
with
a
local
data
filter.
But
this
is
basically
the
end
result
we're
going
for.
Let's
now
back
up
and
talk
about
how
we
got
here.
With
our
data
set
over
here,
we
just
launched
the
Text
Explorer
under
analyze
menu,
we
put
in
our
column
that
we're
interested
in.
In
this
case,
all
three
responses
combined
into
one
column.
We
have
a
bunch
of
options
we
can
use
to
tweak
this,
including
language
and
how
we
tokenize
the
words.
But
we're
going
to
go
ahead
and
just
use
the
defaults.
Here
you
can
see
since
we
have
different
responses
to
different
search
terms,
the
overall
term
and
phrase
list
by
itself
is
not
super
informative.
What
we
would
want
to
do
is
apply
that
local
data
filter
and
the
first
thing
we'll
look
at
is
that
search
term.
Now
we
can
do
something
like
the
economy
or
coronavirus
or
climate
change
and
go
from
there.
Let's
focus
in
on
climate
change
here.
One
thing
that
I
wanted
to
do
was
add
some
sentiment
analysis.
The
first
thing
I'm
going
to
go
ahead
and
turn
on
this
Word
Cloud
so
it
looks
like
it
had
before.
Now
we
can
display
it
this
way
where
you
have
the
most
common
term
in
there
and
you
can
see
it's
climate
and
change.
We
know
that
we're
searching
that,
so I
could
go
in
here
and
add
these
as
stop words
and
now
see
which
ones
come
up
the
most
frequently
when
we're
mentioning
climate
change.
This
is
one
way
to
display
the
Word
Cloud.
I
can
also
go
through
here
and
maybe
change
this
to
something
that
is
a
little
more
appealing
to
the
eye,
but
maybe
less
useful
from
a
quantitative
standpoint.
You
can
always
add
some
arbitrary
colors
if
you
like
that
as
well.
All
right,
so
I've
done
this
to
this
point,
but
now
I
want
to
add
some
sentiment
analysis
to
this.
Are
people
thinking
climate
change
is
natural,
a
good
thing
or
is
it
a
bad
thing?
You
can
see
some
things
in
here
that
maybe
indicate
that,
but
I
wasn't
quite
sure
where
to
find
sentiment
analysis.
With
JMP
17,
we
have
this
new
feature
called
Search
JMP.
If
you're
ever
looking
for
an
analysis
in
JMP,
this
is
a
great
way
to
find
that.
If
I
just
start
typing
in
sentiment,
you
can
see
right
here
that
it
tells
me
how
to
find
this,
I
can
do
the
help,
but
I
can
also
just
hit
show
me
and
it
launches
it
right
there.
If
I'm
ever
wondering,
hey,
how
can
I
do
this,
this
gives
me
the
option.
Now,
a
couple
of
things
you
see
here
it's
identified
some
of
these
default
terms
that
are
providing
sentiment.
Things
like
good.
If
I
click
on
good,
I
get
a
little
summary.
It
looks
like
when
people
are
saying
good,
that
is
actually
a
positive
sentiment.
Now,
what
about
greatest?
Oh,
boy,
almost
everything
that
says
greatest
is
a
greatest
threat.
Maybe
that's
not
actually
a
positive
sentiment
there.
We
might
need
to
do
a
little
bit
of
tweaking.
First
let's
go
in
here
and
say,
okay,
well,
greatest
threat
is
a
phrase
that
we're
seeing
commonly.
I'm
going
to
just
add
that
phrase.
Again,
you
would
do
this
in
your
curation
process,
and
now
you
see
that
that
goes
away.
But
I
think
greatest
threat
is
actually
a
negative
thing.
Let's
look
at
those
sentiment
terms.
You
can
see
JMPs
identified
that
as
something
that
maybe
has
sentiment.
I'm
going
to
just
say,
you
know
what?
That's
a
really
negative
sentiment.
Now
when
we
go
down
here,
you
can
see
that
it's
flagged
those
seven
occurrences
where
they
mentioned
greatest
threat,
and
it
said
that
those
are
a
very
negative.
That's
changed
our
overall
impression
of,
do
most
of
these
search
terms
think
this
is
negative
or
positive?
That's
just
an
example
of
how
you
can
walk
through
that
flow
and
come
up
with
the
end
sentiment
analysis.
I'm
going
to
pass
it
back
over
to
Hadley
and
let
him
wrap
things
up
here.
What
I'd
like
to
say
is
that
we
showed
you,
first
of
all,
how
we
went
about
using
the
MSA
Design
to
help
with
the
data
collection.
We
use
Recode
and
other
items
in
the
Tables
menu
to
help
with
the
data
clean
up.
We
then
used
Distribution,
Graph
Builder,
Text
Explorer,
and
combinations,
all
of
them
together
to
help
with
the
data
exploration,
see
if
we
can
uncover
anything
interesting.
Then
Pete
used
Sentiment
Analysis
together
with
the
Search
and
JMP
17
to
see
what
else
we
can
learn
about
the
data
as
a
whole.
With
that,
I
hope
you
found
this
useful.
I
hope
it's
given
you
some
ideas
about
how
you
can
do
this
on
your
own
data
for
yourselves.
I'd
like
to
thank
you
all
for
listening,
and
I
hope
you
enjoy
the
rest
of
the
JMP
Discovery
Conference.
Thank
you.
