Hello.
Thank
you
for
joining
us
today
for
our
presentation
on,
"Measuring
Change
in
Medication Assisted
Treatment
for
Participants
in
project
ECHO
Using
Administrative
Healthcare
Claims"
We
are
excited
to
share
how
we
use
JMP
as
a
key
tool
in
our
analytic
work.
Next
slide,
please.
My
name
is
Erica
Plante
and
I
am
a
senior
scientific
data
analyst
at
the
Institute
for
Health
Policy
and
Practice
at
the
University
of
New
Hampshire.
I
am
joined
by
Dr.
Michelle
Capozzoli,
Senior
Lecturer
in
the
Department
of
Mathematics
and
Statistics,
also
at
the
University
of
New
Hampshire.
Neither
Michelle
nor
I
have
conflicts
of
interest
to
disclose.
Next
slide,
please.
Before
we
describe
our
work,
I
would
like
to
provide
a
brief
overview
on
Project
ECHO.
Project
ECHO
was
founded
in
2003
by
Dr.
Sanjeev
Arora
at
the
University
of
New
Mexico.
Dr.
Arora
is
a
Physician
specializing
in
Gastroenterology
and
Hepatology.
He
was
seeing
patients
with
Hepatitis
C
die
at
alarming
rates
because
they
could
not
access
care
for
this
treatable
disease
in
a
timely
manner.
He
sought
to
bring
providers
together
to
form
a
community
of
practice
where
doctors
and
other
specialists
can
learn
from
each
other.
ECHO
is
an
"all
teach,
all
learn"
model,
and
the
sessions
are
often
centered
around
a
key
issue
or
condition.
The
University
of
New
Hampshire
launched
its
project
ECHO
hub
in
2018
and
has
since
produced
a
number
of
ECHO
programs,
including
the
Partnership
for
Academic
Clinical
Telepractice,
Medications
for
Addiction
Treatment,
or
PACT-MAT.
Next
slide,
please.
The
primary
goal
of
PACT-MAT
ECHO
is
to
increase
the
number
of
nurse
practitioner
students
in
graduate
and
postgraduate
programs
who
receive
waiver
training,
apply
for
the
waiver,
and
subsequently
prescribe
MAT.
Secondarily,
the
project
seeks
to
increase
provider
self-
efficacy
in
managing
patients
with
Substance-
use
Disorder.
The
program
developed
a
learning
community
that
enabled
a
culture
that
understood
addiction
as
a
chronic
disease
and
was
prepared
to
address
the
range
of
issues
that
emerged
during
the
process
of
treatment.
Specifically,
this
program
focused
on
all
participants
becoming
proficient
and
culturally
competent
in
prescribing
and
treating
SUD,
as
well
as
enhancing
capacity
and
qualities
of
services
available
to
patients
in
their
communities
through
their
providers.
Next
slide,
please.
After
the
completion
of
the
first
PACT-MAT
session,
the
team
was
interested
in
answering
some
questions
about
the
PACT-MAT
ECHO
through
Claims
Data
Analysis.
But
here's
some
core
information
about
the
analytic
project.
The
analytic
period
of
interest
was
from
2018
through
June
2020.
This
was
to
capture
data prior to
and after
the
first
PACT-MAT
ECHO
session.
The
project
was
funded
by
the
Substance
Abuse
and
Mental
Health
Services
Administration (
SAMHSA),
as
part
of
150K 3-year
grant.
The
project's
principal
investigator
was
Dr.
Marcy
Doyle.
We
were
wanting
to
ask
a
few
questions
about
the
actual
ECHO
program
and
how
we
were
able
to
see
if
provider
practices
had
changed
after
actually
participating
in
the
ECHO.
And
the
Center
for
Health
Analytics
of
Informatics
at
the
Institute
for
Health
Policy
and
Practice
at
UNH.
We're
fortunate
enough
to
have
access
to
healthcare,
administrative
claims,
and
enrollment
data
for
commercial
and
New
Hampshire
Medicaid
policies.
Therefore,
the
CHA
team
was
brought
into
the
research
project
to
collect
and
aggregate
the
data,
and
UNH's
Department
of
Mathematics
and
Statistics
was
brought
in
to
build
models
and
perform
analysis.
Next
slide,
please.
And
these
are
the
questions
that
we
asked
as
our
core
research
questions.
Did
the
PACT-MAT
ECHO
Series
have
an
impact
on
participants'
MAT
prescribing
practices?
And
can
we
successfully
perform
a
case/
control
study
on
providers
using
administrative
claims
data?
Next
slide,
please.
When
collecting
health
care
claims,
we
included
all
members,
ages
zero
to
64
who
had
medical
and
pharmacy
enrollment
in
the
month
of
interest.
PACT-MAT
participants
self-
reported
their
name,
titles,
NPI,
organization's
name
and
address,
and
their
waiver
status.
We
cross-
referenced
that
data
against
CMS's
National
Plan
and
Provider
Enumeration
System,
also
known
as
the
NPPES
Registry.
In
the
cases
of
a
mismatch
between
the
self-
reported
data
and
the
NPPES
registry,
the
self
reported
data
was
considered
the
most
up
to
date
and
was
used
for
the
analysis.
Information
on
our
control
group
was
sourced
only
from
the
NPPES.
Next
slide,
please.
And
claims
reflect
if
one
or
more
service
lines
included
an
MAT
procedure
code
or
drug
code,
they
were
also
flagged
if an
Opioid-Related
Disorder
diagnosis
code
was
found.
Medical
providers
were
selected
as
having
billed
for
MAT
if
at
least
one
service
line
included
their
NPI
or
one
of
the
MAT
CPT
codes.
Prescribing
providers
were
selected
as
having
prescribed
MAT
if at least
one
pharmacy
service
line
included
their
NPI
as
the
prescriber
and
at
least
one
of
the
MAT
NDC
codes.
The
case
and
control
populations
each
had
two
pairs
of
datasets.
Next
slide,
please.
One
pair
for
each
insurance
type,
commercial
or
New
Hampshire
Medicaid.
The
first
date
included
there
the
providers
NPI
and
information,
as
well
as
total
aggregates
by
month
of
all
providers,
patients
and
claims.
Patients
with
Opioid-
Use
Disorder
(OUD),
patients
with
any
medication
assisted
treatment,
and
patients
with
both
OUD
and
MAT.
The
second
dataset
provided
the
same
dataset
as
the
first
data
set...
Same
data
as
the
first
data
set,
with
the aggregation
at
the
member
demographic
level,
such
as
age,
category,
county,
and
sex.
No
identifiable
member
data
was
applied
to
the
statisticians.
Now
I'm
going
to
pass
the
presentation
to
my
colleague,
Dr.
Capozzoli.
Thank
you,
Erica.
Once
we
obtain
the
data.
The
data
was
actually
analyzed
by
Rebecca
L. Park.
She
was
a
UNH
master's
student
under
the
supervision
of
myself
and
Dr.
Philip
Ramsey.
We
received
the
three
data
sets.
One
was
the
practitioners
demographics
such
as
name,
National
Provider
Identifier,
title,
practice
address.
The
other
two
were
the
claims
datasets.
One
for
the
Medicaid
and
one
for
the
commercial.
So
the
original
data
was
extremely
large
for
both
the
commercial
and
for
Medicaid
for
each
practitioner,
it
tracked
each
of
their
MAT
patients'
history
over
the
study
period.
So
the
original
thought
was
try
to...
Use
their
pattern
of
behavior
of
their
patients
over
time.
Quite
quickly,
it
became
apparent
that
this
approach
was
a
little
bit
problematic
and
also
we
ran
into
some
privacy
laws
in
trying
to
make
sure
that
all
identifying
markers
were
not
available.
So
what
we
did
was
we
honed
in
on
several
of
the
variables
for
the
demographics.
So
we
honed
in
on
the
National
Provider
Identifier,
their
title,
and
the
city
of
their
practice.
And
then
from
the
claims
data,
we
focused
in
on
the
month
and
year.
So
this
tracks
what
month
and
year
that
we
were
looking
at.
The
phase
of
the
program
was
the
Pre,
so
this
is
Pre
before
ECHO,
Ongoing
is
during
ECHO,
Post is
obviously
after
ECHO.
So
then
we
aggregated
the
data
and
so
instead
of
looking
at
every
single
visit,
what
we
looked
at
were
months
and
we
looked
at
patient
totals.
So
the
first,
we
obviously
looked
at
was
the
number
of
patients
total
for
that
practitioner
during
that
month,
the
number
of
patients
with
Opioid-U
se
Disorder,
the
number
of
patients
who
had
any
MAT,
the
number
of
patients
with
OUD
who
had
any
MAT.
Further,
when
looking
at
the
Medicaid
versus
the
commercial
care,
during
the
exploratory
analysis,
it
became
apparent
that
we
were
going
to
need
to
focus
in
on
the
Medicaid
data
due
to
the
low
patient
numbers
in
the
commercial
care.
So
here
for
example,
when
we
were
looking
at
it,
it
became
apparent;
so
on
average
they
had
one
patient
per
month
with
any MAT.
And
so
what
we
were
doing
was
there's
a
lot
of
sparse
data
here
and
it
was
just
not
conducive
to
trying
to
fit
models.
So
we
focused
in
on
the
Medicaid
data.
Further,
we
initially
had
20
providers,
which
we
had
to
reduce
down
to
nine
providers.
And
the
reason
being
is
that
some
of
the
providers
had
too
many
months
of
missing
data.
Some
of
them,
we
had
to
eliminate
because
the
majority
of
the
months,
they
didn't
have
any
patients
who
had
MAT.
And
then
further,
as
we
started
to
fit
the
models,
it
became
apparent
that
we
also
needed
a
minimum
of
10
total
patients
per
month.
So
we
did
end
up
with
a
very
smaller
sample
size
than
we
had
originally
thought
we
would
have.
The
other
piece,
as
we
were
exploring
the
data,
using
some
of
the
tools
that
John
provides,
we
noticed
that
the
nine
practitioners
had
on
average
the
number
of
patients
or
number
of
total
patients
differed.
So
for
example,
they
could
have,
I
think
it
was
between
a
total
range
of
one
to
161
patients.
So
between
that
and
then
looking
at
the
trend
over
time,
so
we
looked
at
the
average
number
of
patients
total
and
as
you
notice
this,
the
blue
line,
you
notice
that
just
in
general
there's
an
increase
of
patients
over
time.
We
also
looked
at
the
average
number
of
patients
with
the
Opioid-Use D
isorder
and
those
who
had
any
MAT,
and
those
who
were
diagnosed
with
OUD
and
had
any
MAT.
And
if
you
notice,
all
four
have
a
similar
trend
of
an
increasing
trend.
So
to
combat
that,
because
what
we
want
to
know
is,
are the
practitioners
increasing
their
prescribing?
Not
that
they
are
having
more
in patients
over
time,
we
normalized
the
data
by
creating
a
new
variable.
So
what
we
created
was
the
proportion
of
patients
who
had
any
MAT
in
comparison
to
the
total
number
of
patients.
And
the
reason
that
we
did
use
the
patients
who
had
any MAT
is due
to
the
fact
that
a
diagnosis
of
an
OUD
does
have
a
certain
stigmatism.
And
so
we
felt
that
it
would
be
better
to
look
at
any
MAT.
So
Analysis
Considerations.
From
the
beginning
we
knew
that
we
had
a
small
sample
size
of
practitioners,
only
nine.
The
other
thing
that
became
apparent
through
a
lot
of
the
graphical
representations
of
the
data
was
that
we
had
a
lot
of
noise.
And
so
in
taking
this
into
consideration,
we
decided
to
attempt
several
different
approaches.
The
first
approach
was
basically,
your
means
comparison:
ANOVA and Matched
pairs.
Then
we
thought
about
bringing
in
that
time
variable.
And
so
we
did
look
at
them
in
several
different
ways,
from
just
simple
Linear Regression,
from
the
beginning.
We
did
consider
Structural
Equation
Models.
Dr.
Laura
Castro-
Shilo
from
JMP
had
come
to
one
of
Dr.
Ramsey's classes
and
given
a talk
on
these
models.
So
originally,
when...
We
were
looking
at
the
data,
we
thought
that
these
models
might
be
appropriate.
But
it
became
apparent
quickly
because
of
the
difficulties
that
we
had
that
they
just
were
not
working
for
us.
So
the
next,
we
worked
with
Segmented Regression.
That
was
chosen
because
in
some
previous
work
with
claims
data
,
it
was
brought
up
that
maybe
the
Segmented Regression
would
work
well
because
we
had
data
that
was
looking
at
pre
and
post.
So
we
thought
that
with
our
pre, ongoing, and post,
that
maybe
the
Segmented Regression
would
be
appropriate.
We
also
looked
at
Exponential
Regression
and
Logistic
Regression.
We
also
looked
at
their
Generalized
Regression
Models
and
including
just
the
regular
and
then
zero-
inflated
Poisson,
Binomial,
and
Negative
Binomial
Models.
And
the
reason
why
we
decided
to
look
at
the
Poisson,
and
Binomial
and
Negative
Binomial
is
the
data
is
inherent
that
it
has
counts
and
so
we
thought
that
maybe
these
might
be
appropriate.
And
so
what
I'm
going
to
focus
on
today
are
the
Means
Comparison,
the
Segmented Regression
and
the
zero-
inflated
Poisson
and
this
is
going
to
be
for
both
just
looking
at
the
ECHO
group
and
then
the
Matched
Pairs
comparing
the
control
group
to
the
ECHO
group.
So
the
first
analysis
ignores
the
time
variable
and
we're
just
looking
at
averages.
What's
the
average
proportion
of
patients
in
your
pre,
ongoing,
and
post.
And
just
from
the
means,
it
is
quickly
apparent
and
from
the
graph
that
our
pre
phase
is
definitely
lower
than
the
ongoing
and post.
We
also
tested for the...
Because
we
have
small
sample
with
practitioners,
we
did
look
at
the
variance
and
we
noted
that
we
did
have
an
issue
with
non-
equal
variance
and
so
we
did
use
Welch's
test
instead
of
the
traditional
ANOVA
to
conduct
to
see
if
we
had
differences
between
the
phases
and
obviously
we
do.
And
then
to
determine
statistically
which
ones
were
different,
we
did
look
at
the
All
Pairs
Tukey-
Kramer
Test.
And
it
did
confirm
that
our
pre
phase
was
different
than
the
ongoing
and
post,
but
the
ongoing
and
post
were
very
similar.
And
so
this
gave
us
some...
This
obviously
is
indicating
that
we
do
have
some
differences.
The
ECHO
program
is
making
a
difference.
So
this
Segmented Regression,
as
I
noted
before,
was
suggested
because
we
do
have
the
three
phases:
pre,
ongoing,
and
post.
And
we
were
able...
Dr.
Ramsey
had
suggested
that
we
use
a
script
that
is
done
by
David
Burnham,
by
Pega
Analytics.
If
you
are
interested
in
this
script,
the
link
is
here
on
the
website,
it
gives
you
the
code
and
it
also
gives
you
a
very
detailed
description
from
line
to
line,
what
he
is
doing
in
this
code.
And
so
what's
happening
here,
when
we
ran
the
script,
We
were
able
to
fit
separate
regressions
to
the
three
different
phases
as
well
as
get
the
fit,
an
R², and
test
the
significance
of
the
slope.
And
so
again,
we're
seeing
this
pattern
that
we
saw
with
the
ANOVA.
You
do
see
that
the
pre
is
definitely
lower
than
the
ongoing
and post.
Unfortunately,
the
slopes
for
all
of
the
three
phases
were
not
significant.
You
can
also
see
that
we
do
have
a
significant
amount
of
variability.
So
next, what
we
considered
was
the
fact
that
we
had
two
types...
Or
two
categories
of
practitioners;
we
had
those
who
were
nurse
practitioners,
physician
assistants;
and
then
we
had
the
physicians.
Even
though
they
were
small
sample
sizes.
We
decided
to
see,
"Okay,
can
we
see
some
kind
of
a
signal?"
"What
is
going
on
here?"
And
when
we
looked
at
the
nurse
practitioners
again,
you
see
that
behavior
of
the
pre
being
lower
than
the
ongoing
and
the
post.
And
what
we
noted
is
that
for
the
pre
phase,
we
are
seeing
a
little
bit
of
a
significance
even
though
it
is
0.07,
it
is
saying
that
there
is
a
signal
here
and
we
do
have
something
going
on.
Unfortunately
for
the
ongoing
and
post,
we're
not
seeing
that
significance
but
again
we
are
seeing
that
trend.
What's
interesting
is
when
we
looked
at
the
positions.
And
we
noted
that
obviously
we
do
not
have...
The
slopes
are
definitely
not of
significance
and
there
doesn't
seem
to
be
any
difference
whether
they're
in
the
pre,
ongoing,
or
post.
So
it
seems
while
it
may
not
be
a
practical
significance,
I mean it
may
not
be
a
statistical
significance
of
practical
interest
is
the
fact
that
the
ECHO
group
does
seem
to
be
benefiting
this
nurse
practitioner,
physicians
assistants
group.
So
the
next
thing
we
tried
was
the
Zero
Inflation
Poisson
model.
We
chose
the
Poisson
Model
due
to
the
fact
that
we
had
counts
and
we
also
chose
the
Zero
Inflation...
We
did
both
the
regular
and
the
Zero
Inflation
and
we
chose
the
Zero
Inflation
because
we
did
have
a
lot
of
zeros
in
our
data.
And
so
when
we
did
it,
you'll
notice
that
we
do
have
this
slight
trend
of
increasing of
proportion
of
MAT patients over time.
And looking
at
the P
arameter
Estimates,
month
is
significant.
Note
that
the
Zero Inflation
is
zero.
So
if
you
look
at
it,
it wasn't
really
doing
much
for
it
to
have
the
Zero Inflation
but...
It
was
informative
for
us.
Unfortunately,
when
we
went
to
evaluate
the
fit
of
the
model,
it
became
quickly
apparent
that
it
was
a
poor
fit.
So
for
example,
when
we
looked
at
your
generalized
R²,
it's
very
low.
And
then
we
have
this...
When
we
looked
at
the
actual
versus
predicted
plot,
what's
happening
is
that
here
are
the
predicted
values
and
they're
really
ranging
between
0.1
and
0.4
where
the
actual
data
is
ranging
between
zero
and
one.
So
what's
happening
is
that
the
data
between
here
is
really
pushing
this
model.
And
so
obviously,
what
we
would
have
liked to have seen
to
have
is
more
of
the
predictions
following
this
line.
Having
said
that,
we
did
see
some
trends
and
even
though
we
didn't
have
maybe
a
statistical
piece,
we
did
have
some
practical
interpretation.
So
moving
on,
we
did
have
a
control
group.
And
so
what
we
did
is
that
we
took
the
nine
control
providers
were
directly
compared
to
the
nine
ECHO
providers
and
they
were
equivalent
in
title
and
city.
So
we
were
trying
to
match
whether
they
were
a
nurse
practitioner,
physician's
assistant
or
a
physician,
and
then
where
their
practice
mainly
was,
their
primary
city.
And
the
piece
of
that
is
that
there
is
a
very
different
demographic
when
we
go
from
the
South
of
New
Hampshire
up
to
the
North
of
New
Hampshire.
So
we
wanted
to
make
sure
that
we
captured
any
of
those.
So
when
we
did
the
Matched
Pairs
Test,
we
created
a
confidence
interval
for
the
proportion
of
patients
who
had
any
MAT to
the
total
number
of
patients.
We
looked
at
the
difference
between
our
control
and
our
providers
and
we
know
in
that...
First
off,
when
you
look
at
the
confidence,
zero
is
not
in
there.
So
we
do
have
a
difference.
And
we
note
that
when
we
looked
at
the
actual
means
you
have
about
your
treatment,
ECHO
is
about 0.2...
Have 0.2
proportion
of
patients
have
MAT.
And
then
for
the
control
was
only
0.13.
So
we
do
have
a
difference
of 0.07
.
So
we
are
seeing
that
our
ECHO
group
is
prescribing
MAT
more
frequently
than
our
control
group.
So
the
next
was
to
try
to
bring
in
that
again,
that
time
variable.
And
so
we
looked
at
the
Zero Inflation
Poisson
Model
and
so
what
we
see
here,
so
first
off
the
ECHO
group
is
in
red
and
then
the
control
group
is
in
blue.
So
looking
at
it
graphically,
we
are
seeing
an
increase
in
time.
It
is
evident
that
the
ECHO
group
is
slightly
higher
in
prescribing,
a
proportion
of
prescribing
than
our
control
group.
Again
looking
at
the
parameters
in
our
model,
they
are
significant.
Unfortunately
when
we
moved
again
to
assess
our
model,
we
had
a
very
similar
result
as
we
did
with
just
looking
at
the
ECHO,
which
is
somewhat
not
surprising
in
the
sense
that
we
are
using
the
similar
data.
And
so
again
you're
looking
at
your
Generalized
R²,
it's
very
low,
and
again
we're
noting
that
our
actual
versus
predicted,
our
model
is
predicting
probably
between
now
0.05
and
0.4,
whereas
we
were
hoping
that
it
would
predict
along
the
range
from
zero
to
one.
So
findings.
So overall
we
were
able
to
detect
the
difference
in
provider
diagnostic
patterns
before
and
after
they
predicted
in
project
ECHO.
We
did
see
a
small
difference
in
provider
diagnostic
patterns
between
the
providers
that
did
participate
in
project
ECHO
and
those
who
did
not.
One
of
the
things
that
we
did
not
control
on
was
the
number
of
total
patients.
So
that
may
be
something
to
consider
later.
And
then
we
also
noted
that
there
may
be
a
difference
on
the
impact
of
project
ECHO
from
the
different
provider
level
title.
And
we
need
to
maybe
further
delve
into
that.
So
next
steps,
this
is
an
ongoing
project.
These
are
the
next
steps
that
we
are
considering.
So
first
off,
what
we
would
like
to
do
is
to
include
additional
providers
to
increase
the
size
of
the
database.
And
one
of
the
ways
that
we
are
looking
at
doing
that
is
to
include
some
more
of
the
ECHO
periods.
So
we
do
have
one
that
was
finishing
up
this
year.
So
we're
hoping
to
add
practitioners
in
from
that
period.
And
we
also
may
want
to
consider
the
methodology
for
detecting
MAT
and
the
medical
and
pharmacy
claims
data.
Also
we
would
like
to
analyze
at
the
practice
level
with
case
control
studies
to
help
combat
the
small
sample
size.
And
again,
the
overall
goal
is
to
fit
an
appropriate
model.
So
we
wanted
to
thank
you
for
taking
the
time
to
listen
to
our
presentation
on
the
PACT-MAT
ECHO.
If
you
have
any
questions,
please
contact
us
at
our
following
emails.
Enjoy
the
rest
of
your
conference.
Thank
you.
Thank
you.
