|
Speaker
|
Transcript
|
|
|
So welcome everyone. My name is
|
|
00
|
07.933
|
|
|
3
|
|
|
Ambassador with JMP. I am now a
|
|
|
retired professor of Business
|
|
00
|
30.566
|
|
|
7
|
|
|
between a student and a
|
|
|
professor working on a project.
|
|
00
|
49.700
|
|
|
11
|
|
|
12
|
|
|
engage students in statistical
|
|
|
reasoning, teach that
|
|
00
|
12.433
|
|
|
16
|
|
|
to that, current thinking is
|
|
|
that students should be learning
|
|
|
about reproducible workflows,
|
|
00
|
36.266
|
|
|
21
|
|
|
elementary data management. And,
|
|
|
again, viewing statistics as
|
|
00
|
58.800
|
|
|
25
|
|
|
26
|
|
|
wanted to join you today on this
|
|
|
virtual call. Thanks for having
|
|
00
|
20.600
|
|
|
30
|
|
|
and specifically in Manhattan,
|
|
|
and you'd asked us so so you
|
|
00
|
36.433
|
|
|
34
|
|
|
And we chose to do the Airbnb
|
|
|
renter perspective. So we're
|
|
00
|
51.733
|
|
|
38
|
|
|
expensive.
|
|
|
So we
|
|
|
started filling out...you gave us
|
|
00
|
09.166
|
|
|
43
|
|
|
44
|
|
|
separate issue, from your main
|
|
|
focus of finding a place in
|
|
00
|
36.066
|
|
|
49
|
|
|
you get...if you get through the
|
|
|
first three questions, you've
|
|
00
|
54.100
|
|
|
53
|
|
|
know, is there a part of
|
|
|
Manhattan, you're interested in?
|
|
00
|
11.133
|
|
|
58
|
|
|
repository that you sent us to.
|
|
|
And we downloaded the really
|
|
00
|
26.433
|
|
|
32.866
|
|
|
63
|
|
|
thing we found, there were like
|
|
|
four columns in this data set
|
|
00
|
46.766
|
|
|
67
|
|
|
figured out so that was this
|
|
|
one, the host neighborhood. So
|
|
00
|
58.100
|
|
|
71
|
|
|
72
|
|
|
figured out that the first two
|
|
|
just have tons of little tiny
|
|
00
|
13.300
|
|
|
76
|
|
|
Manhattan. So we selected
|
|
|
Manhattan. And then when we had
|
|
00
|
29.700
|
|
|
80
|
|
|
that and then that's how we got
|
|
|
our Manhattan listings. So
|
|
00
|
44.033
|
|
|
84
|
|
|
data is that you run into these
|
|
|
issues like why are there four
|
|
00
|
03.300
|
|
|
88
|
|
|
restricted it to Manhattan, I'll
|
|
|
go back and clean up some
|
|
00
|
18.033
|
|
|
92
|
|
|
data will describe everything we
|
|
|
did to get the data, we'll talk
|
|
00
|
28.400
|
|
|
33.200
|
|
|
97
|
|
|
know I'm supposed to combine
|
|
|
them based on zip, the zip code,
|
|
00
|
47.166
|
|
|
101
|
|
|
102
|
|
|
107 columns,
|
|
|
it's just hard to find the
|
|
00
|
09.366
|
|
|
106
|
|
|
them, so we knew we had to clean
|
|
|
that up. All right, we also had
|
|
00
|
27.366
|
|
|
111
|
|
|
journal of notes. In order to
|
|
|
clean this up, we use the recode
|
|
00
|
45.500
|
|
|
115
|
|
|
Exactly. Cool.
|
|
|
Okay, so we we did the cleanup
|
|
00
|
02.200
|
|
|
119
|
|
|
Manhattan tax data has this zip
|
|
|
code. So I have this zip code
|
|
00
|
19.300
|
|
|
123
|
|
|
day of class, when we talked
|
|
|
about
|
|
|
data types. And notice in the
|
|
00
|
42.300
|
|
|
128
|
|
|
the...analyze the distribution of
|
|
|
that column, it'll make a funny
|
|
00
|
03.200
|
|
|
133
|
|
|
Manhattan doesn't really tell
|
|
|
you a thing.
|
|
|
But the zip code clean data in
|
|
00
|
18.466
|
|
|
23.266
|
|
|
139
|
|
|
just a label, an identifier, and
|
|
|
more to the point,
|
|
|
when you want to join or merge
|
|
00
|
41.833
|
|
|
48.766
|
|
|
145
|
|
|
important. It's not just an
|
|
|
abstract idea. You can't merge
|
|
00
|
03.166
|
|
|
11.266
|
|
|
150
|
|
|
nominal was the modeling type,
|
|
|
we just made sure.
|
|
00
|
26.200
|
|
|
31.033
|
|
|
155
|
|
|
about the main table is the
|
|
|
listings. I want to keep
|
|
00
|
45.533
|
|
|
159
|
|
|
to combine it with Manhattan tax
|
|
|
data.
|
|
|
Yeah. Then what? Then we need to
|
|
00
|
03.266
|
|
|
164
|
|
|
tell it that the column called
|
|
|
zip clean,
|
|
|
zip code clean...
|
|
|
Almost. There we go.
|
|
|
And the column called zip, which
|
|
00
|
33.200
|
|
|
171
|
|
|
172
|
|
|
Airbnb listing
|
|
|
and match it up with anything in
|
|
00
|
57.033
|
|
|
177
|
|
|
178
|
|
|
them in table every row, whether
|
|
|
it matches with the other or
|
|
00
|
13.233
|
|
|
182
|
|
|
main table, and then only the stuff
|
|
|
that overlaps from the second
|
|
00
|
29.600
|
|
|
186
|
|
|
another name like, Air BnB IRS
|
|
|
or something? Yeah, it's a lot
|
|
00
|
50.966
|
|
|
190
|
|
|
do one more thing
|
|
|
because I noticed these are just
|
|
|
data tables scattered around
|
|
00
|
06.666
|
|
|
195
|
|
|
running. Okay. So I'll save this
|
|
|
data table. Now what?
|
|
|
And really, this is the data
|
|
00
|
19.833
|
|
|
22.033
|
|
|
26.266
|
|
|
35.466
|
|
|
203
|
|
|
anything else, before we lose
|
|
|
track of where we are, let's
|
|
00
|
49.733
|
|
|
58.800
|
|
|
01.833
|
|
|
209
|
|
|
or Oak Team?
|
|
|
And then
|
|
|
part of the idea of a project
|
|
00
|
23.700
|
|
|
214
|
|
|
thing. So if you
|
|
|
grab, I would say, take the
|
|
00
|
50.100
|
|
|
218
|
|
|
219
|
|
|
220
|
|
|
two original data sets, and then
|
|
|
my final merged. Okay Now
|
|
00
|
16.200
|
|
|
225
|
|
|
them as tabs.
|
|
|
And as you generate graphs and
|
|
00
|
36.566
|
|
|
229
|
|
|
230
|
|
|
231
|
|
|
even when I have it in these
|
|
|
tabs. Okay, that's really cool.
|
|
00
|
58.833
|
|
|
02.500
|
|
|
236
|
|
|
right, go Oak Team.
|
|
|
Well, hi, Dr. Carver, thanks so
|
|
00
|
19.233
|
|
|
240
|
|
|
you would just glance at some of
|
|
|
these things, and let me know if
|
|
00
|
32.300
|
|
|
244
|
|
|
we used Graph Builder to look at
|
|
|
the price per neighborhood. And
|
|
00
|
45.400
|
|
|
248
|
|
|
help it be a little easier to
|
|
|
compare between them. So we kind
|
|
00
|
01.000
|
|
|
252
|
|
|
have a lot of experience with
|
|
|
New York City. So we plotted
|
|
00
|
18.166
|
|
|
256
|
|
|
stand in front of the UN and
|
|
|
take a picture with all the
|
|
00
|
31.733
|
|
|
260
|
|
|
saying in Gramercy Park or
|
|
|
Murray Hill.
|
|
|
If we look back at the
|
|
00
|
46.566
|
|
|
265
|
|
|
thought we should expand our
|
|
|
search beyond that neighborhood to
|
|
00
|
58.766
|
|
|
269
|
|
|
270
|
|
|
just plotted what the averages
|
|
|
were for the neighborhoods but
|
|
00
|
14.533
|
|
|
274
|
|
|
the modeling, and to model the
|
|
|
prediction. So if we could put
|
|
00
|
30.766
|
|
|
279
|
|
|
expected price. We started
|
|
|
building a model and what we've
|
|
00
|
42.800
|
|
|
283
|
|
|
factors. And so then when we put
|
|
|
those factors into just a
|
|
00
|
58.833
|
|
|
287
|
|
|
more, some of the fit statistics
|
|
|
you've told us about in class.
|
|
00
|
15.466
|
|
|
292
|
|
|
but mostly it's a cloud around
|
|
|
that residual zero line. So
|
|
00
|
30.766
|
|
|
296
|
|
|
which was way bigger than any of
|
|
|
our other models. So we know
|
|
00
|
45.800
|
|
|
300
|
|
|
reasons we use real data.
|
|
|
Sometimes, this is real. This is
|
|
00
|
58.266
|
|
|
304
|
|
|
looking?
|
|
|
Like this is residual values.
|
|
00
|
19.266
|
|
|
309
|
|
|
is good. Ah, cool.
|
|
|
Cool. Okay, so I'll look for
|
|
00
|
34.966
|
|
|
313
|
|
|
is sort of how we're answering
|
|
|
our few important questions. And
|
|
00
|
47.300
|
|
|
317
|
|
|
was really difficult to clean
|
|
|
the data and to join the data.
|
|
00
|
57.866
|
|
|
03.500
|
|
|
322
|
|
|
wanted to demonstrate how JMP
|
|
|
in combination with a real world
|
|
00
|
28.700
|
|
|
327
|
|
|
Number one in a real project,
|
|
|
scoping is important. We want to
|
|
00
|
47.600
|
|
|
331
|
|
|
hope to bring to the
|
|
|
to the group. Pitfall number two,
|
|
|
it's vital to explore the
|
|
00
|
08.033
|
|
|
336
|
|
|
the area of linking data
|
|
|
combining data from multiple
|
|
00
|
27.800
|
|
|
341
|
|
|
recoding
|
|
|
and making sure that linkable
|
|
00
|
45.100
|
|
|
345
|
|
|
346
|
|
|
reproducible research is vital,
|
|
|
especially in a team context,
|
|
|
especially for projects that may
|
|
00
|
05.966
|
|
|
351
|
|
|
habits of guaranteeing
|
|
|
reproducibility. And finally,
|
|
|
we hope you notice that in these
|
|
00
|
32.633
|
|
|
356
|
|
|
on the computation and
|
|
|
interpretation falls by the
|
|
00
|
51.900
|
|
|
360
|
