In the previous blog (Part 1) , we introduced how to use the enhanced Python integration feature in JMP 18 to load the scikit-learn sample data, “California Home Prices,” into a JMP data table.
In this blog (Part 2), we will use the imported data to create a regression model that predicts house prices based on age, number of rooms, latitude, longitude, etc., and visualize the model using the Prediction Profiler. In JMP 18, we will also take advantage of the new feature that allows you to display prediction intervals in the Profiler.
However, it is dangerous to create a model without observing the data, so let's explore the data by graphing and summarizing it. JMP 18 also adds features that are useful for exploring data.
Header statistics and column switcher for data exploration
When exploring data in JMP, we recommend taking advantage of the new features in JMP 18: Header Statistics and Column Manager.
Header Statistics
Clicking the Σ icon in the data table will display summary statistics for each column in the data table, and, as an existing JMP feature, you can also click the histogram icon to display a header graph, allowing for high-level data visualization and summarization.
Column Manager
Clicking on the icon in the columns panel (shown in the green circle above) will launch the column manager as shown below: This tool allows you to modify column properties, display summary statistics, distributions, etc., which is very useful especially for summarizing data with many variables.
Looking at the summary statistics and distributions, we can see that some variables have a right tail and large outliers. For example, the maximum AveRooms (average number of rooms in the block) is 141.9, and the maximum AveBedrms (average number of bedrooms in the block) is 34. I can't imagine living in a house with 141 rooms and 34 bedrooms.
However, because these outliers may affect the model we create later, we decided to calculate multivariate distance (Mahalanobis distance) using the objective variable and explanatory variables and exclude data with large values from model creation.
We removed 17 data points in this process. If you remove these data points from the data table, the header graphs and statistics will be recalculated accordingly.
View prediction intervals in the Prediction Profiler
The report below shows what happens when you use Fit Model to create a model (hereafter referred to as a main effects model) that explains California housing prices (MedHouseVal) using the main effects of other variables (MedInc, House Age, etc.).
The estimated values in the "Parameter Estimates" report are the partial regression coefficients in the regression model, and the "Prediction Profiler" visualizes this model equation. It shows that the higher the average number of rooms (AveRooms), the lower the price, which may seem counterintuitive. However, here we will use the prediction equation we obtained.
Also, when predicting house prices for different blocks, it is a good idea to use prediction intervals to consider the range of possible values for the predicted value. In JMP 18, you can display 95% prediction intervals (the green lines indicate confidence limits) by selecting Prediction Intervals from the red triangle menu to the left of the Prediction Profiler.
If you want to use this prediction formula to predict the housing prices in another block in California based on the parameters of that block (income, year built, number of rooms, etc.), it is desirable to calculate a prediction interval as the range of prediction.
In JMP 18, you can display a 95% prediction interval (the green lines are the confidence limits of the 95% prediction interval) like the one shown below by selecting Prediction Intervals from the red triangle button to the left of the Prediction Profiler.
Additionally, in JMP 18 , you can view confidence and prediction intervals in the Profiler under the Graph menu.
Select Save Columns > Predicted Values and Interval Formulas as shown below.This will add columns for the prediction formula and confidence intervals (mean, individual) to the data table.
*The confidence interval column is set to "not display."
In this process, in addition to the main effects model, we also created a model that included main effects and two-way interactions, and saved the calculation formulas using the same procedure.
To compare the profiles of the "Main Effects Model" and the "Main Effects + 2-way Interaction Model", use [Graph] > [Profiler] and specify the prediction formula column you saved earlier as the [Y, Prediction Formula] column.
The profile report will be displayed. Click the red triangle button to the left of "Prediction Profiler" and select "Prediction Intervals" with "Confidence Intervals" unchecked, as shown below.
The profile above shows the predicted median home price and its 95% prediction interval for a block in California at longitude -122.23, latitude 37.88 with income 8, age 2, number of rooms 7, etc.
| model |
Predicted values and 95% prediction intervals |
| Main effects model |
3.91 [2.53, 5.29] |
| Main effects and second-order interaction models |
4.14 [2.80, 5,47] |
Comparing the two models, there are some differences in the predicted values and prediction intervals, but we can consider the predicted value for house prices to be around 4, with a range of predictions from 2.5 to 5.5.
by Naohiro Masukawa (JMP Japan)
Naohiro Masukawa - JMP User Community
