comparison of slopes<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />After running an ANCOVA model with more than two values of the categorical variable, how can I perform pairwise comparisons of slopes?
Hillary.
Re: comparison of slopesIn recent JMP versions, the Fit Model Platform also has a "Compare Slopes" report option under Estimates under the top red triangle in the Fit Model output.
This report can be used to statistically compare each of the individual slopes (corresponding to each Level of the categorical factor) to the overall average slope, using the Analysis of Means (ANOM) construct in JMP.
Notice how in this example, since none of the three slopes exceed the decision limits of the test (UDL, LDL), we can assert that none of the slopes are statistically significantly different than the overall average slope; therefore, we can infer that none of the slopes are different from each other. This conclusion squares up nicely with what we can assert when conducting pairwise comparisons to compare the slopes as detailed above.
For example, when comparing Japan to USA, we can assert that we do not have evidence to reject the null hypothesis that the slopes are the same, given the Pairwise comparisons test p-value obtained = 0.9985. We can make similar statements when comparing Other vs USA, and Japan vs Other (where p= 0.2193 and p=0.2204, respectively).
Also notice that the "Group Mean" as detailed in the 'Comparisons with Overall Average Summary' header at the bottom of the report is just the slope corresponding to the particular level of Country. For example, the slope corresponding to Japan is 155.5177 and that corresponding to Other is 113.0713. Japan - Other = 155.5177 - 113.0713 = 42.4464 which rounds to 42.45, identical to the analysis above.
Finally, notice that the confidence limits on the observed slopes are indicated by "Lower Limit" and "Upper Limit" respectively. We can graph the observed slopes and their confidence limits in JMP using Graph Builder after right clicking to make into a data table.
By dragging Upper Limit and Lower Limit into the Limits role in Graph Builder, Group Mean into the Y role and Level into X role, we obtain the following graph which is just a graphical representation of the table:
Since all three confidence intervals overlap with each other, we can assert based on the relation between confidence intervals and tests of significance (see Ch. 21 of Experimental Statistics by Mary Gibbons Natrella from the National Bureau of Standards), that we do not have evidence that the Slopes are different from each other (at the 0.05 alpha level, since these limits are constructed with 95% confidence).
Re: comparison of slopesJulian,
I find this post very useful. However, I am struggling with a question. If the categorical variable has many levels for example 18 in my case, JMP is doing a pairwise comparison of the slopes. Is there anyway to get a multiple comparison through Tukey or other tests to avoid the big error rate by having pair wise comparisons of slope. Thank you
Re: comparison of slopesExcellent! I'm so glad we figured this out
Re: comparison of slopesHi Julian,
Brilliant! The add in now works when the categorical variable has numerical
values.
Many, many thanks,
Hillary.
From: julian
Sent: Thursday, June 02, 2016 8:55 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Hillary,
That helped! I think I tracked down one issue and just updated the add-in (v0.05). What happened there is there was only a partial list of levels in the value ordering since you added a level after the ordering had been set. In the new version of the add-in the initial settings for value ordering aren't taken from the column property but are found by making a list of every unique level of the categorical factor. This should be more robust to these issues. Hopefully this is one curiosity settled... let me know if this now works for you.
Julian
Re: comparison of slopesHi Julian,
There wasn't anything in the log. I am attaching the adapted data set, the
Fit Model output and the results after running the add-in.
Thanks,
Hillary.
From: julian
Sent: Wednesday, June 01, 2016 9:20 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Hillary,
Curiouser and curiouser, indeed! I haven't been able to crack this yet, nor can I get this exact behavior to replicate on my side (on mac, or windows 8). It's especially odd that we're getting inconsistent failing! It might help to see what error messages you are getting reported to the log (View menu > Log). It will be most helpful if you clear the contents of the log before running the add-in (Edit > Clear), and then after running the add-in, if it fails, save the log to a file and attach it here. That should give me some sense of where things are failing.
Julian
Re: comparison of slopesHi Julian,
In addition to my last letter:
I tried adding a fourth category to your test data set. When I ran the
add-in I only got results for categories 1, 2 and 3 as reference
categories. Could you use this information to solve the puzzle?
Hillary.
From: julian
Sent: Saturday, May 28, 2016 8:15 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Julian,
Curioser and curioser.
1. Your data set worked perfectly.
2. On my original data set I checked the Value Ordering and found
that is was ordered naturally.
3. I tried removing this ordering, but with or without it I don't get
the full set of comparisons. In some versions I don't get any output from
your add-in and in some I get only the standard reference category.
4. Tried another numerical variable which I defined as Nominal: ditto.
5. Created a new, small data set of my own. Again, the numerical
categorical variable gave me only a single reference category. This was the
same whether I defined it as Character-Nominal or Numerical-Nominal.
6. I am using JMP 12 on Windows 7.
Can you help?
Thanks,
Hillary.
From: julian
Sent: Saturday, May 28, 2016 8:15 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHow interesting! We'll get to the bottom of this. Here are a few things to try:
- On the dataset that is failing, check to see if value ordering is set for your factor (the numbered column)--Right click the column > Column Info, then see if there is a value ordering set under the properties. If Value Ordering it set, does it appear to be blank? Delete that property and try the add-in again. If it now works I think I know what was going on.
- If that doesn't work, do you have another dataset with similar properties that you could try? I'd like to test the possibility that it's something specific to the dataset you're working with.
- If you don't have another data set with this arrangement, would you try with this test data set I've attached here? This has the same arrangement (numeric but nominal factor) and works on my side on both windows and mac.
Re: comparison of slopesHi Julian,
I'm afraid v04 gives the same output as the previous versions for numbered
categories.
Hillary.
From: julian
Sent: Friday, May 27, 2016 9:23 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Hillary,
I just uploaded a new version (v0.04) that should be more robust with numbered levels.
Julian
Re: comparison of slopesHi Julian,
It seems to be the values that are problematical. When I give the
categories names a1, a2, etc. I get the expected output.
Hillary.
From: julian
Sent: Friday, May 27, 2016 12:09 AM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Julian,
I used v03 but got the same output as before.
Hillary.
From: julian
Sent: Friday, May 27, 2016 12:09 AM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Hillary,
I believe I fixed the issue. Download v0.03 and reinstall and please let me know if you get the expected output.
Compare Simple Slopes in One-Way ANCOVA (w Interaction) add-in
julian
Re: comparison of slopesHmm, looks like the script is failing somewhere if that's the output you get. I suspect it might have to do with having numbers as the levels of your categorical factor of Status. When the script looks to make a list of those to shuffle the value ordering it appears to take them in as a matrix not a list. I'll find a workaround for this and post a new version.
Re: comparison of slopesHI Julian,
I see you've uploaded the second version of the Add In and it's working
better. However this is the table I get:
which is different from your description of the Add In" the reference
category isn't listed (should be Status 4) and the table doesn't continue
for other reference categories.
Thanks,
Hillary.
From: julian
Sent: Thursday, May 26, 2016 9:47 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesYes indeed! Thanks for the tip about add ins. However: I installed the Add
In as you instructed and ran the Fit Model. When I click on Add Ins and the
name of your add in I get a message "Select a Fit Model Least Squares
Report". How do I do this?
Hillary.
From: julian
Sent: Thursday, May 26, 2016 9:47 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesAfter some digging...the ANOM in the nonlinear platform appears to be comparing the estimates to the "parallel" model where the model for each group has the same shape but is shifted along the x-axis. So in the case of the example we have been looking at the center line for the slope ANOM of 144.71 is the slope of the model with the same slope but different intercepts for each group.
Re: comparison of slopesAdd-ins extend the functionality of JMP using the built-in scripting language, JSL. Download the add-in from the page, and when you open it JMP will ask you if you wish to install the add-in. Click yes, and you will see a new menu appear called Add-ins. From that menu you can now select the add-in I wrote.
You'll find many other useful add-ins in the file-exchange section of the user community. It's worth a look around!
Julian
Re: comparison of slopesThanks, Julian. I'm not familiar with using add-ins.
Hillary.
From: julian
Sent: Thursday, May 26, 2016 9:12 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesHi Hillary,
I hope you saw my previous post with the add-in to take some of the tediousness out of rerunning the report. Here's that link again if not:
Compare Simple Slopes in One-Way ANCOVA (w Interaction) add-in
Julian
Re: comparison of slopesThank you, Karen and Julian, On the Fit Model platform the parameter
estimates table also gives a t-test for each slope as compared to the
average slope. Or one can use the indicator function parametrization and
get a comparison of each slope to that of the last category. But other
specific pairs can only be compared by changing the order of the categories
and rerunning the model: very tedious if you have, say, 10 categories.
From: julian
Sent: Thursday, May 26, 2016 7:27 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from Julian Parris
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesGreat suggestion, Karen! That's certainly the most direct way to produce those analyses when a comparison to the overall/mean slope is needed.
Edit: Another option if we're after comparisons to the average slope (even though I prefer ANOM) is requesting Expanded Estimates from the original Fit Model report (Top Red Triangle > Estimates > Expanded Estimates). Expanded estimates show all parameters for all levels of the nominal factors:
Unlike the Indicator Parameterization estimates these are all with respect to the average response. "Turning Circle" here is then the average/overall slope, and the estimates for the interaction parameters are the differences in slope for each region compared to the average.
Interestingly... some of these estimates are ever so slightly different than the ANOM in the non-linear platform. The given estimate in the ANOM table for the slope of Japan is 155.5, which is indeed equal to what we recover from the expanded estimates report in Fit Model: 141.35+14.16 = 155.51. However, in the ANOM report, the mean slope is listed/drawn at 144.71, which isn't correct. (155.52+113.07+155.46)/3 = 141.35.)
Re: comparison of slopesAnother option for comparing slopes across groups is the non-linear platform (which has a linear fit option). Using the car data as Julian did if you have weight as the Y, Turning Circle as the X and then Country as the Group and then under the Fit Curve>Polynomials>Fit Linear you will get a linear fit for each group:
Then under the Linear red triangle select "Compare Parameter Estimates" and you will get a comparison of the intercepts and slopes (note they have been estimated for you in the table shown above). This isn't exactly what you were looking for as the comparison of the slopes is done by ANOM so the comparison is to the overall slope: does the slope from any group differ from the overall slope? is the question under consideration with ANOM.
Re: comparison of slopesYou're correct. I've misunderstood the question and showed the pairwise comparison of the estimated effects of the categorical variable.
Re: comparison of slopesHi Hillary,
You're welcome! It looks you replied to my post via email so I hope the screenshots came through.
Let me tackle your questions first:
- Difference between regular parameter estimates and the indicator parameter estimates: This has to do with how JMP parameterizes categorical variables. There are two ways to do this: effect coding, and indicator coding. Effect coding, common in designed experiments, models the parameters such that the estimates are with respect the grand mean across all groups. So, the regular parameter estimates list a difference from average response. Indicator coding means that estimates reflect a difference from some baseline or reference group. Both types of parameterization are useful at times, but effect coding, the default in JMP, has better properties for more complicated models and is thus prefered.
- The naming of the interaction terms with subtracting a constant: this is called centering, or using a centered polynomial. That constant is the mean of that variable, and in constructing the interaction term the cross product between variables is found my multiplying not the original values of the variable, but the value of the variable after subtracting the mean. This also has desirable properties for complicated models, and has the useful effect of keeping lower order terms in the model more interpretable. In short, the parameter estimates of lower order terms, say of Japan vs USA in terms of weight, indicate how much those countries differ for cars of average turning circle. Without centering of the turning circle variable, that lower order term would indicate the estimate when cars have 0 turning circle radius (which doesn't really make any sense). For more information about centering a polynomial and its consequences, see my response and video on this post: estimates in multipule regression
- There are a few ways to get the slopes for each group. Using the indicator parameterization table you can just look at the estimate for your continuous variable, and that will be the estimate for the slope of the reference category. Another way is using Analyze > Fit Y by X. Set this up with just your continuous Y and continuous X, click okay, then once you have the output go to the Red Triangle > Group by, and then select your categorical variable. Now, whatever action you perform will be done separately for each X. Go back to the Red Triangle > Fit Line. You will now have as many linear fits as you have levels of your categorical X. You can read off the individual slopes there.
As for your comment, I agree! After I posted my response I started thinking about how utterly manual this is! If you have many levels of your categorical X this would be a bit tedious. So, I worked on an add-in to automate this: Compare Simple Slopes in One-Way ANCOVA (w Interaction) add-in
I hope this helps!
julian
Re: comparison of slopesHi Julian,
Thank you for your detailed answer. I really appreciate your going through
the stages of the explanation one by one. One comment and one further
question:
If the categorical variable has more than three levels this means changing
the ordering of the values as many times as necessary to cover all the
pairs of comparisons. This isn't difficult, but it would be nice if JMP
could build this into the menu as a single step.
I didn't understand the difference between the regular parameter estimates
and the indicator function parametrization estimates. Nor do I understand
the role of the constant (38.5862) subtracted from the continuous
predictor (turning circle) in obtaining the estimates. When using SAS and
the ESTIMATE statement in PROC GLM one can get an estimate of the
regression slope for each value of the categorical variable. How can I get
these values in JMP?
Thanks again,
Hillary.
From: Julian Parris
Sent: Wednesday, May 25, 2016 12:44 AM
To: hillaryvoet
Subject: You have been mentioned by Julian Parris in Re: comparison of
slopes in JMP User
JMP User Community
<>
You have been mentioned
by Julian Parris
<> *in Re:
comparison of slopes in JMP User* - View Julian Parris's reference to you
<>
Re: comparison of slopesHi hillaryvoet ,
If I'm understanding your question correctly, you have a regression with a categorical variable (X1) and continuous variable (X2) and the interaction between them predicting a continuous variable (Y). Your categorical variable (X1) has more than two levels, and you have a statistically significant interaction between X1 and X2 and would like to follow-up on that to see for which groups the regression of Y on to X2 differs. The methods listed by others here don't seem to be testing the pairwise differences in slopes, but rather the pairwise difference in means, assuming a mean level on the continuous regressor (X2). There is a way to get the tests you want, but it'll take a few extra steps. I'll use the Car Physical Data sample dataset for this example.
First I will fit a model like you're working with. I predict Weight on the basis of Country and Turning circle, and their interaction.
In the regression plot we can see directly what you're asking for, three tests of the differences in slopes between each pair, Japan/Other, Japan/USA, Other/USA.
Tests produced under Effect Details for the factor of Country, (for instance LSMeans Differences Student's t), are not tests of slope differences, but are tests of whether the mean weight of cars differ for those countries assuming the mean level of turning circle.
To reveal a test of slopes, we will go to the top-most Red Triangle > Estimates > Indicator Parameterization Estimates
This will show a table of parameter estimates based on an indicator parameterization (as opposed to effects coding, which is what the regular parameter estimates table is based on) which means the LAST level of your categorical factor is used as a reference category, and estimates are differences from that reference category. This will be useful for us.
Notice that we see no estimates with USA. That's because USA is our reference category, so all estimates are differences from what was estimated for the USA. Country[Japan], with the estimate of 256.78, is how much Japan differs from the USA in terms of weight on average. Notice that is the same thing we saw above in the ordered differences report table.
If we look at the interaction parameter estimates we can use the same logic: Country[Japan]*(Turning Circle-38.5862), and the associated estimates, is how the slope for Japan differs from the slope for the USA (by the way, the slope for the USA is listed under "Turning Circle" since it's the reference category, so 155.46 is the USA). Notice that the slope for Japan is nearly the same as the slope for the USA -- different by only a value of 0.06, p = 0.9985. The slope for Other differs from the USA by -42.38, p = 0.2193.
So we're almost there. We're missing one test, Japan vs Other. To get this we need to make JMP treat Japan, or Other, as the reference category, and then rerun this analysis. I'm going to choose to treat Other as the reference category. To change the reference category we'll change the Value Ordering property for the column Country. Right click on the column in the table and go to Column Info. Click on Column Properties, and then select Value Ordering. To change the ordering, select "Other" and click Move Down. The last category listed will be the reference category. Click okay when you're done.
To rerun the model, go to the top-most red triangle of your existing Fit Model output > script > Redo analysis.
The "Indicator Function Parameterization" now lists the same analyses we saw before, but with "Other" as the reference category.
And now we have our last test, the slope difference between Japan vs Other, which we find in the row "Country[Japan]*(Turning Circle-38.5862)". A difference of 42.45, p = 0.2204.
So we have our three pairwise slope tests.
Japan vs USA, difference = 0.06, p = 0.9985.
Other vs USA, difference = 42.38, p = 0.2193.
Japan vs Other, difference = 42.45, p = 0.2204.
I hope this helps!
julian
Re: comparison of slopesRegression parameter estimates from regression : (I assume the estimates on age groups are what you want to test)
The results from running Multiple Comparisons:
Comparing the pair Age 14 to Age 16: the difference in slope: 2.3656086-0.2714001=2.0942085
Re: comparison of slopesThe multiple comparisons test is giving you a comparison of the Least Square Means, but that is the same as the slopes. Consider a case with 2 levels of your categorical variable (let's call it Q) and your continuous variable is called X. Your model is Y = B0 + B1*Q + B2*X + B12*Q*X.
Suppose I fix Q at the first level, 1. My model simplifies to Y = B0 + B1*(1) + B2*X + B12*(1)*X ==> Y = (B0+B1) + (B2+B12)*X.
Notice that the B12 interaction parameter is the adjustment of the slope for the different levels of the categorical variable. Therefore, testing to see if the least squares means of the interaction terms are different is equivalent to testing if the slopes are different. If B12 for level 1 of the categorical variable is different than B12 for level 2 of the categorical variable, then the slopes for those two categories are different.
Re: comparison of slopesThe ANCOVA is performed by running a model with a categorical variable, a
continuous variable, and â in the case I'm interested in â an interaction
term which tests for equality of slopes. Using Estimates â Multiple
Comparisons gives the differences and significance between the Least Square
Means, notbetween the slopes.
From: jiancao
Sent: Tuesday, May 24, 2016 9:15 PM
To: Hillary Voet
Subject: Re: - comparison of slopes
JMP User Community
<>
comparison of slopes
reply from jiancao
<> in
Discussions - View the full discussion
<>
Re: comparison of slopesIf you use Fit Model's Standard Least Squares for ANCOVA, select Estimates => Multiple Comparisons option from the top red triangle for pairwise comparisons. For more info check out the documentation by scrolling down to All Pairwise Comparisons The following screenshot shows Tukey's all pairwise comparisons for age groups
Re: comparison of slopesI think this kind of analysis (parallel slopes) is consistent with a 'proper' ANCOVA. Julian talks about that in one of her videos here:
https://youtu.be/k3zk45WrmUI?t=1400
Re: comparison of slopesAs part of the JMP 19 release cycle, the Compare Slopes report has been updated to include the ability to compare slopes directly in a student's t or 'multiple comparisons corrected' (e.g. Tukey HSD) framework:
As we can see here, the results presented using the Student's t submenu option (in the JMP 19 EA 5 release, current as of the time of this writing) match what was originally summarized using the method that @julian patiently walked through in the original solution to this post.
