256951_en-US

liliconcom

Regression using absoulte residuals

I have yearly precipitation extremes data (data in frequency, 0 to 15) from 1989 to 2019. My hypothesis is that climate extremes (precipitation) increased in last 16 years (2004 to 2019) than previous 15 years (1980 to 2003).

I have performed regression with this data and my results show that the regression line is downward. Then I performed regression using the absolute residuals and this time the results show that regression line of the absolute residuals is upward.

Am I doing right?

Why two kinds of results?

Which one is right?

Re: Regression using absoulte residuals

I am not sure that linear regression is the correct method to compare two time periods. Regression requires matching a X and Y value, like Precipitation versus Year for a trend. What is your response and what is your regressor?

 

How did you perform both regressions in JMP? Knowing the methods that you used will aid us in helping your with a correct answer.

 

If you are using counts as a response, then a Poisson loglinear regression might be more satisfactory. But I still don't see how you compared the two time periods. Just by comparing the parameter estimates? If so, there are more comprehensive methods for such a case. Assuming that the trend is linear over time, then you might have a linear predictor like Precipitation = constant + beta(1)*Year + beta(Period) + beta(2)*Period*Year. This way, you can test if there is an average difference between the two periods and if the trend is different between the two periods.

Re: Regression using absoulte residuals

Listen to Mark.  I have some other comments:

1. You are regressing what?  Extreme Precipitation vs. Time? Or are there other variables missing in your explanation?

2. Residuals (actual-predicted) is the delta between the prediction model and actual results.  If this is increasing over time, then the model may have an unequal variance issue (it falls apart as time increases).  Not sure why you would take the absolute value of residuals (OK, magnitude), but why wouldn't you care about  sign?  NID(0,variance) basic assumption of quantitative analysis.

3. Realize the regression model is dependent on what Y you are fitting, what terms are in the model and how you got your data. 

Re: Regression using absoulte residuals

Hi Mark,

Thanks so much. I have added a excel file which will show you how I have analyzed the data. Your answer will be highly appreciated.

 

Re: Regression using absoulte residuals

Thanks so much. I have attached a file containing data, and output files.

This will explain you how the data look like and I am sure you will be able to suggest me the right analysis to perform.

Re: Regression using absoulte residuals

1. You are comparing counts of occurrences of extreme precipitation during two periods of time. The permutation test is a good way to consider the observed difference.
2. A permutation test or ANOVA should answer your question. Other non-parametric tests are valid if you are concerned about any difference between the two periods.
3. The simple linear regression is testing if the mean response depends on the predictor Year.
   1. The trend is not significant because the apparent increase in occurrences over time is not unusual while assuming that there is no trend.
   2. The regression of the abs(residuals) versus the original counts is a completely different analysis. You cannot compare the two regressions.
   3. The second regression indicates that your response might be homoscedastic. There is not a mean trend, there is a variation trend.
4. Yes, the permutation test is appropriate for a simple comparison of the two periods.
   1. The statement based on the one-sided permutation test (counts are greater in the second period) is simply that it is estimated that 15% of samples had a difference at least as large as the observed difference.
   2. Where did the operational definition for counting come from? That is occurrence of more than 40 mm rain in 4 consecutive days.

 

I ran my own permutation test with the data that you supplied. I consistently get a different result. Here is one of them:

 

 

So my one-sided p-value would be 1185 / 2501 = 0.4738. That is, it is estimated that 47% of the samples produce a difference at least as large as the observed difference assuming the null hypothesis is no difference (basis for permutations).

Re: Regression using absoulte residuals

In addition, the assumption of linear regression is that the response is unbounded and may vary between negative infinity and positive infinity. Counts are bounded by zero. The regression he performed is not the best model.

Re: Regression using absoulte residualsWhy don't you attach the JMP file?Re: Regression using absoulte residuals

Hi Mark,

Thank you so much for your kind support and advise. The info you provided is very useful. I have done a couple of analyses and looking for some additional advise. Pleased see the attached file. I would appreciate your advsie. Regards

Re: Regression using absoulte residuals

Statman,

I have attached the JMP file. In my previous attachment, I have added a file with some analyses and questions in response of Mark's advise.

Thanks

Re: Regression using absoulte residuals

Your residual analysis indicates that the log Y is better suited to the regression because the variance is now constant. Do you know about the built-in Box-Cox transformation? This automated procedure is a generalization of the ad hoc power transformations. The spectrum of parameter values includes the log. A Box-Cox transformation parameter equal to zero is the same as the log. Now when I say the same, this transform does not yield the same value as the log or the power, but they may be interpreted as such. So a parameter value of 2 is interpreted as the square of Y and a value of -1 is interpreted as the reciprocal.

 

Caution: your wording (thinking) is a bit dangerous. I would not say that there is "no evidence" when the p-value is not below some subjectively chosen threshold (ɑ). I would say that there is insufficient evidence that I would reject the null hypothesis in the test.

 

Anyway, glad that you are able to progress with your analysis.

Re: Regression using absoulte residualsHi Mark,
Thanks so much for your support.
I just tried the Box Cox transformation, and gotten an error "Cannot do Box-Cox because of negative or zero Y value 0".
Can you please advise. Additionally, I wanted to make it sure that I apply Box Cox transformation on the original frequency data and not on residual.Re: Regression using absoulte residuals

Yes, I meant transform the original response. I forgot that you have some count = 0, so this approach won't work. It was just an alternate idea.

Re: Regression using absoulte residuals

Hi Mark,

I performed the permutation test again a few times. Please see the attached file. My results are totall different than yours, means, I am doing a mistake. I run permutation while I think you did bootstarap. Please advise.

Before I contacted you a Mathematics Prof suggsted me  using permutation than bootstrap. Therefore I used  permutationt. Also, I do not have an access of Bootstarap.

Also, how you got value of 1185 / 2501 = 0.4738.

Regards

Re: Regression using absoulte residuals

My result of 1185 / 2501 = 0.4738 is from 1185 samples out of 2501 total bootstrapped samples exhibited a difference greater than your observed difference.

 

JMP does not perform the permutation test as far as I know. How did you perform this test with JMP?

Re: Regression using absoulte residuals

First, I apologize. I misused the bootstrap in a previous post. Ignore the p-value that I stated previously.

 

The bootstrap on the estimated difference produces two-sided confidence intervals. Here is the report:

 

 

The bootstrap was done from the difference reported in the t-test that does not assume equal variance, but only that the sample means are normally distributed. This data is counts and skewed, but perhaps the CLT gives an approximately normal sum. Here is that report, along with the non-parametric exact tests.

 

 

I see general agreement between the confidence intervals and the two-sided p-values across these methods. The p-values range from 0.16 to 0.28. The non-parametric tests should have more power when the distribution assumption of the parametric test is violated, but they seem to suggest the same conclusion in this case.

 

JMP does not have a permutation test. You used an add-in provided in the JMP academic user area. I wrote my own. It produces a one-sided p-value of 0.07-0.08, which is close to the p-value for the upper-tailed t-test (0.08).

 

The Box-Cox transform is intended for strictly positive continuous responses. You could add 1 to every observation and then subtract it off from any predictions. But that is the point of the permutation tests and non-parametric tests. 

Re: Regression using absoulte residuals

Permutation Test for Two Means or Medians 

Hi Mark, I used this link to peform permutation. Thanks
Another questiosn for you - since some of the counts =0, therefore, I cannot use Box-Cox, what is alternative?

Re: Regression using absoulte residuals

Thanks so much,
So, I understand that  an add-in provided in the JMP academic user area (I have used) is similar to that you have written, right?
In my some of the location frequency data, out of 31 data points, 18 have zeros. So, in this kind of frequency data, no transformation can make the data normal. So, any clue, what method should be used? I have attached a JMP file. Thanks for your time.

Re: Regression using absoulte residuals

I am not familiar with the add-in that you are using, but I am sure that it was vetted before it was posted. The case of comparing two samples is so simple that I wrote a script, which I validated.

 

Why do you need to transform the data with the log function? What is the next step?

 

If you need to apply the log, then simply at a small positive number. When the response is on the order of 10^6, then I might add 1. If the response is on the order of 10 or 100, then I might add 0.01 or 0.001. I can always subtract the offset value if I back-transform the predictions from any model or analysis.

Re: Regression using absoulte residuals

Hi Mark,

Thank you so much. In the attached file I have some more questions for you. JMP file is also attached.

Regards

Re: Regression using absoulte residuals

First of all, why not enter your questions directly instead of putting them into an Excel workbook that we must open?

 

Second, there is not JMP data table. I could make one from the columns in the Excel workbook, but you do not seem to be using JMP. This Discussion forum is for JMP users.

 

Here is my answer to your questions.

 

1. Use of permutation test. I already answered this question previously.
2. Use of 'absolute' regression for non-normal data. I already answered this question previously.
3. Compare spread of two periods of time. I already answered this question previously.
4. Suggest another method. I already answered this question previously.

 

We seem to be 'going around in circles.'

Re: Regression using absoulte residuals

Mark,

Thanks for your reply. I use JMP on a regular basis. Sorry, my mistake that I forgot to attach JMP file. I have attached it now that shows that I do use it:)

The dept. I work with have three licenses/users and they are frequent users.