How can income categorized data be normalized?

Question

Hello

I have income categorized data that I need to normalize to use it as a continuous variable. Data come from a willingness to pay survey where respondents selected one out of eleven income categories. For each category I have a number of observations.

As a first step I took the midpoint in each category to use it as the continuous variable. Literature suggests that income follows a log normal distribution. I took the ln of the midpoint and tested it. However, the Shapiro-Wilk test indicates that the data is not normally distributed.

Is there any way to normalize this data?

Also, literature (Hsiao, C., 1983. REGRESSION ANALYSIS WITH A CATEGORIZED EXPLANATORY VARIABLE, in: Amemiya, T., Goodman, L.A. (Eds.), Studies in Econometrics, Time Series, and Multivariate Statistics. Academic Press, pp. 93-129. - Hsiao, C., Mountain, D., 1985. Estimating the Short-Run Income Elasticity of Demand for Electricity by Using Cross-Sectional Categorized Data. Journal of the American Statistical Association 80, 259-265) suggests a two-stage method (pseudo-instrumental variable and conditional mean method) but I have not been able to implement this in JMP.

Would it be possible to use the upper and lower bounds of each category plus the number of observations in each category to obtain a normalized data?

My goal is to have two estimates. One is the simple midpoint of each category and another one is the normalized estimates. I would like to compare the normalized estimates with those from the midpoint in order to assess differences between those two methods.

I would appreciate an interactive explanation since I am new to JMP.

Thanks a lot!

Mark_Bailey · Answer

You can also fit models with predictors using the Generalized Regression platform in JMP Pro with censored responses.

cwillden · Answer

All real data fails those tests with a large enough sample size.  I think the old George Box adage about all models being wrong but some being useful applies just as well to distributional models.  A better question to ask about distributional fit is if it is a good enough model to get you a reasonable answer to your question?

What you have is interval censored data.  You can fit a distribution, or even models, to this type of data using the Reliability and Survival platforms in JMP.  For example, you can use Life Distribution to fit various distributions to your interval censored data and determine which one is the best fit.

Here I generated 300 lognormal(2, 0.5) data and binned the results with bin widths of 3.  The following is the counts in each bin.

I fit this in Life Distribution with Low and High as my response columns and Count as the Frequency column.

The data has a very good lognormal fit (as expected) with estimated parameters 1.975 and 0.43.

I also tested generalized gamma, Weibull, and normal fits.  I get a comparison table that shows lognormal is the best fit according to AICc and BIC.

From here, you could get pretty fancy determining the value to assign to each count within a bin.  For example, you could compute an expected value for the data within each bin using your distributional fit and assign that value to each of your counts for your "normalized" data.  An easy way to do that would be to simulate a really large sample from your fitted distribution and take the mean of the data within each bin.

The midpoints pretty much assume a uniform distribution within each bin, which would not be totally realistic.  However, I would guess the expected values within a bin using the fitted distribution won't be dramatically different from the midpoints (though I'm not sure what you do about the last open-ended bin).  I would be interested to hear how the results turn out.

FGordillo · Answer

hello dale_lehman

thanks a lot for your reply. your approach departs from simulated points and then establishes the bins. unfortunately, I don´t have that info. answers from respondents indicated an income category but they did not give a number as their income. also, income ranges were selected based on miminum salary brackets with an open interval at the end. at this point, I am considering using the standardized values (JMP function) of the mid point, Ln, and Log10 even if the normality tests fails.