I have a customer doing a JMP Pro evaluation and they came up with a good question. Looking at the Diabetes data set, they ran a neural network on the response Y as a function of the factors: LTG, BMI, BP, HDL, Age, Glucose, LDL, TCH, Total Cholesterol, and Gender. The profiler shows that the effect of gender is small compared to the other factors:
From the profiler, the effect of varying Gender on Y is smaller than, say, BMI.
However, looking at the variable importance results, we see that Gender is at the top of the list:
Looking at the scatterplot matrix and the color map on correlations, using the dependent resampled inputs makes the most sense since the factors are correlated (as suggested in the Help).
The question from the customer is this: Why does the profiler show Gender having a small effect vs. the Variable Importance result which shows Gender having the largest effect?
The customer also asked what sample size was needed for the use of the Dependent Resampled Inputs results to be valid. The Help mentions that it doesn't work if the sample size is "too small" but doesn't say how small is small.
Checking the Statistical details for the Variable Importance says that it uses the Variance of the expected value of the response but doesn't give much detail. I assume that the difference between the profiler result and the Variable Importance results has something to do with the profiler using the expected value of Y and the Variable Importance result using the variance of the expected value of Y. However, I'd like more detail so I can explain it to the customer.
thanks!
Charlie
