468953_en-US

liliconcom

Predictive modeling on noisy data

Hello,

does anyone have a good dataset to illustrate the benefit of JMP Pro vs. JMP on noisy data ?

Ideally we should have a variable(s) representing the "truth" and another variable representing "truth + noise" and being able to show that JMP is good enough to modelize "truth" while JMP Pro is needed to modelize "truth + noise".

note: I'm interested on either noise on Y(s) or Xs.

Thanks for the help.

Emmanuel

 

Emmanuel 

Re: Predictive modeling on noisy data

As Phil said it's great to have "VOC", thanks

Re: Predictive modeling on noisy data

I like this dataset. I will try to add noise on the response an see what's happen with ML techniques vs. traditional method.

I did not realized that Tablet is too small for this approach, thanks for letting me know.

Re: Predictive modeling on noisy data

Thanks, Joseph. It is great to get the "voice of the customer". That is what I have always said to prospective customers of JMP Pro: more confidence in your models with validation.

Re: Predictive modeling on noisy data

Hello Emmanuel,

As a customer I always found one of the benefits of JMP Pro to be less about building perfect models and more about having increased confidence in the models with the validation tools. Perhaps you could show in JMP how easily you could overfit models to get a false sense of accomplishment and then show how with JMP Pro and all the validation tools you could more easily identify overfit models and ones that might not generalize well? I suppose it is sort of the opposite of what you wanted but in some cases showing a lower R2 with Pro could be a selling point if you are getting an improved sense of the uncertainty and how well the models perform on validation and testing data. Of course all the extra modeling tools should allow you to build better models but I think most importantly you have more confidence in those models.

Joseph

Re: Predictive modeling on noisy data

I think the Tablet data set is probably too small for validation to be effective. 

Have you looked at this case study:

https://community.jmp.com/t5/Technical-Reference-Library/High-Throughput-Screening/ta-p/385091

I have not had time to rigorously test this. But from quick exploration it seemed that for even relatively small samples of the data (~150 rows), the machine learning methods with validation in JMP Pro outperform the standard models. 

 

Re: Predictive modeling on noisy data

Hello Phil,

Yes this is my goal. I've added random noise to :Dissolution in "Tablet Production" and used Model screening, unfortunately the outcome is disapointing (low r², inapropriate model). I'm considering using PLS that will maybe works better to manage noisy data (noise on multiple correlated responses) but that will not help me selling JMP Pro.

Emmanuel

Re: Predictive modeling on noisy data

Hi @Emmanuel_Romeu ,

I see that you have not got a response. It took me a while to understand what you are asking here. I think I understand now.

I think what you mean is a modelling problem with a simulated Y, and versions of this Y with different amounts of noise. You would expect that as the amount of noise increases, there would be a increasing difference in the performance of JMP vs JMP Pro models. That is, you would expect that with greater noise, models in JMP would overfit. Whereas JMP Pro validation should protect you from overfitting.

I don't have an example like this. It should not be that hard for you to create one.

I am not sure how compelling the difference will be though. In my experience, people don't buy JMP Pro because you show that  they can get an R^2 of 0.9 instead of 0.8.

Cheers!
Phil