189860_en-US

liliconcom

Bootstrap Forest and Boosted Tree Evaluated Probabilities changes upon re-evaluation

I have been trying out  predicitve modeling in JMP Pro to be able to classify observations based on historical data, i am aware that the model boosted tree and bootstrap forest results are always different if seed is not set . but once the model has been trained , and the formula has been generated , the results should not differ for the same set of input right? 

 

I trained various different types of model(eg. nominal logistic , ridge regression ... etc) aside from boosted tree and bootstrap forest , saved formula into the dataset, save the dataset with the formula in it and closed it. when i opened it up a few days later i decided to run this script to re-evaluate the formulas

dt1 = Current Data Table();

dt1 << ReRun Formulas;

I saw that the evaluated probabilities changed just for bootstrap forest and boosted tree and not for the other models. is this normal? or its a bug.

 

Thanks to anyone who can help .

Re: Bootstrap Forest and Boosted Tree Evaluated Probabilities changes upon re-evaluation

Without the actual data, this would impossible to debug. 

• You should specify your JMP version and OS.
• I have used these methods many times and saved the prediction formulas; the formulas contain hard-coded conditions.
• Check if any of your model factors or responses include formulas or constants like :Date - Today(). in other words, the factor values change so the prediction changes.

Those are my suggestions, maybe a JMP developer has other suggestions. 

Re: Bootstrap Forest and Boosted Tree Evaluated Probabilities changes upon re-evaluation

Hi thanks for the reply.

 

i'm on JMP PRO 14.1.0  64bit Windows 10

 

none of my factors or repsonse contain any formulas or such they are pure data.

 

just a side note it only changes once , meaning it does not change to multiple different values each time i rerun the formula.

this only happens the first instance of re-evaluating the formula after reopening the file for the first time after saving.   

 

For example

After saving prediction formula from model : Prob = 0.917825519141285

After opening for the first time after saving and re-evaluating: Prob = 0.901676834571581

2nd time  re-evaluating : 0.901676834571581 (same)

 

I suspect that something went wrong when generating/evaluating the formulas when saving the formula the first time.