Bootstrap Forest and Boosted Tree Evaluated Probabilities changes upon re-evaluation

Question

I have been trying out  predicitve modeling in JMP Pro to be able to classify observations based on historical data, i am aware that the model boosted tree and bootstrap forest results are always different if seed is not set . but once the model has been trained , and the formula has been generated , the results should not differ for the same set of input right?

I trained various different types of model(eg. nominal logistic , ridge regression ... etc) aside from boosted tree and bootstrap forest , saved formula into the dataset, save the dataset with the formula in it and closed it. when i opened it up a few days later i decided to run this script to re-evaluate the formulas

dt1 = Current Data Table();

dt1 << ReRun Formulas;

I saw that the evaluated probabilities changed just for bootstrap forest and boosted tree and not for the other models. is this normal? or its a bug.

Thanks to anyone who can help .

leeqk · Answer

Hi thanks for the reply.

i'm on JMP PRO 14.1.0  64bit Windows 10

none of my factors or repsonse contain any formulas or such they are pure data.

just a side note it only changes once , meaning it does not change to multiple different values each time i rerun the formula.

this only happens the first instance of re-evaluating the formula after reopening the file for the first time after saving.

For example

After saving prediction formula from model : Prob = 0.917825519141285

After opening for the first time after saving and re-evaluating: Prob = 0.901676834571581

2nd time  re-evaluating : 0.901676834571581 (same)

I suspect that something went wrong when generating/evaluating the formulas when saving the formula the first time.

gzmorgan0 · Answer

Without the actual data, this would impossible to debug.

* You should specify your JMP version and OS.
* I have used these methods many times and saved the prediction formulas; the formulas contain hard-coded conditions.
* Check if any of your model factors or responses include formulas or constants like :Date - Today(). in other words, the factor values change so the prediction changes.

Those are my suggestions, maybe a JMP developer has other suggestions.