column contribution missing in bootstrap forest when summary view enabled

Question

Hi All,

I came across an unexpected behavior the other day as a result of defining some preferences, it took me a while to figure out what was driving the column contributions to not display.  It turns out that if you end up specifying a summary view as seen below, column contributions will not show.

Turning the report view to 'Full' will bring back the column contributions.

I was hoping to have just the visuals in the summary view (including column contributions) so that customers who are new to decision trees are not overwhelmed by stat details and instead are first presented with some visual assessments of fit and variables of significance (thinking of a current PG&E pro prospect).  At the end of the day I can always write a quick script to close the dialog boxes.

Has anyone else ran into customers feeling taken aback by the BF default output?

martindemel · Answer

Hi Nate,

I agree with Peter. BF is a statistical advanced method, so it does not hurt to have stats tables to see. However it is what you will talk about. If you have validation column you might want to say a word about the Validation R^2 (or adj. or confusion matrix if categorical response) but then I would go directly into column contributions and if you like the ROC and others, like Actual vs. Predicted.

I highlight the benefits of BF mainly: For holistic DOE analysis I highlight getting the right potential significant factors from historic data. For modeling to get a benchmark model I want to compare against other more simple models I create. If all tell the same story and are similarly accurate you can be more confident to go with a simpler model. If there is a difference in the story you want to dig deeper into your data or augment a design to be sure you understand the process fully.

I would always teach the audiance on comparing differnt models to see if all tell the same story or not, even if BF is great and you could be more confident for finding the right factors. Only then you can be more confident that you have the correct final understanding of the process or system behaviour you are looking at. As well you get an idea how linear the system can be described. May you have to rethink your understanding fo the system.

my bit more than 2cents ;)

Peter_Bartell · Answer

Nate: I think you raise a valid point...which is why when I'm demo'ing ALL the decision tree reports I don't harp too much if at all on summary tabular statistics...after all JMP's hallmark is visualization, not tabular reports. So I mention the tabular stuff in passing VERY, VERY quickly and jump to the the actual vs. predicted plots for continuous responses, ROC curves and confusion matrices for categorical responses. So for me it's more about demo style and the key practical questions that are being addressed. I'm often in demo'ing bootstrap forest within the Holistic DOE framework...who cares about summary statistics anyways? We're after:

1. Key factors for consideration in inclusion in a subsequent designed experiment. Column contributions works here.

2. What levels for these factors should we consider? The Profiler works here.

3. What can we learn about the functional relationship between the factors and responses? The Profiler works here as well.