nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Guide for neural net nodes: inner node(s) versus hidden and outer node(s) ?

Laura_Higgins

I'm helping a customer with a Pro eval and we are going through the "Fundamental of Predicitve Analytics with JMP" second ed. On page 252, they give guidelines for determining the number of nodes. JMP can have the input layer (X's), one or 2 hidden layers, and the output layer (Y). Their recommendations consider the number of output nodes for categorical variables:

"The number of nodes in the output layer depends on the problem. To predict a single continuous or binary variable, use one output node. For a categorical variable with five levels, use five output nodes, one for each level."

They then go on to talk about rules of thumb with input and output node numbers.

So my question is this: Does the user just have to know how many levels they have for a categorical Y to determine the number of nodes? There doesn't seem to be any information present in the neural report in a obvious way, if what I've said above is true. Am I missing something? For example the diagram always has one box for the Y. We see the levels in the profiler as seperate profilers, but that's still just based on the number of levels in the category (so it's implicit but not stated).

Accepted answers

DonMcCormack

Each node in a NN corresponds to a linear function of the inputs transformed with the nonlinear activation function. It makes sense to have one node for each Y level, since any shift or discontinuity associated with the response level can be individually modeled (or, probably more accurately, the contribution from the level can be modeled using a separate node). On an unrelated note, the fitted model will only be able to predict for the levels of Y that are in the model (regardless of the number of nodes used).

All comments

DonMcCormack

Peter_Bartell

Laura: I can't offer any guidance on your original question, but will offer another thought regarding an eval that might resonate with the prospect? These models can be pretty obtuse with lots of terms, transforms, etc. Some of my customers export these models to a production system like a data warehouse for categorization of say loan applicants, upselling services/products, etc. They see the Formula Depot Publish to C, SQL, Python, etc. as something that eases the pain of 'getting the model out of JMP Pro' into some other destination system as a nice capability. Hope this helps.

Laura_Higgins

Thanks Peter- Yes showing the generated C code actually helped explain a lot of what was happening - from dummy variables to how the formulas are being translated. It's a trick worth repeating!

Laura_Higgins

It sounds like it would be helpful if JMP suggested the correct number of nodes for a categorical Y in neural! Or at least the max.

jiancao

Laura,

That part of the guidelines about the output nodes that you quoted from the book is misleading. JMP doesn't ask the user to specify the number of nodes for output layer; it does it automatically per the modeling type of the response variable. That is, for a continuous response, NN makes a numerical value prediction. For a nominal or ordinal response NN makes a set of probability predictions, one for each response level. The remainder of the advice as stated in the book is to guide the user in choosing the number of hidden nodes in a layer.