Weighting Neural Nets?

Question

A customer posed a question to me regarding neural nets:

"I am doing model fitting using neural networks. I would like to use 'weighting' to minimize classification error for certain situations. How do I use 'weight' in JMP? There is a 'Frequency' option in JMP neural network modeling . Is that anything to do with weighting? Any help would be really appreciated."

Admittedly, my understanding of the workings behind neural nets is very basic, and I'm a bit at a loss here. I understand weighting with respect to regression models just fine, but when it comes to neural nets. Can anyone provide me any help or guidance here?

Regards,

Erich

DonMcCormack · Answer

Why not use the profit matrix column property to place costs on incorrect classification? Personally, I wouldn't use the weight, since it will be analogous to duplicating rows.

mia_stephens · Answer

Hi Eric,

Sounds like the customer is dealing with an unbalanced categorical response variable, where the target category is underrepresented.  I tend to stay away from Freq and Weight for models (especially Neural and Partition models).  One general way of dealing with this, if there is enough data, is to sample from the larger data set, selecting all of the observations in the target category and the same number of observations in the other category.  The model is built on balanced data, and the confusion matrix reflects this.  The probabilities are adjusted after the fact to reflect the true probabilities.

There is an add-in that does this, which also handles the balancing of the validation set if desired.  This allows you to build the model on a balanced training set (and validation set), and then an unbalanced test set is used for model comparison and selection: Stratified Data Partitioning (with balancing options) add-in.

Another approach is to explore different cutoffs for the confusion matrix for these situations.  There's an add-in for this as well:  Alternate Cut-off Confusion Matrix Add-In.

These approaches are discussed in Shmueli, Patel and Bruce, Data Mining for Business Intelligence.  The first approach is discussed in Data Mining Techniques by Linoff and Berry.

FYI - we've put a collection of these handy tools at: Collection:  Data Mining Tools Add-Ins

Hope this helps!  Happy to discuss if you'd like.

Mia

martindemel · Answer

Hi Eric,

I won't consider me as an expert in NN but as far I understand them there are different ways of weights in neural nets. The one is having the Penalty function being weighted because you think few of the many factors contribute more to the predictive ability of the model. This could be done in JMP via Setting the Penalty function to absolute or weight decay (Overview of Neural Networks)

What I understood from your description he is interested in weighting the Inputs already for the Transfer function to build artificial neural networks. From my understanding (and I'm happy if someone knows this better than me and share his knowledge ) we do not add weight for the Transfer function used by the activation function. Hmm, although you may achieve kind of weights with the frequency variable in case - under the hood - this is used to add more inputs than just one to simulate a higher importance. But there I'm out of confidence, as I cannot look into the implemented algorithm.

Hope this is of some help. However I'd like to see so who confirms what I've said

Martin