Neural Network's Algorithm

IkjuShin

Hi Everyone

If you have some materials for JMP's neural network's algorithm like below, please let me know.

-whether is normalized or not in input

-The kinds of initialization and parameter setting

-Initialization method of W, b

-Learning rate setting

-Epoch No. setting

-how to check learning curve to visualize optimization process of Validation set

-default value of hyper parameter

jiancao

IkJu,

Chris Gotwalt has written an excellent white paper on the NN algorithm:

https://www.jmp.com/content/dam/jmp/documents/en/white-papers/wp-jmp9-neural-104886.pdf. I base my response below on his paper and JMP documentation.

(SEs: please feel free to chime in. ) 

Jian

 

-whether is normalized or not in input

There is an option called Transform Covariate under in the Model Launch window. Refer to page 5 for more details.

 -The kinds of initialization and parameter setting

“At the beginning, normally distributed random starting values are generated and the penalty parameter is set to zero. After this initial fit, a nonzero candidate value of the penalty parameter is then chosen, and a univariate line search on the penalty parameter is undertaken.” (Page 7)

 -Initialization method of W, b

Normally distributed random values (Pages 4 & 7))

 -Learning rate setting

The default is 0.1, but you can choose any value between (0, 1] in the Model Launch window.

 -Epoch No. setting

The number of tour can be set in the Model Launch window. It is One by default.

 -how to check learning curve to visualize optimization process of Validation set

In NN JMP doesn’t show the cumulative validation. A good suggestion as I’d like to see a report similar to the one in Boosted Trees.

 -default value of hyper parameter

These “default” hyper parameter values are displayed in the Model Launch window. For example, one layer, TanH with 3 nodes, 1 tour, etc.  I’d call the model using these settings a base model as the best practice is fit several models with different settings and then compare them.