37318_en-US

liliconcom

How can I obtain the BIC and AIC form -log likelihood for neural network ?

Hello,

 

I use the JMP pro 12.  

I would like to know, how can I obtain the AIC or the BIC from -likelihood, in neural network analysis?

 

Regards, 

Angelo

 

Re: How can I obtain the BIC and AIC form -log likelihood for neural network ?

Sorry I was pretty sure I'd done it for neural nets, but it was actually for the nonlinear platform.  Here's the code:

 

dt = Open("$SAMPLE_DATA/Nonlinear Examples/Chemical Kinetics.jmp");
fit = dt << Nonlinear( 
 Y( :Name( "Velocity (y)" ) ), 
  X( :Name( "Model (x)" ) ), 
  Newton, 
  Finish   
);
rep = fit << Report;
matRmse = rep[NumberColBox(5)] << Get As Matrix;


/***************************************************
/          calculate AICc                          /
***************************************************/

 // AIC = N.ln(SS/N) + 2k
 // where N = # data points
 //   k = # parameters + 1
 //  SS = sum of squares for the residuals
 
 // AICc = AIC + [ 2k(k+1) / (N-k-1) ]
 

N = NRows(dt);
nParameters = 2;
k = nParameters + 1;
rmse = matRmse[1];
dfSS = N - nParameters;
MS = rmse^2;
SS = dfSS * MS; // this is correct compared to fit model
LL = N*Log(2*Pi()*e()*SS/N);
AIC = LL + 2*k;
AICc = AIC + ( (2*k*(k+1))/(N-k-1) );

/***************************************************
/          calculate BIC                           /
***************************************************/
BIC = LL + k*Log(N);

In this code I had to determine the log-likelihood value whereas the neural net gives you this so it should be easier.  I think you just need to think about what is the value of k.  I assume it would be 1 plus the number of weights that are being estimated (for a single layer network that would be #factors x #nodes); 

Re: How can I obtain the BIC and AIC form -log likelihood for neural network ?Dear David,

It is good for me. I will wait Monday.

Thank you so much,
Angelo
Re: How can I obtain the BIC and AIC form -log likelihood for neural network ?

I have some code that I can share with you to do this - unfortunately I'm travelling at the moment - if you don't get a reply from someone else I'll be able to post it on Monday.

Re: How can I obtain the BIC and AIC form -log likelihood for neural network ?

Is there a way to get k just from a fast column formula for a neural? Or do I have to know how the model was created? For instance

 

Names default to here(1);
dt = open("$SAMPLE_DATA/Boston Housing.jmp");
nn = dt << Neural(
 Y( :mvalue ),
 X(
  :crim,
  :zn,
  :indus,
  :chas,
  :nox,
  :rooms,
  :age,
  :distance,
  :radial,
  :tax,
  :pt,
  :b,
  :lstat
 ),
 Informative Missing( 0 ),
 Validation Method( "Holdback", 0.3333 ),
 Fit( NTanH( 3 ), NLinear( 2 ), NLinear2( 1 ), NGaussian2( 1 ) )
);

nn << (Fit[1] << Save Fast Formulas);
pred = Column(dt, ncols(dt));
pred << Get Formula();

Returns:

 

T#1 = Matrix(
 {{:crim, :zn, :indus, Design Nom( :chas, [0 1], <<ELSE missing=""></ELSE>

 

Thanks in advance.