Non linear Performances

Question

Hi,

I have a customer who use regularly non linear platform.

He defines an equation with 16 parameters and want to estimate them on a relativly big data (106300 rows). It can take around 3hours to  do the estimation which is quite long. 
He has a 4-cores computer but only one is used during the process. 
Except the RAM, does anyone has any advice to improve time issue for his case? and adding RAM will really have good impact in this case?

By default, he is using the Newton method. I'm not sure that changing this option can be useful.

Thank a lot for your suggestions,

Florence

PS: I wish to everybody a good festive break!

martindemel · Answer

I agree with nick‘s approach, especially the starting points and the algorithm used. The starting point is incredibly important when using newton. And even then you might just come up with a local solution based on the method behind. Also Newton has its difficulty if you have a highly nonlinear behavior with multiple turning points then it might jump from one optimum to the other and has a hard time to converge. BFGS or Quasi Neeton should perform better. And they will result more likely in a true optimum . He should give those two suggestions a try, in case he can’t reduce the parameter set in advance. As I also agree that this is a big parameter set. Though in numerical optimization not uncommon. However you have to keep in mind that with Newton you need to calculate in each iteration the 16x16 hessian and it’s inverse which is time consuming. Also for Newton i do not believe it is multithreaded as each step needs to calculate the derivative of the previous step. but in the calculation of the hessian it could be multithreaded. also I’m Not sure if in JMP is implemented a different step size than 1, which might be better for highly nonlinear functions with many extremes. BFGS and e.g. levenberg markwardt are using approximations of the hessian based on the gradient information of the previous iteration,  . Therefore you need less operations.

MikeD_Anderson · Answer

Hi, Florence!

A model with 16 parameters sounds like a lot… is there a reason (physical model, etc.) why they are running 16 parameters?  Also, what version of JMP are they using, OS, Ram size, etc.?

Non-linear is hyper-threaded (see “Using JMP” pg 602) so it should be running on all 4 cores.  If you’re not seeing any load on the other cores while he is fitting then he may have something set up to not allow hyper threading. I’d go after that first, because it will give performance gains all around on JMP.

Best,

M

nick_shelton · Answer

I always share the strategies below with my customers using the non-linear platform, generally these will help in converging to a solution quickly:

-          Use various combinations of the iterative options (Newton, QuasiNewton BFGS, QuasiNewton SR1 and Numeric Derivatives Only)

-          Adjust the starting values of parameters using the sliders to visually modify the curve to fit better

-          Center the polynomials by transforming the equation, this will improve the conditions for optimization

-          Increase the number of iterations allowed

-          Relax one of the convergence criteria

-          Try a different model, if data does not exhibit the same relationship as the model chosen it will be difficult to converge to a solution.

-          Reduce the number of parameters, a model that is over defined is one that has more parameters than are necessary to model the relationship.