57060_en-US

liliconcom

Script to Save Residuals of Spline Fit

Hello,

 

Anyone knows how to write a script to sava residuals (or predicteds) as a new colum for spline fit? It is easy if i do it manually, just need to select Save Residuals in the dropdown red triangle. I know how to write scripts to save residuals as a new column for liner fit using Fit Model. However, I often need to fit time-trend data using standardized x spline fit, and i am not sure if there's a easier way to do so. My final goal is to aquire residuals or predicteds for several varialbles in a loop, so a simple script will help a lot. 

 

Thanks!!

Re: Script to Save Residuals of Spline Fit

Hi Jim

Any resolution/fix to so that we can save residual in v14?

thanks for the help

 

Re: Script to Save Residuals of Spline Fit

Peng,

 

Yes, this seems inconsistent, however, I am guessing this is done on purpose to compare the predicteds (for example, the males are a hold-out set for the predicteds based upon females and vice versa.

 

To create a single column of Predicteds based upon the Group By  model create a new column with a formula or set each value expression

If( :sex=="M", column for the male predicteds, :sex=="F", column for the male predicteds);

 

If you want a column that looks like the residual columns, then create a columnm for males and use an expression like

If( :sex=="M", column for the male predicteds, empty() );

 

Then do the same for females.

 

 

  

Re: Script to Save Residuals of Spline Fit

Cool, this is awesome!! 

 

Just one more question: 

 

When I do  obj <SPAN class="token operator"><</SPAN><SPAN class="token operator"><</SPAN> <SPAN class="token punctuation">(</SPAN>Curve<SPAN class="token punctuation">[</SPAN>i<SPAN class="token punctuation">]</SPAN> <SPAN class="token operator"><</SPAN><SPAN class="token operator"><</SPAN><SPAN class="token messages"> Save Residuals</SPAN><SPAN class="token punctuation">)&nbsp;</SPAN>and group by (:sex), JMP calculated residuals for "M" and "F" and saved in different columns with some empty cells. That's why "transpose" is needed to get all residuals in one column. However, if I do <

 

Thanks

Re: Script to Save Residuals of Spline Fit

I really like the new JMP 13 table subset reference. However, there still is an easy method to get the values with message Get as Matrix. The key syntax is dt << get as matrix( list | index ).  There is no rowwise max function, but there is VMax function which is a columnwise max.  I tested this in JMP 12.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

obj = Bivariate(
 Y( :Weight ),
 X( :Height ),        GroupBy( :sex),
 Fit Spline( 0.1, Standardized )
);

//Find how many in a group. We know it is 2 for Big Class
summarize(_by = By(:sex));

//Find out how many columns before adding residuals
ncpre = ncol(dt);
_nby  = nitems(_by);

for(i=1, i<=_nby, i++,
  obj << (Curve[i] << Save Residuals);
  wait(0);
);

idx = ncpre+1::(ncpre+_nby);
cmat = Transpose(VMax(Transpose(dt << get as matrix(idx)))); //matrix _nby x nrow(dt)
/*the previous line of code is performing these tasks
  - get the columns of residuals into a matrix of size nrow(dt) x _nby
  - create the transpose: coloumns are rows and rows are columns, size _nby x nrow(dt)
  - Vmax is a vertical maximum, it finds the max of each column (previously rows)
    returns a 1 x nrow(dt) vector, a row vector
  - transpose again so it is a column vector
*/

//create a single column of residuals
resids = dt << New Column("Weight v Height Spline Residuals",
 Numeric, Continuous, Values( cmat )
);

  

Re: Script to Save Residuals of Spline Fit

Thank you all for the solutions!! The syntax for multiple subsections of a data table does not work in JMP12. I read another discussion about returning max value of mutiple rows, a similar method was also appllied for JMP13. Is there a better way to acquire max values across mutiple rows in JMP12? Writing a formula to compare row values for unknown number of rows doesn't seem to be efficient. Thanks!

Re: Script to Save Residuals of Spline Fit

Jim,

 

I noticed the same issue for JMP 14.  JMP support states it is fixed in JMP 14.1 (August?).

 

Here is a small modification of Jim's script. It creates a single column of residuals. It uses the new (in JMP 13) syntax for multiple subsections of a data table. This is getting the residual columns by row and finding the max (actually it gets the only non-empty value).

 

 

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

obj = Bivariate(
 Y( :Weight ),
 X( :Height ),        GroupBy( :sex),
 Fit Spline( 0.1, Standardized )
);

//Find how many in a group. We know it is 2 for Big Class
summarize(_by = By(:sex));

//Find out how many columns before adding residuals
ncpre = ncol(dt);
_nby  = nitems(_by);

for(i=1, i<=_nby, i++,
  obj << (Curve[i] << Save Residuals);
  wait(0);
);

idx = ncpre+1::(ncpre+_nby);

//create a single column of residuals
resids = dt << New Column("Weight v Height Spline Residuals",
 Numeric, Continuous, 
   << Set Each Value( max(dt[row(),idx]) )
);

 

 

Re: Script to Save Residuals of Spline Fit

Each separate "Fit" that is specified has to have their residuals saved separately.  The substript on the "Curve[]" reference is what handles this.  The example below has a GroupBy(:Sex) which will produce a Fit for Females, "F" and one for Males, "M".  Thus, 2 "Save Residuals" must be issued.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
obj = Bivariate(
 Y( :Weight ),
 X( :Height ),        
 GroupBy( :sex),
 Fit Spline( 0.1, Standardized )
);
obj << (Curve[1] << Save Residuals);
obj << (Curve[2] << Save Residuals);

As I was testing out this code for your example.  I discovered that JMP 14 does not work for the Save Residuals for a Spline Fit.  I will be passing that on to JMP Support as a bug.

Re: Script to Save Residuals of Spline Fit

Thanks Jim! It works. If I need to add "GroupBy" in Bivariate is it posible to save all the residuals? It looks like JMP only keeps the last fit line. Thanks!

 

<SPAN class="token function">Names Default To Here</SPAN><SPAN class="token punctuation">(</SPAN> <SPAN class="token number">1</SPAN> <SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">;</SPAN>
dt <SPAN class="token operator">=</SPAN> <SPAN class="token function">Open</SPAN><SPAN class="token punctuation">(</SPAN> <SPAN class="token string">"$SAMPLE_DATA/Big Class.jmp"</SPAN> <SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">;</SPAN>
obj <SPAN class="token operator">=</SPAN> Bivariate<SPAN class="token punctuation">(</SPAN>
 Y<SPAN class="token punctuation">(</SPAN> <SPAN class="token punctuation">:</SPAN>Weight <SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">,</SPAN>
 X<SPAN class="token punctuation">(</SPAN> <SPAN class="token punctuation">:</SPAN>Height <SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">,<BR /></SPAN>        GroupBy( :Team),
 Fit Spline<SPAN class="token punctuation">(</SPAN> 0.1, Standardized <SPAN class="token punctuation">)</SPAN>
<SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">;</SPAN>
obj <SPAN class="token operator"><</SPAN><SPAN class="token operator"><</SPAN> <SPAN class="token punctuation">(</SPAN>Curve<SPAN class="token punctuation">[</SPAN><SPAN class="token number">1</SPAN><SPAN class="token punctuation">]</SPAN> <SPAN class="token operator"><</SPAN><SPAN class="token operator"><</SPAN><SPAN class="token messages"> Save Residuals</SPAN><SPAN class="token punctuation">)</SPAN><SPAN class="token punctuation">;</SPAN>

  

Re: Script to Save Residuals of Spline Fit

You can find these types of syntax in the Scripting Index

     Help==>Scripting Index==>Bivariate==>Bivariate Curve

Here is the scripting index example from that entry

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
obj = Bivariate(
 Y( :Weight ),
 X( :Height ),
 Fit Polynomial( 3 )
);
obj << (Curve[1] << Save Residuals);