Re: updating a File everytime loop runs(JSL)

Question

I uploaded the data file(i imported into XL while running my script) that I used in the above script

My script is:-

Clear Globals();
dt = open();
k=5;
collect_means = {};collect_stdevs = {};collect_group_std={};
For( l = 1, l <= 2, l++,
show(k);
For( i = 1, i <= 3, i++, //3 times
 nt = dt << Subset(
  Output Table( "Subset" ),
  Sample Size( k ), //samples of 5
                              
 ); 
 Summarize( m = Mean( :Data ), s = Std Dev( :Data ) );
 Insert Into( collect_means, m,0 );
 Insert Into( collect_stdevs, s,0 );
              // close("subset", nosave); 
                 
    );
   
New Table("Final",
 New Column( "means", set values( collect_means )),
New Column( "stddevs", set values( collect_stdevs )), 
 //New Column("Group Std",Formula(col std dev(:means)))
 );
 //pt << Summary( Group, Std Dev( :means ) );
  pt=Data Table("Final")<< summarize(p=std dev(:means));
     Insert Into(collect_group_std,p,0);   
   //close("Final",nosave);
  k=k+5;
  );

New Table("Groups",New Column( "Summary Group", set values( collect_group_std )));

The "Final" sheet in the above script has to be updated everytime when the outer For loop runs. But now, the "FInal Sheet' is not getting updated and using the previous "Final" sheet (using the sheet from the previous iteration.) Can some one help so that "Final" sheet is updated with new values everytime.

Sample Data.txt.zip

ms · Answer

Btw, using lists and a lot of intermediary tables is not very efficient. Here all data are numeric and the subsets and the Final tabler seem not to be of interest other than acting as temporary holders for data. In this case I would suggest using matrices instead which are much more efficient, especially for large data sets.

Try this and compare the performance with the above script:

Clear Globals();

dt = Data Table( "Sample Data" );

collect_group_std = [];

ks = [];

For( k = 5, k <= 10, k = k + 5,

collect_means = [];

ks = ks |/ k;

For( i = 1, i <= 3, i++, //3 times

sub = Random Index( N Rows( dt ), k );

collect_means = collect_means |/ Mean( dt:Data[sub] );

Show( collect_means );

);

collect_group_std = collect_group_std |/ Std Dev( collect_means );

);

New Table( "Groups", New Column( "k", setvalues( ks ) ), New Column( "Summary Group", set values( collect_group_std ) ) );

ms · Answer

The summarize function should not be used as a message. When working with several  tables that has identical column namesit also a good idea to to explicitly include the table in context (e.d dt:Data instead of just : Data).  And although I am not sure of exactly what you want to achieve here, shouldnt the lists collect_means and collect_stdevs be reset for each outer for-loop?

Below is your code with some minor changes.

Clear Globals();

dt = Data Table( "Sample Data" );

k = 5;

collect_group_std = {};

For( l = 1, l <= 2, l++,

collect_means = {};

collect_stdevs = {};

Show( k );

For( i = 1, i <= 3, i++, //3 times

nt = dt << Subset(

Output Table( "Subset" ),

Sample Size( k ), //samples of 5

);

Summarize( m = Mean( nt:Data ), s = Std Dev( nt:Data ) );

Insert Into( collect_means, m, 0 );

Insert Into( collect_stdevs, s, 0 );

//Close( nt, nosave );

);

pt = New Table( "Final",

New Column( "means", set values( collect_means ) ),

New Column( "stddevs", set values( collect_stdevs ) ),

);

Summarize( p = Std Dev( pt:means ) );

Insert Into( collect_group_std, p, 0 );

//Close( pt, nosave );

k = k + 5;

);

New Table( "Groups", New Column( "Summary Group", set values( collect_group_std ) ) );