83256_en-US

liliconcom

Faster way to read/write a cell

Hello

I have a big data file (250k rows by 2k columns) which contains production data from electronic  Integrated Circuit devices (ICs).

And I have a jsl script which i use to evaluate the results and look for patterns in the data, but in such a big data table it takes hours to complete a run.

To read/write a cell I was using dt:MeasureX[Row] but that instruction seems to be not execution efficient, so i switched to Column( dt, "MeasureX" )[intRow] and this one is like 20% faster to execute.

Are there tricks to improve the execution time? like stopping the table window refresh while the script is executing...

Thanks

 

Re: Faster way to read/write a cell

@oscarmforero,
             Since your data is numeric. It will be singnificantly faster to try and perform most of your actions on a matrix . 

dt = Current Data Table(); 

Mat = dt << Get As Matrix; // fetches all numeric data as a matrix 

            Once you have the matrix, you can use the loc() for selection based on logical conditions. Here is a more detailed example: 

 

dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

// Get as Matrix 
Mat = dt << Get as Matrix; // Note Matrix will only be 3 columns 


// Selecting data in data table 
SR = dt << Select Where(:height <= 65) << Get Selected Rows; 
dt[SR,"height"] = 65; 
dt << Clear Select; 

// Selecting data in matrix 
SR1 = Loc(Mat[0,2] <= 65); 
Mat[SR1,2] = 65;
 

Re: Faster way to read/write a cell

Hi Mark

 

For example, I have the below piece of code (which is the one that takes the longer to execute) where I use loops to read cells and to compare their values agains their spec limits. If the cell value is outside the limits, then read and overwrite the value of different cells.

 

numDataColumns = Current Data Table() << get column names( Numeric );
main_table = Current Data Table();<BR />
//Loop all columns with parametric data
For( intCol = 1, intCol <= N Items( numDataColumns ), intCol++,
    Spec_Lims = Column( numDataColumns[intCol] ) << Get Property( "Spec Limits" );
    Spec_USL_num = Try( Spec_Lims["USL"], . );
    Spec_LSL_num = Try( Spec_Lims["LSL"], . );
    USL_exist = Is Missing( Spec_USL_num ) == 0;
    LSL_exist = Is Missing( Spec_LSL_num ) == 0;
    
    //Loop all the selected rows (fails) and count the number of parametric fails and executed tests.
    For( intRow = 1, intRow <= N Rows( Data Table( main_table ) ), intRow++,
        If( Selected( Row State( Data Table( main_table ), intRow ) ) > 0,        
            If( Is Missing( Column( main_table, numDataColumns[intCol] )[intRow] ) == 0, //Only check for cells with data                                           
                //If both spec limits are available
                If(
                    (USL_exist == 1 & LSL_exist == 1) & (Column( main_table, numDataColumns[intCol] )[intRow] > Spec_USL_num |
                    Column( main_table, numDataColumns[intCol] )[intRow] < Spec_LSL_num),
                    Column( main_table, "parFailCount" )[intRow] = Column( main_table, "parFailCount" )[intRow] + 1;<BR />                    //log names of failing tests
                    Column( main_table, "failListNames" )[intRow] = Column( main_table, "failListNames" )[intRow] || Char( numDataColumns[intCol] )
                     || ",";
                );
                //Considered only as executed tests when any of the limits are available.
                If( USL_exist == 1 | LSL_exist == 1,
                    Column( main_table, "parExecutedTests" )[intRow] = Column( main_table, "parExecutedTests" )[intRow] + 1
                );
            )
        )
    );
);

Re: Faster way to read/write a cell

There are many approaches to improving the performance of a script but the method depends on the actions in the script. Can you describe what the script does to "evaluate the results and look for patterns in the data?"