56743_en-US

liliconcom

Suggest faster & better ways to flatten varying lengths of matrices

All, 
   While I know of a way to do this, I dont think it is the most efficient way. Any thoughts to make this more efficient ? 

 

Matrix1 = [1,2,3,4,5]; 
Matrix2 = [2,3,5]; 
Matrix3 = [7,8,9,10]; 

ColAVals = {};
ColBVals = {}; 
ColCVals = {};
for(i = 1, i <= N rows(Matrix1), i++,
  for(r = 1, r <= N rows(Matrix2), r++,
    for(s = 1, s <= N rows(Matrix3), s++,
      Insert Into(ColAVals,Matrix1[i]); 
      Insert Into(ColBVals,Matrix2[r]); 
      Insert Into(ColCVals,Matrix3[s]); 
       );
     );
   ); 
   
Show(ColAVals);
Show(ColBVals); 
Show(ColCVals);

dt = New Table("Test"); 
dt << New Column("A",Numeric,Continuous, << set Values(ColAVals))
   << New Column("B",Numeric,Continuous, << set Values(ColBVals))
   << New Column("C",Numeric,Continuous, << set Values(ColCVals));

Re: Suggest faster & better ways to flatten varying lengths of matrices

This method does seem to be relatively quick.

nmat = 15;
matrices = {};
For( i = 1, i <= nmat, i++,
 matrices[i] = J( Random Integer( 5 ), 1, Random Uniform() )
);

start = Tick Seconds();
m1 = As Table( matrices[1] );
m2 = As Table( matrices[2] );
dt = m1 << Join( With( m2 ), Cartesian Join, private );
Close( m1, nosave );
Close( m2, nosave );

For( i = 3, i < N Items( matrices ), i++,
 mi = As Table( matrices[i] );
 dt = dt << Join( With( mi ), Cartesian Join, private );
 Close( mi, nosave );
);
mlast = As Table( matrices[N Items( matrices )] );
dt = dt << Join( With( mlast ), Cartesian Join );
Close( mlast, nosave );

end = Tick Seconds();
total = end - start;
Show( total );

 

By the way, be careful with Cartesian joins, they get big really fast.

 

 

Re: Suggest faster & better ways to flatten varying lengths of matrices

So, you want the cartesian join of all of your matrices?

 

Again, I haven't done any timing tests but what about this?

 

Matrix1 = [1, 2, 3, 4, 5];
Matrix2 = [2, 3, 5];
Matrix3 = [7, 8, 9, 10]; 

dts = {};
dts[1] = As Table( matrix1 );
dts[2] = As Table( matrix2 );
dts[3] = As Table( matrix3 );

dt = dts[1] << Join( With( dts[2] ), Cartesian Join );

For( i = 3, i <= N Items( dts ), i++,
 dt = dt << Join( With( dts[i] ), Cartesian Join );
  
);

Re: Suggest faster & better ways to flatten varying lengths of matrices

@Jeff_Perkinson, 
    I am worried about the looping rather than the conversion into the data table. Also, I want my resulting table to have all possible combinations  like 

 

Re: Suggest faster & better ways to flatten varying lengths of matrices

I haven't done any timing tests but using As Table() like this may be faster.

 

matrix1 = [1,2,3,4,5]; 
matrix2 = [2,3,5]; 
matrix3 = [7,8,9,10]; 

maxrows=maximum(nrow(matrix1), nrow(matrix2), nrow(matrix3));

matrix1 = matrix1 |/ j(maxrows-nrow(matrix1), 1, .);
matrix2 = matrix2 |/ j(maxrows-nrow(matrix2), 1, .);
matrix3 = matrix3 |/ j(maxrows-nrow(matrix3), 1, .);

mat=matrix1 || matrix2 || matrix3;

dt=as table(mat);