228036_en-US

liliconcom

Speed Up Tree Box Creation From 4-level Hierarchical Table

Here's a programming challenge for all you JSL nerds out there.

I've got 4 levels of hierarchical data in a table that looks like this:

I'm creating a hierarchical tree structure from this data using tree box that looks like this:

My code creates this output in about 40 seconds.  Is there a faster way?  I tried various approaches and creating subsets seemed to be the fastest.  Here's my code:

dt = data table ("AE Levels");

ae_tree = {};

e0 = hp time();

// Get unique level 1 values
summarize(dt, level1_list = by(:Level 1));

for (i = 1, i <= nitems(level1_list), i++,

 caption("Processing Level1 " || char(i));
 wait(0);

 level1_rows = dt << get rows where(:Level 1 == level1_list[i]);

// Create a tree node for this level
 level1_node = tree node("Level 1: " || level1_list[i]);

// Make a subset of this level1's values
 dt1 = dt << subset(private, rows(level1_rows));
// Get unique level 2 values for this value of level 1
 summarize(dt1, level2_list = by(:Level 2));

 for (k = 1, k <= nitems(level2_list), k++,

// Create a tree node for this level
  level2_node = tree node("Level 2: " || level2_list[k]);

  level2_rows = dt << get rows where(:Level 2 == level2_list[k]);

  dt2 = dt << subset(private, rows(level2_rows));
// Get unique level 3 values for this value of level 2
  summarize(dt2, level3_list = by(:Level 3));

  for (m = 1, m <= nitems(level3_list), m++,
// Create a tree node for this level
   level3_node = tree node("Level 3: " || level3_list[m]);

   l3_rows = dt << get rows where(:Level 3 == level3_list[m]);
   dt3 = dt << subset(private, rows(l3_rows));
// Get unique level 4 values for this value of level 3
   summarize(dt3, l4_list = by(:Level 4));
   close(dt3, nosave);

   for (n = 1, n <= nitems(l4_list), n++,
// Create a tree node for this level
    l4_node = tree node("Level 4: " || l4_list[n]);
    
// Add the level 4 node to the level 3 node
    level3_node << append(l4_node);
   );

// Add the level 3 node to the level 2 node
   level2_node << append(level3_node);
  );
  
  close(dt2, nosave);

// Add the level 2 node to the level 1 node
  level1_node << append(level2_node);
 );
 close(dt1, nosave);
 insertinto(ae_tree, level1_node);
);
caption(remove);
wait(0);

e1 = hp time();
show((e1 - e0) / 1000000);
nw = new window("AE Tree",
 mtree = tree box(ae_tree, Size( 1000, 500 ),
  << Set Max Size( 2000, 4000 ),
  << Set AutoStretching( 1, 1 )
 )
);

Re: Speed Up Tree Box Creation From 4-level Hierarchical Table

I think so...I may not have looked at it long enough, but if this makes sense it is about 10X faster... this is the inner-most loop...the outer loop needs it too. The LinkToOriginalDataTable did not seem to help at all, which was a surprise, but using the smaller subsets for further subsetting helped a bunch.

l3_rows = dt2 << get rows where(:Level 3 == level3_list[m]);
dt3 = dt2 << subset(private, rows(l3_rows),LinkToOriginalDataTable(1));
// Get unique level 4 values for this value of level 3
summarize(dt3, l4_list = by(:Level 4));

 

(e1 - e0) / 1000000 = 3.226879;

Re: Speed Up Tree Box Creation From 4-level Hierarchical Table

Thanks Craige - your solution falls into the category of "why didn't I think of that".  Makes too much sense!