67649_en-US

liliconcom

How to Separate Matching Values in a List

Hi,

 

I have a list which could contain the following, for example: {22, 22, 22, 33, 33}. I want to be able to get the number of each unique value in the list, e.g. the number of 22's and the number of 33's. The problem is, I don't know how long my list will be and how many unique numbers there will be in the list. Is there any way in JSL to generalise a solution for this?

 

Thanks

Automation and ScriptingRe: How to Separate Matching Values in a List

I believe your last edit needs to have the "my_bigger_list" changed to "my_list"

Re: How to Separate Matching Values in a List

Hi,

 

No worries, I solved the ssue and your solution wasn't part of the problem. Thanks.

Re: How to Separate Matching Values in a List

@pmroz's response got me interested.  Because I usually try to avoid data tables if I can just because of the overhead.  I played with timings

 

names default to here(1);
my_list = {44, 33, 22, 33, 44};

repeat_list = [1, 5, 10, 50, 100, 1000,  2500, 5000, 7500, 10000];
dt_timing = new table("Timings", 
 new column("List Size"), 
 new column("Design"), 
 new column("Loop"), 
 new column("Summarize"), 
);
for(ii=1, ii<=20, ii++, 
 for(ir = 1, ir <= nitems(repeat_list), ir++, 
  my_bigger_list = repeat(my_list, repeat_list[ir]);


  // using design
  st = HPTime();
  aa = associative array(my_bigger_list);
  keys = aa << Get Keys;
  d = design(my_bigger_list, keys);
  aa_design = associative array(keys, VSum(d));
  time_design = HPTime() - st;


  //using a loop
  st = HPTime();
  aa_loop = associative array(my_bigger_list, repeat(0, nitems(my_bigger_list)));
  for(i=1, i<=nitems(my_bigger_list), i++, 
   aa_loop[my_bigger_list[i]]++;
  );
  time_loop = HPTime() - st;


  //using a data table
  st = HPTime();
  dt = New Table( "Untitled", invisible, Add Rows( nitems(my_bigger_list) ),
   New Column( "Column 1", Numeric, "Continuous", Format( "Best", 12 ),
    Set Values( my_bigger_list )
   )
  );

  summarize(dt, unique_value_list = by(:column1), unique_count = count(:column 1) );
  close(dt, nosave);

  aa_peter = associative array(unique_value_list, unique_count);
  time_peter = HPTime()-st;
  Eval(EvalExpr(
   dt_timing << Add Rows({List Size = Expr(Nitems(my_bigger_list)), Design = Expr(time_design), Loop = Expr(time_loop), Summarize = Expr(time_peter)});
  ));

 );
);


dt_stack = dt_timing << Stack(
 columns( :Design, :Loop, :Summarize ),
 Source Label Column( "Method" ),
 Stacked Data Column( "Timing" )
);

close(dt_timing, no save);

biv = dt_stack << Bivariate(
 Y( :Timing ),
 X( :List Size ),
 group by(:Method),
 fit line(), 
 SendToReport(
  Dispatch(
   {},
   "Bivar Plot",
   FrameBox,
   {Row Legend(
    Method,
   )}
  ),
 )
);

 

and obviously it depends on the list size.  But I ran it 20 times each on various list sizes and got this

 

Design was the fastest way I could find to do it.  And I definitely agree with @txnelson that I don't like Summarize because of it returning character always.  

 

 

Re: How to Separate Matching Values in a List

I'm a big fan of design.  

names default to here(1);
l = {22, 22, 22, 33, 33};
l = sort ascending(l);
d = VSuM(design(l));
csum = cumulative sum(d);
lvalues = l[csum];

aa = associative array(lvalues, d);

*Edit* I realized that sorting was taking a lot of time so this is a lot faster

names default to here(1);
my_list = {44, 33, 22, 33, 44};
aa = associative array(my_list );
keys = aa << Get Keys;
d = design(my_list , keys);
aa_design = associative array(keys, VSum(d));

 

*Edit again (don't know why this is bothering me so much)* I didn't like the associative array and this is faster still

 

names default to here(1);
my_list = {44, 33, 22, 33, 44};
d = design(my_bigger_list, <<LEVELS></LEVELS>

Re: How to Separate Matching Values in a List

If you're looping through a column in a file why not just use the Tabulate or Summary commands?   If you are grouping things by another column you can use that as a "by" variable.  For example:

ID	Value
A	1
A	2
A	2
B	2
B	3
B	3
B	4
B	4

 

Using tabulate drag the ID and Value columns over the drop zone for rows.  You'll need to change the role of Value to be a grouping variable.  This is the result:

ID	Value	N
A	1	1
A	2	2
B	2	1
B	3	2
B	4	2

So for A you have two distinct values (1, 2), along with the number of occurrences (1, 2).  For B you have three distinct values (2, 3, 4), along with occurrences (1, 2, 2).

Here's the equivalent JSL:

dt = New Table( "Test", Add Rows( 8 ), 
  New Column( "ID", Character, "Nominal",
  Set Values( {"A", "A", "A", "B", "B", "B", "B", "B"} ) ),
 New Column( "Value", Numeric, "Continuous", Format( "Best", 12 ),
  Set Values( [1, 2, 2, 2, 3, 3, 4, 4] ) )
);

dtab = dt << Tabulate(
 Show Control Panel( 0 ),
 Add Table( Row Table( Grouping Columns( :ID, :Value ) ) ),
 invisible
 );

dt2 = dtab << Make Into Data Table;

Re: How to Separate Matching Values in a List

Can you post some data and code?

Re: How to Separate Matching Values in a List

Hi @pmroz,

I tried the first solution you posted and it works fine with one file / list. But when I use your code as a function and loop through it, (despite having initialised all my variables to make sure that nothing from the previous iteration stored) it doesn't seem to be working. I'm referencing to a file because I loop through a column in the file to get the list being used. Any ideas on what's going on as I've made sure to initialise all the lists and variables?

Re: How to Separate Matching Values in a List

@Jaz,
      This is an alternative approach, with the use of Loc().

MyList = {22,33,22,22,22,33,33,44,44,44,33,22}; 
AA = Associative Array(MyList); 
UniqueVals = AA << Get Keys; 
Frequency = {}; 

for(i = 1, i <= N Items(UniqueVals),i++,
  Insert Into(Frequency,N Rows(Loc(MyList,UniqueVals[i])));
   );
Show(Frequency);

 

Re: How to Separate Matching Values in a List

Please post this suggestion in the Wish List area of the Community.

Re: How to Separate Matching Values in a List

My support for the Summary Platform, is that the grouping variable is maintained as the same Data Type in the summary data table as it was in the originating data table.  In the Summarize() function, it is always returned as character values, even if the By column specified was numeric.

Re: How to Separate Matching Values in a List

I agree with @pmroz and @txnelson, data tables make this easy.

 

I prefer the Summarize() function over Tabulate and Summary for this though.

 

 

my_list = {22, 22, 22, 33, 33};
dt = New Table( "Untitled",
 private,
 Add Rows( N Items( my_list ) ),
 New Column( "Column 1",
  Numeric,
  "Continuous",
  Format( "Best", 12 ),
  Set Values( my_list )
 )
);

summarize(dt, group=by(:column 1), n=count(:column 1));

show(group, n);

 

 

 

Re: How to Separate Matching Values in a List

Thanks,

 

Would be useful if JMP included a find matching items / return matching items function for lists / associative arrays as they do for columns & rows.

Re: How to Separate Matching Values in a List

Don't forget summarize.  Even shorter.

my_list = {22, 22, 22, 33, 33};
dt = New Table( "Untitled", invisible, Add Rows( nitems(my_list) ),
 New Column( "Column 1", Numeric, "Continuous", Format( "Best", 12 ),
  Set Values( my_list )
 )
);

summarize(dt, unique_value_list = by(:column1), unique_count = count(:column 1) );
show(unique_value_list, unique_count);

Re: How to Separate Matching Values in a List

Here is my variation on @pmroz solution.  It is more of an old school solution

my_list = {22, 22, 22, 33, 33};
dt = New Table( "Untitled",
 private,
 Add Rows( N Items( my_list ) ),
 New Column( "Column 1",
  Numeric,
  "Continuous",
  Format( "Best", 12 ),
  Set Values( my_list )
 )
);
// Use the Summary Platform to create the aggragate values
dtSum = dt << Summary(
 private,
 Group( :Column 1 ),
 Freq( "None" ),
 Weight( "None" )
);
// Create the new lists
unique_value_list = As List( dtSum:column 1 << get values );
unique_count = As List( dtSum:n rows << get values );
Close( dt, nosave );
Close( dtSum, nosave );
Show( unique_value_list, unique_count );

Re: How to Separate Matching Values in a List

Table operations are probably the fastest way to do this.

my_list = {22, 22, 22, 33, 33};
dt = New Table( "Untitled", invisible, Add Rows( nitems(my_list) ),
 New Column( "Column 1", Numeric, "Continuous", Format( "Best", 12 ),
  Set Values( my_list )
 )
);
dtab = dt << Tabulate(Show Control Panel( 0 ),
 Add Table( Row Table( Grouping Columns( :Column 1 ) ) ));
dt2 = dtab << Make Into Data Table(invisible);
unique_value_list = dt2:column 1 << get values;
unique_count      = dt2:n << get values;
close(dt, nosave);
close(dt2, nosave);
show(unique_value_list, unique_count);

Re: How to Separate Matching Values in a List

I've managed to get a list of the unique values like so {22, 33} using associative arrays and the << get key functionality but I still don't know how to get the number of each of those unique values in the original list.