Optimize for loop within a for loop

Question

Hi all.

I have some JSL code that takes a long time to execute for larger datasets.

The code runs a for loop within a for loop to populate a list of unique conditions. I’m looking for a faster way of doing this.

The list serves multiple purposes. Firstly, it serves as a counter for how many unique conditions are present in a data set, and secondly, I use this list to make smaller lists, to populate columns in a new table. The way this is done is by converting a string to a list by separating each element with a ”,”.

I have attached a dummy data set as well as the code, covering the first ”purpose” of the list.

start=tick seconds();
dt = current data table();
dt << new column("concat_factors");
condition_list = {};
for(x=1, x<=nrows(dt),x++,
	column(dt, "concat_factors")[x] = 
	char(column(dt, "X__2")[x]) || "," || 
	char(column(dt, "X__11")[x]) || "," || 
	char(column(dt, "X__12")[x]) || "," || 
	char(column(dt, "X__15")[x]) || "," || 
	char(column(dt, "X__4")[x]);
	if(x==1,
		insert into(condition_list, char(column(dt, "concat_factors")[x]));
	);
	if(x!=1,
		found = 0;
		for(y=1,y<=nitems(condition_list),y++,
			if(char(condition_list[y]) == char(column(dt, "concat_factors")[x]),
				found = 1;
			);
		);
		if(found == 0,
			insert into(condition_list, char(column(dt, "concat_factors")[x]));
		)
	);
);
end=tick seconds();
runtime=end-start;
print(runtime);

Thanks in advance!

Br. Anders

global_simplified_raw Anonymized.jmp

Mark_Bailey · Accepted Answer

Try this approach:

start=tick seconds();
dt = current data table();
dt << new column("concat_factors");
for each row(
	column(dt, "concat_factors") = 
	char(column(dt, "X__2"))  || "," || 
	char(column(dt, "X__11")) || "," || 
	char(column(dt, "X__12")) || "," || 
	char(column(dt, "X__15")) || "," || 
	char(column(dt, "X__4"));
);
condition_list = Associative Array( column(dt, "concat_factors") ) << Get Keys;
end=tick seconds();
runtime=end-start;
print(runtime);

I think that the optimized For Each Row() function might be faster. Also, it seems like you are spending a lot of time just checking for a new unique value. The Associative Array() construct is very efficient of adding only unique keys.