Transpose List of Lists without a for loopI would like to have a method to transpose a list of lists without using a for loop. That is, some function (call it "Listtranspose" for instance) that flips the "rows" and "columns" of a list of lists, like this:
listlist = {{"cat", "dog", "mouse"}, {"meow", "bark", "squeak"}};
tranlist = Listtranspose(listlist);
Show(tranlist);
/*Result:
{{"cat", "meow"}, {"dog", "bark"}, {"mouse", "squeak}}
An alternate implementation could be something like:
listlist = {{"cat", "dog", "mouse"}, {"meow", "bark", "squeak"}};
nr = NItems(listlist);
col1 = listlist[1::nr][1];
col2 = listlist[1::nr][2];
col3 = listlist[1::nr][3];
translist = EvalList({col1, col2, col3});
I kind of like the latter implementation better as I might only need to transpose just a subset of "columns", not the entire list.
Of course, this can be done using a For Loop, but the reason for asking for this is that For Loops can be slow. I work with large blocks of text and I believe this would help speed things up greatly for me.
Credit to @vince_faller for asking this originally. Transpose List of Lists without a for loop
Automation and ScriptingRe: Transpose List of Lists without a for loopThinking about this some more. I still can't beat the for loop, but figured I'd write this down in the event it helps anyone.
// make a blank data table with the right shape
dt = As Table(J(nr, nc, .), <<PRIVATE></PRIVATE>
Re: Transpose List of Lists without a for loop@Craige_Hales Thanks for the tremendous insight Craig. I agree, I did a speed test of the for loop vs parallel-assign method, and parallel-assign was way slower. My list array was 718r x 14182c, so >10million elements. I ran the script for ~10min and the operation did not complete. This seems to be a little slower than your data above predicts, but too many variables to consider (e.g. computer system, ratio of columns to rows?). For loop is much faster at ~10.1s for the same table.
Still slower than I'd like though. Would be great to have a JMP function that can do this internally.
Re: Transpose List of Lists without a for loopBut, after some testing, I think a simple nested loop will be faster than any other JSL-based approach.
result = {};
for( c = 1, c <= nc, c += 1,
thisrow = {}; // build a row from a column
for( r = 1, r <= nr, r += 1,
thisrow[r] = testList[r][c];
);
result[c] = thisrow;
);@DonMcCormack - Challenge!
The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.The blue-line for-loop is consistently better. The purple JSON-table might overtake it, but I'm running out of memory. The parallel-assign trick begins to fail for large numbers.
The parallel assign is probably blocking threads against each other, all the time. It beats the json data table at first because of different overheads between the two.
Re: Transpose List of Lists without a for loopYes, it would be nice to have more list manipulation functions. I can't come up with a nice or fast way, but after playing a bit came up with this. It needs JMP 15, fails in 14.
list = {{"lion", "tiger", "puma"}, {"lamp", "torch", "pharos"}, {"lark", "toucan", "pigeon"}, {"lawn", "terrace", "patio"}};
jlist = As JSON Expr( list );
dt1 = JSON To Data Table( jlist, invisible(1) );
dt1 << New Column( "row", numeric, setvalues( 1 + Floor( (0 :: (N Rows( dt1 ) - 1)) / N Items( list[1] ) ) ) );
dt2 = dt1 << Split( Split By( :row ), Split( column(dt1,1) ), Sort by Column Property, invisible(1) );
Close( dt1, nosave );
transposed = dt2[0, 0];
Close( dt2, nosave );
Show( list, transposed );list = {{"lion", "tiger", "puma"}, {"lamp", "torch", "pharos"}, {"lark", "toucan", "pigeon"}, {"lawn", "terrace", "patio"}};
transposed = {{"lion", "lamp", "lark", "lawn"}, {"tiger", "torch", "toucan", "terrace"}, {"puma", "pharos", "pigeon", "patio"}};
Unhide the intermediate tables to see what it is doing. A real solution should deal with rows of unequal length.
Re: Transpose List of Lists without a for loop@Craige_Hales wrote a great explanation that I use regularly. Multithreading troubleshooting using Parallel Assign
Re: Transpose List of Lists without a for loop@vince_faller
Mind == Blown.
Never heard of the Parallel Assign function before. I tried this, and yup, it works. Even though I'm working with character strings and this is technically a matrix function, it somehow doesn't care. It will be interesting to play with this and see if it's faster than a for loop. Will try to update here once I know.
Thanks for the idea!
Re: Transpose List of Lists without a for loopI swear up and down I saw someone do something along the lines of
{"cat", "dog", "mouse", "meow", "bark", "squeak"}[[1 1, 1 1]]
// and had it return
{{"cat", "cat"}, {"cat", "cat"}}
during a talk at discovery but I've never been able to recreate it. It could have all been a dream ?
If I remember right, they somehow shaped a new list from a flat list in one line like that. I probably don't remember right.
That being said you can probably do parallel assign really fast
names default to here(1);
listlist = {{"cat", "dog", "mouse"}, {"meow", "bark", "squeak"}};
m = J(nc, nr, -1);
::trans_list = Aslist(m);
Parallel Assign({l = listlist},
m[a, b] = (
::trans_list[a][b] = l[b][a];
)
);
show(::trans_list);Re: Transpose List of Lists without a for loop2024 - JMP18
what is the best/fastest way to transpose a list of lists?
Re: Transpose List of Lists without a for loop@hogi you could run the scripts above and let us know? :smiling_face_with_halo: (lazy/selfish)
Re: Transpose List of Lists without a for loopSorry, I was saying that I was lazy and selfish. I had an angel emoji but it didn't come through. Very sorry.
Re: Transpose List of Lists without a for looprun the scripts: this approach just works if the easiest/fastest/best approach of 2024 can already be found in the list of possibilities of 2021 ; )
Maybe there is now an ultra-fast super-easy listtranspose(list of lists)which just got possible by the improvements of JMP17 or JMP18?
Re: Transpose List of Lists without a for loopwith just a single vote since 2021, I think the is little chance for
Listtranspose(listlist);
If users found this wish, please also take the time to look at Insert Into: Flatten(0|1) - and vote 
Re: Transpose List of Lists without a for loopParallel Assign (as 2-dimensional for loop) has no chance to win against the 1D-For Loop
(... or @vince_faller 's approach via data table manipulations)
winner on my system: DataTable!
Re: Transpose List of Lists without a for loop
The disadvantage of not having Emojis in the community ; )
