nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Removing duplicates from data table in JMP 15

lukasz

Hello Everybody,

I have one script in which I want to remove duplicated values from a table based on the specified column. The following piece of code is used to remove duplicates and that works well with JMP 14.

dt << Select duplicate rows( Match( :name ) );
selected = dt << get selected rows;
if (n rows(selected) > 0,
	dt << delete rows;	
);

In JMP 15, however, that does not seem to work. The rows are selected but later they are not removed. I tried to use "current data table ()" in several locations but it did not help. I would appreciate for hints how to get this part working. Am I missing something? Best regards.

Accepted answers

All comments

txnelson

Here is an example of your code working without error, and as expected

names default to here(1);
dt=open("$SAMPLE_DATA/big class.jmp");

dt << Select duplicate rows( Match( :name ) );
selected = dt << get selected rows;
if (n rows(selected) > 0,
	dt << delete rows;	
);

lukasz

Thank you for reply!

Yes, that is working well, however, what I have noticed is that, that the data table in which duplicates need to be removed is created as a subset of some bigger table. During making a subset an additional option "linked" is used which is helpful, because script creates several diagrams and by selecting data on one graph associated with the bigger table highlights at the same time data on graph from subset table. And that works well in JMP 14.

With this "linked" option in JMP 15 the removal of duplicates does not work. Without it, it works as expected but in this case the functionality of application is noticeably limited. Is there any additional options or commands that need to be considered in order to have that worked?

dt = dt_original << Subset(invisible, linked, Selected Rows( 0 ), selected columns( 1 ), Output Table Name( "Table"));

Byron_JMP

This looks a little ratchety, but it work well and its pretty fast too.

Try joining a table to its self, and drop multiples for both tables.

Match the columns in both tables that create the imperfect unique identifier for duplicates.

here's an example

Data Table( "Big Class.jmp" ) << Join(
	With( Data Table( "Big Class.jmp" ) ),
	By Matching Columns( :name = :name ),
	Drop multiples( 1, 1 ),
	Include Nonmatches( 0, 0 ),
	Preserve main table order( 1 )
)

This approach is menu driven. Just get the source script from the deduped table after you run the join.

Thomas1

This JSL code works fine for me:

Names Default To Here(1);
 
dt = Current Data Table();
 
// Clear Data Table
dt << Clear Column Selection();
dt << Clear Select();
 
//Update Formula Columns
dt << Rerun Formulas;
wait(0);
 
// Delete multiple rows
dt << Select Duplicate Rows();
dt << Delete Rows();

lukasz

Thank you for your suggestions! I have tried all solutions but in all cases removing of duplicates works well if there is no "linked" or "LinkToOriginalDataTable( 1 )" option during creation of a subset from the original table. Joining with the same table and removing duplicates worked well but then no linking with original table is created. Maybe virtual joining will help, I will try.. I would appreciate for further hints and suggestions.