Re: Outlier screening with Jmp Dashboards - how?Have you made sure your brackets are where they should be?
Re: Outlier screening with Jmp Dashboards - how?@jthi , right.
I corrected the code
... and added screenshots how the lists of aliens look on my system for the different cases.
Re: Outlier screening with Jmp Dashboards - how?Depending on what you wish to do in the end you do have some possible workarounds:
- Add "blank" row with :alien = 1 (all other values missing except for the alien being 1)
- Create custom filter using List Box (or something else) and then you can have your own logic and options
- Use some sort of inverse filter if no ones are found
- Use different view if no ones are found
- Don't hide data filter to make it more obvious what is being filtered (or not being filtered in this case)
Re: Outlier screening with Jmp Dashboards - how?Some insights why the approach via Where() is not dynamic
Using Where() to filter down analysis creates new linked private data table :
a private subset of the data table is generated
Re: Outlier screening with Jmp Dashboards - how?Is it a workaround to show the user that there is an issue in Jmp - I want the user to work with Jmp without seeing issues 
Concerning the other workarounds:
I don't like it to reinvent the wheel (create my own Data Filter) - just to prevent Jmp from "doing something wrong".
So what remains is option # 1:
add dummy rows to the data table such that the Data Filter doesn't get cleared by not finding any matching rows.
with some nasty disadvantages: the dummy rows have to be "removed" manually everywhere else...
- Col Number (if(:dummy_row, ., :alien), ...);
- Graph Builder (..., Where(Ismissing(:dummy_row));
- dfRows = myDataFilter << Get Filtered Rows;
NRowsAfterDummyScreening=NItems(Loc(-dt[dfRows,"dummy"]+1));
And it will just work as long as the user doesn't activate
Include missing for grouping columns( 1 ).
For future versions of Jmp, I hope that there will be an option to prevent the Data Filter from resetting when no matches are found. Here is the wish:
Local Data Filter: option to adjust to current data
Re: Outlier screening with Jmp Dashboards - how?I found a way to tackle the issue with the
Include missing for grouping columns( 1 )
Use a Data filter with alien = 1 and dummy = 0 - and in addition set Count Excluded rows(1) [the default].
Then the data filter won't notice that for many age groups there are no rows with alien=1 and dummy=0.
Since it doesn't notice, it doesn't reset the selection
Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "dummy", Nominal );
dt << New Column( "alien", Nominal );
For( myAge = 12, myAge <= 17, myAge++,
dt << add rows( {:name = "xxx", :age = myAge, :dummy = 1, :alien = 1} )
);
:alien[12]=1;
:sex[12]="";
:alien[39]=1;
New Window( "Outlier Screening",
Data Filter Context Box(
H List Box(
dt << Data Filter(
Local,
Add Filter( columns( :age ), Where( :age == 13 ), Display( :age, N Items( 6 ) ) )
),
dt << Tabulate(
Show Control Panel( 0 ),
Include missing for grouping columns( 1 ),
Add Table(Row Table( Grouping Columns( :name, :sex ) ) ),
Local Data Filter(
Close Outline( 1 ),
Mode( Show( 0 ) ),
//Count Excluded Rows( 0 ),
Add Filter( columns( :dummy, :alien ), Where( Is Missing( :dummy ) ), Where( :alien == 1 ) )
)
)
)
)
);
NB: This solution will fail if one day there is not a single alien in the whole data set.
For this case ...
- one could hide an additional alien with dummy = 0 in a dummy age group ("-99")
... and take care that nobody counts the non-dummy aliens for the whole class - or: split the data filter into two:
then both of them get enough matching rows so that they don't reset.
View more...Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "dummy", Nominal );
dt << New Column( "alien", Nominal );
For( myAge = 12, myAge <= 17, myAge++,
dt << add rows( {:name = "xxx", :age = myAge, :dummy = 1, :alien = 1} )
);
:sex[12] = "";
New Window( "Outlier Screening with worarounds",
Data Filter Context Box(
H List Box(
dt << Data Filter( Local, Add Filter( columns( :age ), Where( :age == 13 ), Display( :age, N Items( 6 ) ) ) ),
Data Filter Context Box(
H List Box(
dt << Data Filter( Local, Add Filter( columns( :dummy ), Where( Is Missing( :dummy ) ), Display( :age, N Items( 6 ) ) ) ),
dt << Tabulate(
Show Control Panel( 0 ),
Include missing for grouping columns( 1 ),
Add Table( Row Table( Grouping Columns( :name, :sex ) ) ),
Local Data Filter( Close Outline( 1 ), Mode( Show( 0 ) ), Add Filter( columns( :alien ), Where( :alien == 1 ) ) )
)
)
)
)
)
);
Outlier (:alien:) screening with Jmp Dashboards - how?I want to do some Outlier screening by listing the aliens in my class.
First attempt: Local Data Filter
I was quite shocked with all the aliens in Big Class!
... but then I noticed that the data filter was just ignored (because on day 1 there were no aliens in the class - and with no matching rows, the Data Filter is just reset - instead of showing 0 matching rows !!!).
This seems to be related to Data filter: how to specify the limits :
If a value is just used in the data filter but not in the data table, it is removed from the data filer.
This behavior interferes with another behavior of Jmp: if the last filter value is deselected, all rows get selected (instead of none).
So, with the current behavior of Data Filters, this approach doesn't work 
Next, I used a Where() clause to restrict the Tabulate Report.
Unfortunately, if one day there are no aliens in the class, the Where clause produces an error message - instead of restricting the data set to 0 rows:
This Where-clause for data table 'Big Class' resulted in no rows: :alien == 1
List of "aliens" for the 4 cases:
Please note:
For the case with Where(), there is another issue:
The row selection for the report is static.
After XX changed to non-alien - he is still listed in the report:
Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "alien", nominal, set each value( 0 ) );
myDashboard = Function( {myTitle, withWhereClause},
New Window( myTitle,
<<SHOW menu=""> so many aliens ?!?! no, Data filter was ignored (filter is OK, see below)
myDashboard( "1) Data Filter, no alien", 0 );
// 2) Where Cause + no matching row --> error message instead of a report
myDashboard( "2) Where(), no alien", 1 );
dt << add rows( {name = "XX", alien = 1, sex = "M", age = 200} );
// 3) Data Filter + 1 alien
myDashboard( "3) Data filter, 1 alien", 0 ); // new test: --> filter OK --> works
myDashboard( "4) Where(), 1->0 aliens ...", 1 ); // Where Clause + matching row --> dashboard is created, but ...
// XX gets human -> Dashboard #4 stays the same :(
//dt[41, {"alien"}] = 0;
;</SHOW>
Data Blending and CleanupData Exploration and VisualizationRe: Outlier screening with Jmp Dashboards - how?
@jthi wrote:- Don't hide data filter to make it more obvious what is being filtered (or not being filtered in this case)
?
Re: Outlier screening with Jmp Dashboards - how?@hogi wrote:
NB: This solution will fail *)
... with a quite surprising result - seems that neither Header nor Footer not the Data Filter itself know anything about the reset:
Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "dummy", Nominal );
dt << New Column( "alien", Nominal );
For( myAge = 12, myAge <= 17, myAge++,
dt << add rows( {:name = "xxx", :age = myAge, :dummy = 1, :alien = 1} )
);
:sex[12]="";
New Window( "Outlier Screening",
Data Filter Context Box(
H List Box(
dt << Data Filter(
Local,
Add Filter( columns( :age ), Where( :age == 13 ), Display( :age, N Items( 6 ) ) )
),
dt << Tabulate(
Show Control Panel( 0 ),
Include missing for grouping columns( 1 ),
Add Table(Row Table( Grouping Columns( :name, :sex ) ) ),
Local Data Filter(
Close Outline( 1 ),
Mode( Show( 0 ) ),
//Count Excluded Rows( 0 ),
Add Filter( columns( :dummy, :alien ), Where( Is Missing( :dummy ) ), Where( :alien == 1 ) )
)
)
)
)
);
Re: Outlier screening with Jmp Dashboards - how?I prepared this script for internal discussions about Outlier Screening in Jmp.
It shows most of the aspects and can be used as a starting point for further development of workarounds ...
Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "dummy", Nominal );
dt << New Column( "alien", Nominal );
:alien[12] = 1;
:sex[12] = "";
:alien[39] = 1;
New Window( "alien screeing",
<<SHOW menu=""></SHOW>
Re: Outlier screening with Jmp Dashboards - how?Thanks @hogi - this definitely looks like a bug that we will fix for a future release.
Re: Outlier screening with Jmp Dashboards - how?@danschikore
This video gives a hint what is going wrong:
The Dashboard starts with a wrong list in Tabulate - The Data Filter is restricted to "aliens", but in the list there are many entries with alien=missing.
To remove the wrong entries, the user has to change the selection in the "inner" data filter. (fist part of the video). (*)
On the other hand (second part), concerning the Data Filter in the Data filter Context Box - as soon as the user changes the selection there, the content of Tabulate is reset and the settings of the "inner" data filter are ignored
... till the user corrects it again via (*)
Support Ticket: TS-00046650
Re: Outlier screening with Jmp Dashboards - how?thanks
So, after all, I came back to @jthi 's first suggestion:
@jthi wrote:
- Add "blank" row with :alien = 1 (all other values missing except for the alien being 1)
Just a single blank line with alien=1, NO for loop, NO additional dummy column etc .
Then the dummy alien shows up by the additional Data filter entries sex = missing, name=missing, age=missing
- This makes it easy to remove the dummy alien from the reports by selecting an entry (!= missing) in the respective Data Filter.
- The benefit compared to the approach with the dummy column:
In the case of "no aliens today", the Outlier Screening will NOT be reset
Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
dt << New Column( "alien", Nominal );
dt << add rows( {:alien = 1} );
addAliens=0;
if(addAliens,
:alien[12]=1;
:alien[39]=1;
);
New Window( "Outlier Screening",
Data Filter Context Box(
H List Box(
dt << Data Filter(
Local,
Add Filter( columns( :age, :sex ), Where( :age == 13 ) )
),
dt << Tabulate(
Show Control Panel( 0 ),
Include missing for grouping columns( 1 ),
Add Table(Row Table( Grouping Columns( :name, :sex ) ) ),
Local Data Filter(
Add Filter( columns( :alien ), Where( :alien == 1 ) )
)
)
)
)
);
Re: Outlier screening with Jmp Dashboards - how?With Jmp 17, Data Filter was introduced for Tables/Summary.
Here the issue is much more severe: at the end, there is just the wrong result - no Data Filter on the left of a Dashboard showing the "obviously" wrong setting. 2 bugs with Tables/Summary
Re: Outlier screening with Jmp Dashboards - how?Hi @danschikore ,
@danschikore wrote:
Thanks @hogi - this definitely looks like a bug that we will fix for a future release.
does this comment refer to the special case with the two data filters or to the general case:
A Data Filter forgets its settings, if the value is not in the current data table?
Re: Outlier screening with Jmp Dashboards - how?Hi @hogi - I was referring to the first issue that you show in the video, that Tabulate was not showing the correct subset when the outer filter changes. But I don't see your script for this provided here - did you send it through Tech Support?
-Dan
Re: Outlier screening with Jmp Dashboards - how?The video was based on the code 2 posts before, with the same issue.
In Jmp18 it looks much better. Seems that this issue could be fixed .
On the other hand, there is still the issue with data filters which 'forget' their settings:
https://community.jmp.com/t5/Discussions/Can-animation-of-this-difficulty-be-achieved-with-JSL/m-p/755896#M93798
Re: Outlier screening with Jmp Dashboards - how?fixed in JMP19:
Tiny Traps in Jmp and JSL
Re: Outlier screening with Jmp Dashboards - how?Please watch
Using the Latest Enhancements to JMP and JMP Pro @ 9min / by @brady_brady
for a live demo how Forced Values fix the issues with Data Filters.
So smooth and elegant now : )
