Integratation of GrubbsOutlierTest2 into multivariable data table

Question

Previously stated, I am dealing with a complex set of data tables joined using SQL scripts. With the help from this community I have managed to create an "application" based script that creates plots and distributions based on a user selected conditions.

The last portion of issues is dealing with those pesky outliers. As usual, my problem lies with looping through the test conditions identifying the outliers and then removing them from the data set. I understand that removing them can be risky but given our complex tests our parts are in a tight distribution or catastrophically fails. I am trying to impliment if g > g0 go find the outliers and delete them.

Attached: 3 subsets from the main table and the main table.

I found this THREAD that could do it in a single column but again, how does one loop through everything?

I have quite a few scripts that loop through some of these conditions but I cannot figure out the correct combination.

Again thanks for the help.

subset test_4.jmp

subset test_3.jmp

subset test_1.jmp

sample data _ 6 wafers.jmp

Yngeinstn · Answer

Update: I managed to get GrubbsOutlier to sort of work. I can take one test condition and get what i need. I grabbed a For Loop to capture two test conditions however, it hangs. You can run this script against the sample data _ 1 wafers (test_3).jmp and see the output which i am looking for.. I would appreciate any help with adding the 2nd Trmode and even as far as doing this on a dataset with multiple wafers sample data _ 6 wafers (test_3).jmp After this is resolved i am going to try to do a If( g > g0 ) Loc( < the outliers> ) then delete them from the dataset. Thank You dt = Current Data Table(); dtsum = dt << Summary( Group( :channel, :trmode ), Interquartile Range( :Output ), Freq( "None" ), Weight( "None" ), // invisible ); Current Data Table( dt );
// dist = dt << Distribution( Y( Column( "Output" )), By( Column( "channel" ) ), Normal Quantille Plot( 1 ), Fit Distribution( Normal( Goodness of FIt( 1 ) ) ),
// Added a For() Loop located at the botton of the screen, the script hangs up
// Where( dt :trmode == mode ) Where( dt :trmode == "Tx" )

); distr = dist << Report; bCol = Column( "channel "); Summarize( group = By( bCol )); yy = Column( "Output" ) << Get As Matrix; exRows = dt << Get Excluded Rows(); yy[exRows] = .; For( i = 1, i <= N Items( group ), i++, groupName = Trim( Word( 2, distr[i][OutlineBox(1)] << Get Title, "=" ) ); getRows = dt << Get Rows Where( bCol[] == groupName ); yVal = yy[getRows]; yVal[Loc( Is Missing( yVal ) )] = []; n = N Row( yVal ); a = 0.05; t0Sqr = t Quantile( 1 - a/(2*n), n-2 )^2; g = Maximum( Abs( yVal - Mean( yVal ) ) ) / Std Dev( yVal ); g0 = ((n-1)/Sqrt(n)) * Sqrt( t0Sqr / (n - 2 + t0Sqr) ); distr[i][Outline Box(2)] << Append( Outline Box( "Grubbs' Outlier Test", Table Box( String Col Box( "Statistic", {"G", "G("||Char(a)||")"} ), Number Col Box( "Estimate", Matrix( {g, g0} ) ) ), Text Box( If( g>g0, "Outlier detected", "No outlier detected" ) ) ) ); ); // For Loop I was trying to use // For( i = 1, i <= N Rows( dtsum ), i++, // mode = dtsum:trmode[i]; // < Insert Script From Above > // ); sample data _ 6 wafers (test_3).jmp sample data _ 1 wafers (test_3).jmp

Yngeinstn · Answer

Yes sir, however i would like to use that in an automated form that cacluates the g > g0. Then with that calculation for each test By: channel locates the outliers and then excludes them from the data set instead of manually doing it.

Mark_Bailey · Answer

The script for the Grubb's outlier test can use a grouping variable in the By analysis role.