Missing binomial confidence intervals in Graph Builder and Fit Y by X (only found in Distribution)

Mark_Zwald

Problem statement: It is not convenient in JMP to compare yields (or fail-rates) across different groups for binomial (pass/fail) distributions which is a very common analysis in JMP's core industries. 

It is possible to do this correctly using the Distribution platform and splitting with a By variable, but this is not intuitive for our customers.  The natural expectation is to do this type of analysis in Fit Y by X (Contingency) or Graph Builder, but neither of those platforms support the binomial distribution confidence intervals, only Distribution.  So you have to perform Distribution of the Pass/Fail column split by grouping variable, turn on confidence intervals for probabilities, then make combined into data table and optionally visualize the results in Graph Builder.  Again, oddly difficult and not intuitive.

As a result, I see many engineers do the following (I am guilty of this)... change the pass/fail column into continuous 1/0 values, plot those values versus grouping variable in Graph Builder or Fit Y by X, and compare the calculated means and confidence intervals to determine if one group has a statistically different fail-rate than another.  While this gives a "pretty close" result which is typically good enough for many situations (especially when the rates are not close to 0% or 100%), but they do break down in those situations as shown below.  To illustrate the problem statement, here is a comparison of the same yields (99.0%, 99.5%, 99.9% with 1000 samples each) visualized in GB.  Left shows yield as continuous (1/0 values) and comparing means, and right visualizes the probabilities of Pass/Fail and their confidence intervals calculated from Distribution.

As you can see, similar, but not exact, and this bothers me.  I will be putting this as a feature request, with the hopes that this binomial distribution confidence interval functionality gets included in GB and Fit Y by X.  Customers want to compare binomial probabilities, and right now it is not intuitive to do so with many doing it the incorrect way I have shown.

Fail Rate comparisons.jmp

russ_wolfinger

Hi @Mark_Zwald , How about GenReg?

Fit Model(
	Y( :Result ),
	Effects( :Group ),
	Target Level( "Pass" ),
	Personality( "Generalized Regression" ),
	Generalized Distribution( "Binomial" ),
	Run(
		Fit(
			Estimation Method( "Logistic Regression" ),
			Validation Method( "None" ),
			Multiple Comparisons(
				Effect( :Group ),
				Least Squares Means Plot,
				Comparisons with Control(
					1,
					Control Level( "Group:A" ),
					Comparisons with Control Decision Chart(
						Control Differences Chart(
							1,
							Point Options( "Show Needles" )
						)
					)
				),
				Tukey HSD( 1 )
			)
		)
	)
);