442367_en-US

liliconcom

t-test for results combined from two tests

I received the following email from a customer today.  Please let me know if you have any thoughts.

 

I have a problem where I have two tests that we are running and we are combining them into one metric and we want to compare two tires to see if they are different.

We have Na measurements from each test for tire a and Nb measurements from each test for tire b

m1 is mean of test 1 and m2 is mean of test 2, and s1 is standard deviation of test 1 and s2 is standard deviation of test 2

The combined metric for each tire is calculated with simple linear equation.

f(X,Y)=a*m1+b+c*m2+d where a, b, c, d are constants.

I estimated that the combine standard deviation would be

If the two tests are independent then I can assume s12 = 0 and the formula could be simplified further.

We want to do a t-test between  f(x,y)  on two different tires.

I was thinking I could manually calculate sf from each of the measurements and equation above for each tire.

Then I could estimate the standard error of the difference

Then generate the test statistic as difference of fa and fb divided by standard error of the difference

Then look of the critical t value for a=0.05 and df = Na+Nb-2 and determine if they two values were the same or not based on 95% confidence interval.

I am wondering if there is a way to do this in SAS JMP instead of having to do this manually?

Re: t-test for results combined from two tests

Thanks @Mark_Bailey.  I will ask them and send them documentation about contrasts.  

Re: t-test for results combined from two tests

I suspect that @Chris_Kirchberg might be right (not valid) but have they tried using contrasts?

Re: t-test for results combined from two tests

@karencopeland   Thanks for your explanation.  Totally agree that sometimes we go down the rabbit hole on a solution and a step back to the original problem statement can uncover a simpler solution.

Re: t-test for results combined from two tests

I think you have a solution to their desire to combine both tests into a single metric from which they can make a "black and white" decision. I get that for routine testing and since this is routine we can make the assumption that they understand the 2 metrics that go into the one.  I was pointing out that sometimes people get caught up in methods when they should be solving problems, and sometimes solutions are not that complicated....but we can go with this is routine testing so then they need a black and white repeatable way to make decisions.

Re: t-test for results combined from two tests

Thanks, I was going to guess they must be destructive tests.

I think Sam's approach is probably the way to go.  Reminds me of gene expression. to make the genes comparable across individuals, one has to scale them but they are not usually combined. I would think performing the scaling on the original data and not the means would be more appropriate to make a single value.  Another thought is that this is like what is done in Risk Based Monitoring for JMP Clinical, A weighted average to make a single score for flagging or assessing risk. 

It sounds like this is the approach they want to take. but then test the difference in the scaled (and maybe centered) scores.

Re: t-test for results combined from two tests

@SamGardner   Thanks for this perspective, totally makes sense.  I will share this as an option - along with the option to script the calculations.

@karencopeland  Interesting question about whether they need a test or need to solve a problem.  Can you elaborate on the distinction between the two?  As I understand, this is testing that they do on a routine basis.  The two tests represents different aspects of the same attribute (durability) and want one metric that combines the two.

Re: t-test for results combined from two tests

Do they need a test or do they need to solve a problem?  Analysis by GB shows that the tires differ by test 1 but are relatively similar on test 2.

Re: t-test for results combined from two tests

I don't believe there is a direct way to do this kind statistical hypothesis test without some scripting.  However, my guess is that the a,b,c,d coefficients in the f(x,y) equation are intended to bring the measurements X and Y (Test 1 and Test 2) onto the same relative location and scale.  See the attached table where I calculated a rescaled test 1 and rescaled test 2.  If the two measurements have similar relative error (relative standard deviation), then this would mean that you would expect the rescaled Test 1 and Test 2 results to be from the same distribution.  And if that is true, then combining the rescaled Test1 and Test 2 results together into one column, you can then do a T-test on the combined rescaled results.  Under the assumption that the standard deviations of the rescaled results are the same, then the pooled standard deviation and degress of freedom for the rescaled test results should be the same as if you used the estimate of the standard deviation of the transformed means.  

 

Re: t-test for results combined from two tests

Both of the tests are destructive so each is done on a new tire.  They combine the results for the two tests because both tests provide two aspects to one type of durability. 

Re: t-test for results combined from two tests

Ha, when I responded the page did not refresh to show the data sets, now I see them. The data seems a bit messed up. It looks like they do not perform both tests on the same tire. I don't think what they are trying to do is valid.

 

they could just make column formulas for the second data set (ExampleDataSetMetric). But think it would be better if they did not try to combine the two response and then do a t-test. It would be better to do them separate even if they are independent or dependent. They can combine the p-values into a single probability and use that instead if they are trying to see if there is an overall different.

Maybe our local statisticians can chime in on this.

Re: t-test for results combined from two tests

@Chris_Kirchberg  Correct on the columns but each row is for a tire and either Test 1 or Test .  They happen to have 38 rows - 18 results for Test 1 and 20 results for Test 2.

Re: t-test for results combined from two tests

Hi @andreacoombs1 ,

So they have  Column 1 is Tire type (a or b)  Column 2 is Test 1 and Column 3 is Test 2? Each row is a measurement for each tire (either a or b) for each of the two tests?

Correct?