nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Multiple Hypothesis Testing

bluenose

Hi all,

I will need very soon to perform a big number of hypothesis testing. Since it's well known that in such a case relying on p values is a wrong thing to do, due to type I errors, I would like to use the false discovery rate, and more preciecly the positive false discovery rate (Storey), and to compute q-values as my "adjusted p-values".

My question is divided in two:

1. Is there anywhere a code to calculate the pFDR and the corresponding q-values in JMP 9 ? I saw something on the Internet from which I assume that such functions already exist in JMP Clinical. I only have a licence for JMP 9, and I have no idea how much JMP clinical cost or if I will have any budget. So if there is a simple JSL code somewhere, it can be very usefull for me.

2. In case there is such a code, most likely it will require a vector of p-values as output. Consider that I have a JMP file with 1 categorical variable, and 500 continuous ones, and I want to run a t-test for each one of the 500, and to save in a new file, a variable of the p-values, containing 500 of them. How do I do that in JSL ? Is it possible at all ? That JMP will run automatically 500 tests and save only the p-values ?

Thank you in advance....

Accepted answers

All comments

David_Burnham

I think that pFDR exists in JMP Genomics, but not standard (or Pro) JMP

1. I am not aware of freely availible JMP code but there are several options in R. http://strimmerlab.org/notes/fdr.html lists some FDR-related packages. JMP 9 can connect to and retrieve results from R. That is probably a much simpler task than coding the algoritms in JSL from scratch.

2. That should be possible. Below is an example that may give you some ideas. It works by looping through all numeric y-columns to the right of the fixed x-column in a oneway fit and retrieving the p-value (two-sided t-test) from the report and insert that into a matrix. The matrix is finally added to a column in a new data table.

dt = Open( "$ENGLISH_SAMPLE_DATA/Big Class.jmp" );

m = [];

For( i = 4, i <= N Col( dt ), i++,

ow = Oneway( Y( Column( i ) ), X( :sex ), t Test( 1 ), invisible );

owr = Report( ow );

m = m || owr["t Test"][Number Col Box( 4 )][1];

);

New Table( "t-tests", New Column( "p", numeric, set values( m ) ) );

bluenose

Thank you, that's a good idea...

There is one more thing I need to get it done (accept my apology, since I am too busy I never have time to learn JSL proprely...). From your code I get a end up with a JMP table containing 1 column, of p values, right ? Should I now write a JSL code that connects to R, if so, I'll need a little guidance, or, should I export the table and work directly from R, it should work, isn't it ?

If anyone can write me a quick simple example of how after retreirving the new table, to connect to R and to run even something simple like mean(p), I can modify it for the pFDR later.

and one last question (promise), in the line:

m = m || owr["t Test"][Number Col Box( 4 )][1];

why m or owr[...] ? I mean, why the || is needed ?

Thanks a million, you are saving me lot's of time exporting big datasets to R and coding there....

For matrices || is an horizontal concatenation operator. Concat(m, owr...) is equivalent.

I have no experience of FDR but I have played a little with JMP-R conncention. Below is an example. The first half is identical to the above example. The output is shown in the jmp log. To get the output as a string put the the R Submit inside the Log Capture() function.

dt = Open( "$ENGLISH_SAMPLE_DATA/Big Class.jmp" );

m = [];

For( i = 4, i <= N Col( dt ), i++,

ow = Oneway( Y( Column( i ) ), X( :sex ), t Test( 1 ), invisible );

owr = Report( ow );

m = m || owr["t Test"][Number Col Box( 4 )][1];

);

dtp=New Table( "t-tests", New Column( "p", numeric, set values( m ) ) );

varname="p";

R Init(); // Initialize R connection

R Send( dtp ); //Send data table to R

// Submit a string with R commands

R Submit( "Data <- dtp

names(Data)

attach(Data)

mean("||varname||")" );

Rterm();

David_Burnham

Here are some slides that explain how to use JSL to connect to R

Lots of good information in those slides. Thanks for sharing Dave.

bluenose

Thank you for the help, everyone.

After the weekend I will try to work with the code and will read the slides.

Thank you Dave for the link, it looks like it's just what I need, and thank you everyone for the assitant, I appreciate it !