Hello all,
This is more "spit balling" than anything, so here goes. In a nutshell, I am trying to develop a diagnostic test for reef corals so that we may know which ones will likely die as a result of climate change and which will not. From a series of experiments carried out in the laboratory, I now know the protein signatures associated with resilient and stress-sensitive corals so that, in theory, I may now have the capacity to go out into the ocean, sample new colonies that I don't have any a priori knowledge on, and make predictions about their health.
Analytically, I have tried quite a few things with the proteins I have profiled, large proportions of which are collinear (making standard regression approaches poorly suited). It seems like using the discriminant analysis approach for "healthy" vs. "stressed" coral datasets could give me the answer that I want. I played around with stepwise discriminant analysis and basically tried to identify the minimum number of proteins that would 100% correctly sort the samples by health status. Am I interpreting this correctly? It seems like discriminant analysis is basically a MANOVA with some sort of predictive function (though MANOVA cannot be used in this case since I have 272 proteins measured in only 11 samples).
I saw a great JMP webinar today on generalized regression, so I have played around with that. I set my Y as health status (binomial: healthy vs. stressed) and used the 272 proteins as x variables. In some cases, the final set of "most informative" proteins were similar to those derived from DCA, but there tended to be more (on the order of five proteins could basically be used to build a model that could accurately predict coral health). One issue, of course, is that I am using samples where I already know the answer (i.e., whether the corals lived or died). Therefore, I risk having overfit it. Discriminant analysis seems to be literally just fitting the data entirely, perhaps not leaving any wiggle room for randomness or noise. Since I created a validation column for generalized regression, that should, in part, reduce the possibility of over-fitting, right?
And then there is partial least squares, but every time I've used PLS with OMICS (transcriptome or proteome) datasets, the R2 is always pitifully low, like 0.05-0.07. I saw a really excellent webinar last week on generalized regression in which the presenter basically use Formula Depot to compile the models derived from a number of approaches, which basically gave me the idea to do this sort of analysis. I guess, in closing, my main question is, when you are trying to make predictive models for a binomial (healthy vs. sick, in this case) response with hundreds to thousands of predictors, is any one of these methods considered the "gold standard?" It might also be worth mentioning that the data are typically non-normally distributed. This dataset is very small, and, if you click on the PCA, the unhealthy samples are very easily distinguished.
