Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactorI came across an attached short paper related to predicting real-time spectrum data obtained from a Raman device within a bioreactor using PLS (Partial Least Squares). I've also attached the raw data that can be utilized for this purpose. This raw data is from potential user in Thermofisher scientific Korea.
Is it possible to predict spectrum data like this in JMP? I often encounter USP researchers in biopharma who show a strong interest in using Raman data. and this data and paper is great example for this.
Chris_Kirchberg ) and Ben ( @Ben_BarrIngh @Bingh94 ) , who may have valuable insights on this.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactorHi @HyeWonJung,
I will take a look at the attached PDF to learn a little more about this but it initially sounds like a real-time prediction? So as the probe is measuring values it is making a prediction of what the next point will be? That sounds like forcasting/time series like data with a model that updates as it gets new data.
I will probably have more questions/answers once I read the PDF
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactorHi @HyeWonJung I took a quick look but it appears the data may be incomplete. For example, the paper shows graphs with x-axis values from 2022 but the rama results.jmp table timestamps are all 2023. Also, it looks like we do not have the "offline" data.
From where did you get the data? Is there more available? Is there any code showing how they fit their PLS models and computed prediction statistics? What Xs and Ys did they use?
We can very likely reproduce what they did and probably improve on prediction results using methods more modern than PLS, but we need to start from the same place they did.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactorIs this public data? If it is, then we are OK to use it and need to give the source of it. If it is customer data that is not public, we must delete it immediately and cannot use it, unfortuatnely. And as Russ has mentioned we would need the offline data to build a predictive model using the ground truth (the labels). After reading the paper, it looks like the model was built for calibration purposes so it can correctly display the values. The data in the data table looks like predicted values in units of the metabolite. We would need the original spectra as well to build the predictive model.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactor@russ_wolfinger @Chris_Kirchberg I apologize for not being clear with my previous question. The data and the paper I referred to are unrelated. The data is private and was provided to me by a potential customer from Thermofisher scientific Korea. (As Chris advised, I have deleted the data.)
The potential customer sent me real-time Raman data and asked how it could be utilized within JMP. He did not specify exactly how they want to utilize this data.
I obtained the PDF through a personal channel to find ways to make use of Raman data. After reviewing the materials, I guessed about what they might be looking for.
Anyway, I will ask the customer specifically what they want to achieve and what they expect from this data.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactor@russ_wolfinger @Chris_Kirchberg He(My potential customer) has the data, but he doesn't really know how to utilize it effectively. he's someone far from data analysis. So I searched pdf for practical use cases with raman to show practical cases in a presales meeting to him. However, I think it would be better to ask the customer specifically what they want to achieve. And.. If you have any good ideas related to the utilization of Raman data, please feel free to share them.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactor@HyeWonJung Thanks very much for the clarification--makes sense now! I was allured by the predictive modeling problem as in the title and Thermo-Fisher paper and would love to get hands on that data. For your customer data, it looks like a much better starting point would be a more exploratory and pattern discovery approach.
JMP is very strong here with numerous possibilities: Multivariate, Explore Outliers, Functional Data Explorer, Control Chart Builder.
If the conversation evolves towards predictive modeling, we would need to set up reasonable inputs (Xs) and outputs (Ys). This opens up a whole new set of possibilities and we remain just as powerful, and even have clever approaches like converting batch data into images.
Re: Prediction using PLS of spectrum data (metabolite data) monitored by Raman in a bioreactorI'll discuss the specifics with the customer, and if I encounter any difficulties, I'll reach out to you for help. Thank you!
