Functional data explorer

Peter_Hersh

I have a customer who is attempting to user FDE to determine if their data from censors measured over time is "typical".  They want to use FDE to create FPCs and have these FPCs be used to judge if a run is typical and flag a run that is non-typical by creating specs for the FPCs.  They don't want the model to update with new data just to analyze the data with the current model and create FPCs for the new data.  This seems to work really well by having the new data be validation data, but the issue is the model has to "re-run" on the old data, which takes quite a bit of computing power and time since the model that worked really well for them was a p-sline model with 400 nodes. 

 

Does anyone know of a way to save the spline formula for the functional principal components without having to re-run the model through FDE everytime?

 

Along those same lines is there a way to save the data that has gone through the cleaning steps, aligning and transforming steps?

 

One other word of warning I would not recommend running p-splines in a live demo on a descent sized data set.  Took about 15 minutes then crashed JMP when the customer was looking at ~500,000 rows of data on a 16GB machine.

RyanParker

Unfortunately we don't have a way to save the model fit, so it does need to be re-fit each time. In 14.1 we will have the ability to save the prediction formulas with associated JSL functions that will allow users to better explore the model fits with the Profiler. This will also allow for function re-construction using FPC scores. But this still won't provide a way to directly score new functions without a re-fit of the original model. 

It's possible the user could explore other techniques for making these predictions, such as creating prediction formulas for the FPCs using some observation aspects of the functions. Then, the formulas in 14.1 would certainly help them without being the ideal solution.

For saving data, you can use the top level RTM option "Save Data" to do just that. The user can also "Save Data" after fitting a model to see the modeled response (Y) values (from the model RTM).

 

 

Also if you have any crash repots we could look at please send them in!