nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How does JMP handle Irregular Data in Functional Data Explorer?

statistic_nerd

I have irregular functional data (16 functions each measured at different points of time). I was able to upload the data directly to JMP and without any data preprocessing, JMP chooses the perfect penalized P-spline model and outputs the lambda and the function output's standard deviation in the fit statistics. My question is, how did JMP get the function output's standard deviation? and how did it handle the irregular data? Did it make it regular before fitting the model?

The function's output standard deviation value is different than the value stated in the observation summary from the data so that also confused me.

Accepted answers

Victor_G

Hi @statistic_nerd,

(B- or P-) Splines are piecewise functions using polynomials, they do not require equal/regular timespaces as it interpolation will be done between those points : Spline (mathematics) So JMP can fit B- or P-Splines directly, without any data transformations.

The part you mentioned is about the calculation of specific metrics like Integrated Difference, RISE, RIFS, which require input values from 0 to 1 to calculate integral formula. If your data is not in the range 0-1, JMP will rescale/tranform your input data to have the correct range of input values with a formula like X' = (X - Xmin) / (Xmax - Xmin).

Hope this answer will solve your questions,

All comments

Victor_G

Hi @statistic_nerd,

Welcome in the Community !

When you're mentioning "irregular data", I guess you're mentioning the fact that functional data comes at different time points, not necessarily the same ? Depending on the models family chosen, you may not need to have the data on regular timespaces : Types of Functional Model Fits For models family requiring evenly spaced grid (like Wavelets), JMP do a transformation of the data to have the required input format.

About the function's output standard deviation, which part of the report are you mentioning ? In the model selection panel, the function standard deviation is defined as "the residual sigma from the fitted model." (source : Basis Function Expansion Model Report). In the Summaries, the response standard deviation is the calculation of Std on raw data, there is no fitted model behind. You can look at an example here to see the differences : Additional Examples of the Functional Data Explorer Platform

I hope this answer will help you,

statistic_nerd

Hi Victor,

Thank you for getting back to me. Yes you are right, I meant data that comes at different time points. I have chosen P-spline model but during my course studies I have learnt that fitting "irregular" data to p-spline would require that the data is first regular which peaked my interest because I dont select any specific data preprocessing options and JMP can still perform P-spline fit so I was wondering what are the transformations (if any) that JMP does to the data?

Thank you for the residual sigma that helped clear it out, but another follow up question: In the equation mentioned Statistical Details for the Function Summaries Report it says that it assumes input data is from 0 to 1, does that mean it would still apply these equations if the input is not from 0 to 1?

Victor_G

Hi @statistic_nerd,

Hope this answer will solve your questions,

statistic_nerd

Thank you! That does clear out my doubts!