SEM Experiences: Limit for number of parameters to be used?If I have a data table with lots of parameters (>30) and would like to start the SEM platform, it opens up smoothly within seconds for e.g. 15-20 parameter, but last many minutes if not tens of minutes for e.g. 30+ variables. You can test with e.g. semiconductor capability. Reducing number of rows will fasten up things (e.g. using 200 instead of 1455) and allows for more columns therefore.
Is there a reason for longer opening time (not talking about modeling itself?) Are there any calculations done before opening the platform behind the scenes which are dependent on nr of cols and nr. of rows?
THanks,
Martin
Re: SEM Experiences: Limit for number of parameters to be used?Hi @martindemel,
@LauraCS will need to confirm, but I believe the statistics available under Red Triangle > Univariate Simple Statistics are being calculated upon platform launch, which means the correlation and covariance matrices (pairwise) are computed. Those will take some time for a large number of columns.
@julian
Re: SEM Experiences: Limit for number of parameters to be used?Hi @martindemel,
Your intuition is correct (as is @julian). The univariate and multivariate statistics are computed upon launch. However, the reason you're noticing the delay is because of the additional estimation of 2 models: the (1) Unrestricted and (2) Independence models are also being estimated upon launch. #1 has p(p+1)/2 parameters, where p is the number of columns, so this can make things go slow in 15.0 (with semiconductor capability, 128 columns results in 8,256 parameters for the unrestricted model). Moreover, if you have missing data, then the number of rows also contributes to the slow down but with complete data the rows don't hurt too much.
The good news is that 15.1 has an improved estimation algorithm which is much faster! The caveat is that intrinsic characteristics of the data can make #1 or #2 easier/harder to estimate, so there could still be some examples in which the platform takes a while to launch.
I plan on adding a progress bar and eventually the choice of avoiding the estimation of #1 and #2 to make things go faster (although the latter would require disabling a number of features).
Best,
~Laura
