nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Singular value decomposition outside of JMP

caseylott

Hello- I’m a long-time JMP user (3.2!) and an abysmally untalented beginner in R. Text analysis is new to me, as an ecologist, But I’ve really been enjoying playing with the Text Explorer platform. I have produced a sparse document term matrix that I would like to subject to singular value decomposition in order to do additional analyses in JMP on a dense matrix of singular values for ~100 vectors. I have run into a bit of a wall since I do not have JMP Pro (can’t afford it). I think this means I need to read my sparse document term matrix into R, run some kind of svd function, and then read my new dense matrix back into JMP. This seems like it might be an easy task for someone that is more familiar with text analysis and proficient in R. I’ve struggled to figure out which R package to use to do this (it seems like there are many). When I’ve looked at documentation for different packages, I’ve struggled to figure out which functions and arguments I should use to accomplish this task. I know, it’s time for me to commit to learning R. JMP has shielded me from this inevitability for many years. If there is anyone out there who could share some R advice, or better yet, code, that would read in a sparse matrix as a .csv, create a dense singular value matrix in R, and then write this matrix as a new .csv, I would be very grateful. Thank you!

Accepted answers

G_M

@caseylott a good way to get started using R and JMP is to replicate in R results that you are familiar with in JMP.

Like you, I prefer JMP to R for so many reasons, in fact I was using R back in 2009 when I was presenting results to an executive team and

as always happens, the executive had additional questions, (which I failed to anticipate, I have gotten better at this) and I couldn't just

produce the new analysis in R, live. I asked for 25 mins to go and conduct the analysis and bring results...I lost their attention. That night

I went home and reflected how I could avoid this in the future..1) better anticipate further analysis and 2) identify a more agile software.

JMP re-entered my life (a colleague introduced me to JMP a few years earlier) and has been my chosen software since.

That said, here is some JMP to R syntax that will enable you to get started.

This may be too basic, but I hope is helpful. I use R to estimate non-linear mixed effects models, polytomous IRT models, and recently started estimating

Bayesian models using both R and JAGS all through JMP.

Names Default To Here( 1 );

//Open Big Class in JMP

df = Open( "$SAMPLE_DATA/Big Class.jmp" );

//execute bivariate regression of weight on height in JMP

df << Bivariate( Y( :weight ), X( :height ), Fit Line( {Line Color( {213, 72, 87} )} ) );
//make sure that you have embedded log open in your JSL Scripting window.

//Initiate R
R Init();

//send data frame to R
R Send( df );

//submit R Code for bivariate regression of weight on height
R Submit(
"\[　
fit1 <- lm(weight~height, data = df)　
summary(fit1)　
]\"
);//close R Submit
//Look into the log and see the R Results

All comments

Mark_Bailey

You are welcome to use R with JMP but I don't think that way is necessary. You can save the weighted document-term matrix from the Text Explorer and then analyze the new data columns with any platform in JMP, including those in the Multivariate group. This includes principal components analysis, latent class analysis, and latent semantic analysis without JMP Pro.

Note that the versions in TE are optimized for sparse matrices like the DTM, but you should at least try to use the stand-alone platforms.

caseylott

OK, thanks for the suggestion. I've tried a couple of things so far that haven't worked. Perhaps I'm doing something wrong. I've saved the weighted document term matrix and then entered it in as the y variable for: 1) the principle components platform and 2) the latent class analysis platform. I don't see any options for latent semantic analysis in my non-Pro version of JMP. Where would these be located?
When I try to run the weighted document term matrix in the principle components platform, JMP just spins and inevitably crashes.
When I try to put the document term matrix into the Y role in the latent class analysis platform, I get the message that I am using an unsupported data type.
Any suggestions?

Mark_Bailey

First, I misspoke. It is difficult for me to remember which features are in each version of JMP and all the differences between JMP and JMP Pro. You are correct: only PCA and latent class analysis are available in JMP.

Second, I was afraid that the large and sparse DTM would be too much for JMP outside of the Text Explorer platform. You might try trimming the term list. In general, curating the term and phrase list can lead to much more meaningful analyses than relying on the default parameters. This is one way to reduce the size of the DTM. You can also reduce the maximum number of terms or increase the minimum term frequency when saving the DTM.

Last, are you using 64-bit JMP?

caseylott

Thanks, Mark. I haven't tested how far I need to go with shrinking down my document term matrix to be able to use the latent semantic class analysis in the JMP (not JMP Pro) platform. Perhaps I could make this work, and I'll try, but what about when I have an even larger document set? I predict that I'll run into this problem again.

I think that my original assessment may still be correct. It seems like I still may need a way to transform my sparse document term matrix (created in JMP) into a dense matrix (via singular value decomposition in R) so that I can read the dense matrix back into JMP for additional multivariate analyses.

In other words, I think that my inability to create a dense matrix of singular values in JMP, without JMP Pro, is truly limiting my ability to go very far with text analysis in JMP.

If anyone has any other suggestions for how to deal with this in JMP, or perhaps some R-based solutions, I'd love to hear them.