Topic score interpretationWe are working on the Text Explorer approach and trying to communicate the interpretation of Topic Scores. For example, a negative value compared to a positive value for a term within a topic. Can you direct us to someone who can help with a few examples? The help menu suggests that a negative value term would occur less frequently compared to a positive value, but when we look into actual data sets, this does not seem to be playing out.
So it seems that it is a linear scale where each one end is positive and one end is negative. The more closely a word is towards one end of the scale would indicate a positive or negative usage. Any perspective is appreciated.
Re: Topic score interpretationHi Laura,
Sorry, I read this answer, but I never said thank you. Thanks a lot! This was very helpful.
-Jeff
Re: Topic score interpretationTechnically, the reverse would also apply but, in practice, I wouldn't expect you'll encounter that scenario. The reason is that we only flip the sign when most of the large loadings are negative. With text data, it's not very common to have large positive and negative loadings. Only in those cases you'd expect to find the reverse. In sum, you're doing the right thing in examining your topic loadings along with your scores to make sure you understand the meaning of the topics. You want to be able to label each topic (e.g., "satisfaction with product") so you can then interpret the scores as "higher scores mean more satisfaction with the product" or the reverse.
Best,
~Laura
Re: Topic score interpretationThank you, this helps with my immediate example. A similar question is should I also expect the reverse? For example, a strong negative example in the Topic Score Plots that results in a positive topic score where the text would imply that it should be strongly negative?
Re: Topic score interpretationHi @reese_sj,
Thanks for pointing that out! Indeed, there are cases in which the singular value decomposition leads to topics with all-negative loadings. To facilitate interpretation of topics, the signs of the loadings are flipped so they're always positive. In those cases, the topic scores should also be flipped but they're not currently. We're aware of it and plan to address this in a future release. For now, you could create a new formula column that takes the topic scores and multiplies them by -1. A simple way to figure out which topic scores need to be flipped is to (1) look at the "Topic Score Plots" on the report, (2) select the observations with most extreme scores, (3) click "Show Text" and (4) check the correspondence between documents and terms in the topic.
Here's an example with the "Pet Survey.jmp" data:
I launched Text Explorer, ran a Latent Semantic Analysis and a Topic Analysis with all the defaults. The top loadings for the first two topics are:
We can then look at the score plots and get the text for the highest scores of Topic 1:
The positive topic loadings correspond with the content of the documents. Thus, higher scores in Topic 1 point to documents that are more about "dogs guarding houses."
In contrast, when we examine the text in the most extreme values of Topic 2, we find that lower scores correspond to documents that are more about "huskies enjoying sled rides in winter"
Perhaps the easiest way to identify when scores should be flipped is when the most extreme values are negative but the loadings for that topic are positive:
Hope this enables you to move forward with your analysis!
Best,
~Laura
Re: Topic score interpretationPossible issue with Topic Score interpretation:
I have a topic that "Seems" to have a positive effect on my star rating. I used the DTM and indicator columns to look at the 2 main terms in the topic- "Absolutley" and "Love." As seen, when I use one of the words my star rating goes up and if I use both words, it goes up even higher. Note it is a very small N.
Digging further into this, it seems that when I save my topics, there is a negative correlation between count and how the term is scored. This happens even though the loading is positive. I was able to go through and find it for this topic, but if I was to load this into genreg (or any other modeling approach), I have no idea whether or not the effects are correct- showing positive or negative for the topics without manually going through each and every topic and term to make sure they are being treated appropriately.
Seen below, if I use the words Absolutely and Love, the more they are used or if they are used together, it significantly drives a negative effect as per any modeling approaches.
Re: Topic score interpretationI will definitely try that. Thank you.
Re: Topic score interpretationScott,
If your goal is to screen out the terms that are relevant to your outcome variable, you might just want to run the GenReg with the DTM columns. Since it is a large regression it will take a while to finish. For example, the ouctome is a Yes/No type response and I ran the Lasso with 1,782 binary DTM columns on a data set of 18,000 rows. It took about 20 minutes to complete. I found 483 active (non-zero) terms. If I only keep the statistically significant ones (p-value<0.05) the # of terms are just 203.
A more visual approach is to use the Coloring/By Column Value option in the Word Cloud. Coloring the words by the outcome variable let you see which words are associated with outcomes. The ones with mixed color are not or less associated with the outcome.
Jian
Re: Topic score interpretationBy the way, I used a percent sign, and the entry was recoded as a "Bleep"... perhaps another recode opporunity 
(smiley face).
Re: Topic score interpretationAs I continue to evolve and improve upon my topics, I am looking for guidance on the addition of stopwords. I know each term is correlated with every topic, but I would like a way to reduce the diversity of each topic such that there are fewer terms that have a strong correlation with the topic. This enables me to appropriately take action on select topics of interest. My current data set has an outcome variable.
Can you please comment on the following approach: build the topics (number of topics based on a singular value **bleep** percent cutoff), save the document topic vectors, run genreg with the topics, look at the topics of least impact, identify the lead term from each topic of low impact and add that as a stop word, and then repeat the cycle until I get to a "workable" solution.
Re: Topic score interpretationJeff,
Thank for your question. Let me add to Laura's excellent explanation. Yes, the term topic loading represents the correlation between a term and corresponding latent topic. However, the document topic vectors (i.e., topic scores) are not scaled by JMP out of the concern that a DTM might be too big to do so, so the correlations between the document topic vectors and DTM columns do not match the reported term topic loadings. But, as you found out, they are very close.
Jian
Re: Topic score interpretationHi @JeffSwartzel,
Excellent question! Latent Semantic Analysis (LSA) and Topic Analysis (TA) rely on a sparse singular value decomposition (SVD) to characterize the structure of the data in a very efficient way. That is, despite having a document term matrix (DTM) with hundreds or thousands of columns, the sparse SVD algorithm enables us to get only a fraction of the singular values and singular vectors (the most important ones).
To continue to take advantage of the sparse SVD, the TA scores are computed differently in rotated PCA and TA (note that scores are identical between standard PCA and LSA). In rotated PCA, the scores are scaled by the inverse of the correlation matrix of the DTM and the singular values (inverting the correlation matrix of the DTM would be intractable with a large corpus; imagine inverting a matrix with thousands of columns). Whereas TA scores omit this scaling. The TA scores are simply a projection of the DTM onto the rotated, lower dimensional space.
Moreover, we find that TA scores are highly correlated with PCA scores ( > .90) despite the differences in computations. Importantly, the topic loadings ARE the exact correlation of the DTM column and the corresponding PCA component. Thus, we can think of the TA scores as very good approximations of the PCA scores.
HTH,
~Laura
P.S. These are the formulas:
pcaScores = DTM_s * Inverse(DTM_cor) * evec * Diag(Sqrt(eval)) * rotMat
taScores = DTM_s * evec * rotMat
where DTM_s is the standardized DTM, DTM_cor is the correlation matrix of the DTM, evec are the eigenvectors (or right singular values) of the DTM, eval are the eigenvalues (or squared singular values) of the DTM, and rotMat is the rotation matrix.
Re: Topic score interpretationThanks!Re: Topic score interpretationAnd for anyone else reading this, there's an on-demand version of @jiancao's webcast:
Marketing Analytics and Consumer Research.
Re: Topic score interpretationHi @JeffSwartzel, that was @jiancao at today's Mastering JMP webcast.
By tagging him here he'll be notified about your question and can chime in as appropriate.
Thanks!
Re: Topic score interpretationHi Chris,
I just watched a JMP webinar called "Using JMP Pro for Enhanced Marketing Analytics & Consumer Research (Advanced)".
I'm embarrassed that I missed the speaker's name, but thought that he gave a very intuitive interpretation of the topic loadings. If I understood him correctly, the topic loading for a particular word in a topic could be interpreted as the correlation between that word's vector in the document term matrix and the corresponding document topic vector. To make up a fake example, let's say we have "Topic 1" and the first term in that topic is "oranges". The loading on "oranges" is shown as 0.85361. If calculated the correlation between "Topic 1" and the vector for "oranges" in the document term matrix, it should also be 0.85361.
I thought that this was a great explanation, and I tried to recreate it myself, but I'm having some trouble. The correlations are close but not exact. Do you have any ideas for why this might be the case? If it would help, I could try to provide an example with the "Aircraft Incidents.jmp" file.
Any help is appreciated!
Thanks,
-Jeff Swartzel
Re: Topic score interpretationThank you. I do have a follow up question with regards to the documentation. Currently, it states that within the Topic analysis, negative values indicate a repulsion force and will occur with less frequency compared to positive values.
file:///C:/Program%20Files/SAS/JMPEA/14/Help/index.html#page/jmp%2FBA_Text_Explorer_19.html%23ww428969
Re: Topic score interpretationThank you, this is very helpful.
Scott
Re: Topic score interpretationHi Scott - The best way to understand the SVD-based results in Text Explorer (TE) by understanding that (1) The SVD results are identical to a PCA of the document term matrix (DTM) and (2) The Topic Analysis is a VARIMAX rotation of the SVD results, which means that the Topic Analysis is mathematrically identical to a factor analysis of the DTM.
This means that all of the interpretative tools and rules of thumb that you can glean from a text on factor analysis applies to topic analysis. This includes
(1) The sign of the scores has no meaning. The JMP numerical tools make it so that the largest loading for each dimension (unrotated loadings) is positive. This convention was adopted to make the results reproduceable and unique (otherwise asking for 5 dimensions and then 10 dimensions can lead to results that differ by a factor of -1). The convention for the loadings has to be carried through and applied to the scores.
(2) Topic Analysis based on the Centered and Scaled SVD has the same rules for an upper bound of the number of factors as in traditional factor analysis. For example if there are only 10 singular values that are greater than 1.0, then you would never ask for more than 10 Topics.
Laura Castro-Schilo, JMP R&D's resident expert in multivariate analysis, has written two excellent blogs about understanding and taking advantage of the correspondence between Text Explorer and classical multivariate statistical methods.
JMPer Cable: The new look of Text Explorer
JMPer Cable: Text analysis in the social sciences: A new spectrum of possibilities
@reese_sj wrote:
We are working on the Text Explorer approach and trying to communicate the interpretation of Topic Scores. For example, a negative value compared to a positive value for a term within a topic. Can you direct us to someone who can help with a few examples? The help menu suggests that a negative value term would occur less frequently compared to a positive value, but when we look into actual data sets, this does not seem to be playing out.
So it seems that it is a linear scale where each one end is positive and one end is negative. The more closely a word is towards one end of the scale would indicate a positive or negative usage. Any perspective is appreciated.
