It's been a while, but this is the last of the three-part series. Part 1 Now for the emotional analysis of the lyrics of "The Beatles," Part 2 So, I tried sentiment analysis of the lyrics of "Queen". In Part 3 of this time, we use a function called Term Selection to create a model that predicts which band's song is from the words contained in the lyrics of the two bands, and characterize each band. Look up words (keywords).
What is Term Selection?
In JMP, text analysis is available through the "Text Explorer" platform, but in JMP Pro, you can select "Term Selection" as an option for this platform.
In "word selection analysis", when there is a response variable (Y), it is possible to model the relationship between the word extracted by text analysis and the response variable, What words describe the response variable You can check things like
In this example, we will create a data table that summarizes the song titles and lyrics of "The Beatles" and "Queen" as shown below.
At this time, the response variable (Y) and predictor variable (X) are as follows.
Response variable (Y): "Band" (a variable that distinguishes whether the song is Beatles or Queen)
Predictor variable (X): Words extracted from "Lyrics"
Should refrain be considered?
Let's discuss the predictor variable (X) in a little more detail.
When extracting words from text, Document - Term Matrix (DTM) Words can be put together in matrix form.
For example, Method A. The lyrics of the Beatles song "You Like Me Too Much" on line 166 of , contain (at least one of) the words "know", "time", "just", so the variable "know Binary , "time Binary", and "just Binary" do not contain the value "1" and the words "love", "oh", "yeah", so "love Binary", "oh Binary", "yeah Binary" is given "0" as a value.
Method A. binary (1/0) document word matrix
Extracting the words in this way yields a "rows of data times number of words" matrix (with values of 1 or 0 in this example), and the actual predictors are the words in this document word matrix.
The above constructed the document word matrix with values of 1/0, but the following Method B. It is also conceivable to configure it with "frequency", such as how many times a particular word appears in the document.
Method B. Document word matrix of word frequency
In previous articles, I have mentioned that lyrics are greatly influenced by refrains (using the same words over and over again) in text analysis. Therefore, if we consider word frequency as in Method B above, it will be greatly affected by the refrain.
Therefore, when modeling, try modeling with both method A and method B and compare which model is better.
Method A. Binary (1/0) document word matrix ⇒ unaffected by refrain
Method B. Document word matrix of word frequency ⇒ Affected by refrain
Word choice analysis report
Based on the above points, let's use "word choice analysis". When you select the option Term Selection, you will see a configuration outline similar to the following.
Details of each setting
"Choose Response Column" : Specify the response variable (Y). In this example it is “Band” (Beatles/Queen).
“Target Level” : Specifies the event level in the response variable. Here we specify "Queen" and consider the Queen as the standard. Since Y takes only two values, Beatles and Queen, we fit a logistic regression model assuming a binomial distribution for Y.
"Model Settings" : Specifies the modeling method and validation method. By default, the estimation method is Elastic Net and the validation method is AICc, so we will use the default settings in this example.
"Term Settings" : Specifies which method to use for the document word matrix shown in the previous section. By default it is Binary(1/0). You can also specify the maximum number of words to use in the document word matrix, which is 400 by default. In other words, only the top 400 most frequent words among the extracted words are used as predictors.
Click the Run button in the upper right to view a report of modeling for the specified settings.
You can see the results in Report 1 below.
Report 1 "A Hard Days Night" (Beatles): Predicted correctly
Shows the results of variable selection for a logistic regression model using Elastic Net. In this example, out of 400 predictor variables, 49 variables useful for prediction were selected, and the regression coefficient (Coefficient) for the selected variables (Term) is shown in the upper right of the report.
Positive coefficients are indicated by orange bars and negative coefficients by blue bars. is a word that contributes greatly to predicting , and the larger the negative value, the greater the word that contributes to predicting the Beatles' level.
In the lower left of the report, the lyrics of "A Hard Days Night" (Beatles' song) are colored orange for words that contribute to predicting Beatles, and blue for words that contribute to predicting Queen. In the lyrics, you can see that many words that contribute to the Beatles such as "just", "give", "days" are used. So, even if you don't know which band's song this lyric is from, you can guess it's a Beatles song because of the use of just", "give", and "days".
More precisely, from these lyrics, the probability of predicting it as Queen and the probability of predicting it as Beatles are determined. Queen's predicted probability is obtained from the logistic regression coefficient obtained above using the Logist function ( Logist(x) = 1/(1 + Exp(x)) ) as follows.
Reposting the modeling report, the "Probability of Queen" column below shows, for each song, the probability of predicting the song as Queen, which for "A Hard Days Night" is 32.7%.
So, the probability of being predicted as the Beatles is 67.3% after subtracting from 100%. Than this,
From the words in the lyrics of A Hard Days Night,
Probability of predicting a song by Queen: 32.7%
Probability of predicting a Beatles song: 67.3%
So, I predict that it will be a Beatles song with a high probability.
becomes. That is, the actual value (actual value) and the predicted value (predicted value) match.
I will look at the reports for the other two songs (Report 2, Report 3).
Report 2 "Bohemian Rhapsody" (Queen): Predicting Correctly
Queen's representative song "Bohemian Rhapsody" can be predicted to be a Queen song with a high probability of 88.9%.
Report 3 "Hey Jude" (Beatles): Incorrectly Predicting "Queen"
The Beatles' representative song "Hey Jude" is predicted to be Queen with a probability of 56.3%, so the prediction is wrong.
Some songs are predictable like this, and some are not. When creating a prediction model, the misclassification rate, which indicates the ratio of differences between actual measurements and predictions, is used as an indicator of predictive ability, and the smaller this value, the higher the predictive ability.
Model comparison
In the previous section, we modeled using binary (1/0) as the document word matrix, but what happens when we compare it with modeling using word frequencies in the same way?
In the word selection analysis, the following report comparing the prediction ability of the models is displayed as a "Summary" report when multiple models are applied. Model 1 is binary and Model 2 is word frequency.
AICc used as a validation index, Generalized RSquare (generalized R-square, the larger the value, the better), AUC (the area under the curved surface, the larger the value, the better), Misclassification Rate (the misclassification rate, the smaller the value, the better) Looking at , you can see that the 2 values of Model 1 are good for any index.
From this result, it can be seen that the model with the occurrence of the word (=1)/no occurrence (=0) as the explanatory variable has a higher predictive ability than the model with the frequency of the word as the explanatory variable. .
summary
We summarize the results of the "word choice analysis" described so far.
・We created a model that predicts whether a song is about the Beatles or Queen based on the words contained in the lyrics of the song.
・The following two methods were modeled as document word matrices used as explanatory variables, and the predictive capabilities of the models were compared.
Method A. Indicates whether the lyrics contain the corresponding word : Binary
Method B.Indicates how many times the corresponding word appears in the lyrics: Frequency
⇒ We found that Method A (binary), which does not consider refrains, has a high predictive ability. Therefore, the results modeled by method A were used hereafter.
・Forty-nine words that have a large impact on prediction were extracted. Based on the regression coefficient estimated for these words, the prediction probability of each song was calculated, and the predicted song (predicted value) and the actual song (actual value) were calculated, and the misclassification rate was 8.4%. Met. Considering the R-squared and AUC values, which are other indicators of predictive ability, it is thought that a good predictive model was created.
⇒ With only the information of the 49 selected words, we were able to guess which song it was with an accuracy of 91.6%.
Finally, we show the 49 selected words (Term) and the regression coefficient (Coefficient) drawn with the graph builder. Words in orange contribute to Queen's prediction, and words in light blue contribute to Beatles' prediction.
The word with the highest positive value, "blow," is the defining word in Queen's lyrics. In fact, among the songs analyzed, blow is used in various Queen songs, but it is not used in Beatles songs, which explains why the coefficient of this word is large. can be attached.
We hope that this article will give you a chance to try using word choice analysis.
