It's been a while, but it's the end of the 3rd series. Part 1 Let's analyze the emotions of the lyrics of "The Beatles" Part 2 Then, I tried to analyze the emotions of the lyrics of "Queen". In Part 3 of this time, we will use a function called Term Selection to create a model that predicts which band's song is from the words contained in the lyrics of the two bands, and characterizes each band. Let's look up a word (keyword).
What is Term Selection?
In JMP, the "Text Explorer" platform allows text analysis, but in JMP Pro, you can select "Term Selection" as an option for this platform.
In "word selection analysis", when there is a response variable (Y), it is possible to model the relationship between the word extracted by text analysis and the response variable. What is the word that describes the response variable You can find out such things.
In this example, we will create a data table that combines the song titles and lyrics of "The Beatles" and "Queen" as shown below.
At this time, the response variable (Y) and the predictor variable (X) are as follows.
Response variable (Y): "Band" (variable that distinguishes whether the song is Beatles or Queen)
Predictive variable (X): Words extracted from "Lyrics"
Whether refrain should be considered
Let's take a closer look at the predictor variable (X).
When extracting a word from the text Document --Term Matrix (DTM) You can put the words together in a matrix format.
For example, Method A. The lyrics of the Beatles song "You Like Me Too Much" on line 166 of the song contain the words "know", "time", "just" (at least one), so the variable "know Binary" , "Time Binary", "just Binary" have a value of "1" and do not contain words such as "love", "oh", "yeah", so "love Binary", "oh Binary", "Yeah Binary" is given a value of "0".
Method A. Binary (1/0) document word matrix
Extracting words in this way creates a "row of data x number of words" matrix (in this example, the value is 1 or 0), and the actual predictor is the group of words in this document word matrix.
The above constructed a document word matrix with 1/0 values, but the following method B. It is also possible to configure it by "frequency" such as how many times a specific word appears in the document.
Method B. Document word matrix of word appearance frequency
In previous articles, I mentioned that lyrics are heavily influenced by refrain (using the same word over and over again) in text analysis. Therefore, considering the frequency of word occurrence as in method B above, it is greatly affected by refrain.
Therefore, when modeling, try modeling with either method A or method B and compare which model is better.
Method A. Binary (1/0) document word matrix ⇒ Not affected by refrain
Method B. Document word sequence of word appearance frequency ⇒ Affected by refrain
Word selection analysis report
Based on the above points, let's use "word selection analysis". If you select the option Term Selection, you will see a configuration outline similar to the following:
Details of each setting
"Choose Response Column" : Specify the response variable (Y). In this example it is "Band" (Beatles / Queen).
"Target Level" : Specifies the event level in the response variable. Here, specify "Queen" and consider Queen as the standard. Since Y takes only two values, Beatles and Queen, we apply a logistic regression model assuming a binomial distribution to Y.
"Model Settings" : Specifies the modeling method and validation method. By default, Elastic Net is specified as the estimation method and AICc is specified as the verification method, so the default setting is used in this example.
"Term Settings" : Specifies which method to use for the document word matrix shown in the previous section. The default is Binary (1/0). You can also specify the maximum number of words to use in the document word matrix, which defaults to 400. That is, only the top 400 most frequently extracted words are used as predictors.
Click the Run button in the upper right to see a modeling report for the specified settings.
We will see the results in Report 1 below.
Report 1 "A Hard Days Night" (Beatles): Correctly predictable
The result of variable selection of the logistic regression model using Elastic Net is shown. In this example, 49 of the 400 predictors are selected to help predict, and the upper right corner of the report shows the Coefficient for the selected variable (Term).
If the coefficient is positive, an orange bar is shown, if it is negative, a blue bar is shown, but since Queen is specified as the event level here, the larger the positive value, the higher the Queen level. It is a word that greatly contributes to predicting the level of Beatles, and the larger the negative value, the more it contributes to predicting the level of Beatles.
At the bottom left of the report, the lyrics of "A Hard Days Night" (Beatles song) are painted orange for the words that help predict Beatles and blue for the words that help predict Queen. You can see that the lyrics often use words that contribute to the Beatles, such as "just", "give", and "days". Therefore, even if you don't know which band's song the lyrics are, you can predict that it's a Beatles song because just "," give "," days ", etc. are used.
More precisely, from this lyrics, the probability of predicting Queen and the probability of predicting Beatles are calculated and judged. The prediction probability of Queen is calculated by the following formula using the Logist function (Logist (x) = 1 / (1 + Exp (x))) from the regression coefficient of the logistic regression obtained above.
When I repost the modeling report, the "Probability of Queen" column below shows the probability that a song is expected to be Queen for each song, which is 32.7% for "A Hard Days Night".
So, on the contrary, the probability of being predicted as Beatles is 67.3% by subtracting from 100%. Than this,
From the words in the lyrics of A Hard Days Night
Probability of predicting Queen song: 32.7%
Probability of predicting Beatles song: 67.3%
Therefore, I predict that it is a Beatles song with a high probability.
It will be. In other words, the actual value (measured value) and the predicted value (predicted value) match.
Let's take a look at the reports on the other two songs (Report 2, Report 3).
Report 2 "Bohemian Rhapsody" (Queen): Correctly predictable
Queen's representative song "Bohemian Rhapsody" can be predicted to be Queen's song with a high probability of 88.9%.
Report 3 "Hey Jude" (Beatles): Falsely predicts "Queen"
Beatles' representative song "Hey Jude" is predicted to be Queen with a probability of 56.3%, so the prediction is wrong.
Some songs are predictable like this, while others are not. When creating a predictive model, the Misclassification Rate, which indicates the rate at which the actual measurement and the prediction differ, is used as an index to indicate the predictive ability, and the smaller this value is, the higher the predictive ability is.
Model comparison
In the previous section, we modeled using binary values (1/0) as the document word matrix, but how does it compare to the case of modeling using word frequency in the same way?
In word selection analysis, as a "Summary" report, when multiple models are fitted, a report comparing the predictive power of the models is displayed as follows. Model 1 is for Binary and Model 2 is for Word Frequency.
AICc used as a verification index, Generalized RSquare (generalized R-squared, larger value is better), AUC (area under curved surface, larger value is better), Misclassification Rate (misdiscrimination rate, smaller value is better) If you look at, you can see that the 2 values of Model 1 are good for any index.
From this result, it can be seen that the model with the occurrence (= 1) / non-appearance (= 0) of the word as the explanatory variable has higher predictive power than the frequency of the word as the explanatory variable. ..
summary
I will summarize the results of the "word selection analysis" explained so far.
-From the words contained in the lyrics of the song, we created a model that predicts whether the song is Beatles or Queen.
-As a document word matrix used for explanatory variables, we modeled the following two types of methods and compared the predictive power of the models.
Method A. Indicates whether or not the word corresponding to the lyrics is included: Binary value
Method B.Indicates how many times the word corresponding to the lyrics appears: Frequency
⇒ It was found that the prediction ability of Method A (binary value) without considering refrain is high. Therefore, the results modeled by Method A are used thereafter.
・ Forty-nine words that greatly affected the prediction were extracted. When the predicted probability of each song was obtained based on the regression coefficient estimated for these words, and the predicted song (predicted value) and the actual song (actual measurement value) were obtained, the misdiscrimination rate was 8.4%. Met. Considering both the R-squared and AUC values, which are indicators of other predictive capabilities, it is considered that a good prediction model could be created.
⇒ With only the information of the selected 49 words, it was possible to guess which song it was with 91.6% accuracy.
Finally, here is a diagram of the 49 selected words (Terms) and their regression coefficients (Coefficient) drawn in Graph Builder. Orange is the word that contributes to Queen's prediction, and light blue is the word that contributes to Beatles' prediction.
"Blow", which has the highest positive value, is the word that characterizes Queen's lyrics. In fact, among the songs analyzed, blow is used in various Queen songs, but since it is not used in Beatles songs, it is explained that the coefficient of this word is large. You can attach it.
I hope this article will give you an opportunity to use "Word Selection Analysis".
