In the previous Part 1 , we demonstrated an example of using the "categorical" platform to aggregate various types of responses (single answer, multiple answer, score) from survey data all at once.
That being said, you can still do the tabulation using spreadsheet software if you put in the effort, and if you're using a survey system, the system will display the results in tables and graphs.
So, what are the advantages of using JMP's "Category" feature?
This allows us to compare the response distributions between the groups we will be discussing.
Specify the group variable in "Category"
In survey analysis, in addition to aggregating individual responses, responses are also aggregated by respondent attributes (gender, age group, etc.). This is known as cross-tabulation.
Let's consider the example of SQC training we covered in Part 1. In this example, there is an attribute called "position" (general employee, person with a position), so we will collect and compare other responses by position.
1.png
Let's perform the following analysis on the data table above.
Job title × Reason for taking the course (multiple answers): Multiple responses
Job title × Experience using statistical software (single answer): Nominal scale
Job title × Time allocation, text structure, practicality (score): Ordinal scale
Job title × Satisfaction level (score): Ordinal scale
In this case, under "Category," specify "Job Title" as [X, Grouping Category] as follows:
nao_masukawa_0-1681374861319.png
Homogeneity test of proportions (Do the response proportions differ between groups?)
First, let's look at the report for the multiple response "reason for taking the course" and the nominal measure "experience with statistical software" (yes/no). A cross-tabulation table is displayed, separated into "general" and "with a managerial position," and the corresponding share chart is also displayed.
3.png
When comparing the share of responses regarding "the reason for taking the course" between general participants and those with managerial positions, there appear to be differences in the following items.
Instructions from a supervisor: General staff (24.7%), Staff with managerial positions (16.8%)
Interest in SQC: General public (8.9%), those in management positions (16.1%)
Regarding experience with statistical software, there appears to be a difference as well, with 39.6% of general users having experience and 50.9% of those with managerial positions having experience.
What I've described here is merely that there *seems* to be a difference. Since I'm using statistical software, I want to investigate whether there is a statistically significant difference. Therefore, I'll use the hypothesis testing option available in the "Category" category.
Homogeneity test for non-multiple response responses
If the responses are not multiple responses, as in experience with statistical software, run the [Test for homogeneity of responses] option.
The homogeneity of responses between groups is tested, and here, the hypothesis that the proportion of responses is equal across job titles is tested. This is equivalent to the chi-squared test used to examine the independence of cross-tabulation tables.
nao_masukawa_1-1681375602235.png
Since the p-values for the report's likelihood ratio (chi-squared) and Pearson's chi-squared are less than 0.05, a significance level of 5% indicates that the responses are not homogeneous, meaning there are differences in the proportion of statistical software experience among different job titles.
Homogeneity test of multiple responses
For the multiple-choice question "Reasons for taking the course," we will use a [multiple response test] . Here, we present a report of a [binomial distribution homogeneity test] , assuming a binomial distribution of selected/not selected responses for each category of response.
nao_masukawa_2-1681375648352.png
The report, "Binomial Tests for Each Response," displays statistics comparing the response rates for each question across different groups.
For example, the total number of responses and the number of respondents for the answer "instructions from my boss" are as follows:
In the general population, 144 out of 392 people fit the description (36.7% = 144/392).
Of those with a managerial position, 26 out of 111 people fit the description (23.4% = 26/111).
At this point, we calculate the chi-squared value and p-value (=0.0074) using a logistic model where Y is the response rate (assuming a binomial distribution) and X is the general/positional response.
Referring to the report "Binomial Tests for Each Response," significant differences were found for two items: "Supervisor's Instructions" and "Interest in SQC." This suggests that general employees attended the seminar because their supervisors told them to, while those in managerial positions were genuinely interested in SQC itself.
While there are differences in the proportions of other items between general employees and those with managerial positions, these differences do not appear to be statistically significant.
Compare scores
Questions regarding product satisfaction often involve responses using a five-point scale or similar scoring system. In such cases, it's desirable to check whether there are significant differences in scores among different groups.
In this example, participants were asked to rate the seminar's time allocation, text structure, and practicality on a 5-point scale, and their overall satisfaction with the seminar on a 10-point scale. To compare these evaluation scores across different job titles, we would use the [Comparison of Average Scores] function as shown below.
nao_masukawa_0-1681431984784.png
This report details the seminar's time allocation, text structure, and practicality. The mean and standard deviation of responses 1-5 (considered as scores) are displayed to the right of the cross-tabulation table.
nao_masukawa_1-1681432150602.png
Referring to the last column, "Comparison of Means," you'll see the letter "f" in row 5 (the score for general utility, displayed as E). This indicates that it is statistically significant at a significance level of 0.1 compared to row 6 (the score for utility with a specific position, displayed as F). (Please refer to the explanation enclosed in the red box in the figure.)
Regarding practicality, the average score for the general public was 3.52, while for those with managerial positions it was 3.37. However, when comparing these averages in a pairwise test (using a t-test), the p-value was less than 0.1. Those with managerial positions may have been slightly more dissatisfied with the practicality of the seminar compared to the general public.
Let's also refer to the report on seminar "satisfaction levels."
nao_masukawa_0-1681432780154.png
The average score for the general public was 7.23, and the average score for those with managerial positions was 6.91. The letter "B" is displayed in the "Comparison of Averages" column. Since it is shown in uppercase, it indicates that the result is statistically significant at a significance level of 0.05. Regarding satisfaction with the seminar itself, there was a significant difference in scores between the general public and those with managerial positions.
Effective for group comparisons across many questions.
In "Category" mode, you can specify a grouping variable to perform group-based aggregation and comparison. In actual surveys, there are often many questions and many answer choices. In such cases, many tables and graphs are displayed, making it difficult to examine each answer individually to find out which answers have differences.
Using the "test of response homogeneity" and "comparison of mean scores" explained here, you can quickly identify items with significant differences, allowing for efficient comparison of responses between groups.
In the next installment, Part 3, we will explain in detail "multiple response" questions in surveys.
