An annual user survey provides an opportunity to collect standardized feedback about our brand and software from our global userbase. The 2023 release is the third edition of the annual survey. This article will discuss some insights from the data and any referenced reports can be found in JMP Live. The anonymized survey results have also been made available for your personal exploration.
Methodology
Salesforce contacts were used to construct the sample of 72,181 users who would receive a survey invitation. The number of survey invitations was increased from last year to maximize data collection. Survey invitations were sent through Qualtrics on November 7, 2023, and the survey closed after two weeks on November 21, 2023. In total, 1,071 responses were received resulting in a response rate of 1.48%.
Limitations
As users could self-select to take the survey, the results of the analysis have a few limitations. Any conclusions drawn from the data should be relative to the sample and not generalized to the broader JMP population. Also, when filtering the data by an attribute the data can become much smaller which limits the statistical insights that can be drawn from such subgroups.
Respondent Demographics
The survey respondents represented JMP users around the world with the majority of responses coming from the United States (43%) and Japan (24%). Users tended to report longer years of software interaction with more than 10 years of JMP use representing 37% of responses. Most users reported being intermediate (51%) followed by advanced (26%) and novice (26%). Years of JMP use was visualized using experience level as the overlay which revealed that intermediate users remained a substantial portion of responses even as years of use increased.
Country representation of survey respondentsCountry representation of survey respondentsCountry representation of survey respondents
Dashboard of JMP use and experience levelDashboard of JMP use and experience levelDashboard of JMP use and experience level
Respondents had the option of answering additional demographic questions at the end of the survey. Among those who chose to respond, 77% identified as male and 20% identified as female. Regarding racial identity, the majority of users identified as either white (50%) or Asian (39%). Ninety percent of respondents reported not being of Hispanic, Latino, or Spanish origin. Of our four core industries, 17% of users reported working in the semiconductor industry, followed by the chemical (9.6%), pharmaceutical (8.2%), and consumer products (6.4%) industries. Generationally, millennials represented 32% of users followed by baby boomers (28%), Generation X (28%), Generation Z (2.4%), and the traditionalists/silent generation (1.5%).
Optional Demographics DashboardOptional Demographics DashboardOptional Demographics Dashboard
Industry DashboardIndustry DashboardIndustry Dashboard
Product Usage
Among the respondents most were either JMP (67%) or JMP Pro (31%) users. Only 0.7% and 0.3% reported using JMP Clinical and JMP Live respectively. Of respondents using either JMP or JMP Pro, 58% reported using version 17 and 22% using version 16. Regarding frequency of JMP use, respondents reported using JMP a few times a week(35%) followed by everyday (24%), a few times a month (27%), every few months (10%), and a few times a year or less (4.1%). The following responses answer the question “How well do the following statements describe JMP?” summarized by three categories: UI/UX, trust/reliability, and competitive insight.
UI/UX Insight
- Is easy to use: 25.2% extremely well & 40.1% very well
- Is intuitive to use: 20.3% extremely well & 39.3% very well
- Is enjoyable to use: 28.4% extremely well & 42.5% very well
Trust/Reliability Insight
- Produces analysis results that I trust: 45.8% extremely well & 40.8% very well
- Produces graphics that look professional: 32.8% extremely well & 37.6% very well
- Is dependable: 31.6% extremely well & 46.8% very well
- Has good documentation: 23.4% extremely well & 38.7% very well
Competitive Insight
- Is the market leader: 18.1% extremely well & 31.9% very well
- Has more advanced features than competitors: 22.9% extremely well & 37.3% very well
- Is a premium product: 27.1% extremely well & 44.6% very well
- Is a good value for the money: 17.7% extremely well & 34.7% very well
- Is the best software: 21.8% responding extremely well & 45.9% very well
- Is a product people love: 18.3% extremely well & 39.3% very well
This year, respondents’ capability usage was modeled after the Analytics Maturity Assessment. Each of the 14 capabilities were visualized with their self-reported hours of usage per week. This visual provides insight into how the respondents are spending their time interacting with data. Specific capabilities can also be examined to understand weekly usage by experience level.
Capability vs. UsageCapability vs. UsageCapability vs. Usage
Level of Expertise vs. Usage Filterable by CapabilityLevel of Expertise vs. Usage Filterable by CapabilityLevel of Expertise vs. Usage Filterable by Capability
Competition
Data from the survey also provides insight into how users perceive us competitively. The respondents were asked the question “How often do you use tools other than JMP to prepare, analyze, and/or visualize data?” For each of these categories, 'sometimes' was the most likely response. Notably, users in the sample were most likely to use other tools (9.5% responding never) when preparing data as compared to analyzing (23% responding never) and visualizing data (22% responding never). Among provided options, respondents reported Excel (42%), Python (9.6%), and R/R-studio (8.4%) as the top three tools used in addition to JMP for preparation, analysis, and/or visualizations needs.
CompetitionCompetitionCompetition
Education, Community, and Support
Users’ perceived satisfaction of the following groups were captured: JMP education resources/training, JMP User Community, and JMP support (e.g., technical support, customer engagement). For each of these categories, ‘extremely satisfied’ and ‘satisfied’ represented over 50% of responses.
Education, Community, and Support SatisfactionEducation, Community, and Support SatisfactionEducation, Community, and Support Satisfaction
Net Promoter Score (NPS)
The Net Promoter Score (NPS) is a metric used to measure customer perception. NPS is based on asking a question of the following framework: how likely are you to recommend [organization/product] to a friend or colleague? Respondents answer this question on a scale of 0 (not at all likely) to 10 (extremely likely). This numeric scale corresponds to three groups: promoters (9 or 10), passives (7 or 8), and detractors (0 to 6). NPS is calculated by subtracting the percentage of detractors from the percentage of promoters. Companies can then use this score to compare against industry benchmarks or compare scores against itself year over year. The annual user survey captures two Net Promotor Scores: one for our company (cNPS) based on the question “How likely are you to recommend our company to a friend or colleague?” and another for our product (pNPS) based on the question “How likely are you to recommend JMP software to a friend or colleague?”
Overall Company and Product NPS
Using the entire sample, the cNPS equated to 29, with 48% of responses being associated with promoters and 19% of responses associated with detractors. This represents a 12-point decrease from the 2022 annual user survey. The pNPS equated to 50 with 60% of responses belonging to promoters and 10% of responses being detractors. This represents a two-point increase from the 2022 annual user survey. Notably, the pNPS is higher than our cNPS suggesting that people may be more likely to promote our software than our company. Further investigation revealed why this may be the case: in the survey, an open-ended follow-up question asked why the users chose their rating for the cNPS. When reviewing these comments, it seems users may not perceive a distinct separation between our company/brand and software. Therefore, the lower ratings are likely attributed to a neutral sentiment towards our company stemming from not understanding our brand as opposed to a negative sentiment about our company.
Company NPSCompany NPSCompany NPS
Product NPSProduct NPSProduct NPS
Two variables that may have a strong influence on the NPS is years of JMP use and level of experience with the software. Two ordinal logistic regressions were performed using these variables to explore their impact on the numeric ratings associated with the company and product net promoter score questions. For both models, level of expertise was statically significant while years of use was not.
NPS by Region
Both the company and product NPS were filtered by the ‘Sales Region’ attribute in Salesforce. The table below shows the cNPS and pNPS for the Americas, Japan, APXJ, Europe, and Korea regions.
| |
Company NPS
|
|
Product NPS
|
Region
|
cNPS
|
% Promoters
|
% Detractors
|
|
pNPS
|
% Promoters
|
% Detractors
|
Americas
(N = 407 )
|
48 |
59% |
11% |
|
64 |
71% |
7% |
Japan
(N = 232)
|
-12 |
25% |
37% |
|
12 |
35% |
23% |
APXJ
(N = 69)
|
33 |
46% |
13% |
|
56 |
59% |
3% |
Europe
(N = 69)
|
45 |
54% |
9% |
|
66 |
70% |
4% |
Korea
(N = 31)
|
38 |
45% |
7% |
|
55 |
55% |
0% |
NPS by Sales Region DashboardNPS by Sales Region DashboardNPS by Sales Region Dashboard
NPS by Generation
NPS was also explored through the generations. The three generations with the most data representation were baby boomers, Generation X, and millennials. For the cNPS, baby boomers had the highest score (38) followed by Generation X (37) and millennials (29). For the pNPS, Generation X had the highest score (59) followed by baby boomers (58) and millennials (50).
| |
Company NPS
|
|
Product NPS
|
Region
|
cNPS
|
% Promoters
|
% Detractors
|
|
pNPS
|
% Promoters
|
% Detractors
|
Traditionalists
(N = 11 )
|
28 |
64% |
36% |
|
82 |
82% |
0% |
Baby Boomers
(N = 198)
|
38 |
55% |
17% |
|
58 |
67% |
9% |
Gen X
(N = 203)
|
37 |
53% |
16% |
|
59 |
66% |
7% |
Millennials
(N = 231)
|
29 |
47% |
18% |
|
50 |
60% |
10% |
Gen Z
(N = 31)
|
29 |
53% |
24% |
|
53 |
65% |
12% |
NPS by Generation DashboardNPS by Generation DashboardNPS by Generation Dashboard
Future Improvements
Future improvements to the annual user survey will be considered to improve the statistical validity of the data and thereby the generalizability of the results. Strategies to improve the survey completion rate will also be considered to combat year over year survey fatigue.
