Reliability Survey ResultsIn April of this year, we asked SEs, SE Managers, and Technical Enablement Members (N=76) to respond to a survey about reliability platforms and supporting resources. The survey was conducted by the Product Mtg and Technical Enablement teams. The goal of the survey was to better understand how and where the reliability platforms are being used, and to identify reliability-related resources that might be in need of improvement.
For those of you who responded, thank you for your time!
The return rate was approximately 71% (54/76). While the response rate appeared lower in Europe, there was not sufficient evidence to suggest effects due to location or tenure.
The three platform-specific items - comfort demoing, demo frequency, and customer use - had similar results. This lead us to believe they may measure the same underlying characteristic.
Beginning with demoing comfort, we aggregated the five response levels into three, grouping the bottom two (Not comfortable demoing and Can perform basic demo) to Basic and the top two (Can answer most customer questions and Can create examples) to Advanced.
Life Distribution, Fit Life by X, and Degradation had fewer Basic and more Advanced responses. The four repairable systems platforms (Recurrence Analysis, Reliability Growth, Reliability Block Diagram, and Repairable Systems Simulation) and Cumulative Damage had more Basic and fewer Advanced responses.
Similar patterns were seen for demoing frequency and customer usage.
Satisfaction with support materials was consistent across sources, with the exception of Detailed Examples/case studies, where there was a smaller percentage of Satisfied responses.
As is probably expected, there are some differences in comfort level (in showing/discussing reliability tools) across our core industries, with people most comfortable discussing reliability within the pharma and semiconductor industries followed by oil and gas. The lowest comfort level was within the Consumer Products, Chemical, and Biotech/Medical industries.
Seven of the responders (~13%) indicated they have lost sales to a competing product. Four people indicated it was Reliasoft, two Minitab, one R. Five individuals (~9%) indicated they lost sales due to gaps in functionality. No single specific gap was identified.
In conclusion, we found:
- People were most comfortable with, demoed most frequently, and had the most customer usage with four non-repairable system platforms (Life Distribution, Fit Life by X, Degradation, and Survival).
- People were least comfortable with, demoed least frequently, and had little/no customer usage for with all repairable system platforms (Recurrence, Reliability Growth, Reliability Block Diagram, and Repairable Systems Simulation) and Cumulative Damage.
- Overall, people were more neutral than satisfied for the various support resources. The largest proportion of neutral responses were for detailed examples/case study, suggesting room for improvement in this area.
- Approximately 12% of sales were lost to a competitor with Reliasoft making up 2/3rds of the cases.
Our next steps are to meet with the SEs to discuss the results and better understand the differences between the platforms. Particularly, to understand why the repairable system platforms are not used more often.
We would also like to discuss potential gaps and improvements to resources, along with possible resources to help guide SEs and their customers to the proper reliability platform.
Feature StoryProduct ManagementRe: Reliability Survey Results@peng_liu
Thanks very much for your comments!
I will recheck about the case of reliability growth program.
Re: Reliability Survey ResultsThank you all for your inputs. According to the survey results and extra comments here, I should believe that the reliability platforms have a sizeable group of users. Our client facing colleagues, you are our ears and eyes, thank you for helping to promote our reliability platforms. Please don’t hesitate to contact Don, Mia, or me, if you have observations, concerns, thoughts, recommendations regarding the reliability platforms. Thank you!
Thanks for sharing some of the challenges that you had with customers. I wish that some inputs could have been more specific. I firmly believe there are important implications behind the comments or attitudes from existing customers and prospective customers regarding reliability platforms. Although those implications may not be exactly clear, I hope that they can be addressed by the effort of this survey and subsequent ones. I believe that one goal is to identify the knowledge gap between what reliability platforms offer and what our client facing colleagues know, so that our marketing folks can get our customer facing colleagues prepared to identify more opportunities and be more confident in persuasion.
I would like to express my own views on the challenges mentioned here. It is not the first time that I heard some of them. I would like to offer my perspectives on how the effort of this survey and subsequent ones can help to address these challenges. But please don’t hesitate to express your objections, disagreement, or provide your own. And, if you have success stories, please share with us. Your generosity will help us to understand our reliability platforms and our customers more comprehensively. Thank you!
First, Excel users! I cannot agree more about how hard it is like about convincing an Excel user. As the developer of reliability platforms, I have seen what a skilled reliability engineer can do with just Excel, and what kind of terrible mistakes a less skilled engineer can make with just Excel. With the skill engineers, conversations usually include the benefit of time saving, they know that well. But now I can see changing habit might be another important topic. For the less skilled engineers, it is important to understand what data they have. And more often than you might imagine, they are not clear about the data that they have. Reliability data usually involve censoring, which may not be a familiar topic if one has not been trained. The effort of this survey and subsequent ones may help you to spot opportunities.
Second, TMI. Or maybe, please allow me to talk about “complexity” instead. Sometimes, it is not clear which word they mean. I think TMI can be addressed by cutting back unimportant information if there is any, but addressing complexity is much harder. I am not sure that reliability platforms have a TMI problem. I won’t deny it, but I wish the complaints could have been more specific. Some reliability platforms are complex, I agree. And there are reasons. The effort of this survey and subsequent ones may help you to understand some of the reasons behind the complexity and be more confident while explaining them to prospective customers.
Third, compete against Minitab where they are established. I don’t have direct experience in dealing with the situation. But I think it is not uncommon. It reminds me of the days when we competed against them on quality control programs. How did we stand out then?
Fourth, reliability growth program. It is the first time that I heard such a case. It sounds quite legit. Who want a whole house renovation because a light bulb is out? Wait a minute, did they say that they are in a reliability growth program? What are they developing? What kind of issues does their growth program monitor? Does their test involve other types of data analysis, other than tracking reliability growth? Are they trying to optimize their design? Can DOE be applied? Are they doing a whole house renovation indeed? Oh, BTW, did they say what software that they use for analyzing reliability growth data? It can be done all in Excel because it is quite simple. If they say Excel, good for them. They might have the Excel sheet from AMSAA, the birthplace of the subject. If they say Minitab, then do they just use Minitab for reliability growth? Anyway, those two are our weakest competitors in reliability software market. If we ever set those two as our only opposing forces, we will not be anywhere near where we are today. You should have seen the one who shall not be named. And the effort of this survey and subsequent ones may get you prepared: know key subjects, connect related topics, ask relevant questions, and uncover demands!
Re: Reliability Survey Results@peng_liu I think the pharma shelf-life determination in Degradation is the most valued aspect of it. I have some ideas on how we could expand our suite of tools for pharma stability analysis. I will setup some time to talk with you about it.
Re: Reliability Survey ResultsI just wanted to share what I heard from potential customer and JMP Partner.
Hope that helps.
- A Sr. Director of Quality from a big electronics components manufacturer said in 2018 that I have currently been receiving about 200 Reliability Tests Reports per week. All the reports are based on Reliability Analyses using Minitab. Please tell me why I need to change the standard procedure of using Minitab to JMP.
- A JMP Korea Partner has recently consulted on reliability for a mid-size automotive part manufacturer. The company was requested to design and conduct a new reliability growth test by its customer (maybe Hyundai/Kia) recently. In fact, there is a very strong movement or/and initiative related with reliability from the automobile companies. The JMP Partner recommended them to use JMP. They replied that it is not needed to use JMP because we are just set the new plan and needed to conduct it only. We have no resources to be charge of reliability job.
Re: Reliability Survey ResultsJMP Education is here to help! Our course on Reliability Analysis for Non-Repairable Systems is scheduled for end of July, please let customers and users know. SEs are also welcome. You can see the topics covered on the course page. We have repairable system reliability material available, but it's waiting until I have time to make it into a real course before we schedule it.
Re: Reliability Survey ResultsFWIW, I agree completely with @PatrickGiuliano when he says ' It puts a greater onus on us to step through the meaningful interpretation of the information that is presented', but somewhat disagree with the statement 'It's up to the customer to do that digging themselves'.
In saying this, I've deliberately taken the second phrase out of context to accentuate the point - Namely that, using every means possible, we have to assume the burden of supporting users in a way that make s sense to them. Of course this is a 'big' problem, which is why establishing a baseline like this is so important.
As others have done already, I'd like to thank @DonMcCormack and @mia_stephens for taking it on, and everyone who gave input.
Re: Reliability Survey ResultsThanks for sharing @DonMcCormack, interesting results.
As you know I have experience with my former company and what I noticed at that time is that a good number of potential users was the reluctance to move out from their comfort zone (more specifically excel files) to JMP mainly related to the fact that the reliability platform outcome was different in terms of format and/or info what they expected/wanted to stick with what done in the past (to have evergreen info of equipment or units of interest).
Re: Reliability Survey ResultsRE: @IkjuShin's comments -- It's tricky because JMP in general provides lots of very important statistical information all at once, but it does increase the chances of overwhelming the user. If you take a step back and step into the Customer's shoes (and assume you know almost nothing about Reliability statistics as a discipline), then yes, it can be overwhelming. To me, it puts a greater onus on us to step through the meaningful interpretation of the information that is presented. We cover a lot of detail in the support documentation - it's up to the customer to do that digging themselves, and it's up to us to make it as clear as we can (in whatever package we feel is most appropriate for that vital knowledge transfer to the user base).
Re: Reliability Survey ResultsThanks Don for sharing this survey result.
Although there is no exact survey result, it seems that Minitab has the No. 1 market share in terms of reliability in Korea.
A general engineer who is not a reliability expert often finds it difficult because the JMP analysis results display too much information. TMI (too much information)!. It seems necessary to increase accessibility and intimacy by displaying the analysis results step-by-step, etc. Of course, some say Minitab's analysis is 'too less information'.
Re: Reliability Survey ResultsThanks, @DonMcCormack for sharing these data and results.
Re: Reliability Survey Results@peng_liu I was surprised by that, too, and came to a similar conclusion as to why. This is why it would be helpful to discuss this with the folks demoing Degradation.
Re: Reliability Survey ResultsThanks for sharing the report, Don!
The patterns shown in this report are generally what I would expect.
One platform is surprising though: Degradation. It seems to be more well received by both customers and our colleagues than I expected. If this platform's numbers reflect what I think they reflect about the usage in pure "Degradation", we may want to promote a little more on the standalone Destructive Degradation (which seem to be less well received) and the upcoming standalone Repeated Measures Degradation. But I am wondering whether the Stability part in the Degradation platform might have contributed significantly!
