R / Python Training12.8.2020 R/Python Training Session
Presenter: Ben Francis
Watch Recording | Journal
Key Takeaways
- "Do you want your people to be coders, or do you want them to be people who solve problems?" SME is often not the R or Python coder.
- JMP can be seen as a threat to those who are R/Python coders. Move away from gatekeeper (R/Python users) towards enablement (I want to keep my job so therefore R and Python need to stay)
- Cost of software vs cost of people/time
- One of the most critical quality issues in software development, especially for analytical software, is the accuracy and reliability of results. Quality statement vs open source solutions. JMP provides trust and security.
- R and Python are competitors for all of our products (JMP, JMP, Clinical, JMP Live...)
What is open source software?
The term open source refers to something people can modify and share because its design is publicly accessible. Open source software is software with source code that anyone can inspect, modify, and enhance.
Who uses R/Python?
- Any company
- Roles: data scientists, statistician, higher level biologists, chemists, technologists
- Those that want to be trendy
- Those with initiative
- Often not management
- Not data driven modelers
- Not in highly regulated areas
- What is open source? What does that mean to users? Cost, access, support, etc?
- Learning curve for R/Python vs JMP JSL? Action recorder.
What are the pros of R/Python?
- It's a free, data science tool accessible to all.
- Mature and extensive "partner" in the market.
- Sites such as Stack overflow offer great help to the R/Python communities.
- New cutting edge functions/features added frequently by users.
- Quality of output can be very high.
- Straightforward interfacing with other programming languages (C, C++, Java).
- If a company has been doing data science, likely they will have material in R/Python scripts.
- Can be cloud-based.
- Having Python programming knowledge can be a differentiator for scientists/engineers.
What are the cons of R/Python?
- The focus is on the software rather than the people using the software.
- Further the software can be seen as a panacea to messy/lack-of/non-existent data.
- Code-first approach, with often more than one "coder" required.
- "Pie in the Sky" thinking.
- Poor security options and no/very little quality and accuracy assurances (JMP's Quality Statement)
How to Position JMP vs R/Python:
General Suggestions:
- Is there an actual problem they are trying to solve? Walk away from "window" shoppers.
- Do not get into deep technical deep conversations or use buzz words yourself.
- Do not do functions and features, open source wins virtually every time.
- Avoid narrowing the customer's focus on any one JMP product.
For the User Buyer:
- Qualifying Questions:
- What aspects of your current work are challenging?
- Do you wish you had more time for innovation data analysis?
- What do you wish you could improve about your current scripts/applications?
- Are you part of a central technical support team? If so, how do you provide support?
For the Technical Buyer:
- Work away from the gatekeeper mentality towards enablement.
- Be clear about the quality and numerical assurances of JMP.
- Qualifying Questions:
- How many within your organisation are involved in knowledge/process development?
- Would a software with regularly updated numerical accuracy and quality assurances be important to you?
For the Economic Buyer:
- Hear the grand vision out. The grander it is, the more painful they have it coming.
- Encourage them to consider that open source could require a higher level of training (and therefore cost).
- Qualifying Questions:
- Where do you want to be a year from now? What is your strategy?
- How are you going to scale their data analytics?
When and how should you position JMP with R/Python?
JMP is a standalone, full-featured data visualization and statistical analysis software package from SAS for the Windows and Mac desktop. JMP has the interactivity and dynamic linkage that makes data exploration exciting and insightful and contains many advanced analytical options, fully satisfying the needs of data explorers and analysts. Still, there may be occasions where you’ll want (or need) to use JMP in conjunction with open source tools, like Python or R. Consider the following examples:
- A corporate statistician prohibited from using non-validated software like R for primary analysis may nevertheless want to explore new methodology in an R package side-by-side with validated methods in JMP.
- A Python or R programmer may want to take advantage of the dynamic linkage, advanced visuals or powerful analytics available in JMP.
- A company’s data analysis department may want to develop an add-in to JMP that allows some R or Python code to be run as an add-in within JMP, so that future users can interact with the GUI add-in without needing to write the R or Python code themselves.
- A statistical educator may want to introduce specific open source packages in the classroom but within a point-and-click environment that wouldn’t require coding by the student.
Resources:
- Mastering JMP: JMP and R Integration: Learn to use basic JSL scripting commands to connect to R, send data between R and JMP and submit R code.
- White Paper: Using JMP and JMP Pro with R and Python: Examples of when users may want (or need) to use JMP in conjunction with open source tools, like Python or R.
- Addin: XG Boost: Provides a point-and-click interface to the popular XGBoost open-source library for predictive modeling with extreme gradient boosted trees. Includes profiling export of JMP Scripting Language (JSL) and Python code for reproducibility.
- Addin: JMP to R addin Builder: Allows R coders to house their R code in a JMP add-in without writing JSL code. Enabling JMP users to access and alter options within the R code.
- Wiley Paper - Better Together - Extending JMP with Open-Source Software: In this paper, we will explore use cases that illustrate how JMP users can take advantage of the broad support and rapid pace of change of open-source software while still benefiting from JMP's streamlined interface and robust capabilities.
- JMP Can Be Shiny Too: Americas 2021 Discovery Paper that talks about the dashboard functionality in JMP is a fantastic tool that can serve as a good alternative to R Shiny. Compared to R Shiny, JMP dashboards are easier and faster to build and can save lots of time in app development. In this poster, I talk about how JMP dashboards can be a tool for:
- Sharing the excitement of data discovery with others.
- Personalizing the presentation of graphics.
- Creating a visual storyboard to convey information.
Competitor Info (SWOTS)TrainingRe: R / Python TrainingThanks for sharing @martindemel! This sounds like a nice use case. Do you know if they are using the Python code mostly as it is provided by JMP, or if it requires a lot of modifications? Also, do you know which models they are using?
Re: R / Python Training@mia_stephens Few years ago I had a Pro deal at Osram where they used JMP Pro to build a model they can use in their python generated web-application. The goal was to have a better estimation of how long a project will last including all the different project steps depending on the resources (people), innovation class (new product, build on previous, ...) and current workload (and some other factors). So all together a project planning model. As the webapp was build in Python they found a decent good model wuite easy in JMP and used the formel depot to generate Python code they then embedded into the application. Worked almost like a charm. From time to time they recheck the model with the new acquired data from past project timelines and adjust if necessary. I haven't heard back from them for some time but I think it is still in productive use.
Re: R / Python Training@malcolm_moore1 @nascif_jmp
What questions can we ask to identify whether someone is a gate opener or gate keeper?
My main thinking would be along the lines of
- How do you include domain expertise in the data analytics? Why is this important?
- Does working with a team on a data analytics challenges lead to success or failure?
Re: R / Python Training@malcolm_moore1 while data literacy is an important question, I think it is orthogonal to the specific question that Mia just asked about. If anything, the experts that allow JMP models to be deployed in production to generate business value are not gate keepers, but gate openers.
If the problem the business is trying to solve requires the creation of a web site to serve predicted values or an automated, end-to-end monitoring process in a factory, there has got to be a way to get our models out of JMP and make them part of the required processes. It is not a question of "who" as much as a question of "where" and "how".
Maybe if one day we make it as easy to deploy models to production environments as we have made it easy to "deploy" reports and dashboards using JMP Live, we would reduce the need for data analytic experts in the analytic workflow. That would be an interesting way to combine these threads in a productive way.
@mia_stephens here are some of the people I talked to over the years regarding Python scoring code generation by the Formula Depot, in no particular order.
- Ping Liu - Advanced Analytics at Bain & Company
- Matthew Flynn - AIG Science
- Philip O'Leary - Steering Group member at University of Eastern Finland
- Bradford Foulkes - General Electric
- Thomas Slezak - Brigham Young University
- Burak Bagdatli - Georgia Tech
- Gul Ege, David Duling - SAS (ESP)
- Lucy Luo - Conoco Philips
Re: R / Python TrainingOne additional aspect to consider might be other aspects of the data analytic workflow: the need to be alert to the potential limitations in our data and the need to challenge our data and models using subject matter knowledge. Recognise that our processes are rarely in a steady state, or put another way: learning from data is a continual process that is better done by enabling the integration of subject matter knowledge with data literacy and data analytic know-how. It is easier for a company to achieve data analytics at scale by enabling subject matter experts with regard to data literacy and data analytics than it is to scale data analytics by transferring subject matter knowledge to our few data analytic experts.
If the data analytic experts are currently gate keepers of the analytic workflow then we may need to look to have this conversation with other buying influencers.
More aspects of the data literacy arguments are in Dark Data by David Hand, see his Stats Speaking keynote Dark Data some aspects will also be covered by Ron Kennet and Chris Gotwalt in their upcoming Discovery paper: Maximising Data Science Success
Re: R / Python TrainingHi Mia, Jerry Winter at Unilever has used this functionality. I don't have the full story but we worked together to figure out a section that hadn't translated over from JMP to Python. Let me know any next steps that would be helpful!
Re: R / Python TrainingGreat discussion here! Curious about the use of Formula Depot to generate Python code. Anyone have named customers using this functionality, and examples you can share?
Re: R / Python TrainingMight be good to have some context from how some of our competitors (even if partial ones) are dealing with integration (not replacement)
* https://www.analyticsvidhya.com/blog/2020/08/integrating-python-in-power-bi/
* https://towardsdatascience.com/tabpy-combining-python-and-tableau-511b10da8175
* https://looker.com/blog/data-science-with-looker-and-python-part-two
Re: R / Python TrainingThis is a brilliant discussion, great back and forth.
Re: R / Python TrainingVery useful resources @mzwald and @nascif_jmp, definitely helps to broaden the page. I did mention about @ruthhummel, @MJ and @nascif_jmp work as a resource but didn't want to subject it to a 2nd hand explanation!
FWIW, my experience is a user of R/Python in Academia and Unilever. In Unilever, whilst understanding that JMP can be synergized with R/Python is a fantastic tool, there was a small number (<10) in the company who could do this. A drop in the ocean compared to 5000 (mostly Scientists and Engineers) in just R&D worldwide. JMP enabled me to work/collaborate with those 5000 rather than as a service to them, R/Python did not. The focus should always be around identifying challenges of the current situation and enablement towards a vision (either comprehensively with JMP products or in part).
Being honest, I've struggled with being savvy enough with R/Python packages, both in terms of the numerical accuracy of the package itself and whether my usage is erroneous. In JMP, I have a Quality Statement and Commercial SE and Technical Support. Whilst they are discretionary sales points, we should not be shy to bring these up when relevant.
Re: R / Python Training"Synergy is an interaction or cooperation giving rise to a whole that is greater than the simple sum of its parts."
Re: R / Python Training@nascif_jmp and @MJ and I wrote a paper showing more detail about how to use opensource with JMP, underscoring that idea of using them together rather than replacing one with the other. The paper is available at https://onlinelibrary.wiley.com/doi/abs/10.1002/sta4.336. It's under the journal paywall right now, but as soon as our request to make it OpenAccess goes through, this link should allow you (or a customer to whom you send this link) to view the full paper for free.
In the paper there are links to four supporting short videos. You can also send these links directly or use them in a demo. The first two videos show the Housing Price application that Nascif built using generated Python scoring code from JMP incorporated deployed in AWS. These two videos show (1) building the model in JMP and generating the Python code, and (2) the final application that was built in AWS.
Movie 1: JMP Pro Listing Price Predictor Model fitting and Python code generation:
https://players.brightcove.net/1872491364001/default_default/index.html?videoId=6214684063001
Movie 2: Final Model Deployment on AWS of the Python scoring code generated by JMP Pro:
https://players.brightcove.net/1872491364001/default_default/index.html?videoId=6214680112001
The final two videos show MJ's use of t-SNE and UMAP in R, built into a JMP add-in so that the user interacts only with JMP point-and-click dialogs (and the R code is invisible behind the scenes, with respect to the add-in user). Movie 3 shows how to use the add-in. Movie 4 shows how JMP brings features that are very difficult in R, and R brought the algorithms that weren't in JMP, and both software systems together can do this cool thing.
Movie 3. Application of the Embedding add-in on the MNIST data set:
https://players.brightcove.net/1872491364001/default_default/index.html?videoId=6214686689001
Movie 4: t-SNE result exploration with image labeling in JMP:
https://players.brightcove.net/1872491364001/default_default/index.html?videoId=6214680173001
Re: R / Python Training@saradoudt what @mzwald said. I am surprised that @ruthhummel 's white paper is listed as a resource but not fully explored as a viable approach to sell JMP without trying to displace Python/R - there is space for both and we can show that the sum is greater than the parts. Also, you might want to list the Discovery presentations on the Formula Depot as they have examples of how to deploy the Python scoring code that could be helpful when making the case for using JMP in a Python shop.
* Scoring Outside the Box - JMP User Community
* Sharing the Ultimate Boon: A Journey From Modeling to Scoring - JMP User Community
Also, I think the vision of Open Source described here is missing the point of what we are going against:
"The term open source refers to something people can modify and share because its design is publicly accessible. Open source software is software with source code that anyone can inspect, modify, and enhance."
In the data science space, "people" and "anyone" include large companies like Google (TensorFlow) and IBM (Spark). They pay developers to maintain and enhance these frameworks, as they are part of their cloud-based growth strategy (the software stack is free, you just pay for the cloud usage). You can be sure they pay testers as well. So the old SAS FUD argument that you can't trust their quality as much as ours is not going to fly with users that know where they come from - it might even backfire.
Even for smaller packages such as XGBoost, when you check the list of contributors you see researchers from universities and developers from major corporations. These are not people working in garages. So sure, there are small, poorly maintained open source packages out there; but the scientists and engineers we are going after are savvy enough to know which ones to pick and which ones to avoid. We should be as savvy as they are when having this conversation.
Re: R / Python Training@mzwald Thanks for the added comments. Always love hearing different experiences!
Re: R / Python TrainingSome thoughts on my experience positioning JMP against R/Python (Python being much more common in NWT2 region than R). We've found it's best to position JMP (or more likely JMP Pro if they are using Python), as a complimentary tool to Python rather than making it an either/or decision. If you make the choice "use JMP Pro rather than Python", it will not be effective as Python has too much inertia and infrastructure behind it, especially for predictive modeling (ex. TensorFlow library). But if you position JMP Pro as a complimentary tool to Python where data models can be prototyped and compared in JMP Pro faster and then deployed as Python score code, that is a much more compelling reason and has successfully landed a number of JMP Pro sales, even where Python is still heavily used.
