Welcome to the JMP Development Update for August 26, 2022!
Final installer image candidates were created this week and QA teams continue to test many of our installer configurations today. We have halted testing of Mac JMP and JMP Pro 17.0 so that a change can be made to the Mac JMP and JMP Pro installer license file search path. This change will make it easier as we transition to delivering JMP and JMP Pro annual license software to My JMP for 17.0.
We didn’t meet this week, but John Sall will do a practice run of his Discovery keynote during next Thursday’s dev meeting slot. The following week, Luciano Silva from the JMP life sciences team has volunteered to share a demo of the new Marker Simulation platform in JMP Pro 17. With the transition of genomics features into JMP Pro and the imminent retirement of the SAS/Genetics codebase that used to support genetics features in JMP Genomics, it’s certainly a time of change for our genetics features. I appreciate Luciano taking the time to share his JMP 17 work with us!
Thanks to Tonya Mauldin for sharing the details on JMP Beta tracks this week with crashes selected.
JMP Beta Report 8-26-2022 | JMP Live (sas.com)
And finally, thanks very much to stat writer and Colorado resident Karen Copeland for sharing her experience volunteering for a USFS forest survey data collection effort in a burn scar about 35 miles out of her town. She sent along the following pictures and details about her experience which brings new meaning to "exercise science."
This was the 4th year since the fire and at least the 2nd year that they have done a survey of the area to collect data to use to inform where to reseed. The area of interest was divided into “units” and then “plots” were selected within each unit. We traversed forest service roads and then hiked into the various plots following our leader who had an ipad with a location. Once at the location, you put a stake in the ground that had a string tied to it, then traversed the circle defined by the string and counted any saplings (lodge pole pine, spruce, fir, or aspen) that you saw. Not high tech, but sufficient data to answer the question: where should I spend my resources reseeding? This is an example of hard to collect data, in the sense that it was time/people intensive. We had 4 groups, 1 USFS and 2 or 3 volunteers per group who worked all day (travel time was 1 hr to the initial location that we dispersed from) and they had 3 days planned and this was one fire area (and not a huge fire). My group did 10 – 12 plots and hiked about 4 miles (with some driving between units). What did we find? Some areas had seed sources left and we saw saplings growing. Other areas had nothing (besides weeds) growing.
General Announcements
- If you’re willing to be a featured employee on Inside JMP, please download a copy of the Get to Know form. Simply fill out the questions and when you’re done, upload it, along with 3 photos of you to the folder. Thank you for helping to make our JMP family a little more familiar!
Important Upcoming Dates
- SAS Commit to Ship – 9/19/22
- JMP QA Signoff - 9/20/22
- JMP 17 Ship Date – 10/18/22
- Please visit Confluence for a full list of JMP 17 dates.
JMP Development Updates
Host
- Aaron: Added virtual Platform::getAllDataTables() for platforms that use multiple data tables. This will enable such platforms to be published to JMP Live, restored in session scripts, and saved in projects. So far Choice and ReliaForecast have been updated to implement the new API. If anyone owns knows of additional Platforms that use more than one data table, please let me know.
- Michael H: Worked with Bryan Boone on two R Connect issues:
- Cannot connect to Apple Silicon R on Mac (should be resolved next week when we get a fresh TKEXT from SAS, JMP-8901)
- Most R source submissions cause an error on Mac or Windows (resolved this week by Bryan, JMP-13901).
- John Schroedl: With help from Evan McCorkle found cause of (and protected against) a very old exception with taking abs(INT_MIN) which is undefined behavior (JMP-13811, reported by Craige Hales and Intel). Changing the way host code handles private clipboard formats and doing that afforded using the nice new Uuid class from David W. for telemetry needs.
- Paul N: Working on the Open StreetMap replacement server. I Now have the WMS proxy server demo running using locally generated map tiles on RHEL 8 Virtual Machine. Still verifying if updated map data is properly coming from the database. Right now, it looks like I’m just getting continental outlines. SAS IT is setting up the replacement production machine. Once that’s up, the VM setup needs to be replicated on the ‘production’ machine with world mapping data. The development VM has just a small subset of the map data.
- Some history, Open StreetMap development community primarily uses Ubuntu Linux. It is ‘An Exercise Left to the Reader’ to get it to work on Red Hat Enterprise Linux (RHEL) or other Linux distributions. SAS IT only supports use of RHEL, due to support contracts and familiarity. There are no complete installable pre-compiled packages for Open StreetMap running on RHEL. So, several of the individual pieces needed to be built from source including library dependencies. Pre-built packages for some of the component pieces do exist, but most of them are too old, to work with the current Open StreetMap software stack.
- An Open StreetMap server is a large collection of software, all of which has to work together. Linux, Python 3, PostgreSQL database, PostGIS additions for Postgresql, Apache web server, 2 Apache modules: mod_tile and mod_wsgi_python3, GDAL – Geospatial Data Abstraction Library, Projection library Proj, and it’s Python3 module pyproj. Mapnik and renderd create the image tiles, and finally MapProxy which is layer that provides the WMS protocol that JMP wants to talk to. Most of the component pieces have their own configuration and log files.
HTML5
- Praveena: Fixed the following bugs, Hui is helping test them:
- Paul Spychala: This week was spent merging many changelists into 17.1. For 18.0 I have committed the initial implementation for Confidence Intervals into HTML5 (JMPHTML-1741, JMPHTML-1742, JMPHTML-1721). The individual platforms are not ready, but will soon be there. I also added the Quantile Profiler for Parametric Survival into HTML5 (JMPHTML-1763). I also have added support for categorical response profilers in the Standalone Profiler in HTML5 (JMPHTML-869). Lastly I have added Poisson Generalized Linear Mixed Model Profiler into HTML5 (JMPHTML-1763).
JMP Live
- Aurora: For 18.0, remove the flagging & moderation feature from JMP Live entirely. Use database transactions more consistently (ongoing effort). A few small bug fixes. Miscellaneous: Prep for Discovery.
Statistics
- Jianfeng: resolved some crash reports from technical supports. I am mainly working a feature of one-tailed Dunnett test requested by customers JMP-7951. I found SAS Proc GLM has this feature (means/DUNNETTL, DUNNETTU). I have made some progress but haven’t finalized them yet. Here is a peek of the output. Thanks!
- John Sall: Fix for Explore Outliers for excluded rows (JMP-13830, JMP-13847). Fix Hierarchical Cluster CCC for missing value situation (JMP-13819). Fix Surface Profiler for repeated names crash (JMP-13888). Fix Model Screening for ROC target and Ordinal script (JMP-13945, JMP-13947). Work on Keynote with Julian, Joseph, Aurora.
- Laura C-S: Fixed a bug involving equality constraints in multiple group models and another where a menu option needed to be disabled. I also pushed a change for improving starting values for regressions between latent variables and started implementing a shortcut for linear measurement error models (inspired by a customer asking for instructions on how to do this manually).
- Annie: In addition to fixing control chart bugs and integrating them into JMP v17.1, I’ve been working on a new interface for handling Cusum control chart parameter conflicts.
Stat Documentation
- Michael C: Installed VS2022; wrote a JSL script to assist a testing project that Colleen is working on; answered questions from the doc team about branching the documentation; updated the Numerical Accuracy white paper for 17.0.
- Karen: Defect triage and 17.1 planning.
Stat Testing
- Mark: Completed the Product Security Training courses. Discussed the Easy DOE tutorial (November 10) with Tom Donnelly, the moderator. Converted the script for the usability test of Easy DOE from its original journal format to a JSL application, and extended its behavior to address both moderated and un-moderated tests in preparation for the Usability Lab at the in-person portion of the JMP Discovery Summit next month.. Answered questions in the JMP Community.
- Jacob: Investigating JMP 16/17 crashes in the Prediction Variance Surface. Identifying cases that result in duplicate factor names being used in DOE. Updating performance machines to begin running tests in 18/17.1. Adding new performance tests for Mixed Model, Sort, and Summary.
- Tonya: Testing the ZI SHASH, SHASH with detection limits, and the overall Edit MSA Metadata in MSA. Bug verification. Blog posts for Version 17 new features. Figuring out Service Cloud reporting.
Technical Support
- Duane: TS remains busy with cases. As schools/universities begin, our volume will increase. This week, we have been involved in meetings/discussions regarding several projects: My JMP and My JMP ID, software order emails, JMP Support space in the Community, and our relationship with the SAS Customer Engagement Center. Lots of wheels are turning. As usual, below is a plot of our case volume. The JMP Live reports are still being modified for JMP 17.
Development Operations
- Jim: Working through the Perforce integration of docsource files from Main to r17.0 with Qiana and Susan – thanks to both of them for their willingness to learn about the process, while also trying to improve it! Working through the My JMP Portal installer changes for Macintosh with Jeff, Bob and Audrey. We have a plan moving forward. Now that we are in the Final Candidate stage for JMP 17.0, respective Jenkins jobs are disabled and daily unit test machines are being migrated to run JMP 18.0 and its respective test suite.
- Myron: Worked on several doc/doc-related items and crash-related issues this week:
- JMPTOOL-528 – help map not updating
- JMP-13604 – incorrect OSIPI help redirect
- JMPTOOL-544 – latest source not included in help project builds
- JMPTOOL-347 – make help key report more accessible
- JMPTOOL-564 – update links to defects/Jira from crash summaries
- Created Jenkins job to fix IMAGE-less Mac crash files
- Shannon: Reviewed open crash Jira issues in 17.0, retried test cases and assigned to testers if issues appeared to be fixed. Moved case/issues that were still crashing ahead to 18.0. Tested a few crash system fixes. Reviewed common crash stacks from 16.X and 17 to make sure they are attached to issues and entered new crash investigation Jiras. Completed security training.
Documentation
- Susan C: Ran the help build and visually inspected 6 books in the updated help. Pushed the help to production. Gave some training on FrameMaker & JMP books to Tom, Qiana, and Chris. Working on student subscription updates from Ross. Troubleshooting issues with content and help.
- Chris B: Created and checked the big ‘JMP Documentation Library’ PDF. Reviewed help of 4 books before passing to translation. Mocked up an alternative internal doc site. Finished the security training.
System/Host Testing-Cary
- All: JMP 17.0 Installer Testing
- Audrey: Installer Testing, working with Nick Ruva on getting some good AWS machines for testing purposes, evaluating potential R and Excel issues, evaluating usage of personal spaces in Confluence by our JMP staff and testing out a method for migrating them in the future.
- Mandy: Worked several days on Performance issue with Ernest a bit from Nvidia – with sort and summary and LARGE tables, tracking how long it took on Mac (he did Windows) - came up with some recommendations. Talked to Jacob about adding a scaled back test case for this performance issue. Answered a number of TS tracks, but nothing looks like a show stopper, 2 issues were crashes that might be able to be fixed in 17.1. Investigated JMP Clinical issue with Tabulate and WIDE tables with long names in columns and cells (JMP-13950). Began Mac installer testing for 17.0 but halted at the moment for a license file fix on Mac. Submitted my Workflow talk for Discovery Summit Europe and considering an advanced workflow talk with Michael H.
- Nick: Verifications for JMP 18 bugs and small features, adding several unit tests to the framework. Discussed with Ernest what may be feasible going forward in making our preference strings between the Preferences GUI and JSL more consistent as well as expanding what we write out to Get Preferences() as default. Filed a bug and a couple of feature requests related to With Window Handler and how it handles file picker dialogs. Filed Jiras for the following 18.0 crashes: (1) Test case related to Global Handler in testWindowHandler crashing. (2) Crashes when running a number of project tests in the Unit Test GUI. (3) Crash specifically related to closing a Project window after moving a script into it and running that script (2&3 could possibly be related).
- Hunter: Completed the final part of the security training, Installed Ventura Beta 6 on MDA692 to confirm JMP-13655 remains, and started JMP 17 installer testing before halting due to a possible change for detecting the license file.
- Mike M: Installer testing. Opened 8 new bugs across installer testing, image diffs and new features. Spent a bit of time verifying/closing defects as many folks were updating their Jiras.
- Josh: JMP Live 17 Regression testing, JMP Live 17.1 and 18.0 story testing. Keeping JMP Live 17.1 and 18.0 servers up to date with the latest build. Still working out some the kinks with my silent install scripts.
System/Host Testing-Beijing
- JMP Jira: 17 Bugs / Stories created ( Link ), 8 Bugs / Stories verified ( Link)
- JMP Live Jira: 3 Bugs / Stories created ( Link ), 14 Bugs / Stories verified ( link )
- Beijing Team: JMP 17 Installer Testing, details can be found in JMPTESTBJ-1679.
- Eric Wang: (1) JMP Live 17.1/18 bug verification, added end-to-end tests for bugs. (2) Working on JMP17 installer testing.
- Da Li: (1) Bugs verification which are related to OSIPI, MFI component. (2) Tested OSIPI Timeout parameter. And enter bugs. (3) Provide feedback to Tonya for TS 00023726. (4) Working on JMP17 installer testing.
- Mike Wu: (1) 1. Started the investigation on deploying Trino and created this confluence page. (2) Working on JMP17 installer testing.
- Kimi: (1) Done for updating Formula Depot bench files for JMP 17.1 and JMP 18.0, since the “code generated” version change. (2) Working on JMP17 installer testing. (3) Done for support python init/connect testing (via python tool). Working on support OLE Tests, 17.1 works now. (4) Create some confluence pages under Beijing team JSL Unit Tests Framework (5) Help Mike deal with Trino deployment on CentOS. Create a confluence page: Basic Commands for Linux
- Nala: (1) Update OLEAutomation batch file to make the tests run automatically. Rewrote Confluence page Win+Automation. (2) Working on JMP17 installer testing.
Data Discovery
- Dave B: Updated jmprunner to work with both jmp 18 per files, fixed broken image test, and currently working on adding excel specific settings to MFI.
DOE
- Peng: Continue making changes in 17.1, thank Caleb for reviewing my changes. Fulfilled a recently opened request (JMP-13877) for allowing Degradation to fit build-in models directly through JSL without going through UI interaction to setup initial values. Continue working on prototypes of Response and Factor List UI component for DOE. Continue working with Bill Meeker on research. Finished reviewing six chapters of the third edition of Predictive Analytics using JMP for SAS Press.
- Caleb: Finalizing bug fixes slated for 17.1 for the sample size explorers. I’ve also been revising a couple papers for resubmission and helping prepare for the upcoming Fall Technical Conference.
- Brad: Working on the nomination for the 2023 Shewhart Medal for Prof. David Banks of Duke University.
Note from Shannon
Thanks to John Sall for sharing that he ordered some sodas in response to reading the JMP success story about soda company Phyzz. John says, “When they arrived, the package included a personal note of thanks for JMP. The drink is good, actually.” Phyzz sodas are for sale on their website if you’d like to try them out ($24 for 12 cans) and are coming to selected stores soon.
