Welcome to the JMP Development Update for December 1, 2023!
JMP 18 Approval Mode is in full swing. Developers, testers and others continue to collaborate to identify the critical issues that need to be resolved before we ship. As a reminder, the Doc Team is requesting that non-Stat developers update Jira to identify new feature items that should be surfaced to customers on the New in JMP page. If a feature item merits inclusion, please set the Release Notes Affected? field to Yes and provide a summary by Wednesday, December 20. Otherwise, set the field to No. If you have questions or concerns, please contact Qiana Eaglin.
JMP 18 EA7 is planned to freeze next Friday, 12/8/23. Please update your feature pages with any new details that should be shared with customers and identify the items as relevant for EA7. In addition to pointing EA customers in the right direction, feature pages provide crucial details for internal training usage, as Hyde Miller shared this week in this thank-you message:
I wanted to pass along my appreciation to the developers for all they do that makes it easier to understand and illustrate the new features in JMP. The feature pages are extremely helpful, many times I copy and paste directly into the SE training materials. Also, there has been a lot of really great example data that illustrates the new features. The organization of the late features is also very helpful and has avoided rewriting training material because it wasn’t fully ready to be documented. Going back a couple releases, we would select the top 12-15 features for training. This year I’m developing material for 39 topics, and I’m on schedule with 10 to go. That’s only possible because of the excellent work from the team and their efforts to facilitate the knowledge transfer. It’s been an amazing experience for me to learn about all the really great new features and I know our customers will appreciate them too. Please pass along my gratitude and know that it is appreciated.
Speaking of new feature work, we are nearing our JMP 19 branching on 12/20/23. Once JMP 18.0 work is wrapped up in mid-January, we will also add branches for 18.1 (maintenance fixes) and 18.0.1 (approved critical security and showstopper level fixes only). Just a heads up that we will likely have an EA8 at some point in January once customers have had a chance to test authorization with EA7. More details will be coming as those plans solidify.
Thanks very much to Krzysztof Jozefowicz and Sohail Lokhandwala for presenting at this Thursday’s JMP dev meeting! If you missed the meeting, you can find a recording here. They presented two tools that the L10N team has created that can be used by others at JMP. The JMP Translation Memory Search Interface can be used to query English strings, translations and related fields in JMP’s current translation database of user interface and documentation strings. The other tool supports machine translations of any text using Google AutoML language models trained with our existing JMP translations. Sohail and Krzysztof are interested in feedback on these tools, so please send along any ideas!
Also, thanks very much to Tonya Mauldin for sharing the beta/crash tracks for the last 2 weeks. Crashes are selected below.
Beta/Crash reports for 11/17/23 through 12/1/23 | JMP Live (sas.com)
General Announcements
- If you’re willing to be a featured employee on Inside JMP, please download a copy of the Get to Know form. Simply fill out the questions and when you’re done, upload it, along with 3 photos of you to the folder. Thank you for helping to make our JMP family a little more familiar!
Important Upcoming Dates
- JMP 18/JMP Clinical/JMP Live 18 EA7 – 12/8/23
- JMP 19 Branch Creation – 12/20/23
- JMP 18 EA8 – Date TBD in January 2024
- JMP 18 Code Freeze – 1/16/24
- Please visit Confluence for a full list of JMP 18 dates.
JMP Development Updates
Statistics
- John Sall: For a future release of Fit Response Screening, I implemented an all-effects all-pairs-sliced LS Means Difference test facility, supporting REML also. (JMP-19632) It works for up to 3-way interactions, testing all LSMeans differences across levels for one term for all combinations of levels of the other terms in the effect. The facility can easily be adapted to other fitting platforms. In the image below, lower right label is within a 3-way interaction testing Sex difference within certain levels of Age and Line. For that interaction it is also testing all Age differences for all levels of Sex and Line, and Line differences for all levels of Sex and Age, so all slices are performed. This type of mass-production sliced testing should prove valuable in digging out which combinations are driving interactions.
- Laura C-S: Fixed a few 18.1 jiras in my workspace and picked back up work on MIIV-2SLS estimator. Researching various topics to determine potential submissions for upcoming academic conferences. Met with DEI metrics committee. Met with professor at Stockton U. to discuss SEM analyses.
- Annie: Continuing to fix issues in Control Chart Builder.
Stat Documentation
- Michael C: Updating documentation for various Fit Model platforms, including Least Squares, Mixed Models, and GLMM; reviewed Easy DOE and Screening Design doc updates for Karen.
- Karen: Sent Easy DOE and Screening chapters out for review. CSALT up next.
Stat Testing
- Mark: Closed tickets for Easy DOE in JMP 18. Attended Metrics committee meeting for DEI Action Team to prepare presentation to leadership next year. Met to discuss the distribution of the Usability Study Tool to JMP users. Reviewed draft of chapter documenting Easy DOE. Answered questions in JMP Community.
- Jacob: Testing approved DOE fixes in JMP 18. Unit test cleanup. Preparing a covering array/draft of a unit test to help test the Torch Deep Learning add-in.
- Tonya: Reviewing documentation and working on bug verification.
Technical Support
- Duane: I am extremely happy to say the JMP Community transition to My JMP accounts went smoothly. We’ve had very few cases opened on the change (one RPS case, one inquiring about the change itself). The transition seems to be a big success! As for Support, here are the TS JMP Live reports and the link to the Jira filter showing the tickets reported by TS for the last 14 days.
Case Volume by week:
Case Topics for Engagement queue, total:
Case Topics for Engagement queue, last week:
Development Operations
- Jim: Coordinated mac installer changes of sample data location for multiple JMP products with Michael Hecht (JMP-10455). Performed mac OS system updates on many unit test machines to bring them into SAS compliance. Worked with Kimi to get new Python modules installed and configured on all daily unit test machines. Planning, with Bob’s assistance, to test batch installs and authorization of JMP on WinOS over this weekend. Reported Jiras for JMP Pro 18.0 crashes and assertion failures.
- Melanie: Got the helpmap creation and upload portions of the help processes updated and running on the new AWS vm. With consulting and testing from Nick, Myron, Chris Barrett, Qiana, and Michael Crotty, everything should be in place for EA7.
- Myron: Working on AWS replacement for JMPTOOL (web server, crash database, etc.). JiraBot owner updates.
Localization
- Krzysztof: Worked on a presented to JMP Dev (together with Sohail) the TM Search and MT interfaces. Helping with launching and processing deliveries of L10N projects. Setting up new projects for L10N automation. Continiued working on enhancements to UI translation validation scripts
- Sohail: Implementing enhancements to the TM Search and MT web apps. Demoed the two interfaces to JMP Dev.
- Sandy: Working on validation enhancements for the project creation dashboard.
Documentation
- ChrisB: Corrections to examples in "Discovering JMP" and "Essential Graphing": JMP-15512, JMP-15514, JMP-15515, JMP-13142, JMP-15042, JMP-12286, JMP-15510, and JMP-18841. Looks like Google indexing caught up and the problem of Google returning a featured page that links to an English page that’s in the Chinese help has been solved, but I'm still confirming (JMP-663).
- Qiana: Completed the work on the Keyboard Shortcuts and Menu Description Cards and handed them off to L10N. Continued working on the deprecated sample data. Attended the DEI Leadership meeting. New Features planning. Project planning and coordination.
- Susan C: Working on JMP Live/Public help, both user/admin and install, to finish the updates for 18. I sent both PDFs of the help for review on Tuesday. Reviewers have until Monday to provide feedback. I also updated the help templates for JMP Live/Public.
- Devon: Adding examples to the Python documentation. Added Column Dialog() to the JMP Scripting Reference. Updating HelpKeys and Preferences chapter of Using JMP doc.
System/Host Testing-Cary
- Mandy: Finished setting up my MacBook Pro so Xcode and a few other settings for symbolizing crash files and P4 would work for me like it did before (thanks to Michael H for the help). Worked on a few TS tracks for 18. Tested out a Crash fix where Michael H. added some additional information to be added when a customer sends in a crash file to help determine more about the crash. Worked some more on the data table editing changes – met with Mia to discuss a bit. Testing fixes for workflow builder and running through a few scenarios for 18.0.
- Nick H: Finished up performance tests for Uniform Where in JMP 18.0. Those should be running now as a part of Jacob’s performance test stream. Found a few interesting bugs: one of which is related to tool tips not showing up on Mac OS Sonoma brought to our attention by an EA track – this is likely a Mac OS issue, and another where 2 two-line code block does not delete properly with code folding enabled. Also working on 18.0 bug/small feature verifications as well as a few EA tracks.
- Hunter: Tested the latest SAS Download Manager 3.02 on several Macs at the request of the SAS 9 Deployment staff, sent updates and reviewed the latest keyboard shortcuts and menu description cards for Qiana, and JIRA verifications.
- Audrey: Jira admin tasks, Defect verifications, review of existing authorization bugs, preparing for end-of-release tasks and updating those.
Web Testing – Cary
- Anna: Investigating interval error bar issues, color problems and selection problems with HTML5 output. Working with developers to get information about how to generate reports for late new features for JMP 18.
- Josh L: Reviewing documentation, updating the test matrix from JMP Live 18.0 installer testing. Trouble shooting firewall connection issues and getting firewall tickets approved.
System/Host Testing-Beijing
- Beijing Team:
- JMP 18.0 3rd party software regression testing round 2 (JMPTEST-BJ-1800) - DONE
- Da Li: (1) DONE: JMP 18.0 3rd party software regression testing round2. (2) DONE: JMP Live Migration Automation case writing tips sharing JMP Live Migration Automation case writing tips sharing. (3) Migration automation UT cases writing continued and submit into jmp-live / jmp_live / Branch scndli-migration-jest-ut-writing — Bitbucket. (4) Update jenkins job config to implement 16.2>17.2>18.0 migration automation. (5) Bug verification.
- Nan Wu: (1) Bugs verification. (2) Testing for JMP Data Connectors. (3) JMP 18.0 3rd party software regression testing round2. (4) Technical support issues investigation and provided feedback on: 00076332.
- Kimi(Yu) Wang: (1) Bug verification. (2) JMP 18.0 NF – Tested Python sample scripts. (JMP-18914) Sort out the possible new features of python, ask Paul for the plans. (3) Assisted Jim to set up Python and R env. Python packages are not installed successfully. R set up successfully, testing now. (4) E-mail with Audrey about the new R connection testing via python. (5) Fix UT failures. (6) Jenkins: Check results for Beijing Daily UT. (JMPTESTBJ-2201) – Done. Discuss with Da Li about JMP Live migration path and add a new path 16.2>17.2>18.0 in Jenkins. (7) Others: Create a new confluence page Python Setup and Unit Tests - JMP 18 and above. Update some confluence pages according to Audrey’s suggestion.
- Da Zhang: (1) Bug verification. (2) Fixed E2E test failures. (3) Added new tests for WEBJMP-6764. (4) Investigated slow test running. (5) Fix WinAutomation test failures.
- Nala: (1) Bug Verification. (2) JMP 18.0 3rd party software regression testing round 2. (3) Reviewed Beijing team’s unit tests and prepared for the upcoming 18.1 and 19 testing Tests in Beijing Team's Jenkins - Prepare for JMP 18.1 and JMP 19. (4) Learning playWright and trying to write tests for JMPC non-English testing validation.
Data Discovery
- Dan S: Group Platform order-by stats are now limited based on column type. Fixed regression in JSL output for SendToEmbeddedScriptable. Platform Preset icon will no longer show on outlines that are closed. Fixed crashes in App Builder (report with custom graphics script) and Data Filter (deleted during rs callback).
- Dave B: Fixed a couple of problems with jmprunner “copy local” and jmp clinical support that Evan and Dan noticed, worked on 18.1/19 fix for tablebox sizing, working with Jasean on possible improvements to editing menu items when editing an add-in.
- Morgan: Working on increasing JMP Notebook functionality with other parts of JMP as well as prepping for my dev demo next week.
- Xan: Fixing 18.1 and 19.0 bugs in a branch.
DOE
- Caleb: Worked on a manuscript with Sky and coauthors from Jordan. Fixed a last minute bug in CSALT and starting to plan for the next versions of JMP. On a weather-related note, got some snow flurries on Tuesday!!
Host
- John Schroedl: Updated policy code to use an array instead of unordered_map for performance reasons. Fixed issue where non-auth builds would always use debug.user@jmp.com for the employee id. Crash investigations, code reviews and bug triage. Opened process Jira to move more ancillary binaries to be built nightly.
- Michael H: Wrapping up odds and ends on the “Allow side-by-side installation of multiple JMP Products on Mac” issue (JMP-10455). Changed crash reporting to include additional diagnostics for help tracking down a specific class of crash. Fixed various other JMP 18 issues.
- Paul N:
- Security Update: Fixed High vulnerabilities in zlib, by updating to zlib 1.3. The following applications are affected: JMP 18, JMPPreview, and JMPDtPreview.
- Fix to macOS version of aes_encrypt() that was adding an extraneous 16-byte block of zeros to the plaintext. This caused an issue for data connectors when credentials saved using JMP’s SecureString. Credentials saved on the Mac were unable to be properly read and accepted on Windows.
- Researching Python package compilation issues. There are some Python packages that need to compile local shared libraries as part of their install process. HDBSCAN is one of those packages. Some work has been done. One issue is the need to include the Python include files in our distribution; this has been pushed on the Mac. Pure Python based packages, and those that have precompiled binaries have no issue.
- The current workaround is to install a full Python 3.11.x distribution and use that to compile and install the package in the system default location, then run the install from JMP’s jpip. Or directly copy the files from the system default site-packages/ directory into JMP’s site-packages/ directory. There is still some additional work needed here, as this is my one of my priorities. Security updates to the newest Python 3.11 are also necessary.
- Updating some of the Python sample scripts with input from Russ. He found a shorter code path to create a JMP DT in Python from a pandas data frame, for most use cases. I also revisited and validated JSL code that creates a data table from a pandas data frame using CSV files as the transport mechanism.
dt = Python Get( data_frame );
It returns a ‘Scriptable[]’ data table object, that can be brought forth with dt << New Data View;
This will be just one of the many goals for JMP 19. Making this conversion in memory instead of through the filesystem.
Example code:
# Use run_jsl to create data table by Python Get( df )
# NOTE: This uses CSV file in the filesystem to transfer
# df is a data frame created from Big Class.jmp in Python
jmp.run_jsl('''
df = Python Get(df);
print( df:name[1] ); // Get 'KATIE'
df << new data view; // Make table visible.
''')
Interactive HTML
JMP Live
- Aurora: 18.0: Gave feedback on user help doc, installer doc. Investigating penetration testing results. 18.1: Bug fixing (moving image post from one space to another). 19.0: Allow larger user avatar uploads, downsize prior to storage. Assisted with JMP Live moving auto-integration from 18.0 to 19.0 (wrote Git tips for devs, updated versions in source code).
- Adrianna: Working on things related to JMP Pool authorization.
Note from Shannon
Thank goodness - only a few more weeks of getting dark sooner and we’ll be on the way back to longer days! I hope you all enjoy the weekend!
