Welcome to the JMP Development Update for June 7, 2024!
The JMP 18.0.2 track is open for security fixes, with nightly builds, installers and unit tests now running. Meanwhile, authorization changes for JMP 19 EAs are in early testing. We expect to freeze code for JMP 19 EA1 next Friday, 6/14/24. Installers will be built early the next week and verified by QA teams.
We are now in JMP 18.1 Approval Mode. If you have a critical fix that must be included in 18.1, please send a request to the JMP Approvers email list with the name of your reviewer and information on the proposed change, including a short description of the issue with Jira link and Swarm link (if applicable). Please be sure to comment on any expected impact of the change for testing, documentation and other downstream teams.
Dev managers will discuss 18.0.2 and 18.1 freeze dates for security fixes next week. All security fixes in 18.0.2 must also be included in 18.1, since 18.0.2 will ship before 18.1. Currently, we do not have any JMP code fixes or security fixes pushed to 18.0.2.
Mia Stevens, Shannon Leslie, Sarah Sylvestre and I met this week to discuss implementing a reflective sprint assignment for Jira story type items after we freeze JMP 19 EA1. Any story that was marked with a fix version of 19.0 before our EA1 freeze date will be assigned an EA1 value in the sprint field. As we move into our EA2 development period, as new items get assigned a fix version of 19, they will be assigned a sprint value of EA2 automatically using Jira automation.
We’ll be using this sprint assignment mechanism to help product management to get a better view of items that will be in the code base for each EA, and they’ll use this list of items list to ping developers to ask about whether fixed items are ready to feature in the EA notes. We will also have available sprint values for future EAs (EA3, EA4, etc.) so developers can pre-assign unfinished Story items to timeframes where you expect to complete them. We hope that the sprint assignment can be used as a lightweight planning tool to help allocate work into EAs and provide more information to our PM team. Mia and the Shannons will talk about this (along with other EA related topics) at the June 20th Dev Meeting.
Thanks to John Sall for sharing this article surveying the teaching of intro statistics using an instrument called the Statistics Teaching Inventory (STI).
John notes that it seems that we still have work to do to catch up to Excel, but usage of GUI-based tools like JMP has passed Syntax-driven tools. The Other and Excel combined category is even more dominant.
From the article:
For instructors who have students use software to analyze data, we also asked which specific software(s) were used in the course. To understand more about the types of tools instructors have students use, the software types were collapsed into three categories: Syntax-Driven (Python, R GUI, RStudio, RStudio Cloud, SAS, SAS Studio), Pedagogical (CODAP, Fathom, TinkerPlots, Statkey), and GUI-Based (JMP, Minitab, SPSS, Stata, Statcrunch). We left Excel and Other as separate categories in this analysis. The results are displayed in Figures 6.
Fig. 6 Percentage (and 95% CI) of STI respondents in 2019 who indicated using Excel, GUI-Based software, Pedagogically-based software, Syntax-Driven software, or Other software not listed.
Of the instructors who have students use software to analyze data, Excel is being used more than any other software tool. There is a fairly even split among instructors using GUI-based, Pedagogical, Syntax-Driven, and Other software. (For those who indicated Other, the most commonly mentioned software used in courses but not listed were the Rossman Chance applets and Google Sheets.)
General Announcements
Important Upcoming Dates
- JMP 18.1 Code Freeze – 6/28/24
- Please visit Confluence for a full list of JMP 18 and JMP 19 dates.
JMP Development Updates
DOE and Reliability
- Caleb: Helped finalize a manuscript with Sky and Zoe at WildTrack and some of their co-authors from Lebanon. They believe they’ve found evidence of what could be a variant or possibly even a new species of dormouse in the country. It’s been an exciting project to help with! Outside of that, been fixing some bugs with the reverse search in the converted explorers.
Hosting and Interfaces
- Paul N:
- Echo() supports echoing or suppressing sending the Python source code into the log.
- Expand() supported internally calling Eval Insert() on the Python source code to look for ^delimited^ JSL code and insert the result of that JSL into the Python source code, just before submitting the Python code to execute.. This seems likely to be error prone. If the user really needs this capability, they can preprocess the string themselves using Eval Insert(). The user can then utilize other delimiter characters as well. The expand() parameter only allowed use of the default ‘^’ delimiter.
- JMP-6803 – Named arguments differed for Python Submit() and Python Execute() even though documentation said they took the same optional arguments, echo( True | False ) and expand( True | False ). Both now support echo(), support for expand( ) was removed from both. JMP 19.0
- JMP-20343 – Suppress JSL statements to the log from Python jmp.run_jsl( ‘JSL code’ ) is called. A new optional echo = True | False | None parameter has been added to run_jsl() to add back the functionality of echoing JLS code to the log. Using jmp.run_jsl( ‘JSL code’, echo=True) will exhibit the behavior of JMP 18 in always sending JSL to the log. echo = False | None will suppress the JSL to the log. (JMP 19.0)
- JMP- 21037 – Python Send File ( non-datatable ) => has incorrect behavior, if you call a second time a hidden script window opened on the first call suddenly pops to the top. Fixed a general issue down in jmpOpen() where an existing hidden windown matched a file spec from a jmpOpen() would disregard the visibility status and force the window visible. The code now honors the visibility status. Additionally, the original Python code tries not to open the file if it’s not a data table importable file, and closes the file if we opened but it was not a data table. For JMP 19.0 – the error message was fixed in 18.1, but still pops window on 2nd JSL attempt.
- JMP-20811 – Python script does not find jmp_score. Trying to run generated Python scoring code, would wind up with a module not found error when run from a JMP script window. Added the install location of jmp_score.py to the Python environment’s module search path. JMP 18.1 & 19.0
- JMP-20057 – jmp.current() should return None if no data tables are opened instead of massage that the data table pointer is stale. JMP 18.1 & 19.0. Updated the value returned by jmp.__version__ in JMP 18.1 to 1.0.1 since some behavior has changed, such as the return from jmp.current()
- JMP-18386 – Python files location in windows installers. Moves all the .pyd and python dependent dll other than python311.dll to the DLLs directory. This also solves some customer issues of not finding module _socket when doing some installs. The Python install processes on Windows apparently have built into their scripts, looking for the DLLs/ directory, since we dumped them into JMP’s directory, scripts running in JMP could find them, but not the install scripts running from the shell. JMP 18.1, 19.0
- John Schroedl: Pushed updated spin button control (JMP-21454). Helped Daniel with a Help > Deauthorize menu which will force hard check-in on next run. Updated libxml2, WebView2 Runtime, and WebView2 SDK to latest releases. Reported false positive static analysis result to Microsoft. Helped Jim with JMPPROLive stuck build. Investigating metafile clipping issue related to << Get Picture (JMP-21026).
- Daniel Powell: Authorization changes are nearly complete for 18.1, with user logout and dynamic error messaging being the final 2 remaining.
- Michael H: Fixing annoyance and crashing bugs on Mac.
Interactive HTML
- John Powell: Dealt with some build machine issues after an outage last weekend with help from Nick Ruva and Alex Taylor. Investigated ways to permit embedding JMP 17 Interactive HTML files in web pages (other than JMP Live) using iFrames in response to a JMP User Community discussion with help from Michael Goff. Improved bubble plot drawing performance (JMPHTML-13817). Wrapping up our performance improvement sprint documenting findings, efforts, and results. Began preparing for our first JMP 19 new feature sprint to be led by Praveena.
- Paul Spychala: Submitted a changelist with a rework of how selection is done on HTML. Before we used to let each segment be in charge of selecting rows and sending out RowState event changes and redraw requests and selection states, and now they only return the rows that they are interacting with and pass the logic to a Frame element JMPHTML-13809:. This rework prevents individual segments from fighting over rowstate and redraw events.
- Praveena: Pushed code changes to handle the shapes with respect to the triage status and the rowState change behaviour due to exclude/hide & include and show all. Addressed WEBJMP-8624 and WEBJMP-8625. Spent some time with Sam for Pie Chart segment work. Met with John Powell to plan Sprint 19-3.
- Mike Stickles: Completed work on JMPHTML-2018 (Bar Segment takes a long time to load with multiple X-columns having many possible values each). Began and completed work on JMPHTML-1049 (Page slow to display in some instances with backslashes in X-value names). Updated performance metrics for our performance sprint.
- Michael Senoyuit: Completed an optimization for shapeSegment to prevent unnecessary calculations.
JMP Live
Statistics
- John Sall: For Categorical,
- improve Means Differences report and saved table (JMP-19882, JMP-21495),
- fix the reversed column names in situations with "Save tTests and pValues" (JMP-21676),
- color red significant comparison (uppercase) letters (JMP-21688),
- fixed drag/drop for when multiple spaces were in column name (JMP-21703),
- for "Specify Comparison Groups" supported broadcast mode and added a Favorites feature (JMP-21315),
- suppressed Exact tests for comparison groups that shouldn't have them (JMP-21704),
- changed four save commands to save multiple crosstabs (JMP-21731).
- Elizabeth: Investigating a crash with the scatterplot matrix in KMeans that is almost certainly due to running out of memory as I watched the usage hit 98% on my machine before I had to kill the process (>2000 columns, so >2mil pairwise scatterplots); waiting to hear whether capping the number of columns for the matrix to be created would be acceptable since we can’t magically give users more memory.
Helped Meichen with getting sandwich standard errors from GLMM to Causal. Running simulations as part of some research for stability estimates with mixed models (and thanks to Beth Evans for procuring a book with a chapter written by a JMP user related to this topic).
- Laura C-S: Finally pushed all the code I had for the MIIV-2SLS estimator in SEM. The estimator option in the launch menu is still disabled because the feature isn’t ready for prime time, but I’m down to the last few details. I’m working through all the RTM options of the platform to figure out which apply to this new estimator and to update the code where options do apply.
- Kasia: Made progress on the new Bayesian Optimization platform with Chris. I added kernel selection to the platform dialogue and the ability to edit model parameters to the platform itself.
Stat Testing
- Mark: Finished a prototype of a Bayesian Optimization platform with Chris. I introduced a decomposition of the Expected Improvement function into the contributions by exploitation and exploration to aid in understanding the benefit of adding the next run. I am writing the poster with Tonya for JMP Discovery Summit this Fall.
- Jacob: Testing Categorical, adding tests for Cell Chisq Pval FDR and the new Mean Score Comparisons table. Testing updates to OC Curves. Working on scripts and add-ins for Bill Fisher’s usability study, following up on the Easy DOE study from Discovery Europe.
Life Sciences Updates
- Wenjun: Our AI paper, collaborating with FDA researchers, has been published in the Frontiers in Artificial Intelligence. The paper describes two important aspects of AI, Text Explorer and XGBoost in JMP Pro, that are highly effective in classifying Drug Induced Liver Injury with very setup and no coding required, of course.
- Drew: Closed out as many last remaining Jiras for 18.1 prior to approval mode and deferred the rest to appropriate planned versions. Continued fixing bugs and addressing new features at main. Participated in retrospective for JMP Clinical development processes.
- Rebecca: JMP Clinical retrospective was held on Thursday. Many thanks to Shannon Conners and Shannon Leslie for moderating for us! It was a very successful event and resulted in some good suggestions we’ll implement for 19.
- Richard: Participated in retrospective for JMP Clinical 18. Working on SMQ/FMQ manuscript. Prototyping AE Recurrence report.
Product Management and Product Marketing
- Bryan F: Continued the work associated with an alternative and optional JMP theme. Dan Schikore and Evan McCorkle are preparing the associated code so that it can be delivered to the main branch after the work is considered sufficiently complete and after EA1. Also, I continued to work with Sarah Sylvestre on the data table report. We intend to present the findings next week.
Technical Support and Engagement
- Duane: Support moves along. I’ve enhanced the usual first plot a bit and broke it down to two visuals: one for Tech Support, and one for Engagement Support. This shows the drastic increase in the number of cases Wendy’s team is handling. As normal, here are the TS JMP Live reports and the link to the Jira filter showing the tickets reported by TS for the last 14 days.
Case Volume for Technical Support by week:
Case Volume for Engagement Support by week:
Case Topics for Engagement queue, total:
Case Topics for Engagement queue, last week:
Development Operations
- Jim:
- Due to recent changes made to the way EAs are authorized, the EA branch was created to allow for interim installers to be built and authorization tested. The branch will be refreshed once we reach 6/14. Thanks to John Schroedl for his assistance and patience in overcoming unusual file attribute issues/gremlins while investigating WinOS EA compiler errors.
- Thanks to Nick Ruva for all the assistance on Jenkins UT nodes, allowing only single projects to be run.
- Thanks to Hunter Weaver for his thorough installer and authorization testing skills in confirming specific Retail and Authorization setting behaviors.
- Thanks to Daniel Powell for getting the JMP 19 authorization changes in place and making necessary modifications, allowing unit test runs to launch unassisted.
- Perforce support.
- Nick:
- Created the JMP_18.0.2_Nightly_Build_Logs [Jenkins] and JMP_19.0.0_Nightly_Build_Logs [Jenkins] projects to move nightly build logs into our Jenkins dashboard and to automate some of the log checks Jim does nightly for our JMP build process.
- DailyUT work and updates to track down some stubborn dual-agent issues for running jobs
- Some updates to the Daily_UT_Watcher [Jenkins] (jmp.com) project to handle the new 18.0.2 DailyUT machine runs and optimize how this script runs through the various nodes and AWS machines.
- A few other Jenkins troubleshooting issues to recover from an outage over the weekend. Everything appears to be running fine now.
- Melanie: Stringbot Migration - Documented and described the initial testing plan; nailed down some configuration plans with Myron. Jira: Worked with a few people on how to manage Jira and Jirabot to get the assignments they want; I will be making a change next week to one of the automation rules to exempt JMPDOC from changing the assignment when the components field is updated. DevOps Processes: General investigation, testing, troubleshooting for Unit Test streams; participated in build migration and EA planning meetings; experimented with some different ways to visualize Jira data in JMP.
- Bob: Created code for new JMP EA installer. Built draft installers for JMP EA, JMP Pro EA, and JMP Clinical EA for authorization testing – these are NOT the final EA 1 installers. Worked with Legal to get final license agreements for EA installers.
- Shannon C: Facilitated JMP Clinical team’s first retrospective on Thursday with Shannon L. Thanks very much to all for offering feedback and great discussions about improvements to the Clinical development process.
- Shannon L: Held the JMP Clinical retrospective with Shannon C; reviewed June 11 upcoming Jira Cloud and Confluence Cloud changes - will provide highlights soon.
Localization
- Allison: Sent out GPO documentation for translation into Euro languages and Korean; looked into cross-ref link issue in 18.0 UG and communicated with Doc team and localization teams; responded to queries from translation teams for JMP Documentation localization.
- Trish: Onboarding tasks for Sohail (who starts full time with our team 16June). Query triage. Project planning and scheduling.
- Howard: Continued testing some dB build features - QA Report, Backend Task engine, and Jira condition/triggering. Also adjusted memoQ return job for rel 18.1.
- Sandy: Tested Project Dashboard export to json file function. Worked on file comparison validation programs.
- Krzysztof: Processing translation returns (validations and pushes) of 18.0 Books and 18.1 UI (Portable and SampleData) projects. Localization workflow support and big fixes.
- Sohail: Bug fixes and new feature enhancements to translation search interface.
Documentation
- Khawar Latif Khan: Removing manual steps from screenshots (JMPDOC-26
finished and closed with help from Chris Barrett and Michael C. UG edits (JMP-21471 Susan helped with some edits; moved to test-ready.
- Qiana: Participated in Global Inclusion Week activities at SAS. Installed Oxygen. Did some sprint maintenance for the JMPDOC and Oxygen projects. Provided a copy of our Designer Projects folder and info about our context sensitive help linking (help keys) to Tony at WebWorks. Discussed JiraBot automation rule changes for JMPDOC with Melanie and Michael. Addressed JMPDOC-2072. Created Jiras for updates needed to the DE, ES, FR, IT JMP Documentation pages.
- Susan C: Working on JMP Live 18.1 help updates and also on Jira management (updating dates for 18.1, updating sprints on Jira boards, creating a few new Jiras).
- Devon: JMPDOC-2202 and JMPDOC-2209: Fixing images that were positioned over text and broken links, cleaning up errors in produced XML output from FrameMaker source files.
- ChrisB: String reviews. JMP-16705, path changes in 18 and impact on add-ins. Finishing tool to compare map files.
System/Host Testing-Cary
- Adnane El Hachami: Attended JMP Academy. Also finished working on my third project which aims to generalize some preset tests to more languages other than English.
- Mandy: Working on all things Data table – Forced Values, tags, nesting of column groups and scripts and found some crashes and fixes along the way – hoping to get a journal created for a short EA video by middle of next week. Worked with Mike M. on a Tabulate issue that was broken at 19, but Dan S. has pushed a fix. Writing 3 short Image tests for Group/Ungroup Cols and then Scripts, and also adding a few tags – just to make sure as changes pop up with the visual part of things, we don’t break it with new pushes.
- Nick H: Worked on verification for Platform Preset bug fixes, adding some new unit tests as well as some other bug verifications for 19.0. Fixed some more failing unit tests following Jitter changes, and changes made to Report() error handling. Assisted Adnane in getting started with his latest project of trying to get our Platform Preset example unit tests running in other languages. Added a Confluence page for the Platform Preset Test GUI that I made during the 18 cycle, hopefully Adnane will be able to use this to find some gaps in platform command coverage for presets throughout his time this summer.
- Hunter: Worked with Daniel and Jim to report authorization issues with the nightly JMP 19.0 on Mac non-retail builds that have authorization enabled. Verified JMP 18.1 project fixes and installed the first test builds of JMP 19 EA1 (Standard, Pro, and Clinical) on Windows and reported an issue to Bob with a crash when launching Clinical EA1.
- Mike M: Continue updating Sikuli tests to run on 19. Found another regression with Tabulate not working with their version of Border Box padding. Thanks to Dan Schikore for fixing so fast.
- Audrey: Participated in a JMP Academy day for interns and hosted a Q&A for them with John and Jon. Atlassian admin and consults. Preparing for EA1.
System/Host Testing-Beijing
Core
- Parker Dingman: Pushed faster map image anti-aliasing, new loading mouse for slow map API operations, and new tiling to show one globe (instead of multiple) in zoomed out view.
- Dave B: Reproduced an anonymous crash in BarSeg and fixed it in 19.0 and 18.1. Found a test case where textbox was ignoring non-breaking spaces and have a fix almost ready. Worked with John Schroedl on an erroneous build warning with jmprunner builds that Andrew V found.
- Xan: Hex jitter testing; Graph Builder bug fixes and enhancements (allowing more color/size/interval variables); higher resolution ellipse calcs. Trying a development blog in Confluence, since January, here.
- Dan S: Group Platform now restricts a shared column switcher from switching the By-column or multiple-Y columns. Fixed a few padding issues related to the Visual Styling work.
- Evan: Preparing VisualStyling branch to go to main.
- Morgan: Getting Notebooks ready for JMP 19 EA1!
Note from Shannon
Today was my son Hunter’s last day of sixth grade before summer vacation! He’s already been to the pool and sorted Pokemon cards…and is looking forward to a break from classes and homework for a few months. Hope you all have a great weekend!
