All information is SAS company confidential. Do not disclose.
Welcome to the JMP Development Update for April 2, 2021!
A reminder that we will enter Code Approval mode for JMP 16.1 on 4/27/21. (I know-how can that possibly be coming up at the end of this month! But it is.)
April 27 is also the deadline for making changes to JMP 16.1 strings. Please continue to review your outstanding strings in the weekly StringBot emails for 17.0 and 16.1. If you have pushed changes to 17.0 that were integrated into 16.1 at a later date, please don’t forget to double-check your Stringbot results for 16.1 to make sure you haven’t missed anything.
You can check the ScanBot page within a few hours of making string edits to make sure that any stuck items are correctly marked for extraction. Any strings added after the Stringbot run the weekend of May 1 will ship in English for 16.1.
We are continuing to make progress in our Defects to Jira transition planning. You can find pages covering various JMP Jira topics in the Defects to Jira Conversion section of the JMP Confluence space. This week, the Jira discussion group meet for a second time (notes for these meetings can be found here).
Thanks to a lot of work by Amy and Audrey, all JMP components have now been populated in Jira and default roles populated in JiraBot. I encourage everyone from development to make sure that you have logged into R&D Jira using the details in my email from this morning. You won’t be able to be added as a Watcher (interested party) till you do.
If you have a set of defects that you would like to move to Jira, please let me know and I’ll work with you to get them transitioned. Thanks very much to Justin Chilton for helping create a cleanup process to resolve various issues I encountered while using the Jira CSV import route to migrate items.
Thanks to Peng Liu and Duane Hayes for sharing the news that Donna Woodward from the SAS tech support team is retiring, and next Tuesday will be her last day at SAS. Although Donna is not part of Duane’s team, she has long supported both SAS time series features and the JMP time series platforms. Peng says, “Over the years, she helped JMP customers to better use the Time Series platform, and gave wonderful suggestions that helped the platform to improve. I will miss her. Wish her all the best!”
Thanks very much to Xan Gregg for presenting at this week’s dev meeting! Xan presented several customization examples using graph builder features and highlighted some improvements that are new in JMP 16. If you missed the meeting, you can see a recording here.
Thanks to Tonya for sharing this week’s details on Beta/Crash/Telemetry Tracks! (Beta/Crash/Telemetry Tracks for the week of 4/2/21 | JMP Live (sas.com)
Company
Product Release
Topic
Subtopic
Platform
General Announcements
- If you’re willing to be a featured employee on Inside JMP, please download a copy of the Get to Know form. Simply fill out the questions and when you’re done, upload it, along with 3 photos of you to the folder. Thank you for helping to make our JMP family a little more familiar!
Important Upcoming Dates
- JMP 16.1 Code Approval Mode Begins – 4/27/21
- JMP 16.1 Code Freeze – 5/14/21
- Please visit Confluence for a full list of JMP 16 and JMP 17 dates.
JMP Development Updates
Statistics
- Jianfeng: Thanks to a helpful discussion with Chris, Mark and Jeremy, the design of the equivalence tests pop-up dialog was decided. Thanks to Xan for fixing a reference line width issue! The screen shots of these plots reflect the equivalence, superiority and non-inferiority tests. There is still a long way to go, but I am happy with our progress!
- John Sall: Explore Missing Values now has replaced the "Multivariate SVD Imputation" with "Multivariate RPCA Imputation" using the same mechanism used in Explore Outlier for Robust PCA. This will be important in making the imputed values robust with respect to outliers. The old command is still available through scripting (S1586294, S1587397).
- Peng: Researching repeated measures degradation and answering questions from the field.
- Elizabeth: Integrated fixes for 3 SVM bugs into 16.1. Continued work on GLMMs, which will be tracked under Jira epic JMP-87.
- Laura L: Working on the Type 1 Gauge platform, which will tracked under Jira epic JMP-293.
Stat Testing
- Mark: Settling in with Perforce. Learning unit testing with Nicole. Refreshed knowledge of XML Path Language. Assisted Clay with new pharmacokinetic models for Fit Curve. Assisted Mia with dissolution curves. Assisted Jeremy with spectral analysis. Assisted Jianfeng with new equivalence testing. Shadowing Jacob for MSA Design testing. Answered questions in JMP Community.
- Jeremy: Getting caught up verifications and unit test clean up for 16.1. Giving Jianfeng feedback on her new equivalence test additions. Organizing notes on spectral data analysis in JMP, preparing example workflows. Provided DOE team with materials on the covering array tests for disallowed combinations which they will use in their DATAworks presentation.
- Jacob: Tested new replicate runs options in MSA Design. Tested Multivariate RPCA Imputation. Tested OC Curves launched from legacy Control Charts. Worked on time series cross-validation add-in.
- Tonya: Cleaning up unit tests, testing new process screening features (sort, new subgroups, time as a subgroup for CCB), and bug verification.
Life Sciences Updates
- John Sall: If JMP Genomics removes SAS in the future, they will need Fit Mixed to support many responses representing tens or even hundreds of thousands of gene expression values. A simple fix was applied to run multiple Fit Mixed models in one window. However, huge performance bottlenecks involving scrolling support in TableBox and host features in the constructor for GFont became apparent while creating reports for a problem with 49,041 responses. Overall, numerical processing was a very small fraction of total cputime. So some options were added to collect the results into output data tables, suppressing reports, allowing two different test cases to complete in about 10 minutes (49,041 responses by 449 rows, 20,124 responses by 7,266 rows)-previously, each would have taken days of cputime. When run in SAS, this was done by stacking into by groups, so ultimately in JMP it will be much smoother, done directly from wide data. Chris Gotwalt has ideas on how to make this dramatically faster. Separately, it became important to shorten saved scripts for many responses by using column groups (S1393891, S1588101). Here are the output data tables from 49,041 responses fit in Fit Mixed:
Technical Support
- Duane: Working with John Jones on JMP Student Subscription issues regarding the MySubscription page. After communication with Dan V., I have updated select Sirius email reports to include JMP 16. TS reviewed an updated set of Activation Tool pages. They are view-only and intended for use by the Customer Contact Center. Bobby coordinated the update of version-dependent TS collateral to include JMP 16.
- Wendy has been working with Mandy, Audrey, John Schroedl, Chung-Wei, Ernest and others on a performance issue with deleting rows in large data tables.
- Wendy and Duane will be working with Audrey to set up a JMP TS Confluence area off of the existing JMP Confluence pages. We hope to populate it with information that is currently spread over SharePoint, Teams, Outlook. (Thanks, that will be terrific!-Shannon)
- Here are the TS JMP Live reports, and here is information regarding Customer Reported Defects filed by TS.
Development Operations
- Jim: Worked with Justin to add a JiraIssues status file creation project to Jenkins (https://rndjira.sas.com/browse/JMPTEST-44). Removed “Check for Updates” link on the summary page for the JMP 17 installers on Mac (https://rndjira.sas.com/browse/JMP-288). Perforce support.
- Shannon: Jira discussions about various issues, Confluence pages for various Jira topics, working with developers on epics for new features, reviewing unit test DNCs and crashes needing further investigation.
Documentation
- Michael C: Updated the Imbalanced Classification add-in on the JMP Community to a newer version (with Colleen and Marie G); the original version didn’t work in JMP 16 due to a change in JSL behavior. Continued work on Jira conversion investigations.
- Karen: Working on minor defects for 16.1. Provided a list of user feedback suggestions for DOE platforms to the DOE team after working on a complex DOE project for a client.
- Sheila: Worked on defects. Generated the 16.1 help. Regenerated and published the 15.2 JA help to fix a search bug.
Host/System Testing
- Eric Wang: Testing JMP Live Jira items, testing JMP Live database / regeneration.
- Da Li: Testing Google Sheets NF (BRDSTG-3084). Testing JMP Live Jira items, testing download entire package (WEBJMP-2533). Verified defects. Preparing knowledge sharing about the testing of downloading JMP Live Content as a JMP Project.
- Mike Wu: Tested JMP 16.1 Student Subscription and JMP 16.1 Sub/Student Subscription reset tool on Mac. Verify defects.
- Nala: Testing JMP Live Jira items, tested the APIs of access records count, tested URLs on Publish dialog. Verify defects. Learning the new Jira workflow.
- Mandy: Reviewed Tabulate defects as preparation for moving to JIRA. Met with JMPLS team to demo a new JMP 17 Tabulate feature. They had additional requests - opened 3 new defects. Worked on a Select Duplicates performance issue reported to JMP TS by NVIDIA while using a 54 million+ row table - tested possible improvements with Ernest. Verifying defects and adding JSL and image tests for Tabulate NF.
- Hunter: JMP 16.1 and 17 defect verification, worked on misc projects items, applied updates to Big Sur host testing Macs in the lab, and tried to reproduce JMP 16 Trial installer issue on several Macs from a CRP.
- Lisa: Verified several hover label crashes and wrote tests for Cumulative Percent summary statistic in GB & new legend item wrap/max items feature. Also got some how-to training in Jira from Audrey.
- Mike M: Trying to clean up the noisy image diffs. Added check to fix a noisy baseline test. Updated Sikuli to run 16.1. Verifying 16.0 and 16.1 JMP Help defects.
- Justin: Continued work on open Jira issues feature for JSL unit testing framework. Created UI for easily running Jira issue cleanup script. Cleaning up and fixing open defects for Scripting Index content.
- Audrey: Jira Jira Jira.
Data Discovery
- Chung-Wei: Looking at speeding up delete rows and continued fixing tabulate bugs. Implemented JSL statements to read a subset of a data table without bringing in the entire table (most recently requested at Discovery). The first statement will the file and read in the first 10 rows of the table. The second statement will read in the last 10 rows of the table. The third statement will read 10 random rows from the table:
dt = Open( "$Sample_Data/Hollywood Movies.jmp",first(10));
dt = Open( "$Sample_Data/Hollywood Movies.jmp",last(10));
dt = Open( "$Sample_Data/Hollywood Movies.jmp",random(10));
- Dan S: 17 - Fixed 6 cases where "Sync to Data Changes" item was showing up with the recent Auto-Recalc-off changes. 16.1 fixes - journal toolbar for narrow widths, GB contour journaling, slow performance changing tabs.
- Ernest: Performance fixes in Select Duplicates/Term for Nvidia. Adding preview to Tables menu items.
- Dave B: Fixed surface plot journal defect, fixed jsl sib append, looking into performance issues, fixed marker expression not magnifying with crosshairs cursor.
DOE
- All: Met with John Szarka and Maria Lanzerath of Gore for them to describe their AddIn for designing MSA studies – more data for our own improvements of MSA for JMP 17.
- Brad: Reviewed a paper for the Journal of Quality Technology (JQT). The new editor of JQT is Allison Jones-Farmer. She asked me to continue serving on the Editorial Review Board and I agreed.
- Joseph: Worked on JMP 16.1 MSA bugs and updates. Continued work on the new EZDOE platform which will be tracked in stories under Jira epic JMP-292. Worked on DataWorks 2021 tutorial slides. Also, edited the proof of the Combinatorial Testing paper, coauthored with Ryan, for Statistical Theory and Related Fields journal (to appear later this month).
- Caleb: Fixed a few more bugs and adding run order/replicate columns in MSA designs for JMP 16.1.
Host
- John Schroedl: Checked in WebView2 runtime and SDK. JMP code changes are pending installer update. Adding project tabs to Ctr+Tab window switcher. Reviewing Win2D update from MS.
- Paul N
- Interviewing candidate for Licensing position. Debugging S1587999 – a bug dating back to 13.1 or earlier where saved axis settings in an OpenGL journal were not being read in properly. Thanks to Dave Barbour for the fix. Investigated S1588080 – where a missing value was causing a crash in an OpenGL script. Could not replicate the issue on Mac, but added code to validate inputs anyway for JMP 17.
- Continuing work on Metal. Added Apple’s SceneKit and test view into JMP. Apple has 3D graphics frameworks that include Metal, a very low level framework that both MetalKit and SceneKit are based on. Metal is lower level than OpenGL, SceneKit is higher level than OpenGL (and is based on Metal). Exploring the tradeoff of dramatically easier coding and use vs performance on large data. Metal is almost a graphics assembly language in that the full power of the GPU is available, but every minute detail has to be specified and painstakingly correct.
Demo in JSL in JMP 17 on the Mac – Display Box 3D() use bare bones Metal, this just shows a rotatable 3d grid.
Several hundred lines of code to do all of the scene setup, establish the graphics pipeline, including GPU shader code, draw the grids, and do the quaternion math to handle the arcball rotation.
d3d = display Box 3D(800,800);
d3d << autostretching("on");
d3d << Background Color( {240,240,240 });
new window ("box3D", d3d);
d3d << set size(1200, 900);
Display Scene 3D brings up a SceneView. Lighting, shadows, basic shapes like cube and plane built in.
About 2 dozen lines of code for the entire scene, arcball support is built into SceneKit (one line of code to enable). No additional shader code needed for this level of complexity. (Clipping planes will likely require custom shader code, either or SceneKit or Metal)
s3d = display Scene 3D(800,800);
s3d << Demo("cube");
new window ("Scene3D", s3d);
s3d << set size(1200, 900);
HTML5
- The team held their JMP 17.0 Interactive HTML5 Kickoff meeting this week. Bryan Fricke posted the kickoff meeting notes here.
- Josh M
- S1586056 Made a large push for Interactive HTML profiler in 17 to fix the formatting of the response variable to match JMP, follow the orientation of the Y axis label, and adjust the layout to accommodate larger edit box and select controls which now follow material design. There will be several changes coming in preparation for categorical profilers. Made an additional improvement to profilers to simplify processing in browser by discarding unneeded axis contents at export time in C++
- Fixed a line bug found in whitebox testing marked as a high for 16.1 and 17 (S1587810: Value label appearing in % of total value)
- Fixed a data filter tooltip bug found in last round of whitebox testing related to performance work for categorical data filter that is still only in 17, but on the table to pull back to 16.1 if verified (S1587218), along with an additional improvement (S1588085). I have now let the testers know to perform additional testing so we can make a final decision on pulling it back.
- Bryan F and Mayowa: Collaborated to fix an issue where unselected boxplots look darker when exported to HTML5 (S1587815).
- Bryan F: Resolved a regression issue where a large legend caused the HTML5 export to display a HUGE blank report. Fixed an issue where information could appear twice in a tooltip. Prioritized ideas for refactoring projects and presenting Scrum process at HTML5 JMP 17 kickoff meeting.
JMP Live
- Michael G: Re-added regenerate post option. 16.1 bug fixing. Added a right-click context menu on JMP Live reports allowing you to perform some of the HTML5 report actions quickly.
- Aurora: Downloading a JMP Live post as a JMP Project: bug fixing. Package access records: bug fixing. Whitebox testing.
Note from Shannon
We have new additions to the JMP family this week! No, I’m not talking about our JMP 16.0 product releases, now available for all customers to request and download…
Russ and Anisa shared the good news that JMP Clinical developer Nora Varga had her baby! Sophie Emese Paton was born March 31 at 4:57 am. She weighs 7 lbs 9 oz. Mom and baby are both doing well. Big congratulations to the happy family, and I hope to have pictures to share at a later date.
Congratulations also go out to Mandy and Jeff Chambers on the arrival of their new grandson Regan Sharp Womble, 9 lbs, 5 oz, 21 ½ inches tall. While Mandy and Jeff could not be there this time due to COVID restrictions, they were able to Facetime later to meet the family’s new bundle of joy. Mandy says that mom, baby and the rest of the family doing well and they look forward to meeting him on Monday!
Regardless of whether you celebrate the Easter holiday, everyone can enjoy the seasonal candy and the memes that result. Have a great weekend!
