Regression testing JSL scripts on new releases of JMP

firesale

Does anyone have any thoughts on or methodology for regression testing JSL scripts on new releases of JMP? Specifically for the purpose of 1) ensuring the script compiles and 2) verifying that the resulting reports are unchanged if run on the same data set. An automated approach would be preferred to manually inspecting the script output. Thanks!

michael_jmp

There is a prototype of a regression testing framework written in JSL that might be of use to you:

JSL-Unit: Automated JSL Testing

There is also a JMPer Cable newsletter article from 2007 that might help you get started as well:
http://www.jmp.com/about/newsletters/jmpercable/pdf/23_fall_2007.pdf

Basically, if you have a set of scripts that produce known results in one version of JMP, you would need to do a little bit of work to instrument them in such a way that they not only produce the known results, but also verify that the known results are still being produced. Then, you could put this set of scripts (test cases) into a framework like the one available on the file exchange. The JSL-Unit framework then provides you a way to run your set of test scripts and produce a report showing the results. You would run this set of tests on a new release of JMP to perform the regression testing. The framework is flexible and extendable if you need to customize it to your needs. Incidentally, the internal testing teams at JMP use a customized version of this framework to perform automated regression testing during the development of JMP.

I hope that's enough to get you started in the right direction!

Best,
Michael