JMP® 18: Going beyond simply satisficing PythonistasOne year ago, JMP users who also used Python made it clear they thought our Python interface was less than optimal. Ouch! We thought the JMP and Python integration we introduced more than five years ago was hitting the mark. Realizing it was only satisficing (in other words, that it was good enough but not the best), we decided to address the challenge head on.
JMP Principal Systems Developer Paul Nelson spearheaded the effort to create a consistent, full-featured, integrated development environment for writing and executing Python scripts within JMP in JMP 18.
I recently spoke with Paul about the upgraded JMP and Python integration. Paul emphasized that the goal was to provide a significantly more productive environment for Python developers. Some key improvements include:
A consistent environment that will work immediately when installed, without configuration. Python supports unlimited customer Python environments. Adding new JMP and Python capabilities that could not be tested in unlimited customer Python environments was untenable. The solution? JMP 18 includes an embedded version of Python that is the basis for a fully tested environment that will work immediately upon installation.
Direct memory access to Python data tables in a live environment. In previous versions, Python users accessed a copy of a JMP data table. Now, JMP 18 provides direct access to JMP data tables from the Python environment, so Python users can access, create, and modify JMP tables from Python scripts within JMP.
A new Python Script Editor that uses Python commenting and code coloring, as well as an embedded log. The previous integration used JMP Scripting Language (JSL). JSL was familiar to JSL developers but was often uninviting to JMP users with Python experience but no JSL experience. In JMP 18, users can open and run Python scripts directly in the JMP script editor and run JSL from their Python scripts. The JMP Scripting Index documents and gives samples of the new functionality. Having a consistent environment for Python scripts also provides a foundation for writing add-ins using Python as well as JSL.
Hear more from Paul:
Consider registering for Paul's March 21 live webinar Developer Tutorial: New Python Integration and Scripting Capabilities in JMP 18 to hear him provide full details and demonstrate the interface. And, if you attend JMP Discovery Summit in Manchester, UK, you can hear Paul speak as part of the morning plenary on March 6.
By the way, not being a Python programmer, I hadn’t heard the rumor that Python originator, Guido van Rossum, was reading scripts from “Monty Python's Flying Circus” when he started working on the new language. He wanted a short, unique, intriguing name, so he decided to call it Python. Sounds plausible!
JMP Principal Systems Developer Paul Nelson spearheaded the effort to create a consistent, full-featured, integrated development environment for writing and executing Python scripts within JMP in JMP 18. The result? A more productive environment for Python developers that works immediately when installed, gives direct memory acces to Python data tables in a live environment, and an integrated Python Script Editor that uses Python commenting and code coloring and an embedded log.
Automation and ScriptingRe: JMP® 18: Going beyond simply satisficing PythonistasWow, this sounds awesome!
Re: JMP® 18: Going beyond simply satisficing PythonistasComment / Question on bringing data back to JMP from Python.
Currently we can do this :
dt = Python Get( df )
where 'df' is a Python dataframe.
I have noticed that what the above command actually does is write the dataframe out to a CSV in the $TEMP folder, then invoke a JSL to read it in, then delete that temporary CSV. For large files there's obviously an overhead to both writing the file and scanning it / reading it in.
It would be very nice if this could be an in-memory transfer direct from Python to JMP
Re: JMP® 18: Going beyond simply satisficing PythonistasOut of curiosity, will JMP v18 support formula columns with python code?
Re: JMP® 18: Going beyond simply satisficing PythonistasIn JMP 14-17 both the Python Send ( dt ) and Python Get( df ) operated by saving CSV through a temporary file. This is one of the major changes in JMP 18. With JMP 18, the Python Send ( dt ) now creates a jmp.DataTable object in Python. This is not a copy it is a live in memory reference to the data table. It is able to be read, and modified directly from Python. From JSL if you do a Python Get ( dt ), the data table you will get the in-memory data table reference. Left for compatibility concerns was the original Python Get( df ); method of getting a pandas dataframe into a JMP data table as a copy.
You now have the power to take a data frame in Python and simply iterate on the data frame columns and create in-memory, the equivalent JMP data table. See jmp.DataTable. under the new Python category in JMP 18's scripting Index. There is also a Python subdirectory to Sample Scripts. In there you will find a JMP2pandas.py script that shows two ways to create in-memory a pandas.DataFrame from a jmp.DataTable object, and one example of creating the jmp.DataTable object from the pandas.DataFrame ( in-memory )
# create JMP DataTable from pandas DataFrame
dt2 = jmp.DataTable('BC2',df.shape[0])
for j in range( df.shape[1] ):
dt2.new_column(dt[j].name, dt[j].dtype)
dt2[j] = list(df.iloc[:,j])
Where df is a pandas dataframe, and dt2 is the new JMP DataTable. ( note dt is an also a JMP jmp.DataTable object, the type parameter is a jmp.DataType enumeration value. ). The example file opened 'Big Class.jmp' as dt, creates a pandas.DataFrame, the creates dt2 from the data frame.
Re: JMP® 18: Going beyond simply satisficing PythonistasFormula columns do not support Python code in JMP 18. It is something that is being investigated, and would be a great feature. There was an enormous amount of infrastructure that needed to be added to make JMP 18 robust, work on startup, support a Python aware script editor, enhanced error reporting, add scripting index, and opening up a bridge from Python to JMP. Many things that had been lacking for some time, have now been enhanced to make what be believe to be an inviting and productive Python environment within JMP.
Now that the heavy lifting has been done, we are not calling it a day. Much more to come.
