请注意,内容编辑器从 defs.py 的内容中删除了一个回车符,应该显示为:
def f(x):
return x*x
在假期休息期间,我有时间修补 Python。 我们在 Python/JSL 接口的并行处理中遇到的问题是 Python 通常会启动 JMP 的多个副本,而不是 Python。 原因是我们在 JMP 中嵌入了 Python,因此可执行文件是 JMP 而不是 Python。 我发现 multiprocessing 包允许将可执行文件设置为在运行并行代码时启动。 这是拼图中缺失的部分。
下面的脚本我已经在 Mac 和 Windows 上使用 Python 3.9 在 JMP 15.2.1 上进行了测试。在我的 Windows 机器上,我必须使用 Python Init() 的 Path() 参数。 我有多个版本的 Python,并希望确保选择正确的版本。 在我的 Mac 上,不带参数的 Python Init() 适用于我的配置。 我在 Windows 上遇到了一个问题,我不得不硬编码我的 pythonw.exe 的路径。 您将需要更改它以匹配您的 Python 位置。 搜索修复我:
除了 Python Init() 参数和 Windows 上的 pythonw.exe 路径之外,这应该在 JMP 15.2.1 和更新版本的 Mac 或 Windows 上运行没有问题。 可以在具有 Python 支持但尚未经过测试的旧版 JMP 上运行。
您需要在与此脚本相同的目录中创建一个 defs.py 文件。 多处理包要求并行运行的代码来自可导入的代码,而不是脚本的本地代码。 所以这个脚本中的大部分工作是确保 sys.path 设置正确,以便脚本可以简单地导入 defs.py 文件。
defs.py 文件的全部内容都在文件注释中,便于粘贴到新文件中。 下面重复的内容是:
def f(x):
return x*x
该脚本接受一个数字列表 [1,2,3] 并并行调用 f(x) 来平方给出结果 [1,4,9] 的值。
pyParallel.jsl
Names Default To Here( 1 );
/* Description: Python mulltiprocessing from JMP JSL
* File: pyParallel.jsl, and defs.py
* Author: Paul Nelson
* JMP Statistical Discovery, LLC
*
* Python multiprocessing from an embedded context adds additional challenges.
* Just attempting to run Python parallel processing modules, typically launches
* multiple copies of JMP, not executing the Python code in parallel.
*
* The multiprocessing module allows setting the executable to launch, since we
* are loading Python into JMP, it's an embedded case thus sys.executable => JMP, not Python.
*
*
* Note: Functions to be run in parallel must come from importable code, not local
* code definitions.
* A multiprocessing 'Feature', not bug, See: https://bugs.python.org/issue25053
*
* Known Bugs:
* On Windows within JMP, sys.exec_prefix is empty, so path to pythonw.exe has to be set explicitly
*
* Create a defs.py file with the contents below. Place in the same directory as this JSL file.
def f(x):
return x*x
*/
// On my windows machine since I have multiple versions of Python, I have to specify the
// full path to the python39.dll, on my Mac it just works.
If( Host Is("Windows"),
Python Init(Path("C:\\Users\\panels\\AppData\\Local\\Programs\\Python\\Python39\\python39.dll")),
Python Init()
);
// send the current working directory where we loaded this script, down to Python so we can find defs.py
script_path = Get Default Directory();
Python Send(script_path);
Python Submit("\[
import sys
import os
import multiprocessing
from multiprocessing import Pool
import platform
# The loaded Python does not have the current working directory initialized, so change it
# to the directory of the script, as sent in from JSL
print('script_path: ' + script_path)
hostPlatform = platform.system()
print(hostPlatform)
try:
if 'Windows' == hostPlatform:
# strip leading / off from path if it exists
if '/' == script_path[0]:
script_path = script_path[1:]
norm_path = os.path.normpath(script_path)
print(norm_path)
os.chdir(norm_path)
else:
os.chdir(script_path)
except:
print("Unable to change current working directory", sys.exc_info())
finally:
print(os.getcwd())
# if '.' is not at the beginning of sys.path, prepend so we can look for modules within CWD
if sys.path[0] != '.':
sys.path.insert(0, '.')
print( sys.path )
# here argv is [''] empty string, doesn't seem to bother us in this use case.
#print( 'argv: ', sys.argv )
# prints the path to JMP, not Python
print('sys.executable: ' + sys.executable)
# build up path prefix to python3 exectable
if 'Darwin' == hostPlatform:
pyPath = os.path.join(sys.exec_prefix, 'bin', 'python3')
elif 'Windows' == hostPlatform:
# Bug - on windows sys.exec_prefix is empty in Python within JMP!
pyPath = None
if sys.exec_prefix:
pyPath = os.path.join(sys.exec_prefix, 'pythonw.exe')
else:
# FixMe: hardcode for now
pyPath = 'C:\\Users\\panels\\AppData\\Local\\Programs\\Python\\Python39\\pythonw.exe'
print ('Python Executable: ' + pyPath)
# Finally down to running something in parallel.
#
# Import the function code that will be executed in parallel.
# JMP is not reporting ImportError or ModuleNotFound errors, the scripting just returns -1
# on an import failure, so wrap it within our own try: except: block
try:
import defs as d
# print( d.f(2) )
# Tell multiprocessing to use the Python executable, not JMP to spawn the workers.
multiprocessing.set_executable(pyPath)
# Run the tasks in parallel - (separate Python instances)
with Pool(5) as p:
print(p.map(d.f, [1,2,3])) # Results in [1,4,9] the square of the numbers
except ModuleNotFoundError as mnf:
print(mnf)
print('Check that the moudule is located on your sys.path:')
print( sys.path )
except ImportError as error:
print(error)
except Exception as ex:
print(ex)
]\");