請注意,內容編輯器從 defs.py 的內容中刪除了一個回車符,應該顯示為:
def f(x):
return x*x
在假期休息期間,我有時間修補 Python。 我們在 Python/JSL 接口的並行處理中遇到的問題是 Python 通常會啟動 JMP 的多個副本,而不是 Python。 原因是我們在 JMP 中嵌入了 Python,因此可執行文件是 JMP 而不是 Python。 我發現 multiprocessing 包允許將可執行文件設置為在運行並行代碼時啟動。 這是拼圖中缺失的部分。
下面的腳本我已經在 Mac 和 Windows 上使用 Python 3.9 在 JMP 15.2.1 上進行了測試。在我的 Windows 機器上,我必須使用 Python Init() 的 Path() 參數。 我有多個版本的 Python,並希望確保選擇正確的版本。 在我的 Mac 上,不帶參數的 Python Init() 適用於我的配置。 我在 Windows 上遇到了一個問題,我不得不硬編碼我的 pythonw.exe 的路徑。 您將需要更改它以匹配您的 Python 位置。 搜索修復我:
除了 Python Init() 參數和 Windows 上的 pythonw.exe 路徑之外,這應該在 JMP 15.2.1 和更新版本的 Mac 或 Windows 上運行沒有問題。 可以在具有 Python 支持但尚未經過測試的舊版 JMP 上運行。
您需要在與此腳本相同的目錄中創建一個 defs.py 文件。 多處理包要求並行運行的代碼來自可導入的代碼,而不是腳本的本地代碼。 所以這個腳本中的大部分工作是確保正確設置 sys.path 以便腳本可以簡單地導入 defs.py 文件。
defs.py 文件的全部內容都在文件註釋中,便於粘貼到新文件中。 下面重複的內容是:
def f(x):
return x*x
該腳本接受一個數字列表 [1,2,3] 並並行調用 f(x) 來平方給出結果 [1,4,9] 的值。
pyParallel.jsl
Names Default To Here( 1 );
/* Description: Python mulltiprocessing from JMP JSL
* File: pyParallel.jsl, and defs.py
* Author: Paul Nelson
* JMP Statistical Discovery, LLC
*
* Python multiprocessing from an embedded context adds additional challenges.
* Just attempting to run Python parallel processing modules, typically launches
* multiple copies of JMP, not executing the Python code in parallel.
*
* The multiprocessing module allows setting the executable to launch, since we
* are loading Python into JMP, it's an embedded case thus sys.executable => JMP, not Python.
*
*
* Note: Functions to be run in parallel must come from importable code, not local
* code definitions.
* A multiprocessing 'Feature', not bug, See: https://bugs.python.org/issue25053
*
* Known Bugs:
* On Windows within JMP, sys.exec_prefix is empty, so path to pythonw.exe has to be set explicitly
*
* Create a defs.py file with the contents below. Place in the same directory as this JSL file.
def f(x):
return x*x
*/
// On my windows machine since I have multiple versions of Python, I have to specify the
// full path to the python39.dll, on my Mac it just works.
If( Host Is("Windows"),
Python Init(Path("C:\\Users\\panels\\AppData\\Local\\Programs\\Python\\Python39\\python39.dll")),
Python Init()
);
// send the current working directory where we loaded this script, down to Python so we can find defs.py
script_path = Get Default Directory();
Python Send(script_path);
Python Submit("\[
import sys
import os
import multiprocessing
from multiprocessing import Pool
import platform
# The loaded Python does not have the current working directory initialized, so change it
# to the directory of the script, as sent in from JSL
print('script_path: ' + script_path)
hostPlatform = platform.system()
print(hostPlatform)
try:
if 'Windows' == hostPlatform:
# strip leading / off from path if it exists
if '/' == script_path[0]:
script_path = script_path[1:]
norm_path = os.path.normpath(script_path)
print(norm_path)
os.chdir(norm_path)
else:
os.chdir(script_path)
except:
print("Unable to change current working directory", sys.exc_info())
finally:
print(os.getcwd())
# if '.' is not at the beginning of sys.path, prepend so we can look for modules within CWD
if sys.path[0] != '.':
sys.path.insert(0, '.')
print( sys.path )
# here argv is [''] empty string, doesn't seem to bother us in this use case.
#print( 'argv: ', sys.argv )
# prints the path to JMP, not Python
print('sys.executable: ' + sys.executable)
# build up path prefix to python3 exectable
if 'Darwin' == hostPlatform:
pyPath = os.path.join(sys.exec_prefix, 'bin', 'python3')
elif 'Windows' == hostPlatform:
# Bug - on windows sys.exec_prefix is empty in Python within JMP!
pyPath = None
if sys.exec_prefix:
pyPath = os.path.join(sys.exec_prefix, 'pythonw.exe')
else:
# FixMe: hardcode for now
pyPath = 'C:\\Users\\panels\\AppData\\Local\\Programs\\Python\\Python39\\pythonw.exe'
print ('Python Executable: ' + pyPath)
# Finally down to running something in parallel.
#
# Import the function code that will be executed in parallel.
# JMP is not reporting ImportError or ModuleNotFound errors, the scripting just returns -1
# on an import failure, so wrap it within our own try: except: block
try:
import defs as d
# print( d.f(2) )
# Tell multiprocessing to use the Python executable, not JMP to spawn the workers.
multiprocessing.set_executable(pyPath)
# Run the tasks in parallel - (separate Python instances)
with Pool(5) as p:
print(p.map(d.f, [1,2,3])) # Results in [1,4,9] the square of the numbers
except ModuleNotFoundError as mnf:
print(mnf)
print('Check that the moudule is located on your sys.path:')
print( sys.path )
except ImportError as error:
print(error)
except Exception as ex:
print(ex)
]\");