大多数网络都使用 Restful API 将数据移入和移出服务器。 休息这是一个简单的概念，与睡眠无关；休息方法代表性状态转移，但本文不是关于休息。 本文介绍了一种笨拙的机制，用于解决当您确实需要从站点检索某些数据时缺乏 REST API 的问题。
网站可能出于各种原因不希望您这样做：数据带宽需要花钱，数据许可需要花钱，不看广告也可能要花钱。 这个工具，硒，名义上用于测试网站，而不是用于在减速带上加速。 大多数网站都有使用条款；您可以在本页底部找到 JMP 的条款。
附上完整的 JSL 文件。 它是为 Firefox、Windows 和 JMP 网站编写的，如 2022 年 5 月 8 日所示。 Firefox 部分可能可以更改为许多其他浏览器。 它也可能在 Mac 上运行，未进行测试。 JMP 网站会随着时间的推移而发生变化，JSL 也需要调整。 这就是不使用官方 API 的缺点。
在开始之前，请下载驱动程序并安装 selenium，如下面的评论所示。 您还需要 Firefox，或者对您喜欢的浏览器的驱动程序进行一些研究。
// JMP + Python + Selenium + Firefox (probably works with Chrome or Edge with a few tweaks)
// https://firefox-source-docs.mozilla.org/testing/geckodriver/index.html
// https://selenium-python.readthedocs.io/
// downloaded https://github.com/mozilla/geckodriver/releases -- geckodriver-...-win64.zip (pick latest)
// expand on desktop geckodriver.exe
// installed selenium python -m pip install selenium
// the following code is welded into the JMP.COM's HTML of 8May2022. It could change at anytime.
// the functions are sprinkled through the JSL below, near where I first needed them, reused later.
// you'll need similar functions, probably, and want to consult the Python Selenium bindings to write them.
// I only wrote enough to make this simple example work. There is no actual need to be logged in to JMP.
//
// Using try...except on snips of python code makes debugging much easier. JMP loses the exception message without it.
//
// the JSL functions are thin wrappers around the python calls to selenium which (I believe) is
// a thin wrapper to generate an API call to the gecko (etc) driver. I *think* it might be possible to
// remove the python layer and load/call the driver directly from JSL. But Selenium is documented.
Python Init(); // one-time startup...
// 0: startup
xrc = Python Execute( {}, {By_ID, By_XPATH, rc},
"\[
try:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
#
options = webdriver.FirefoxOptions()
#options.add_argument("--private") # example. you most likely don't want private, some things don't work.
#
service=Service(r'C:\Users\v1\Desktop\geckodriver.exe')
# sometimes people use "browser" rather than "driver". It will be used below.
driver = webdriver.Firefox(service=service,options=options)
# return two magic values. you may need some others, just add them in the same way...
By_ID = By.ID
By_XPATH = By.XPATH
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
If( xrc != 0 | rc != "ok", Throw( "start up Selenium failed" || Try( ": " || Char( rc ), "" ) ) );
PythonInit() 只需要执行一次；它将 JMP 连接到 Python，第一次需要几秒钟。 您可以再次调用，不会受到任何处罚。 PythonExecute(...) 不发送任何变量，但从它运行的代码中返回三个变量。 加载所有内容并启动浏览器需要一些时间。
You are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMP您正在查看由 JMP 控制的空浏览器
接下来打开 JMP.COM 页面。 您可能会看到通常不会被注意到的重定向。
// 1: navigate to jmp.com
nav = Function( {url}, {rc},
Python Execute( {url}, {rc},
"\[
try:
driver.get(url)
rc = "ok"
except Exception as e:
rc = repr(e)
]\" );
return(rc);
);
rc = nav( "https://www.jmp.com/" );
if( rc != "ok", throw("nav: "||char(rc)));
nav 函数返回“ok”或错误消息。 JMP 网页在浏览器中加载。 忽略屏幕截图中的人。
The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.该图标表示浏览器已被远程控制。
编写 JMP 站点登录脚本。 右键单击“登录”按钮以查找该按钮的 HTML id 值。 记住如何做到这一点；最后我会跳过这个解释......
F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 可能会将您带到下一个屏幕，但这样控件就已经被选中。
开发者控制台将打开并显示控件的 id。 再往下，将有一些具有类但没有 id 的控件。 Selenium 的 XPATH 可以处理它。 这个技巧类似于显示框导航——找到一条不太脆弱且仍然足够具体的路径。
Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.使用后面 JSL 中的按钮 ID。
ID 通常是可用的最佳选择，因为它们在页面上是唯一的。 WaitID 最多等待 10 秒，直到登录按钮出现。 可能不需要等待。 如果它已经在那里，那么不需要时间。
// 2: login with userid/password credentials
// wait for an id to be available
waitID = Function( {id, timeout = 10, BYformat=By_ID}, {rc},
Python Execute( {id, timeout, BYformat}, {rc},
"\[
try:
myElem = WebDriverWait(driver, timeout).until(EC.presence_of_element_located((BYformat, id)))
rc = "ok"
except TimeoutException:
rc = "timeout"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
rc = waitID( "loginButton", 5 );
If( rc != "ok", Throw( "no login button: " || char(rc) ) );
// click a button ID
clickID = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
driver.find_element(BYformat, id).click()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = clickID( "loginButton" );
if( rc != "ok", throw("login button: "||char(rc)));
凉爽的！ 屏幕上会弹出标志。 接下来找到用户名字段。
Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...现在通过右键单击获取用户名字段的 id...
再次等待预期的字段。 现在需要一个按键功能......
rc = waitID( "idp-discovery-username", 10 );
If( rc != "ok", Throw( "no username field: " || char(rc) ) );
// type a value into a field
keysToID = Function( {id, txt}, {rc},
Python Execute( {id, txt}, {rc},
"\[
try:
driver.find_element(By.ID, id).send_keys(txt)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = keysToID( "idp-discovery-username", Include( "$documents/UserID.jsl" ) ); // file contains "AliBaba@1000&OneNights.com", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID username: "||char(rc)));
Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.假用户名配假密码。
我的用户 ID 滚出屏幕，下一个按钮可见...找到它的名称...
Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.单击“下一步”按钮以获取密码提示。
单击它，然后等待密码字段...
// click Next button
rc = clickID( "idp-discovery-submit" );
if( rc != "ok", throw("click submit user name: "||char(rc)));
// wait for password field
rc = waitID( "okta-signin-password", 5 );
If( rc != "ok", Throw( "no password field: " || char(rc) ) );
然后输入密码
// enter password
rc = keysToID( "okta-signin-password", Include( "$documents/password.jsl" ) ); // file contains "OpenSesame", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID password: "||char(rc)));
并重复该过程...找到登录按钮...
After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.输入密码后，单击登录按钮。
点击它
// click Sign In
rc = clickID( "okta-signin-submit" );
if( rc != "ok", throw("click signin submit: "||char(rc)));
我们已登录。
Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.必须登录，有一个编辑个人资料选项。
图片中有一个搜索字段，输入JSL，然后单击放大镜。 “searchField”是 id。 放大镜可以被单击，但 selenium 有一个提交表单机制，可以在 searchField 之外工作，searchField 是表单中的输入字段。
The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.搜索字段需要更大的窗口才能可见，就在这里。
有一些异步 JavaScript 可以加载页面的某些部分。 如果字段作为页面的一部分加载，则可能不需要等待任何特定字段。
// 3: query for articles about JSL
rc = waitID( "searchField" );
If( rc != "ok", Throw( "no search field: " || char(rc) ) );
// enter a search string. "jsl" currently retruns 3 pages
rc = keysToID( "searchField", "jsl" );
if( rc != "ok", throw("keysToID search field: "||char(rc)));
submitForm = Function( {id}, {rc},
Python Execute( {id}, {rc},
"\[
try:
driver.find_element(By.ID, id).submit()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = submitForm( "searchField" ); // submit form works OK off of this input field.
if( rc != "ok", throw("submitForm searchField: "||char(rc)));
现在准备好翻阅结果。 多页列表元素如下所示
The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.粉色外部元素包含三个内部部分：标题、描述、链接。
有一个跨越多个页面的粉色圆圈数据项列表。
// 4: page through the results to capture them
getElements = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
list = driver.find_elements(BYformat,id)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
getNElements = function({},{n},
Python Execute( {}, {n},
"\[
try:
n = len(list)
except Exception as e:
print(repr(e))
n = -1
]\"
);
return(n);
);
getElementItext = function({i,id, BYformat=By_ID},{txt},
Python Execute( {i, id, BYformat}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).text
except Exception as e:
txt = "Error: getElementItext: " + repr(e)
]\"
);
return(txt);
);
getElementIattribute = function({i,id, BYformat=By_ID, attr},{txt},
Python Execute( {i, id, BYformat, attr}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).get_attribute(attr)
except Exception as e:
txt = "Error: getElementIattribute: " + repr(e)
]\"
);
return(txt);
);
上图：下面循环中使用的一些函数。 页面底部有按钮可以进入下一页；他们运行一些 JavaScript 来销毁并重新创建项目列表。 再次调用这些函数以重新捕获新列表。 对于本示例来说，JSL 和 Python 已经足够了。 如果有多个列表需要同时跟踪，它们就会崩溃——参见 Python列表多变的。 我很确定有必要在每个下一页之后等待数据加载......
dt = New Table( "articles",
New Column( "link", character,
Set Property("Event Handler",
Event Handler(
Click(JSL Quote(Function( {thisTable, thisColumn, iRow}, Web( Char( thisTable:thisColumn[ iRow ] ) ); );)),
Tip(JSL Quote(Function( {thisTable, thisColumn, iRow}, "Open " || Char( thisTable:thisColumn[ iRow ] ) || " in your browser."; );)),
Color(JSL Quote(Function( {thisTable, thisColumn, iRow}, RGBColor("link"); );))
)
)
),
New Column( "title", character ),
New Column( "description", character )
);
// this while loop will grab screens of answers and break() when the NextScreen button goes dim
while(1,
// not sure, yet, what to wait for. maybe the card(s)...
// this html is updated in the background by a json ajax mechanism that
// is hard to see.
rc = waitID("//div[@id='searchresults']//div[@class='result-card']",10,By_XPATH);
If( rc != "ok", Throw( "no search results" ) );
// get the current set
rc = getElements("//div[@id='searchresults']//div[@class='result-card']",By_XPATH);
if( rc != "ok", throw("getElements: "||char(rc)));
n = getNElements();
for(i=0,i<N></N>
此时浏览器已打开，并且该表显示在屏幕上。
Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.今天，三页纸上有 57 个条目。
是时候关闭浏览器了。
// 5: quit
Python Submit(
"\[
driver.quit() # close the browser
]\" );
Python Term();
最后有一个 XPATH
rc = clickID("//ul[@id='pager']//a[@class='pager-next']",By_XPATH);
这意味着
// - somewhere below the root of the document find a
ul - a tag (some sort of HTML list)
[@id='pager'] - the list has this id
// - more nested tags, followed by...
a = a tag (link)
[@class='pager-next']
可能不需要那么复杂。 这是一个使用唯一 id 来查找如果仅考虑类则可能不唯一的项目的示例。
