许多 Web 使用 RESTful API 将数据移入和移出服务器。 休息是一个与睡眠无关的简单概念；休息方法表征状态转移, 但这篇文章不是关于休息. 这篇文章是关于一种笨拙的机制，用于在您确实需要从站点检索一些数据时解决缺少 rest API 的问题。
网站可能出于各种原因不希望您这样做：数据带宽要花钱，数据许可要花钱，不看广告也可能要花钱。 这个工具，硒，名义上是用于测试网站，而不是用于超速通过减速带。 大多数网站都有使用条款；您可以在本页底部找到 JMP 的条款。
附上完整的 JSL 文件。 它是为 Firefox、Windows 和 JMP 网站编写的，它在 2022 年 5 月 8 日的样子。 Firefox 部分可能会更改为许多其他浏览器。 它也可能在 Mac 上工作，没有进行测试。 JMP 网站会随着时间的推移而改变，JSL 也需要调整。 这是不使用官方 API 的缺点。
在开始之前，下载驱动程序并安装 selenium，如下面的评论所示。 您还需要 Firefox，或者研究一下您喜欢的浏览器的驱动程序。
// JMP + Python + Selenium + Firefox (probably works with Chrome or Edge with a few tweaks)
// https://firefox-source-docs.mozilla.org/testing/geckodriver/index.html
// https://selenium-python.readthedocs.io/
// downloaded https://github.com/mozilla/geckodriver/releases -- geckodriver-...-win64.zip (pick latest)
// expand on desktop geckodriver.exe
// installed selenium python -m pip install selenium
// the following code is welded into the JMP.COM's HTML of 8May2022. It could change at anytime.
// the functions are sprinkled through the JSL below, near where I first needed them, reused later.
// you'll need similar functions, probably, and want to consult the Python Selenium bindings to write them.
// I only wrote enough to make this simple example work. There is no actual need to be logged in to JMP.
//
// Using try...except on snips of python code makes debugging much easier. JMP loses the exception message without it.
//
// the JSL functions are thin wrappers around the python calls to selenium which (I believe) is
// a thin wrapper to generate an API call to the gecko (etc) driver. I *think* it might be possible to
// remove the python layer and load/call the driver directly from JSL. But Selenium is documented.
Python Init(); // one-time startup...
// 0: startup
xrc = Python Execute( {}, {By_ID, By_XPATH, rc},
"\[
try:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
#
options = webdriver.FirefoxOptions()
#options.add_argument("--private") # example. you most likely don't want private, some things don't work.
#
service=Service(r'C:\Users\v1\Desktop\geckodriver.exe')
# sometimes people use "browser" rather than "driver". It will be used below.
driver = webdriver.Firefox(service=service,options=options)
# return two magic values. you may need some others, just add them in the same way...
By_ID = By.ID
By_XPATH = By.XPATH
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
If( xrc != 0 | rc != "ok", Throw( "start up Selenium failed" || Try( ": " || Char( rc ), "" ) ) );
PythonInit() 只需执行一次；它将 JMP 连接到 Python，第一次需要几秒钟。 您可以再次调用它而不会受到任何惩罚。 PythonExecute(...) 不发送任何变量，但从它运行的代码中获取三个变量。 加载所有内容并启动浏览器需要一些时间。
You are looking at an empty browser controlled by JMP您正在查看由 JMP 控制的空浏览器
接下来打开 JMP.COM 页面。 您可能会看到通常不会引起注意的重定向。
// 1: navigate to jmp.com
nav = Function( {url}, {rc},
Python Execute( {url}, {rc},
"\[
try:
driver.get(url)
rc = "ok"
except Exception as e:
rc = repr(e)
]\" );
return(rc);
);
rc = nav( "https://www.jmp.com/" );
if( rc != "ok", throw("nav: "||char(rc)));
nav 函数返回“ok”或错误消息。 JMP 网页加载到浏览器中。 忽略屏幕截图中的人。
The icon means the browser is remote controlled.该图标表示浏览器处于远程控制状态。
编写 JMP 站点的登录脚本。 右键单击“登录”按钮以找出该按钮的 HTML id 值。 记住如何做到这一点；最后我会跳过这个解释......
F12 might bring you to the next screen, but this way the control will already be selected.F12 可能会将您带到下一个屏幕，但这样控件就已经被选中了。
开发人员控制台打开，并显示控件的 ID。 再往下，将有具有类但没有 ID 的控件。 Selenium 的 XPATH 可以处理它。 这个技巧类似于显示框导航——找到一条不太脆弱但仍然足够具体的路径。
Use the button id in the JSL that follows.在随后的 JSL 中使用按钮 ID。
如果可用，ID 通常是最佳选择，因为它们在页面上是唯一的。 WaitID 最多等待 10 秒，以便登录按钮出现。 可能不需要等待。 如果它已经存在，它不需要时间。
// 2: login with userid/password credentials
// wait for an id to be available
waitID = Function( {id, timeout = 10, BYformat=By_ID}, {rc},
Python Execute( {id, timeout, BYformat}, {rc},
"\[
try:
myElem = WebDriverWait(driver, timeout).until(EC.presence_of_element_located((BYformat, id)))
rc = "ok"
except TimeoutException:
rc = "timeout"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
rc = waitID( "loginButton", 5 );
If( rc != "ok", Throw( "no login button: " || char(rc) ) );
// click a button ID
clickID = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
driver.find_element(BYformat, id).click()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = clickID( "loginButton" );
if( rc != "ok", throw("login button: "||char(rc)));
凉爽的！ 屏幕上的标志弹出。 接下来找到用户名字段。
Now get the id for the user name field by right-click...现在通过右键单击获取用户名字段的 id...
再次等待预期的字段。 现在需要一个击键功能...
rc = waitID( "idp-discovery-username", 10 );
If( rc != "ok", Throw( "no username field: " || char(rc) ) );
// type a value into a field
keysToID = Function( {id, txt}, {rc},
Python Execute( {id, txt}, {rc},
"\[
try:
driver.find_element(By.ID, id).send_keys(txt)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = keysToID( "idp-discovery-username", Include( "$documents/UserID.jsl" ) ); // file contains "AliBaba@1000&OneNights.com", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID username: "||char(rc)));
Fake user name for fake password.假密码的假用户名。
我的用户标识滚出屏幕，下一个按钮可见……找到它的名字……
Click the Next button to get the password prompt.单击“下一步”按钮以获取密码提示。
单击它，然后等待密码字段...
// click Next button
rc = clickID( "idp-discovery-submit" );
if( rc != "ok", throw("click submit user name: "||char(rc)));
// wait for password field
rc = waitID( "okta-signin-password", 5 );
If( rc != "ok", Throw( "no password field: " || char(rc) ) );
然后输入密码
// enter password
rc = keysToID( "okta-signin-password", Include( "$documents/password.jsl" ) ); // file contains "OpenSesame", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID password: "||char(rc)));
并重复该过程...找到登录按钮...
After entering the password, click the sign in button.输入密码后，点击登录按钮。
点击它
// click Sign In
rc = clickID( "okta-signin-submit" );
if( rc != "ok", throw("click signin submit: "||char(rc)));
我们已登录。
Must be signed in, there is an edit profile choice.必须登录，有一个编辑配置文件选项。
图片中有一个搜索字段，输入 JSL 并单击放大镜。 “searchField”是 id。 可以单击放大镜，但 selenium 有一个提交表单机制，可以在 searchField 之外工作，这是表单中的输入字段。
The search field needs a bigger window to be visible, here it is.搜索字段需要一个更大的窗口才能看到，就在这里。
有一些异步 JavaScript 加载页面的某些部分。 如果字段作为页面的一部分加载，则可能不需要等待任何特定字段。
// 3: query for articles about JSL
rc = waitID( "searchField" );
If( rc != "ok", Throw( "no search field: " || char(rc) ) );
// enter a search string. "jsl" currently retruns 3 pages
rc = keysToID( "searchField", "jsl" );
if( rc != "ok", throw("keysToID search field: "||char(rc)));
submitForm = Function( {id}, {rc},
Python Execute( {id}, {rc},
"\[
try:
driver.find_element(By.ID, id).submit()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = submitForm( "searchField" ); // submit form works OK off of this input field.
if( rc != "ok", throw("submitForm searchField: "||char(rc)));
现在准备翻阅结果。 多页列表元素如下所示
The pink outer element holds three inner parts: title, description, link.粉红色的外部元素包含三个内部部分：标题、描述、链接。
有一个跨越多个页面的粉红色圆圈数据项列表。
// 4: page through the results to capture them
getElements = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
list = driver.find_elements(BYformat,id)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
getNElements = function({},{n},
Python Execute( {}, {n},
"\[
try:
n = len(list)
except Exception as e:
print(repr(e))
n = -1
]\"
);
return(n);
);
getElementItext = function({i,id, BYformat=By_ID},{txt},
Python Execute( {i, id, BYformat}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).text
except Exception as e:
txt = "Error: getElementItext: " + repr(e)
]\"
);
return(txt);
);
getElementIattribute = function({i,id, BYformat=By_ID, attr},{txt},
Python Execute( {i, id, BYformat, attr}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).get_attribute(attr)
except Exception as e:
txt = "Error: getElementIattribute: " + repr(e)
]\"
);
return(txt);
);
上图：在下面的循环中使用的一些函数。 页面底部有进入下一页的按钮；他们运行一些 JavaScript 来破坏和重新创建项目列表。 再次调用这些函数以重新捕获新列表。 JSL 和 Python 足以胜任此示例。 如果要同时跟踪多个列表，它们将崩溃——参见 Python列表多变的。 我很确定有必要在每个下一页之后等待数据加载......
dt = New Table( "articles",
New Column( "link", character,
Set Property("Event Handler",
Event Handler(
Click(JSL Quote(Function( {thisTable, thisColumn, iRow}, Web( Char( thisTable:thisColumn[ iRow ] ) ); );)),
Tip(JSL Quote(Function( {thisTable, thisColumn, iRow}, "Open " || Char( thisTable:thisColumn[ iRow ] ) || " in your browser."; );)),
Color(JSL Quote(Function( {thisTable, thisColumn, iRow}, RGBColor("link"); );))
)
)
),
New Column( "title", character ),
New Column( "description", character )
);
// this while loop will grab screens of answers and break() when the NextScreen button goes dim
while(1,
// not sure, yet, what to wait for. maybe the card(s)...
// this html is updated in the background by a json ajax mechanism that
// is hard to see.
rc = waitID("//div[@id='searchresults']//div[@class='result-card']",10,By_XPATH);
If( rc != "ok", Throw( "no search results" ) );
// get the current set
rc = getElements("//div[@id='searchresults']//div[@class='result-card']",By_XPATH);
if( rc != "ok", throw("getElements: "||char(rc)));
n = getNElements();
for(i=0,i<N></N>
此时浏览器已打开，并且该表显示在屏幕上。
Today there were 57 entries spanning three pages.今天有 57 个条目，跨越三页。
是时候关闭浏览器了。
// 5: quit
Python Submit(
"\[
driver.quit() # close the browser
]\" );
Python Term();
最后有一个 XPATH
rc = clickID("//ul[@id='pager']//a[@class='pager-next']",By_XPATH);
那意味着
// - somewhere below the root of the document find a
ul - a <ul> tag (some sort of HTML list)
[@id='pager'] - the list has this id
// - more nested tags, followed by...
a = a <a> tag (link)
[@class='pager-next']
它可能不需要那么复杂。 这是一个使用唯一 id 查找项目的示例，如果仅考虑类，该项目可能不是唯一的。
