大多數網絡都使用 Restful API 將數據移入和移出服務器。 休息這是一個簡單的概念，與睡眠無關；休息方法代表性狀態轉移，但本文不是關於休息。 本文介紹了一種笨拙的機制，用於解決當您確實需要從站點檢索某些數據時缺乏 REST API 的問題。
網站可能出於各種原因不希望您這樣做：數據帶寬需要花錢，數據許可需要花錢，不看廣告也可能要花錢。 這個工具，硒，名義上用於測試網站，而不是用於在減速帶上加速。 大多數網站都有使用條款；您可以在本頁底部找到 JMP 的條款。
附上完整的 JSL 文件。 它是為 Firefox、Windows 和 JMP 網站編寫的，如 2022 年 5 月 8 日所示。 Firefox 部分可能可以更改為許多其他瀏覽器。 它也可能在 Mac 上運行，未進行測試。 JMP 網站會隨著時間的推移而發生變化，JSL 也需要調整。 這就是不使用官方 API 的缺點。
在開始之前，請下載驅動程序並安裝 selenium，如下面的評論所示。 您還需要 Firefox，或者對您喜歡的瀏覽器的驅動程序進行一些研究。
// JMP + Python + Selenium + Firefox (probably works with Chrome or Edge with a few tweaks)
// https://firefox-source-docs.mozilla.org/testing/geckodriver/index.html
// https://selenium-python.readthedocs.io/
// downloaded https://github.com/mozilla/geckodriver/releases -- geckodriver-...-win64.zip (pick latest)
// expand on desktop geckodriver.exe
// installed selenium python -m pip install selenium
// the following code is welded into the JMP.COM's HTML of 8May2022. It could change at anytime.
// the functions are sprinkled through the JSL below, near where I first needed them, reused later.
// you'll need similar functions, probably, and want to consult the Python Selenium bindings to write them.
// I only wrote enough to make this simple example work. There is no actual need to be logged in to JMP.
//
// Using try...except on snips of python code makes debugging much easier. JMP loses the exception message without it.
//
// the JSL functions are thin wrappers around the python calls to selenium which (I believe) is
// a thin wrapper to generate an API call to the gecko (etc) driver. I *think* it might be possible to
// remove the python layer and load/call the driver directly from JSL. But Selenium is documented.
Python Init(); // one-time startup...
// 0: startup
xrc = Python Execute( {}, {By_ID, By_XPATH, rc},
"\[
try:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
#
options = webdriver.FirefoxOptions()
#options.add_argument("--private") # example. you most likely don't want private, some things don't work.
#
service=Service(r'C:\Users\v1\Desktop\geckodriver.exe')
# sometimes people use "browser" rather than "driver". It will be used below.
driver = webdriver.Firefox(service=service,options=options)
# return two magic values. you may need some others, just add them in the same way...
By_ID = By.ID
By_XPATH = By.XPATH
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
If( xrc != 0 | rc != "ok", Throw( "start up Selenium failed" || Try( ": " || Char( rc ), "" ) ) );
PythonInit() 只需要執行一次；它將 JMP 連接到 Python，第一次需要幾秒鐘。 您可以再次調用，不會受到任何處罰。 PythonExecute(...) 不發送任何變量，但從它運行的代碼中返回三個變量。 加載所有內容並啟動瀏覽器需要一些時間。
You are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMPYou are looking at an empty browser controlled by JMP您正在查看由 JMP 控制的空瀏覽器
接下來打開 JMP.COM 頁面。 您可能會看到通常不會被注意到的重定向。
// 1: navigate to jmp.com
nav = Function( {url}, {rc},
Python Execute( {url}, {rc},
"\[
try:
driver.get(url)
rc = "ok"
except Exception as e:
rc = repr(e)
]\" );
return(rc);
);
rc = nav( "https://www.jmp.com/" );
if( rc != "ok", throw("nav: "||char(rc)));
nav 函數返回“ok”或錯誤消息。 JMP 網頁在瀏覽器中加載。 忽略屏幕截圖中的人。
The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.The icon means the browser is remote controlled.該圖標表示瀏覽器已被遠程控制。
編寫 JMP 站點登錄腳本。 右鍵單擊“登錄”按鈕以查找該按鈕的 HTML id 值。 記住如何做到這一點；最後我會跳過這個解釋......
F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 might bring you to the next screen, but this way the control will already be selected.F12 可能會將您帶到下一個屏幕，但這樣控件就已經被選中。
開發者控制台將打開並顯示控件的 id。 再往下，將有一些具有類但沒有 id 的控件。 Selenium 的 XPATH 可以處理它。 這個技巧類似於顯示框導航——找到一條不太脆弱且仍然足夠具體的路徑。
Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.Use the button id in the JSL that follows.使用後面 JSL 中的按鈕 ID。
ID 通常是可用的最佳選擇，因為它們在頁面上是唯一的。 WaitID 最多等待 10 秒，直到登錄按鈕出現。 可能不需要等待。 如果它已經在那裡，那麼不需要時間。
// 2: login with userid/password credentials
// wait for an id to be available
waitID = Function( {id, timeout = 10, BYformat=By_ID}, {rc},
Python Execute( {id, timeout, BYformat}, {rc},
"\[
try:
myElem = WebDriverWait(driver, timeout).until(EC.presence_of_element_located((BYformat, id)))
rc = "ok"
except TimeoutException:
rc = "timeout"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
rc = waitID( "loginButton", 5 );
If( rc != "ok", Throw( "no login button: " || char(rc) ) );
// click a button ID
clickID = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
driver.find_element(BYformat, id).click()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = clickID( "loginButton" );
if( rc != "ok", throw("login button: "||char(rc)));
涼爽的！ 屏幕上會彈出標誌。 接下來找到用戶名字段。
Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...Now get the id for the user name field by right-click...現在通過右鍵單擊獲取用戶名字段的 id...
再次等待預期的字段。 現在需要一個按鍵功能......
rc = waitID( "idp-discovery-username", 10 );
If( rc != "ok", Throw( "no username field: " || char(rc) ) );
// type a value into a field
keysToID = Function( {id, txt}, {rc},
Python Execute( {id, txt}, {rc},
"\[
try:
driver.find_element(By.ID, id).send_keys(txt)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = keysToID( "idp-discovery-username", Include( "$documents/UserID.jsl" ) ); // file contains "AliBaba@1000&OneNights.com", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID username: "||char(rc)));
Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.Fake user name for fake password.假用戶名配假密碼。
我的用戶 ID 滾出屏幕，下一個按鈕可見...找到它的名稱...
Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.Click the Next button to get the password prompt.單擊“下一步”按鈕以獲取密碼提示。
單擊它，然後等待密碼字段...
// click Next button
rc = clickID( "idp-discovery-submit" );
if( rc != "ok", throw("click submit user name: "||char(rc)));
// wait for password field
rc = waitID( "okta-signin-password", 5 );
If( rc != "ok", Throw( "no password field: " || char(rc) ) );
然後輸入密碼
// enter password
rc = keysToID( "okta-signin-password", Include( "$documents/password.jsl" ) ); // file contains "OpenSesame", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID password: "||char(rc)));
並重複該過程...找到登錄按鈕...
After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.After entering the password, click the sign in button.輸入密碼後，單擊登錄按鈕。
點擊它
// click Sign In
rc = clickID( "okta-signin-submit" );
if( rc != "ok", throw("click signin submit: "||char(rc)));
我們已登錄。
Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.Must be signed in, there is an edit profile choice.必須登錄，有一個編輯個人資料選項。
圖片中有一個搜索字段，輸入JSL，然後單擊放大鏡。 “searchField”是 id。 放大鏡可以被單擊，但 selenium 有一個提交表單機制，可以在 searchField 之外工作，searchField 是表單中的輸入字段。
The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.The search field needs a bigger window to be visible, here it is.搜索字段需要更大的窗口才能可見，就在這裡。
有一些異步 JavaScript 可以加載頁面的某些部分。 如果字段作為頁面的一部分加載，則可能不需要等待任何特定字段。
// 3: query for articles about JSL
rc = waitID( "searchField" );
If( rc != "ok", Throw( "no search field: " || char(rc) ) );
// enter a search string. "jsl" currently retruns 3 pages
rc = keysToID( "searchField", "jsl" );
if( rc != "ok", throw("keysToID search field: "||char(rc)));
submitForm = Function( {id}, {rc},
Python Execute( {id}, {rc},
"\[
try:
driver.find_element(By.ID, id).submit()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = submitForm( "searchField" ); // submit form works OK off of this input field.
if( rc != "ok", throw("submitForm searchField: "||char(rc)));
現在準備好翻閱結果。 多頁列表元素如下所示
The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.The pink outer element holds three inner parts: title, description, link.粉色外部元素包含三個內部部分：標題、描述、鏈接。
有一個跨越多個頁面的粉色圓圈數據項列表。
// 4: page through the results to capture them
getElements = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
list = driver.find_elements(BYformat,id)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
getNElements = function({},{n},
Python Execute( {}, {n},
"\[
try:
n = len(list)
except Exception as e:
print(repr(e))
n = -1
]\"
);
return(n);
);
getElementItext = function({i,id, BYformat=By_ID},{txt},
Python Execute( {i, id, BYformat}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).text
except Exception as e:
txt = "Error: getElementItext: " + repr(e)
]\"
);
return(txt);
);
getElementIattribute = function({i,id, BYformat=By_ID, attr},{txt},
Python Execute( {i, id, BYformat, attr}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).get_attribute(attr)
except Exception as e:
txt = "Error: getElementIattribute: " + repr(e)
]\"
);
return(txt);
);
上圖：下面循環中使用的一些函數。 頁面底部有按鈕可以進入下一頁；他們運行一些 JavaScript 來銷毀並重新創建項目列表。 再次調用這些函數以重新捕獲新列表。 對於本示例來說，JSL 和 Python 已經足夠了。 如果有多個列表需要同時跟踪，它們就會崩潰——參見 Python列表多變的。 我很確定有必要在每個下一頁之後等待數據加載......
dt = New Table( "articles",
New Column( "link", character,
Set Property("Event Handler",
Event Handler(
Click(JSL Quote(Function( {thisTable, thisColumn, iRow}, Web( Char( thisTable:thisColumn[ iRow ] ) ); );)),
Tip(JSL Quote(Function( {thisTable, thisColumn, iRow}, "Open " || Char( thisTable:thisColumn[ iRow ] ) || " in your browser."; );)),
Color(JSL Quote(Function( {thisTable, thisColumn, iRow}, RGBColor("link"); );))
)
)
),
New Column( "title", character ),
New Column( "description", character )
);
// this while loop will grab screens of answers and break() when the NextScreen button goes dim
while(1,
// not sure, yet, what to wait for. maybe the card(s)...
// this html is updated in the background by a json ajax mechanism that
// is hard to see.
rc = waitID("//div[@id='searchresults']//div[@class='result-card']",10,By_XPATH);
If( rc != "ok", Throw( "no search results" ) );
// get the current set
rc = getElements("//div[@id='searchresults']//div[@class='result-card']",By_XPATH);
if( rc != "ok", throw("getElements: "||char(rc)));
n = getNElements();
for(i=0,i<N></N>
此時瀏覽器已打開，並且該表顯示在屏幕上。
Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.Today there were 57 entries spanning three pages.今天，三頁紙上有 57 個條目。
是時候關閉瀏覽器了。
// 5: quit
Python Submit(
"\[
driver.quit() # close the browser
]\" );
Python Term();
最後有一個 XPATH
rc = clickID("//ul[@id='pager']//a[@class='pager-next']",By_XPATH);
這意味著
// - somewhere below the root of the document find a
ul - a tag (some sort of HTML list)
[@id='pager'] - the list has this id
// - more nested tags, followed by...
a = a tag (link)
[@class='pager-next']
可能不需要那麼複雜。 這是一個使用唯一 id 來查找如果僅考慮類則可能不唯一的項目的示例。
