許多 Web 使用 RESTful API 將數據移入和移出服務器。 休息是一個與睡眠無關的簡單概念；休息方法表徵狀態轉移, 但這篇文章不是關於休息. 這篇文章是關於一種笨拙的機制，用於在您確實需要從站點檢索一些數據時解決缺少 rest API 的問題。
網站可能出於各種原因不希望您這樣做：數據帶寬要花錢，數據許可要花錢，不看廣告也可能要花錢。 這個工具，硒, 名義上是用於測試網站，而不是用於超速通過減速帶。 大多數網站都有使用條款；您可以在本頁底部找到 JMP 的條款。
附上完整的 JSL 文件。 它是為 Firefox、Windows 和 JMP 網站編寫的，正如它在 2022 年 5 月 8 日的樣子。 Firefox 部分可能會更改為許多其他瀏覽器。 它也可能在 Mac 上工作，沒有進行測試。 JMP 網站會隨著時間的推移而改變，JSL 也需要調整。 這是不使用官方 API 的缺點。
在開始之前，下載驅動程序並安裝 selenium，如下面的評論所示。 您還需要 Firefox，或者研究一下您喜歡的瀏覽器的驅動程序。
// JMP + Python + Selenium + Firefox (probably works with Chrome or Edge with a few tweaks)
// https://firefox-source-docs.mozilla.org/testing/geckodriver/index.html
// https://selenium-python.readthedocs.io/
// downloaded https://github.com/mozilla/geckodriver/releases -- geckodriver-...-win64.zip (pick latest)
// expand on desktop geckodriver.exe
// installed selenium python -m pip install selenium
// the following code is welded into the JMP.COM's HTML of 8May2022. It could change at anytime.
// the functions are sprinkled through the JSL below, near where I first needed them, reused later.
// you'll need similar functions, probably, and want to consult the Python Selenium bindings to write them.
// I only wrote enough to make this simple example work. There is no actual need to be logged in to JMP.
//
// Using try...except on snips of python code makes debugging much easier. JMP loses the exception message without it.
//
// the JSL functions are thin wrappers around the python calls to selenium which (I believe) is
// a thin wrapper to generate an API call to the gecko (etc) driver. I *think* it might be possible to
// remove the python layer and load/call the driver directly from JSL. But Selenium is documented.
Python Init(); // one-time startup...
// 0: startup
xrc = Python Execute( {}, {By_ID, By_XPATH, rc},
"\[
try:
from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
#
options = webdriver.FirefoxOptions()
#options.add_argument("--private") # example. you most likely don't want private, some things don't work.
#
service=Service(r'C:\Users\v1\Desktop\geckodriver.exe')
# sometimes people use "browser" rather than "driver". It will be used below.
driver = webdriver.Firefox(service=service,options=options)
# return two magic values. you may need some others, just add them in the same way...
By_ID = By.ID
By_XPATH = By.XPATH
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
If( xrc != 0 | rc != "ok", Throw( "start up Selenium failed" || Try( ": " || Char( rc ), "" ) ) );
PythonInit() 只需執行一次；它將 JMP 連接到 Python，第一次需要幾秒鐘。 您可以再次調用它而不會受到任何懲罰。 PythonExecute(...) 不發送任何變量，但從它運行的代碼中獲取三個變量。 加載所有內容並啟動瀏覽器需要一些時間。
You are looking at an empty browser controlled by JMP您正在查看由 JMP 控制的空瀏覽器
接下來打開 JMP.COM 頁面。 您可能會看到通常不會引起注意的重定向。
// 1: navigate to jmp.com
nav = Function( {url}, {rc},
Python Execute( {url}, {rc},
"\[
try:
driver.get(url)
rc = "ok"
except Exception as e:
rc = repr(e)
]\" );
return(rc);
);
rc = nav( "https://www.jmp.com/" );
if( rc != "ok", throw("nav: "||char(rc)));
nav 函數返回“ok”或錯誤消息。 JMP 網頁加載到瀏覽器中。 忽略屏幕截圖中的人。
The icon means the browser is remote controlled.該圖標表示瀏覽器處於遠程控制狀態。
編寫 JMP 站點的登錄腳本。 右鍵單擊“登錄”按鈕以找出該按鈕的 HTML id 值。 記住如何做到這一點；最後我會跳過這個解釋......
F12 might bring you to the next screen, but this way the control will already be selected.F12 可能會將您帶到下一個屏幕，但這樣控件就已經被選中了。
開發人員控制台打開，並顯示控件的 ID。 再往下，將有具有類但沒有 ID 的控件。 Selenium 的 XPATH 可以處理它。 這個技巧類似於顯示框導航——找到一條不太脆弱但仍然足夠具體的路徑。
Use the button id in the JSL that follows.在隨後的 JSL 中使用按鈕 ID。
如果可用，ID 通常是最佳選擇，因為它們在頁面上是唯一的。 WaitID 最多等待 10 秒，以便登錄按鈕出現。 可能不需要等待。 如果它已經存在，它不需要時間。
// 2: login with userid/password credentials
// wait for an id to be available
waitID = Function( {id, timeout = 10, BYformat=By_ID}, {rc},
Python Execute( {id, timeout, BYformat}, {rc},
"\[
try:
myElem = WebDriverWait(driver, timeout).until(EC.presence_of_element_located((BYformat, id)))
rc = "ok"
except TimeoutException:
rc = "timeout"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
rc = waitID( "loginButton", 5 );
If( rc != "ok", Throw( "no login button: " || char(rc) ) );
// click a button ID
clickID = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
driver.find_element(BYformat, id).click()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = clickID( "loginButton" );
if( rc != "ok", throw("login button: "||char(rc)));
涼爽的！ 屏幕上的標誌彈出。 接下來找到用戶名字段。
Now get the id for the user name field by right-click...現在通過右鍵單擊獲取用戶名字段的 id...
再次等待預期的字段。 現在需要一個擊鍵功能...
rc = waitID( "idp-discovery-username", 10 );
If( rc != "ok", Throw( "no username field: " || char(rc) ) );
// type a value into a field
keysToID = Function( {id, txt}, {rc},
Python Execute( {id, txt}, {rc},
"\[
try:
driver.find_element(By.ID, id).send_keys(txt)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = keysToID( "idp-discovery-username", Include( "$documents/UserID.jsl" ) ); // file contains "AliBaba@1000&OneNights.com", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID username: "||char(rc)));
Fake user name for fake password.假密碼的假用戶名。
我的用戶標識滾出屏幕，下一個按鈕可見……找到它的名字……
Click the Next button to get the password prompt.單擊“下一步”按鈕以獲取密碼提示。
單擊它，然後等待密碼字段...
// click Next button
rc = clickID( "idp-discovery-submit" );
if( rc != "ok", throw("click submit user name: "||char(rc)));
// wait for password field
rc = waitID( "okta-signin-password", 5 );
If( rc != "ok", Throw( "no password field: " || char(rc) ) );
然後輸入密碼
// enter password
rc = keysToID( "okta-signin-password", Include( "$documents/password.jsl" ) ); // file contains "OpenSesame", in quotation marks, possibly encrypted
if( rc != "ok", throw("keysToID password: "||char(rc)));
並重複該過程...找到登錄按鈕...
After entering the password, click the sign in button.輸入密碼後，點擊登錄按鈕。
點擊它
// click Sign In
rc = clickID( "okta-signin-submit" );
if( rc != "ok", throw("click signin submit: "||char(rc)));
我們已登錄。
Must be signed in, there is an edit profile choice.必須登錄，有一個編輯配置文件選項。
圖片中有一個搜索字段，輸入 JSL 並單擊放大鏡。 “searchField”是 id。 可以單擊放大鏡，但 selenium 有一個提交表單機制，可以在 searchField 之外工作，這是表單中的輸入字段。
The search field needs a bigger window to be visible, here it is.搜索字段需要一個更大的窗口才能看到，就在這裡。
有一些異步 JavaScript 加載頁面的某些部分。 如果字段作為頁面的一部分加載，則可能不需要等待任何特定字段。
// 3: query for articles about JSL
rc = waitID( "searchField" );
If( rc != "ok", Throw( "no search field: " || char(rc) ) );
// enter a search string. "jsl" currently retruns 3 pages
rc = keysToID( "searchField", "jsl" );
if( rc != "ok", throw("keysToID search field: "||char(rc)));
submitForm = Function( {id}, {rc},
Python Execute( {id}, {rc},
"\[
try:
driver.find_element(By.ID, id).submit()
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
return(rc);
);
rc = submitForm( "searchField" ); // submit form works OK off of this input field.
if( rc != "ok", throw("submitForm searchField: "||char(rc)));
現在準備翻閱結果。 多頁列表元素如下所示
The pink outer element holds three inner parts: title, description, link.粉紅色的外部元素包含三個內部部分：標題、描述、鏈接。
有一個跨越多個頁面的粉紅色圓圈數據項列表。
// 4: page through the results to capture them
getElements = Function( {id, BYformat=By_ID}, {rc},
Python Execute( {id, BYformat}, {rc},
"\[
try:
list = driver.find_elements(BYformat,id)
rc = "ok"
except Exception as e:
rc = repr(e)
]\"
);
Return( rc );
);
getNElements = function({},{n},
Python Execute( {}, {n},
"\[
try:
n = len(list)
except Exception as e:
print(repr(e))
n = -1
]\"
);
return(n);
);
getElementItext = function({i,id, BYformat=By_ID},{txt},
Python Execute( {i, id, BYformat}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).text
except Exception as e:
txt = "Error: getElementItext: " + repr(e)
]\"
);
return(txt);
);
getElementIattribute = function({i,id, BYformat=By_ID, attr},{txt},
Python Execute( {i, id, BYformat, attr}, {txt},
"\[
try:
txt = list[int(i)].find_element(BYformat, id).get_attribute(attr)
except Exception as e:
txt = "Error: getElementIattribute: " + repr(e)
]\"
);
return(txt);
);
上圖：在下面的循環中使用的一些函數。 頁面底部有進入下一頁的按鈕；他們運行一些 JavaScript 來破壞和重新創建項目列表。 再次調用這些函數以重新捕獲新列表。 JSL 和 Python 足以勝任此示例。 如果要同時跟踪多個列表，它們將崩潰——參見 Python列表多變的。 我很確定有必要在每個下一頁之後等待數據加載......
dt = New Table( "articles",
New Column( "link", character,
Set Property("Event Handler",
Event Handler(
Click(JSL Quote(Function( {thisTable, thisColumn, iRow}, Web( Char( thisTable:thisColumn[ iRow ] ) ); );)),
Tip(JSL Quote(Function( {thisTable, thisColumn, iRow}, "Open " || Char( thisTable:thisColumn[ iRow ] ) || " in your browser."; );)),
Color(JSL Quote(Function( {thisTable, thisColumn, iRow}, RGBColor("link"); );))
)
)
),
New Column( "title", character ),
New Column( "description", character )
);
// this while loop will grab screens of answers and break() when the NextScreen button goes dim
while(1,
// not sure, yet, what to wait for. maybe the card(s)...
// this html is updated in the background by a json ajax mechanism that
// is hard to see.
rc = waitID("//div[@id='searchresults']//div[@class='result-card']",10,By_XPATH);
If( rc != "ok", Throw( "no search results" ) );
// get the current set
rc = getElements("//div[@id='searchresults']//div[@class='result-card']",By_XPATH);
if( rc != "ok", throw("getElements: "||char(rc)));
n = getNElements();
for(i=0,i<N></N>
此時瀏覽器已打開，並且該表顯示在屏幕上。
Today there were 57 entries spanning three pages.今天有 57 個條目，跨越三頁。
是時候關閉瀏覽器了。
// 5: quit
Python Submit(
"\[
driver.quit() # close the browser
]\" );
Python Term();
最後有一個 XPATH
rc = clickID("//ul[@id='pager']//a[@class='pager-next']",By_XPATH);
那意味著
// - somewhere below the root of the document find a
ul - a <ul> tag (some sort of HTML list)
[@id='pager'] - the list has this id
// - more nested tags, followed by...
a = a <a> tag (link)
[@class='pager-next']
它可能不需要那麼複雜。 這是一個使用唯一 id 查找項目的示例，如果僅考慮類，該項目可能不是唯一的。
