622199_zh-TW

liliconcom

一週中最不受歡迎的一天

星期六並不是一週中最不受歡迎的一天。

 

大約一年前，我開始從 Twitter 收集趨勢詞資料。 除了製作藝術影片之外，沒有任何真正的目標。 除了幾次斷電之外，數據相當完整，每五分鐘採樣十個地方。

A Linux shell window logging the captures.A Linux shell window logging the captures.A Linux shell window logging the captures.A Linux shell window logging the captures.記錄捕獲的 Linux shell 視窗。

Twitter API 的典型結果如下所示：

["2022-10-06 21:20:00.366761", 
"#SheHulk", "Matt Ryan", "Colts", "Colts", "Mario", "Hines", 
"#TNFonPrime", "Bowser", "#ThursdayNightFootball", 
"Russell Wilson", "#FanDuelisTheGREATEST", "#INDvsDEN", 
"Judy Tenuta", "Quavo", "Graphic B", "Hunter Biden", "Saweetie", 
"Matty Ice", "Luigi", "Phillip Lindsay", "Caden Sterns", "Jovic", 
"Charles Martinet", "Marijuana", "Offset", "Kathy", "Dank Brandon", 
"Carina", "Pittman", "Sasse", "Brooklyn", "Commanders vs Bears", 
"Frank Reich", "Primetime", "Elvis", "Mushroom Kingdom", "Jamal Cain", 
"Toad", "Sara Lee", "Sam Ehlinger", "Darren Bailey", "Norville", 
"Deon Jackson", "Rodney Thomas", "Ryan Kelly", 
"Mumbles McTouchdown", "Melvin Gordon", "Ranboo", "Illumination", ""]

通常每個位置都會有 50 個熱門術語。 美國的九個地點有很多共同點，趨勢會持續五分鐘。 即使是一個微小的趨勢也會受到很多關注。 這是我用來載入文件並製作單字和頻率表的 JSL。

// for this quick test I copied the json files to my desktop.
// probably should put them in their own directory...
dir = "$desktop/";
names = Filter Each( {j}, Files In Directory( dir ), Ends With( j, ".json" ) );
// I'll need a list of open data tables
dtfiles = {};
For Each( {file}, names, Insert Into( dtfiles, Open( dir || file, JSON Settings( Col( "/root", Type( "JSON Lines" ) ) ) ) ) );
// I'll use an associative array to gather the frequency info
WordToFrequency = [=> 0];
For Each( {dt}, dtfiles,
 For Each Row( dt,
  // there are a lot of ways to do this; here it makes a list of col2..col51 from the current row
  row = dt[Row(), 2 :: N Cols( dt )];
  // then process the list
  For Each( {word}, row, WordToFrequency[word] += 1 );
 )
);
// then make a data table from the associative array
dtWF = New Table( "wordfreq",
 addrows( N Items( WordToFrequency ) ),
 New Column( "word", character, values( WordToFrequency << getkeys ) ),
 New Column( "freq", numeric, values( WordToFrequency << getvalues ) )
);

該表按單字排序：

There are currently 127K words. The blank one is a filler when there are <50 trends.There are currently 127K words. The blank one is a filler when there are <50 trends.There are currently 127K words. The blank one is a filler when there are <50 trends.There are currently 127K words. The blank one is a filler when there are <50 trends.目前有 127K 字。 當趨勢 <50 時，空白是填充項。

第一個有趣的問題是“最流行的詞是什麼？” 按頻率排序：

Caturday seems to be about cats.Caturday seems to be about cats.Caturday seems to be about cats.Caturday seems to be about cats.星期六似乎是關於貓的。

就在那裡。 星期六是一周中最不受歡迎的一天。

如中所提到的WebSocket ，我之前對這些數據做了一個無聊的視覺化。 數據按城市（或美國或世界）排列，並具有時間戳記。 我有一個單獨的程序，可以從維基百科上抓取新聞項目，這可能會有所幫助。 也許不會;推特熱門詞彙似乎與實際新聞鬆散（充其量）有關。

（使用此腳本時請注意：它濫用了 html 和 xml 之間的相似性，將 html 轉換為 JMP 的 xml 閱讀器可以處理的內容。 在使用這個想法之前，您需要弄清楚為什麼任何特定的 html 看起來不夠像 xml 並修復它。 請參閱第 10 行周圍的正規表示式——它刪除了導致 xml 閱讀器混亂的 html 位元。 使用 python 和 beautiful soup 可能會更容易，但我還沒有這樣做。）

dtBig = New Table( "news",
 New Column( "date", Numeric, "Continuous", Format( "yyyy-mm-dd", 12 ), Input Format( "yyyy-mm-dd" ) ),
 New Column( "text", Character, "Unstructured Text" )
);

For( ddd = 1apr2023, ddd < Today(), ddd += In Days( 1 ),
 url = "https://en.wikipedia.org/wiki/Portal:Current_events/" || Format( ddd, "Format Pattern", "<YYYY>_<MONTH>_<D>" );
 txt = Load Text File( url );
 Length( txt );
 xml = Regex(
  txt,
  "||&nbsp;|(<script.>)|(<FORM.>)|(<OPTION.>)|crossorigin|data-mw-ve-target-container|<LINK.>|<meta.>|<INPUT.>|<IMG.>",
  "",
  GLOBALREPLACE
 );
 dt = Open(
  Char To Blob( xml ),
  XML Settings(
   Stack( 0 ),
   Row( "/html/body/div/div/div/main/div/div/div/div/div/div/ul" ),
   Row( "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul" ),
   Row( "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul" ),
   Row( "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul/li/ul" ),
   Row( "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul/li/ul/li/ul" ),
   Col(
    "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li",
    Column Name( "text" ),
    Fill( "Use Once" ),
    Type( "Original Text" ),
    Format( {"Best"} ),
    Modeling Type( "Continuous" )
   ),
   Col(
    "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li",
    Column Name( "text" ),
    Fill( "Use Once" ),
    Type( "Original Text" ),
    Format( {"Best"} ),
    Modeling Type( "Continuous" )
   ),
   Col(
    "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul/li",
    Column Name( "text" ),
    Fill( "Use Once" ),
    Type( "Original Text" ),
    Format( {"Best"} ),
    Modeling Type( "Continuous" )
   ),
   Col(
    "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul/li/ul/li",
    Column Name( "text" ),
    Fill( "Use Once" ),
    Type( "Original Text" ),
    Format( {"Best"} ),
    Modeling Type( "Continuous" )
   ),
   Col(
    "/html/body/div/div/div/main/div/div/div/div/div/div/ul/li/ul/li/ul/li/ul/li/ul/li",
    Column Name( "text" ),
    Fill( "Use Once" ),
    Type( "Original Text" ),
    Format( {"Best"} ),
    Modeling Type( "Continuous" )
   ),

  ),
  XML Wizard( 0 )
 );
 For Each Row( dt, dt:text = Regex( dt:text, "<[^>]+>", "", GLOBALREPLACE ) );
 dt << New Column( "date", Numeric, "Continuous", Format( "yyyy-mm-dd", 12 ), Input Format( "yyyy-mm-dd" ), <<SETEACHVALUE></SETEACHVALUE></IMG.></INPUT.></LINK.></OPTION.></FORM.></D></MONTH></YYYY>

This is recent at the time the picture was made. Sometimes these entries are updated/corrected/deleted later.This is recent at the time the picture was made. Sometimes these entries are updated/corrected/deleted later.This is recent at the time the picture was made. Sometimes these entries are updated/corrected/deleted later.This is recent at the time the picture was made. Sometimes these entries are updated/corrected/deleted later.這是拍攝照片時最近的照片。 有時這些條目稍後會更新/更正/刪除。

 

好的，我研究了一下星期六多一點。 這可能與星期六有關。 我們來分析一下。 試著收集所有以 ...day 結尾的流行詞，並記住它發生的那一天...

days = [=> ];
For Each( {dt}, dtfiles,
 (dt:col) << datatype( "numeric" );
 For Each Row( dt, 
  row = Filter Each( {trend}, dt[Row(), 2 :: N Cols( dt )], (Ends With( Lowercase( trend ), "day" )) );
  dow = Day Of Week( dt:col );
  For Each( {trend}, row,
   If( !Contains( days, trend ),
    days[trend] = [0, 0, 0, 0, 0, 0, 0]
   );
   days[trend][dow] += 1;
  );
 );
);

哇。 有一堆（1568）。 看起來星期六和星期六有很強的連結。 也許星期四實際上是最不受歡迎的。

Sunday is 1, Monday is 2, ...Sunday is 1, Monday is 2, ...Sunday is 1, Monday is 2, ...Sunday is 1, Monday is 2, ...週日是 1 點，週一是 2 點，...

蔓延到週日可能是時區問題。

 

歡迎任何想法...感謝您的閱讀。

首先來看看十一個月的熱門詞彙。 星期六並不是一週中最不受歡迎的一天。