Case-sensitive processing in Text ExplorerA user in academic area asks me whether there is an option to pick up words with case sensitive mode in the Text Explorer platform.
Please run the script below as an example. He wants to show words as "JMP" in the report, but JMP's Text Explorer returns "jmp".
dt = New Table("Test", New Column("Y", Character, Values({"JMP JMP JMP JMP JMP JMP"})));
dt << Text Explorer(
Text Columns( :Y ),
Set Regex( Custom( Title( "Test" ), Regex( "\[(.+?(\s|\Z))]\" ), Result( "\[\1]\" )))
);
[Background]
Almost all characters in his data are in Japanese. Only very small portion of characters are alphabets. He picks up words by other software, and create lists of words. In his tokenized data, each words is separated by a space. (I mean that he tokenizes his data by other software than JMP.) He wants that JMP treats these tokenized words as they are.
Re: Case-sensitive processing in Text ExplorerThe first step in parsing the text is to convert it to all lower case, before either parsing method begins. I tried to apply a function (e.g., Title Case( string ) ) to the return result from Regex, but then you get an expression instead of a string. (Huh?)
If the list of such tokens is small, then Recode rules in Text Explorer might be the way to go. They can be scripted, so it might be possible to build the code from a list of exceptions.
Re: Case-sensitive processing in Text ExplorerThank you very much for your advice.
I also have tried
Title Case(\1)
and
Upper Case(\1)
for the result in regex. It returns "title case(jmp)" and "upper case(jmp)" as you see.
The Recode option seems to also treat characters by case-insensitive manner. Even if I change "jmp" to "JMP" in the Recode window, the Text Explorer changes back to lowercase characters (i.e. "jmp").
Anyway, I thank you very much for your advice. I come to think strongly that there is no option to treat characters as they are in the Text Explorer.
In general, the tokenization of Japanese texts is a very tough task. So, the user uses other software for tokenization. The user must agree on that the case-sensitive treatment is a very rare usage (especially for Western languages).
