Looking for some good text cleaning/extraction examples

DonMcCormack

I'm presenting a workshop on JMP text tools at next months US Discovery and I'm looking for good, realistic examples of extracting information from text. I'll likely spend more time on structured text but I do plan on discussing regular expressions and unstructured text. If you have examples you'd like to share, I'm interested in hearing about them. They don't have to be challenging, simple but interesting works, too. My plan is to email the participants the text prior to the workshop with information on what needs to be accomplished. We'll spend the workshop going over solutions.

Ross_Metusalem

The Beige Book economic reports data (attached) could be useful. Starting in 1981, the reports have individual sections set off by headings in double curly brackets (e.g., {{ Banking }} or {{ Employment }}). You could use regex on these to grab only text from certain sections for further analysis. 

One challenge is that there is variation in the header text for a given concept/topic, for example {{ Finance }} vs {{ Banking }}. But that could be just another layer to the challenge.

Beige Book - District Reports.jmp