Word density

Question

Hey there,

Does anyone have a way to create a word density distribution for a column of textual data in JMP?  Basically I'd like to create a summary table that has every word in the data file and a column showing how many times it appears.

Thanks!

Matt

michael_jmp · Answer

This script on the File Exchange might help you: Word Counts to Columns.

-Michael

pmroz · Answer

Here's a JMP solution.  This is the JMP discussion forum after all.

dt = New Table( "Word Table",

Add Rows( 9 ),

New Column( "Column 1",

Character,

Nominal,

Set Values(

{"The quick red fox jumped over the brown log",

"The quick red fox jumped over the brown log",

"Apples peaches pumpkin pie.  If you're not ready holler I.",

"Apples peaches pumpkin pie.  If you're not ready holler I."}

)

);

dtcol = column(dt, "Column 1");

first_pass = dtcol << get values;

// Convert the column to one big string

first_string = concat items(first_pass, " ");

// Remove periods, change two spaces to one

first_string = substitute(substitute(first_string, ".", " "), "  ", " ");

// Parse all values out of this big string, delimited by a space

all_list = words(first_string, " ");

// Add these values to a new table

wordsdt = New Table("Parsed Words",

New Column("listvals", Character, Nominal, Set Values(all_list)));

// Run tabulate to get a frequency count

words_tab = wordsdt << Tabulate(

Show Control Panel( 0 ),

Add Table( Row Table( Grouping Columns( :listvals ) ) )

);

antony_allianz · Answer

use

proc means data =dsname n;

class varnm;

run;

give the varnm of the word variable name

otherwise use first.varnm and last.varnm