233659_en-US

liliconcom

Random Sampling with Replacement

Is it possible to do this?  It does not appear the Tables --> Subset function allows for this.  

 

I recognize that you can get Bootstrap estimates in various parts of the JMP Pro platform but I am wondering specifically about generating data tables.

 

Thank you!

Re: Random Sampling with Replacement

Hi Wendy,

 

Yes, see script below. It takes a 4 line table and pulls 10 samples with replacement. The AsList( ) function is needed to pull this off, otherwise you get an error, unfortunately.

 

Names Default To Here( 1 );

dt = astable((1::4)`);

nSamples = 10;

dt2 = dt << subset( rows( As List( J( nSamples, 1, Random Integer( 1, N Row( dt ) ) ) ) ) );

Re: Random Sampling with Replacement

Hi Wendy, you can do this from the data table. Right-click on a column head, select New Formula Column, Random, and then Sample With Replacement.

Re: Random Sampling with Replacement

Hi Wendy, I should clarify.

 

The New Formula Column technique Mia mentions will work as long as the user wants a sample whose size is less than or equal to the number of rows in the table. If you need a larger sample, I don't know of a way to do this (efficiently, that is--concatenating a table with itself until its size is large enough, is inefficient but will work) via point-and-click. Maybe others will chime in with a way to do that.

 

The script will work for any sample size, but needs the As List() function wrapped around the row matrix whenever the number of rows requested exceeds the number of rows in the original table.

 

 

Re: Random Sampling with Replacement

Thank you both!  Yes - I think the client may have a need to oversample so it would be helpful to know how others have accomplished this.

 

The other related question is around stratification.  Is it possible to resample with replacement ensuring a distribution of another attribute?

Re: Random Sampling with Replacement

Hi @wendytseng,

There is a Sample With Replacement Add-In that can perform oversampling, but not stratification with respect to another variable. 

@julian 

Re: Random Sampling with Replacement

Hi Wendy,

 

Not sure whether your customer is willing to script, but the idea can readily be extended to stratify wrt a categorical variable. The short script below does this for the Car Physical Data dataset and the "Type" column. 1000 rows are requested, where the original table has 116. Note that 1002 rows are returned, due to rounding up to the nearest integer the sample size of each category.

 

This isn't the fastest way to do this, but it is relatively straightforward to understand. Perhaps your customer will be able to script what they want if you talk them through this?

 

Cheers,

Brady

 

Names Default To Here( 1 );

dt = Open( "$Sample_Data\Car Physical Data.jmp" );

stratCol = Column( dt, "Type" );

nSample = 1000;

//determine the (over)sampling ratio
sRatio = nSample / N Row( dt );

//get a list of the stratification column's values
categories = ((Associative Array( stratCol << get values ) << get keys));

// initialize the master row matrix
rowMat = [];

//For each category
For( i = 1, i <= N Items( categories ), i++,

 // Get the table rows belonging to this category
 catRows = dt << getrowswhere( dt:stratCol == categories[i] );
 
 // Determine how many rows belong to this category
 catN = N Items( catRows );
 
 // Oversample these row values by the specified ratio, and append them to the master row list
 rowMat |/= catRows[As List( J( Ceiling( catN * sRatio ), 1, Random Integer( 1, catN ) ) )];
);

// Subset the data table, using the final master row list
dt << subset( rows( As List( rowMat ) ) );

Re: Random Sampling with Replacement

For oversampling, there's also this add-in:  Stratified Data Partitioning (with balancing options) add-in

Haven't tested in JMP 15...

Re: Random Sampling with ReplacementThat's the one we spec'd for the prof at UC Denver, right? I looked over the code for it and couldn't think of a way to get it to work without breaking into Train/Test/Validate.