Using Stratification and Grouping in Validation ToolI've got an interesting question from a marketing company. They have a pool of stores - to make the math easier let's say 5000. Of the 5000 stores, 2000 are designated as "control" stores - they are off limits to advertising campaigns, etc. The other 3000 are "test" stores. The problem they are facing is that they want to select 200 stores from the control and test groups with similar demographics.
I think I found a way to do this by using stratification columns and grouping columns in the Validation Column Tool, and pairing that with the random stratified subsample capabilities in Subset. But, that seems a little complex and I'm wondering if someone has run into this before. It also has the problem that using stratification and grouping columns in the Validation Column Tool only allows you to have one column in stratify and one group column - they have between 3 and 5 that they want to consider for stratification.
Thoughts?
Best,
M
Re: Using Stratification and Grouping in Validation ToolHave you considered using one of the clustering tools and pairing/matching stores from the same group?
Re: Using Stratification and Grouping in Validation ToolYeah - I had them try hierarchical first to see if they could use the clusters in place of all the demographic information, but the distributions from some of the categorical variables (like the store brand distribution) weren't as close to matching as they'd like. It might just be that the population they're sampling from doesn't have enough stores to make two identical subsets from the test and control pools. ¯\_(ツ)_/¯
Re: Using Stratification and Grouping in Validation ToolIn addition to the clustering scoring for the selection of a comparison group that Don mentioned, Another popular method is the propensity score matching https://en.wikipedia.org/wiki/Propensity_score_matching and https://www.scalestatistics.com/propensity-score-matching.html
SAS has a Proc dedicated to this method this https://go.documentation.sas.com/doc/en/statug/15.2/statug_psmatch_overview.htm
Re: Using Stratification and Grouping in Validation ToolSo you're matching two like groups and not store to store?
Re: Using Stratification and Grouping in Validation ToolCorrect.
Re: Using Stratification and Grouping in Validation Tool@MikeD_Anderson have you thought of or tried using covariate DOE? You select the factors to be part of the "DOE" and let the DOE platform select the rows to be part of the subset. You may have to subset the Test and Control stores into separate tables to get this technique to pull 200 from each group, but I can imagine, with a little trial and error of course , that there would be a way to make this happen in one table.
Best,
Bill
Re: Using Stratification and Grouping in Validation ToolThanks @Bill_Worley - I'd had a question from a user about tools in JMP to create balanced groups, e.g. initial weight of mice assigned to cages in groups of 5, and had inferred this was one application of the covariate DoE with cage as block and initial weight as covariate. A little scripting to test out the hypothesis was initial encouragement I was on the right track.
(Admittedly I don't recall seeing the word 'balanced' in the reading I dug up in our docs so was hesitant.)
