Grouping by fixed number(i.e. 4 or 5) of products in cluster.Hi,
I got some question from some customer, I have tried to find solution but couldn't find until yet.
Hope your idea sharing.
Here is the situation.
They produce product B and measure 5 points in every products(lots) like below.
And, they want to group 5 most similar products based on the value of 5 points
(Assume there are 100 products, 5 points, and 500 measurement data)
1. We can group based on some statistics(mean, stdev and other methods to calculate flatness etc.) for 5 points.
But I think that these methods are less reasonable compared to Clustering.
2. I tried Hierarchy and K-Means. In this case, these methods has also some limitations.
They want to group by fixed number(i.e. 4 or 5) of products in every cluster.
I saw something similar about the grading of agricultural products, but I couldn’t remember.
Is there any other ideas ?
Thanks in advance
Ikju
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.Ikju: What’s the limitation of clustering? You didn’t get the equal-sized groups?
If so, you can use this JSL* after you run kmeans for a one-cluster solution and save the distances:
((Col Rankc(:Distance Formula ) / Col Number( :Distance Formula )) * N Row() / :Group Size)
(*adapted from Phil Kay's post .)
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.Hi Jian,
Thanks for your valuable comments.
I can make equal-sized cluster according to your comments.(data size 135 = 27*5)
Ceiling(
(Col Rank( :"Distance Formula(K Means)"n ) /
Col Number( :"Distance Formula(K Means)"n )) * 27
)
However, clustering using k-means distance does not seem to be an appropriate method. The key is to group similar things by equal-size. I don't think this is the proper way to group similar things.
I attached 3 gif animation
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.Due to the earnest request of the customer, I could not share the original data, so I created a similar data structure.
Please give your ideas
Thanks
Ikju
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.I see that you used the distances saved from a 20-cluster solution, which are the distances from each cluster’s centroid. Try using the scaled distances from 0 (you can run a one-cluster and then remove the means). However, that's is just a way of calculating similarity among objects.
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.@jiancao Would this be another instance of the use of the covariate DoE, using blocks as mentioned in that thread about the mice? I haven't compared what is happening behind the scenes in covariate DoE with the clustering methods, but it seems the application is similar: optimally create equivalent groups of defined size.
Apologies, related thread here. I've also attached the file used for exploring the idea. Only 1 covariate, weight, in this case (though the source table has 2 so i could explore different distributions).
Re: Grouping by fixed number(i.e. 4 or 5) of products in cluster.Christian,
Covariate DoE, as I understand it, can be used to create similar groups, while clustering is to assign similar objects into different groups. it's like within-variability vs between-variability.
Before your data came in I just did a comparison: grouping using distance versus. grouping using covariate DoE on Big Class. Below is the ANOVA on the three covariates or clustering variables used in grouping: age, height and weight:
As you can see, grouping by distances creates four groups with different means (left column), while grouping via covariate DoE produces four groups with similar means (right column) ;
Does this make sense?
