Text Explorer for Pareto Analysis of Non-compliance

Phil_Kay

Hi,

 

A customer is interested in using Text Explorer to look at a QA database. They want to see if they can find insight into non-compliance issues that have been raised by looking at the free-text comments associated with each entry in the database - these are the documents in this case. It is analogous to looking for topics in the Trucks data.

 

The main thing they want to do is to categorise the documents using topic analysis. This would allow them to do a Pareto analysis to focus improvement activity on the most common non-compliance types.

 

A couple of questions that I don't currently have good answers for:

1. Any guidance for how you can decide on the most useful number of topics? Any diagnostics that would help with this? Or is it a case of using knowledge of the domain to guide when you are not adding useful topics?

2. Having identified topics, is there a simple way to count the number of documents associated with each topic?

 

Looking forward to your comments.

Regards,

Phil

Daniel_Valente

Phil -- 

For #1, generally: 

You need enough topics to get enough separation. But not too many so that you have too much overlap between topics. The LCA has a useful plot, which can be used for helping aid in determining this number. It's the MDS plot. With trucks, you can see with 3 topics, there is perhaps too much separation, with 100, there is too much overlap.

 

For your specific question, I think a good workflow might be to use the topic analysis (SVD or LCA) and then bin the clusters as causes that relate to the 6Ms (Method, Mother Nature, Manpower, Measurement, Machines, Materials).  That would be a more useful label vs. cluster 1, cluster 4 etc. Then you can see top terms that are relating to a specific category. LCA reports the BIC and also term probabilities by Cluster and Cluster Mixture probability. Sorting the Term Probabilities by Cluster is useful as well. In the case of Trucks. Either the LCA or SVD topic analysis will help with the binning process much faster than the alternative (reading through each indivudal documents) or requiring engineers to fill out a form a bucket each non-compliance at the time of reporting the issue.

 

Heath rushing goes through another example of FDA medical device non-compliances in the Text Analytics article in the JMP Foreward. He also covers it in the webcast and tech speaking demo. The data table is here.

 

For #2, using the LCA, you can save most likely cluster to the data table. And tabulate from there. Or save a data table of table of cluster mixture probabilities from the context menu > make into data table. You can get total docs from the probability and then use domain expertise to flag each cluster with a theme.

 

 

 

Again, in the case of the FDA inspection citations example, going from 27k case records to top topics in a few minutes is a huge time savings vs. reading each long descripting and binning manually. Should there be 5 or 7 topics? Well that is going to be a judgment call based on domain expertise and looking at the MDS plots/cluster probabilities/top terms per topic etc. 

Either way, it's going to point me in the right direct to find out where I need to focus my attention next, which I think, is the ultimate goal.