A customer noticed that the Model Comparison report for kNN models with categorical responses is empty for all metrics but Misclassification Rate. I have little experience with kNN but assumed this was because kNN predicts classes without assigning probabilities, which looks right according to the documentation. But in reading on, I noticed the Category Bias parameter for biasing predicted probabilities away from zero. I think this was introduced in 16 and can be traced back to this defect. So it looks like as of 16, kNN does compute predicted probabilities (and now even includes Entropy RSquare in the model report) but isn't yet able to save the probabilities to the data table for purposes of Model Comparison.
Assuming I've got this right (and please correct me if I don't!), does anybody know why the probabilities are being computed inside the platform but are not able to be saved out? I'm not sure how best to explain to the customer why they're seeing Entropy RSquare values in the platform but they can't get them and other probability based metrics in Model Comparison. Thanks!
