Level: Beginner
Marie Gaudard, Senior Data Scientist, Nhg
Philip Ramsey, Senior Data Scientist, Predictum
Rosemary Danaher, Graduate Student, University of New Hampshire
Christopher Gotwalt, Director, JMP Statistical R&D, JMP
Artificial neural networks (ANN) and their deep learning can generate highly accurate predictive models. However, they carry with them a hidden risk of scientific disaster: ANNs are very sensitive to the spatial distribution of the input variables. For data with clumps or sizeable gaps, ANNs tend to fit well within the clumps but predict gibberish in the gaps. We use case studies and simulations to show the dangers of applying ANNs to data with poor spatial distribution. We also demonstrate that improvements in predictive accuracy on new data are obtained by designing validation and test sets with high generalizability performance explicitly in mind. By using the popular PyTorch and TensorFlow ANN libraries, in addition to the algorithms that are native to JMP Pro, we demonstrate that the problem of poor neural network generalizability exists independently of the neural network implementation.
