A customer asked me about neural networks the following two questions. As far as I know this is not available yet (please correct if I'm wrong). Do we have plans to enhance the functionality in that direction that a user could tweak the activation functions more?
1. Are there plans to add ReLu as activation function (Rectified Linear Unit), saw the internal wishlist entry from @IkjuShin
2. Do we have the ability to use regularization methods? (l1/l2, batch normalization, drop out, residual connection, etc). As far I understand this relates to deep networks, which we do not do, and likely going into, correct?
