Influence of the Neural Network Hyperparameters on its Numerical Conditioning

Abstract

In this paper, the task of assessment of numerical conditioning of multilayer perceptron, forecasting time series with sliding window method, has been considered. Performance of the forecasting perceptron with various hyperparameters sets, with different amount of neurons and various activation functions in particular, has been considered. Main factors, influencing on the neural net conditioning, have been revealed, as well as performance features, when using various activation functions. Formulas for assessment of condition numbers of individual components of the forecasting perceptron and of the neural network itself have been proposed. Comparative analysis of results of training the forecasting perceptron with various hyperparameters on modeled time series has been performed. Conditions, providing the best stability and conditioning for the neural network, have been formulated.