The Effect on Modelling Performance of Different Activation Functions for Feed Forward and Feedback Network Structures in Modeling of Chen Chaotic System

Abstract

Activation functions is an important parameter that affects the performance of the network in the process of training of Artificial Neural Network (ANN) structures. This paper presents the modelling of Aizawa Chaotic System (ACS) using the structures of Feed Forward Neural Network (NN) and Feedback NN. Runge Kutta 5 Butcher (RK-5-B) algorithm has been used for the numeric solution describing ACS. Nonlinear activation functions like Radial Basis (RadBas), Logarithmic Sigmoid (LogSig) and Tangent Sigmoid (TanSig) have been used in the modelling process and the analysis study has been performed related to the modelling performance of ACS by using these functions in the created network structures. It has been observed that the TanSig activation function which is one of the actication functions used in the modelling with FFNN structure has produced more sensitive results than others and the LogSig activation function which is one of the actication functions used in the modelling with RNN structure. has produced more sensitive results than others.