Complex valued neural network (CVNN) has been developed to process complex valued data directly. In CVNN, one of the most important factors is selecting the node’s activation function. Choosing the right activation function for each layer is also crucial and may have a significant impact on metric scores and the training speed of the model. This paper introduces three new activation functions for CVNNs which is closely related to the activation function complex swish. These new activation functions are complex modified swish, complex E-swish and complex Flatten-T swish. In order to verify the validity and practicability of the proposed three new activation functions are tested and compared with complex swish activation function on complex valued four bit XOR problem, three inputs symmetry detection and the fading equalization problems. We show that complex E-swish ( β=1.4)  has the best overall performance when compared to other networks using complex swish, complex modified swish and complex Flatten-T swish activation functions on the considered tasks.


Complex valuedartificial neural network, Swish, Modified swish, E-swish, Flatten-T swish

Full Text:



M. Ceylan, “Improving an algorithm with complex-valued artificial neural network and applications,” MS Thesis, Selcuk University Gradute School of Natural and Applied Sciences, Konya, 13-14, 2004.

A. Hirose, “Complex-Valued Neural Networks, World Scienific Publishing,” Singapore, 2003.

A. Hirose, “Complex-Valued Neural Networks, Series on Studies in Computational Intelligence,” Vol. 32, Springer-Verlag, New York 2006.

T. Nitta, “Complex-Valued Neural Networks: Utilizing High-Dimensional Parameters,” Information Science Reference, Hershey, PA, 2009.

T. Nitta, “Solving the XOR problem and the detection of symmetry using a single complex valued neuron,” Neural Networks, 16:1101-1105, 2003.

T. Nitta, “Orthogonality of Decision Boundaries in Complex-Valued Neural Networks,” Neural Computation 16, 73–97, 2004.

M. Celebi, M. Ceylan, “The New Activation Function for Complex Valued Neural Networks: Complex Swish Function,” ISAS WINTER-Samsun, 2019.

N. Özdemir, B. İskender, N. Özgür, “Complex valued neural network with Möbius activation function,” Commun Nonlinear Sci Numer Simulat 16, 2011.

H. Leung, S. Haykin, “The complex backpropagation algorithm,” IEEE Trans Signal Process,39:2101–4, 1991.

N. Benvenuto, F. Piazza, “On the complex backpropagation algorithm,” IEEE Trans Signal Process 40:967–9, 1992.

D.L. Birx, S.J. Pipenberg, “Chaotic oscillators and complex mapping feed forward networks (CMFFNS) for signal detection in noisy environments,” Proceedings of IEEE international jt. conf. neural networks. II, p. 881–888, 1992.

G. Kechriotis, E.S. Monalakos, “Training fully recurrent neural networks with complex weights,“ IEEE Trans Circuits Syst II, 41:235–8, 1994.

M. Kinouchi, M. Hagiwara, “Learning temporal sequences by complex neurons with local feedback,”Proceedings of IEEE international conference on neural networks IV, p. 3165–3169,1995.

A. Hirose, “Applications of complex-valued neural networks to coherent optical computing using phase-sensitive detection scheme,” Inf Sci Appl, 2:103–17, 1994.

I. Aizenberg, “Complex-Valued Neural Networks with Multi-Valued Neurons, International Joint Conference on Neural Networks,” IJCNN, 2013.

N. Monning, S. Manandhar, “Evaluation of Complex-Valued Neural Networks on Real-Valued Classification Tasks,” CoRR abs/1811.12351, 2018.

P. Ramachandran, B. Zoph, “Searching for activation functions,” In International Conference on Learning Representations Q. V. L., 2018.

E. Alcaide, “E-swish: Adjusting activations to different network depths,” arXiv preprint arXiv:1801.07145, 2018.

H. Chieng, N. Wahid, O. Pauline, S. Perla, “Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning,” International Journal of Advances in Intelligent Informatics, 2018.

M. Ceylan, “Combined complex-valued artificial neural network (CCVANN),” Proceedings of the World Congress on Engineering. 2:955-959, 2011.

X. Chen, “An Modified Error Function for the Complex-value Backpropagation Neural Networks,” Neural Information Processing - Letters and Reviews Vol.8, No.1, 2005.

Y. Acar, M. Ceylan, E. Yaldız, “An examination on the effect of CVNN parameters while classifying the real-valued balanced and unbalanced data, International Conference on Artificial Intelligence and Data Processing (IDAP),” 2018.

T. Nitta, “Orthogonality of Decision Boundaries in Complex-Valued Neural Networks,” Neural Computation 16, 73–97, 2004.

T. Nitta, “Complex-Valued Neural Network and Complex-Valued Backpropagation Learning Algorithm,” Advances in Imaging and Electron Physics,Volume 152, ISSN 1076-5670, 2008.

T. Nitta, “The Computational Power of Complex-Valued Neuron,” ICANN/ICONIP, 2003.

D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning internal representations by error propagation, Parallel Distributed Processing: Explorations in the microstructures of cognition,” vol. 1, pp. 318–362. Cambridge, MA: MIT Press, 1986.

D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning representations by back-propagating errors,” Nature 323, 533–536, 1986.

H.A. Jalab, R. W. Ibrahim, “New activation functions for complex-valued neural network,” International Journal of the Physical Sciences Vol. 6(7), pp. 1766-1772, 4 April, 2011.

T. Enomoto, K. Kakuda, S. Miura, “New Activation Functions in CNN and Its Applications,” ICCES, vol.1, no.2, pp.36-39, 2019.

M. Ceylan, “Combined complex-valued artificial neural network (CCVANN),” Proceedings of the World Congress on Engineering. 2:955-959, 2011.

H. Gürüler, M. Peker, “A Software Tool for Complex-Valued Neural Network: CV-ANN,” 23nd Signal Processing and Communications Applications Conference (SIU), 2015.

X. Chen, Z. Tang C. Variappan, S. Li and T. Okada, “A modified error backpropagation algorithm for complex-value neural networks,” International Journal of Neural Systems, 15, 435-443, 2005.



  • There are currently no refbacks.

Copyright (c) 2020 International Journal of Advanced Research in Computer Science