Prof. Benabdellah Yagoubi
Laboratory of Signals and systems, Department of Electrical Engineering, University of Mostaganem, Algeria

Title: Neural Network Training using the Orthogonality Principle

Abstract: When modeling any real system, we start first by selecting a linear model due to its mathematical simplicity. This linear modeling corresponds to the representation of the real system by its linear component. However, if the linear modeling error corresponding to the resultant of the nonlinear and the observation error components, is important, then the real system should be better represented by its nonlinear component. Therefore, in this case, it is more convenient to look for a nonlinear model to represent the real system. The search for a corresponding nonlinear model, is however usually a very difficult task. The idea to try to overcome this problem, is to mix many different models to produce one that approximately fits the real system, and leave the remaining computing task to the computer. Geometrically speaking, the real system nonlinear component could be seen as a variable hyper-surface not as a hyper-plane as in the case of a linear modeling. Therefore, to obtain the nonlinear model that best represents the real one, the nonlinear modeling error should be locally orthogonal to the nonlinear model component. This means that for each neural network input training data, we have to make the nonlinear modeling error locally orthogonal to the curve (or surface) representing the nonlinear neural network output component. In summary, for a neural network to fit locally a real system, the local ,orthogonality principle must be satisfied. In this lecture, we will show how to train a neural network using the orthogonality principle and suggest, consequently, a simple algorithm and a python code for modeling a nonlinear system.

BIO: Dr B. Yagoubi received the M. Sc degree in Electrical Engineering in 1985 from Bel-Abbes University, Algeria and the Ph. D degree (amorphous thin films) (1986-1989) in the Faculty of Sciences from Brunel University (UK). He was the head of the Signals and Systems Laboratory (1999-2003) and the head of the Department of Electrical Engineering (2005-2006) and recently (since 2023)as an ethic officer. He is lecturing the theory of digital signal, systems modeling and identification, random processes and detection at Mostaganem University, Algeria. He has been involved in some national projects such as forest fire detection, heart rate variability in the LF and HF bands to characterize the autonomous nervous system, and study and application of random processes. He has been involved in developing techniques such asthe inverse of the belonging probability to extract useful information from mono-dimensional and multidimensional signals as well as on detection of a rare event in a Gaussian realization, as part of a national project. Currently, he is focusing on the nonlinear modeling and nonlinear identification using neural networks as well as on windowing for FIR filter design enhancement and spectral leakage mitigation. Further research interests are in OFDM and OCDMA transmissions.