DATA DENOISING PROCEDURE FOR NEURAL NETWORK PERFORMANCE IMPROVEMENT

Miroslav B. Milovanović, Dragan S. Antić, Saša S. Nikolić, Miodrag D. Spasić, Staniša Lj. Perić, Marko T. Milojković

DOI Number
-
First page
173
Last page
182

Abstract


This paper will present training data denoising procedure for neural network performance improvement. Performance improvement will be measured by evaluation criterion which is based on a training estimation error and signal strength factor. Strength factor will be obtained by applying denoising method on a default training signal. The method is based on a noise removal procedure performed on the original signal in a manner which is defined by the proposed algorithm. Ten different processed signals are obtained from the performed method on a default noisy signal. Those signals are then used as a training data for the nonlinear autoregressive neural network learning phase. Empirical comparisons are made at the end, and they show that the proposed denoising procedure is an effective way to improve network performances when the training set possesses the significant noise component.

Full Text:

PDF

References


F. Scarselli, A. C. Tsoi, "Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results," Neural Networks, vol. 11, no. 1, pp. 15–37, 1998. [Online]. Available: http://dx.doi.org/10.1016/S0893-6080(97)00097-X

T. Jayalakshmi, A. Santhakumaran, "Statistical normalization and back propagation for classification," International Journal of Computer Theory and Engineering, vol. 3, no. 1, pp. 89–93, 2011. [Online]. Available: http://www.ijcte.org/papers/288-L052.pdf

K. Shihab, "A backpropagation neural network for computer network security," Journal of Computer Science, vol. 2, no. 9, pp. 710–715, 2006. [Online]. Available: http://docsdrive.com/pdfs/sciencepublications/jcssp/2006/710-715.pdf

S. Solanki, H. B. Jethva, "Modified back propagation algorithm of feed forward networks," International Journal of Innovative Technology and Exploring Engineering, vol. 2, no. 6, pp. 131–134,2013. [Online]. Available: http://ijitee.org/attachments/File/v2i6/F0816052613.pdf

P. J. Werbos, "Backpropagation through time: What it does and how to do it," Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990. [Online]. Available: http://dx.doi.org/10.1109/5.58337

O. Al-Allaf, "Improving the performance of backpropagation neural network algorithm for image compression/decompression system," Journal of Computer Science, vol. 6, no. 11, pp. 1347–1354, 2010. [Online]. Available: http://94.142.32.98/conferences/ITC12/AllPapers/PA_13.pdf

M.-B. Rădac, R.-E. Precup, E. M. Petriu, "Model-free primitive-based iterative learning control approach to trajectory tracking of MIMO systems with experimental validation," IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 11, pp. 2925–2938, 2015, [Online]. Available: http://dx.doi.org/10.1109/TNNLS.2015.2460258

M.-B. Rădac, R.-E. Precup, E. M. Petriu, St. Preitl, "Application of IFT and SPSA to servo system control," IEEE Transactions on Neural Networks, vol. 22, no. 12, part 2, pp. 2363–2375, 2011. [Online]. Available: http://dx.doi.org/10.1109/TNN.2011.2173804

M. Z. Rehman, N. M. Nawi, "Improving the accuracy of gradient descent back propagation algorithm (GDAM) on classification problems," International Journal on New Computer Architectures and Their Applications, vol. 1, no. 4, pp. 838–847, 2011. [Online]. Available: http://sdiwc.net/digital-library/improving-the-accuracy-of-gradient-descent-back-propagationalgorithm-gdam-on-classification-problems.html

J. R. Dorronsoro, V. López, C. S. Cruz, J. A. Sigüenza, "Autoassociative neural networks and noise filtering," IEEE Transactions on Signal Processing, vol. 51, no. 5, pp. 1431–1438, 2003. [Online]. Available: http://dx.doi.org/10.1109/TSP.2003.810276

C. Wang, J. C. Principe, "Training neural networks with additive noise in the desired signal," IEEE Transactions on Neural Networks, vol. 10, no. 6, pp. 1511–1517, 1999. [Online]. Available: http://dx.doi.org/10.1109/72.809097

D. G. Khairnar, S. N. Merchant, U. B. Desai, "Radar signal detection in non-Gaussian noise using RBF neural network," Journal of Computers, vol. 3, no. 1, pp. 32–39, 2008. [Online]. Available: http://www.ojs.academypublisher.com/index.php/jcp/article/view/03013239/366

X.-P. Zhang, "Thresholding neural network for adaptive noise reduction," IEEE Transactions on Neural Networks, vol. 12, no. 3, pp. 567–584, 2001. [Online]. Available: http://dx.doi.org/10.1109/72.925559

C. L. Giles, S. Lawrence, A. C. Tsoi, "Noisy time series prediction using a recurrent neural network and grammatical inference," Machine Learning, vol. 44, no. 1–2, pp. 161–183, 2001. [Online]. Available: http://dx.doi.org/10.1023/A:1010884214864

I. Kaastra, M. Boyd, "Designing a neural network for forecasting financial and economic time series," Neurocomputing, vol. 10, no. 3, pp. 215–236, 1996. [Online]. Available: http://dx.doi.org/10.1016/0925-2312(95)00039-9

V. Nedeljković, M. Milosavljević, "On the influence of the training set data preprocessing on neural networks training," in Proceedings of 11th IAPR International Conference on Pattern Recognition, Conference B: Pattern Recognition Methodology and Systems, Hague, Netherlands, pp. 33–36, 1992. [Online]. Available: http://dx.doi.org/10.1109/ICPR.1992.201716

O. E. de Noord, "The influence of data preprocessing on the robustness and parsimony of multivariate calibration models," Chemometrics and Intelligent Laboratory Systems, vol. 23, pp. 65–70, 1994. [Online]. Available: http://dx.doi.org/10.1016/0169-7439(93)E0065-C

J. de Witt, "Adaptive filtering network for associative memory data preprocessing," in Proceedings of the World Congress on Neural Networks, San Diego, USA, vol. 4, pp. 34–38, 1994.

H. H. Nguyen, C. W. Chan, "A comparison of data preprocessing strategies for neural network Modeling of oil production prediction," in Proceedings of the Third IEEE International Conference on Cognitive Informatics, Victoria, Canada, pp. 199–209, 2004. [Online]. Available: http://dx.doi.org/10.1109/COGINF.2004.1327476

L.-O. Fedorovici, "A comparison between two character recognition approaches," Facta Universitatis Series: Automatic Control and Robotics, vol. 10, no. 2, pp. 125–140, 2011. [Online]. Available: http://facta.junis.ni.ac.rs/acar/acar201102/acar20110201.html

S. Geman, E. Bienenstock, R. Doursat, "Neural networks and the bias/variance dilemma," Neural Computation, vol. 4 no. 1, pp. 1–58, 1992. [Online]. Available: http://dx.doi.org/10.1162/neco.1992.4.1.1

L. Badri, M. Al-Azzo, "Modelling of wavelength detection of objects using Elman network modified covariance combination," The International Arab Journal of Information Technology, vol. 5, no. 3, pp. 265–272, 2008. [Online]. Available: http://ccis2k.org/iajit/PDF/vol.5,no.3/8-201.pdf

H. Jaeger, "A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "Echo state network" approach," GMD Report 159, German National Research Center for Information Technology, pp. 1–48, 2002. [Online]. Available: http://www.pdx.edu/sites/www.pdx.edu.sysc/files/Jaeger_TrainingRNNsTutorial.2005.pdf

D. Antić, M. Milovanović, S. Nikolić, M. Milojković, S. Perić, "Simulation model of magnetic levitation based on NARX neural networks," International Journal of Intelligent Systems and Applications, vol. 5, no. 5, pp. 25–32, 2013. [Online]. Available: http://dx.doi.org/10.5815/ijisa.2013.05.04

D. Antić, M. Milovanović, S. Nikolić, S. Perić, M. Milojković, "Primena NARX neuronske mreže za simulaciju rada sistema magnetne levitacije," TEHNIKA, vol. 62, no. 3, pp. 473–479, 2013.


Refbacks

  • There are currently no refbacks.


Print ISSN: 1820-6417
Online ISSN: 1820-6425