Home   >   CSC-OpenAccess Library   >    Manuscript Information
Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradient Training Algorithms For Image Compression Using MLP
Devesh Batra
Pages - 412 - 422     |    Revised - 07-10-2014     |    Published - 10-11-2014
Volume - 8   Issue - 6    |    Publication Date - November / December 2014  Table of Contents
Image Compression, Artificial Neural Network, Multilayer Perceptron, Training, Levenberg-Marquardt, Scaled Conjugate Gradient, Complexity.
The Internet paved way for information sharing all over the world decades ago and its popularity for distribution of data has spread like a wildfire ever since. Data in the form of images, sounds, animations and videos is gaining users’ preference in comparison to plain text all across the globe. Despite unprecedented progress in the fields of data storage, computing speed and data transmission speed, the demands of available data and its size (due to the increase in both, quality and quantity) continue to overpower the supply of resources. One of the reasons for this may be how the uncompressed data is compressed in order to send it across the network. This paper compares the two most widely used training algorithms for multilayer perceptron (MLP) image compression – the Levenberg-Marquardt algorithm and the Scaled Conjugate Gradient algorithm. We test the performance of the two training algorithms by compressing the standard test image (Lena or Lenna) in terms of accuracy and speed. Based on our results, we conclude that both algorithms were comparable in terms of speed and accuracy. However, the Levenberg- Marquardt algorithm has shown slightly better performance in terms of accuracy (as found in the average training accuracy and mean squared error), whereas the Scaled Conjugate Gradient algorithm faired better in terms of speed (as found in the average training iteration) on a simple MLP structure (2 hidden layers).
1 Google Scholar 
2 CiteSeerX 
3 refSeek 
4 Scribd 
5 SlideShare 
6 PdfSR 
Anderson, Dave, and George McNeill. "Artificial neural networks technology." Kaman Sciences Corporation 258 (1992): 13502-462.
Andrei, Neculai. "Scaled conjugate gradient algorithms for unconstrained optimization." Computational Optimization and Applications 38.3 (2007): 401-416.
Costa, Saverio, and Simone Fiori. "Image compression using principal component neural networks." Image and vision computing 19.9 (2001): 649-668.
Cyriac, Marykutty, and C. Chellamuthu. "A near–lossless approach for medical image compression using visual quantisation and block–based DPCM."International Journal of Biomedical Engineering and Technology 13.1 (2013): 17-29.
Dony, Robert D., and Simon Haykin. "Neural network approaches to image compression." Proceedings of the IEEE 83.2 (1995): 288-303.
Durai, S. Anna, and E. Anna Saro. "Image Compression with Back-Propagation Neural Network using Cumulative Distribution Function." International Journal of Applied Science, Engineering & Technology 3.4 (2007).
Ebrahimi, Farzad, Matthieu Chamik, and Stefan Winkler. "JPEG vs. JPEG 2000: an objective comparison of image encoding quality." Optical Science and Technology, the SPIE 49th Annual Meeting. International Society for Optics and Photonics, 2004.
MATLAB Product Help. Available: http://www.mathworks.in/help/matlab/.
Mohamad, N., et al. "Comparison between Levenberg-Marquardt and scaled conjugate gradient training algorithms for breast cancer diagnosis using MLP."Signal Processing and Its Applications (CSPA), 2010 6th International Colloquium on. IEEE, 2010.
Møller, Martin Fodslette. "A scaled conjugate gradient algorithm for fast supervised learning." Neural networks 6.4 (1993): 525-533.
Multilayer Perceptron. Available: http://neuroph.sourceforge.net/tutorials/MultiLayerPerceptron.html
Nguyen, Derrick, and Bernard Widrow. "Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights." Neural Networks, 1990., 1990 IJCNN International Joint Conference on . IEEE, 1990.
Ranganathan, Ananth. "The levenberg-marquardt algorithm." Tutoral on LM Algorithm (2004): 1-5.
Riedmiller, Martin. "Advanced supervised learning in multi-layer perceptrons—from backpropagation to adaptive learning algorithms." Computer Standards & Interfaces 16.3 (1994): 265-278.
Steihaug, Trond. "The conjugate gradient method and trust regions in large scale optimization." SIAM Journal on Numerical Analysis 20.3 (1983): 626-637.
Wei, Wei-Yi. "An Introduction to Image Compression." National Taiwan University, Taipei, Taiwan (2009): 1.
What is feedforward neural network. Available: http://www.fon.hum.uva.nl/praat/manual/Feedforward_neural_networks_1__What_is_a_feed forward_ne.html
What is lossless image compression. Available:http://dvd-hq.info/data_compression_1.php.
What is lossy image compression. Available: http://dvd-hq.info/data_compression_2.php.
Yu, Hao, and B. M. Wilamowski. "Levenberg-marquardt training." The Industrial Electronics Handbook 5 (2011): 1-15.
Zurada, Jacek M. "Introduction to artificial neural systems." (1992).
Mr. Devesh Batra
Stanford University - India