TWO-DIMENSIONAL GMM-BASED CLUSTERING IN THE PRESENCE OF QUANTIZATION NOISE

Aleksandra Jovanović, Zoran Perić

DOI Number
https://doi.org/10.22190/FUACR210321008J
First page
099
Last page
110

Abstract


In this paper, unlike to the commonly considered clustering, wherein data attributes are accurately presented, it is researched how successful clustering can be performed when data attributes are represented with smaller accuracy, i.e. by using the small number of bits. In particular, the effect of data attributes quantization on the two-dimensional two-component Gaussian mixture model (GMM)-based clustering by using expectation–maximization (EM) algorithm is analyzed. An independent quantization of data attributes by using uniform quantizers with the support limits adjusted to the minimal and maximal attribute values is assumed. The analysis makes it possible to determine the number of bits for data presentation that provides the accurate clustering. These findings can be useful in clustering wherein before being grouped the data have to be represented with a finite small number of bits due to their transmission through the bandwidth-limited channel. 

Keywords

Unsupervised learning, clustering, Gaussian mixture model, expectation-maximization algorithm, quantization noise

Full Text:

PDF

References


C. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.

T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, 2016.

I. H. Witten, E. Frank, M. A. Hall, C. J. Pal, Data Mining: Practical Machine Learning Tools and Techniques, 4th ed., Morgan Kaufmann Series in Data Management Systems, 2016.

A. P. Dempster, N. M. Laird, D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society, vol. 39, no. 1, pp. 1–38, 1977. [Online]. Available: https://web.mit.edu/6.435/www/Dempster77.pdf

X. Lin, X. Yang, Y. Li, "A deep clustering algorithm based on Gaussian mixture model," ISAI 2019, Journal of Physics: Conference Series, 1302, 2019. [Online]. Available: https://iopscience.iop.org/article/10.1088/1742-6596/1302/3/032012/pdf

M. S. Yang, C. Y. Lai, C. Y. Lin, "A robust EM clustering algorithm for Gaussian mixture models," Pattern Recognition, vol. 45, no. 11, pp. 3950–3961, 2012. [Online]. Available: https://www.sciencedirect.com/science/article/abs/pii/S0031320312002117

A. Cedric, L. A. John, V. Michael, "On convergence problems of the EM algorithm for finite Gaussian mixtures," in Proceedings of 11th European Symposium on Artifical Neural Networks, Bruges, Belgium, pp. 99–106, 2003. [Online]. Available: http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/esann_ca03.pdf

A. Jovanović, Z. Perić, D. Aleksić, J. Nikolić, "The effect of uniform data quantization on GMM-based clustering by means of EM algorithm", 20th International Symposium INFOTEH-JAHORINA, VRT-2.7 (45), March 17-19, 2021, Jahorina, RS, B&H.

I. H. Tseng, O. Verscheure, D. S. Turaga, U. V. Chaudahari, "Quantization for adapted GMM-based speaker verification," in Proceedings of IEEE ICASSP, Toulouse, France, May 2006. [Online]. Available: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.332.4539&rep=rep1&type=pdf

I. H. Tseng, O. Verscheure, D. S. Turaga, U. V. Chaudhari, "Optimized one-bit quantization for adapted GMM-based speaker verification," in Proceedings of INTERSPEECH, Antwerp, Belgium, August 27–31, pp.786-789, 2007. [Online]. Available: https://www.isca-speech.org/archive/archive_papers/ interspeech_2007/i07_0786.pdf

N. S. Jayant, P. Noll, Digital Coding of Waveforms, Englewood Cliffs, NJ: Prentice-Hall, 1984.

D. Hui, D. L. Neuhoff, "Asymptotic analysis of optimal fixed-rate uniform scalar quantization," IEEE Transaction on Information Theory, vol. 47, no. 3, pp. 957–977, 2001. [Online]. Available: https://www.researchgate.net/publication/3080367_Asymptotic_analysis_of_optimal_fixed-rate_uniform_scalar_quantization

S. Na, D. Neuhoff, "Monotonicity of step sizes of MSE-optimal symmetric uniform scalar quantizers," IEEE Transaction on Information Theory, vol. 65, no. 3, pp. 1782−1792, 2019.

S. Na, D. Neuhoff, "On the convexity of the MSE distortion of symmetric uniform scalar quantization," IEEE Transaction on Information Theory, vol. 64, no.4, pp. 2626−2638, 2018.

Z. Eskić, Z. Perić, J. Nikolić, "A method of designing adaptive uniform quantizer for LPC coefficient quantization," Przeglad Elektrotechniczny, vol. 87, no. 7, pp. 245–248, 2011. [Online]. Available: https://www.researchgate.net/publication/228520367_A_method_of_designing_an_adaptive_uniform_quantizer_for_LPC_coefficients_quantization

A. Jovanović, Z. Perić, J. Nikolić, "Iterative algorithm for designing asymptotically optimal uniform scalar quantization of the one-sided Rayleigh density," IET Communications, pp. 1–7, February 2021. [Online]. Available: https://doi.org/10.1049/cmu2.12114

Z. Perić, J. Lukić, J. Nikolić, D. Denić, "Application of mean-square approximation for piecewise linear optimal compander design for Gaussian source and Gaussian mixture model," Information Technology and Control, vol. 42, no. 3, pp. 277–285, 2013. [Online]. Available: https://doi.org/10.5755/ j01.itc.42.3.4349

A. Ž. Jovanović, Z. H. Perić, "Geometric piecewise uniform lattice vector quantization of the memoryless Gaussian source," Information Sciences,vol. 181, no. 14, pp. 3043–3053, 2011.

A. Ž. Jovanović, Z. H. Perić, J. R. Nikolić, "An efficient iterative algorithm for designing an asymptotically optimal modified unrestricted uniform polar quantization of bivariate Gaussian random variables," Digital Signal Processing, vol. 88, pp. 197–206, May 2019.

A. Ž. Jovanović, Z. H. Perić, J. R. Nikolić, M. R. Dinčić, "Asymptotic analysis and design of restricted uniform polar quantizer for Gaussian sources," Digital Signal Processing, vol. 49, pp. 24–32, February 2016.

Z. Perić, J. Nikolić, "Asymptotic analysis of switched uniform polar quantization for memoryless Gaussian source," IEEE Signal Processing Letters, vol. 20, no. 1, pp. 75–78, 2013.

Z. H. Perić, M. D. Petković, J. R. Nikolić, A. Ž. Jovanović, "Support region estimation of the product polar companded quantizer for Gaussian source," Signal Processing, vol. 143, pp. 140 145, 2018.




DOI: https://doi.org/10.22190/FUACR210321008J

Refbacks

  • There are currently no refbacks.


Print ISSN: 1820-6417
Online ISSN: 1820-6425