ONE-BIT QUANTIZER PARAMETRIZATION FOR ARBITRARY LAPLACIAN SOURCES

Danijela Aleksić, Zoran Perić

DOI Number
https://doi.org/10.22190/FUACR220321004A
First page
037
Last page
046

Abstract


In this paper we suggest an exact formula for the total distortion of one-bit quantizer and for the arbitrary Laplacian probability density function (pdf). Suggested formula additionally extends normalized case of zero mean and unit variance, which is the most applied quantization case not only in traditional quantization rather in contemporary solutions that involve quantization. Additionally symmetrical quantizer’s representation levels are calculated from minimal distortion criteria. Note that one-bit quantization is the most sensitive quantization from the standpoint of accuracy degradation and quantization error, thus increasing importance of the suggested parameterization of one-bit quantizer.

Keywords

Laplacian source, one-bit quantization, symmetric quantizer design

Full Text:

PDF

References


N. S. Jayant, P. Noll, Digital Coding of Waveforms, Englewood Cliffs, NJ: Prentice-Hall, 1984.

D. Hui, D. L. Neuhoff, "Asymptotic analysis of optimal fixed-rate uniform scalar quantization," IEEE Transaction on Information Theory, vol. 47, no. 3, pp. 957–977, 2001, doi: 10.1109/18.915652.

S. Na, D. Neuhoff, "On the convexity of the MSE distortion of symmetric uniform scalar quantization," IEEE Transaction on Information Theory, vol. 64, no.4, pp. 2626−2638, 2018, doi: 10.1109/TIT.2017.2775615.

S. Na, D. Neuhoff, "Monotonicity of step sizes of MSE-optimal symmetric uniform scalar quantizers," IEEE Transaction on Information Theory, vol. 65, no. 3, pp. 1782−1792, 2019, doi: 10.1109/TIT.2018.2867182.

J. Lee, S. Na, "A rigorous revisit to the partial distortion theorem in the case of a Laplacian source," IEEE Commun. Lett., vol. 21, pp. 2554–2557, 2017, doi: 10.1109/LCOMM.2017.2749218.

A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, K. Keutzer, "A survey of quantization methods for efficient neural network inference," arXiv:2103.13630

Y. Guo, "A survey on methods and theories of quantized neural networks," arXiv:1808.04752

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, Y. Bengio, "Quantized neural networks: training neural networks with low precision weights and activations," J. Mach. Learn. Res., vol. 18, pp. 6869–6898, 2017.

D. Liu, H. Kong, X. Luo, W. Liu, R. Subramaniam, "Bringing AI to edge: from deep learning’s perspective," arXiv:2011.14808.

S. Kim, A. Gholami, Z. Yao, N. Lee, P. Wang, A. Nrusimha, B. Zhai, T. Gao, M. W. Mahoney, K. Keutzer, "Integer-only zero-shot quantization for efficient speech recognition," arXiv:2103.16827v2

R. Banner, Y. Nahshan and D. Soudry, "Postraining 4-bit quantization of convolutional networks for rapid-deployment," 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, pp. 7948 7956, 2019.

S. Gazor and W. Zhang, "Speech probability distribution," IEEE Signal Processing Letters, vol. 10, no. 7, pp. 204 207, 2003, doi: 10.1109/LSP.2003.813679.

S. Kotz, T. Kozubowski and K. Podgorski, "The Laplace distribution and generalizations," Birkhäuser, Boston, 2001.

J. Nikolić, D. Aleksić, Z. Perić and M. Dinčić, "Iterative algorithm for parameterization of two-region piecewise uniform quantizer for the Laplacian source," Mathematics, vol. 9, pp. 3091, 2021, doi: 10.3390/math9233091.

Stefan Tomić, Jelena Nikolić, Zoran Perić, Danijela Aleksić, "Performance of Post-Training Two-Bits Uniform and Layer-Wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice,",Mathematical Problems in Engineering, Volume 2022, Article ID 1463094, 2022.J.

Nikolić, Z. Perić, D. Aleksić, S. Tomić, A. Jovanović, "Whether the support region of three-bit uniform quantizer has a strong impact on post-training quantization for MNIST dataset?," Entropy, vol. 23, pp. 1699, 2021, doi: 10.3390/e23121699.

W. Zhao, M. Teli, X. Gong, B. Zhang, D. Doermann, "A review of recent advances of binary neural networks for edge computing," IEEE J. Miniat. Air Space Syst., vol. 2, pp. 25–35. 2021, doi: 10.1109/JMASS.2020.3034205.

P. E. Novac, G. B. Hacene, A. Pegatoquet, B. Miramond, V. Gripon, "Quantization and deployment of deep neural networks on microcontrollers," Sensors, vol. 21, pp. 2984, 2021, doi: 10.3390/s21092984.

D. Aleksić, Z. Perić, "Analysis and design of robust quasilogarithmic quantizer for the purpose of traffic optimisation," Inf. Technol. Control, vol. 47, pp. 615-622, 2018, doi: http://dx.doi.org/10.5755/j01.itc.47.4.20668.

Z. Perić, J. Nikolić, D. Aleksić, A. Perić, "Symmetric quantile quantizer parameterization for the Laplacian source: qualification for contemporary quantization solutions, " Math. Probl. Eng., vol. 2021, Article ID 6647135, 2021, doi: 10.1155/2021/6647135

Z. Perić, D. Aleksić, "Quasilogarithmic quantizer for Laplacian source: support region ubiquitous optimization task," Rev. Roum. Sci. Techn., vol. 64, pp. 403−408, 2019.

G. Dehaune, "A deterministic and computable Bernstein-von Mises theorem," arXiv:1904.02505.

S. Kullback, R. Leibler, "On information and sufficiency. Annals of mathematical statistics," vol. 22, pp. 79–86, 1951.

S. Tomić, Z. Perić, J. Nikolić, "Modified BTC algorithm for audio signal coding," Advances in Electrical and Computer Engineering, vol. 16, 2016.




DOI: https://doi.org/10.22190/FUACR220321004A

Refbacks

  • There are currently no refbacks.


Print ISSN: 1820-6417
Online ISSN: 1820-6425