ARTICLE | doi:10.20944/preprints201802.0008.v1
Subject: Computer Science And Mathematics, Probability And Statistics Keywords: Optimal Bayesian detection, information geometry, minimal error probability, Chernoff/Bhattacharyya upper bound, large random tensor, Fisher information, large random sensing matrix
Online: 1 February 2018 (16:32:04 CET)
The performance in terms of minimal Bayes’ error probability for detection of a high-dimensional random tensor is a fundamental under-studied difficult problem. In this work, we consider two Signal to Noise Ratio (SNR)-based detection problems of interest. Under the alternative hypothesis, i.e., for a non-zero SNR, the observed signals are either a noisy rank-R tensor admitting a Q-order Canonical Polyadic Decomposition (CPD) with large factors of size Nq R, i.e, for 1 q Q, where R, Nq ! ¥ with R1/q/Nq converge towards a finite constant or a noisy tensor admitting TucKer Decomposition (TKD) of multilinear (M1, . . . ,MQ)-rank with large factors of size Nq Mq, i.e, for 1 q Q, where Nq,Mq ! ¥ with Mq/Nq converge towards a finite constant. The detection of the random entries (coefficients) of the core tensor in the CPD/TKD is hard to study since the exact derivation of the error probability is mathematically intractable. To circumvent this technical difficulty, the Chernoff Upper Bound (CUB) for larger SNR and the Fisher information at low SNR are derived and studied, based on information geometry theory. The tightest CUB is reached for the value minimizing the error exponent, denoted by s?. In general, due to the asymmetry of the s-divergence, the Bhattacharyya Upper Bound (BUB) (that is, the Chernoff Information calculated at s? = 1/2) can not solve this problem effectively. As a consequence, we rely on a costly numerical optimization strategy to find s?. However, thanks to powerful random matrix theory tools, a simple analytical expression of s? is provided with respect to the Signal to Noise Ratio (SNR) in the two schemes considered. A main conclusion of this work is that the BUB is the tightest bound at low SNRs. This property is, however, no longer true for higher SNRs.
ARTICLE | doi:10.20944/preprints201610.0086.v1
Subject: Computer Science And Mathematics, Geometry And Topology Keywords: information geometry; mixture models; log-sum-exp bounds
Online: 20 October 2016 (10:35:57 CEST)
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using costly Monte-Carlo stochastic integration, approximated, or bounded using various techniques. We present a fast and generic method that builds algorithmically closed-form lower and upper bounds on the entropy, the cross-entropy and the Kullback-Leibler divergence of mixtures. We illustrate the versatile method by reporting on our experiments for approximating the Kullback-Leibler divergence between univariate exponential mixtures, Gaussian mixtures, Rayleigh mixtures, and Gamma mixtures.