Kullback-Leibler (K-L) Distance
Kullback Leibler Distance between Probabilistic Context Free Grammars and Probabilistic Finite Automata
... the Kullback-Leibler distance, also called the relative entropy, between a probabilistic context-free grammar and a probabilistic fi- nite ...the Kullback-Leibler distance, ...
7
A New Upper Bound for the Kullback-Leibler Distance and Applications
... the distance between two ...D(p k q) is a measure of the inefficiency of assuming that the distribution is q when the true distribution is ...D(p k q) bits on the average to describe the random ...
11
New Estimates of the Kullback-Leibler Distance and Applications
... The relative entropy is a measure of the distance between two distributions. In statistics, it arises as an expected logarithm of the likelihood ratio. The relative entropy D(pkq) is a measure of the inefficiency ...
12
On the Importance of the Kullback Leibler Divergence Term in Variational Autoencoders for Text Generation
... Quantitative analysis of generated text without gold reference sequences (e.g. in Machine Trans- lation or Summarization) has been a long-standing challenge. Recently, there have been efforts to- wards this direction, ...
10
Learning Marginalization through Regression for Hand Orientation Inference
... contour distance features from hand silhouette images ...a Kullback-Leibler di- vergence based optimization for estimating weights for ex- pert ...
10
Gaussian approximations for probability measures on Rd*
... We then adopt the abstract measure approximation theory to understanding the posterior consistency of finite dimensional Bayesian inverse problems. In particular, we give an alterna- tive (and more analytical) proof of ...
31
Algorithms for Kullback Leibler approximation of probability measures in infinite dimensions
... Abstract. In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference ...
26
A Kullback Leibler Divergence for Bayesian Model Diagnostics
... [3]; Kullback and Leibler [4]; Lindley [5]; Bernardo [6]; Schwarz ...The Kullback- Leibler distance (KLD) is perhaps the most commonly used information criterion for assessing model ...
13
Zipf–Mandelbrot law, f divergences and the Jensen type interpolating inequalities
... are Kullback–Leibler divergence, Hellinger distance, Bhattacharyya distance (via coefficient), χ 2 -divergence, total variation distance and ...
20
Discrimination between Gamma and Log-Normal Distributions by Ratio of Minimized Kullback-Leibler Divergence
... of Kullback-Leibler Divergence (information or distance) based test statistic and its usage in practice, but its application on discrimination of two known and overlapping distributions has less ...
11
On preferred point geometry in statistics
... Abstract. A brief synopsis of progress in differential geometry in statistics is followed by a note of some points of tension in the developing relationship between these disciplines. The preferred point nature of much ...
18
Kullback Leibler divergence based wind turbine fault feature extraction
... 𝑓(𝑥) = 𝑠𝑔𝑛((𝑤, 𝑥) + 𝑏) (5) The sign of 𝑓(𝑥) depends on the side of the hyperplane where the sample lies. An optimal hyperplane is the one that maximises the distance between the hyperplane and the nearest points ...
6
Sensitivity to Prior Specification in Bayesian Identification of Autoregressive Time Series Models
... the Kullback-Leibler divergence to measure the distance between the posteriors of autoregressive (AR) model order, aiming to evaluate mathematically the sensitivity of the model identification to ...
15
Adverse Factors for the Technology Modernization: Local Public Management
... The reason for using the term “divergence” rather than “distance” is that Kullback-Leibler measure is neither symmetric nor obeys triangle inequality. Therefore Kullback-Leibler measure ...
16
Dimensional Reduction of Statistical Structural of a Paper by Information Geometry
... performance. This improvement can be understood from the point of view of information-theoretic bounds: information divergence is generally more relevant to statistical discrimination performance than Euclidean ...
9
Strong Consistency of the Prototype Based Clustering in Probabilistic Space
... In this paper we formulate in general terms an approach to prove strong consistency of the Empirical Risk Minimisation inductive principle applied to the prototype or distance based clustering. This approach was ...
11
Approximations of Shannon Mutual Information for Discrete Variables with Applications to Neural Population Coding
... Information theory is a powerful tool widely used in many disciplines, including, for example, neuroscience, machine learning, and communication technology [1–7]. As it is often notoriously difficult to effectively ...
21
Survey on Change Detection in SAR Images with Image Fusion and Image Segmentation
... In order to improve the accuracy of the binary change map, the data samples obtained by fusing the log-ratio and mean-ratio images are projected to a higher dimensional feature space, in which a linear algorithm can be ...
7
Some Inequalities For The Kullback-Leibler And x²−Distances In Information Theory And Applications
... We consider the mutual information, which is a measure of the amount of infor- mation that one random variable contains about another random variable.. It is the reduction of uncertainty[r] ...
10
Derivation of the PHD and CPHD Filters Based on Direct Kullback-Leibler Divergence Minimization
... In this paper, we present novel derivations of the PHD and CPHD filters that do not require the use of PGFLs or functional derivatives. We believe that the new derivations are more accessible, thereby bringing the ...
10