International Journal of Mathematics and Mathematical Sciences Volume 2012, Article ID 315686,10pages
doi:10.1155/2012/315686
Research Article
Some Coding Theorems for Nonadditive
Generalized Mean-Value Entropies
Satish Kumar,
1Gurdas Ram,
2and Vishal Gupta
31Department of Mathematics, Geeta Institute of Management and Technology, Kurukshetra,
Kanipla 136131, India
2Department of Applied Sciences, Maharishi Markandeshwar University, Solan 173229, India
3Department of Mathematics, Maharishi Markandeshwar University, Mullana 133203, India
Correspondence should be addressed to Satish Kumar,drsatish74@rediffmail.com
Received 8 July 2011; Accepted 7 December 2011
Academic Editor: Andrei Volodin
Copyrightq2012 Satish Kumar et al. This is an open access article distributed under the Creative
Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
We give an optimality characterization of nonadditive generalized mean-value entropies from suit-able nonadditive and generalized mean-value properties of the measure of average length. The results obtained cover many results obtained by other authors as particular cases, as well as the
ordinary length due to Shannon 1948. The main instrument is thelnifunction of the word lengths
in obtaining the average length of the code.
1. Introduction
Given a discrete random variableXtaking a finite number of valuesx1, x2, . . . , xnwith prob-abilities P p1, p2, . . . , pn,pi ≥ 0,ni1pi 1, the Shannon’s entropy of the probability
distribution is given by
HP −n
i1
pilogpi, 1.1
where the base of the logarithm is in general arbitrary.
that is encoded into words of lengthsN {n1, n2, . . . , nn} forming an instantaneous code, then
n
i1
D−ni ≤1, 1.2
whereDis the size of the code alphabet.
The average lengthL, for the instantaneous code is such that
Lpini≥ −n
i1
pilogDpi 1.3
with equality if and only if for eachi
piD−ni. 1.4
Result1.4,refer Shannon5, characterizes Shannon’s entropy as a measure of opti-mality of a linear function, namely,L, under the relation1.2.
Several generalizations of Shannon’s entropy have been studied by many authors in different ways. Here we will need the nonadditive generalized mean value measures such as
Hp1, p2, . . . , pn; 1, α, βDβ−1/α−1−1Dβ−1/αni1pilogDpi−1 , α, β >0, β /1,
1.5
Hp1, p2, . . . , pn;α, β
D1−β/α−1−1
⎡ ⎣
n
i1
pα
i
1/αβ−1/α−1
−1
⎤ ⎦,
β >0, β /1, α >0, α /1; α /β.
1.6
These quantities satisfy the “nonadditivity”
HP∗Q HP HQ D1−β/α−1HPHQ. 1.7
2. Nonadditive Measure of Code Length
Let us consider two independent sourcesX {x1, x2, . . . , xn}andY {y1, y2, . . . , ym}with
associated probability distributionP p1, p2, . . . , pnandQ q1, q2, . . . , qm. Then the
prob-ability distribution of the productXY {xi, yi}isP∗Q p1q1, p1q2, . . . , p1qm, . . . , pnqm. Let the sourceYbe encoded with a code of lengthM{m1, m2, . . . , mm}and the pairxi, yj
be represented by a sequence forxi andyj put side by side, so that the product source has
code length sequence:
NM{n1m1, . . . , n1mn, . . . , nnmm}. 2.1
The additive measure of mean length is required to satisfy the requirement, refer Campbell
2,
LP∗Q, NM, φLP, N, φLQ, M, φ, 2.2
where
LP, N, φφ−1
n
i1
piφni
, 2.3
φbeing a continuous strictly monotonic increasing function.
Campbell 3proved a noiseless coding theorem for Renyi’s entropy of order α in terms of mean length2.3defined forφni Dni.
The mean length concerning Shannon’s entropy and orderαentropy of Renyi are both additive as they satisfy additivity of type2.2.
Here we deal with nonadditive measures of length denoted byL∗which satisfy “non-additivity relation”
L∗PQ, NM, φL∗P, N, φL∗Q, M, φλL∗P, N, φL∗Q, M, φ,
2.4
and the mean value property
L∗P, N, φφ−1ni1piφlni
n
i1pi
, 2.5
wherelniis the function of length of a single element with code word length ni, which is
nonadditive.
3. Characterization of Nonadditive Measures of Code Length
First of all we will determine the nonadditive length functionlof the code word length in satisfying the nonadditivity relation
lnm ln lm λlnlm, λ /0. 3.1
This by takingfn 1λlngives
fnm fnfm. 3.2
The most general nonzero solution of3.2is
fn D1−β/αn, 3.3
whereα >0,β /1 are arbitrary constantswe have taken baseDwith a purpose here. So that
ln D1−β/αnλ −1, λ /0, 3.4
at this stage we make a proper choice of the constantλ. By analogy, its value is dictated as
λD1−β/α−1, β /1 asλ /0. 3.5
Another purpose served with this value ofλis that when it tends to zero, that is, when
β → 1, the function of lengthlnshould reduce to additive one which isln n. This value ofλcan be obtained by imposing a boundary conditionl1 1 also.
So that finally we have
ln D1−β/αn−1
D1−β/α−1 , β /1, α >0. 3.6
Next we proceed to determineL∗P, N, φby first evaluating the values ofφ. To achieve this we put the value oflnfrom3.6in2.5and then use the relation2.4withλ D1−β/α− 1,β /1 to get
φ−1
n
i1mj1piqjφ
D1−β/αnimj−1/D1−β/α−1
n
i1
m
j1piqj
φ−1
n
i1piφD1−β/αni−1/D1−β/α−1
n
i1pi
φ−1
m
j1qjφ
D1−β/αmj−1/D1−β/α−1
m
j1qj
D1−β/α−1φ−1
n
i1piφD1−β/αni−1/D1−β/α−1
n
j1pi
×φ−1
m
j1qjφD1−β/αmj−1/D1−β/α−1
m
j1qj
.
3.7
Now let us takeQ{q}andM{m}so that forP p1, p2, . . . , pn, pi≥0, ni1pi1,3.7
gives after some simplification
φ−1
n
i1
piφ
D1−β/αnim−1
D1−β/α−1
φ−1
n
i1
piφ
D1−β/αni−1
D1−β/α−1
D1−β/αmD1−β/αm−1
D1−β/α−1 ,
3.8 or ψ−1 m n i1 piψm
D1−β/αni−1
D1−β/α−1
φ−1 n i1 piφ
D1−β/αni−1
D1−β/α−1
D1−β/αmD1−β/αm−1
D1−β/α−1 ,
3.9
where
ψm
D1−β/αni−1
D1−β/α−1
φ
D1−β/αnim−1
D1−β/α−1
. 3.10
Now, refer Hardy et al.4, there must be a linear relation betweenψmandφ, that is,
ψmn Amφn Bm, 3.11
whereAmandBmare independent ofn. Using3.10and3.11, we have
gnm Amgn Bm, 3.12
where
gn φ
D1−β/αn−1
D1−β/α−1
or
Gnm AmGn Gm, 3.14
where
Gn gn−g0 gn−a, ag0. 3.15
From the symmetry of3.14, we get
AmGn Gm AnGm Gn
⇒ AGn
n−1
Gm
Am−1
1
K
say. 3.16
Thus,
An−1KGn 3.17
for all real values ofn.
There are two cases, namely,K0 andK /0. IfK0,An 1, and3.14gives
Gnm Gn Gm, 3.18
the most general continuous solution of which is given by
Gn cn, 3.19
wherecis an arbitrary constant. This, by3.15and3.13, gives
φ
D1−β/αn−1
D1−β/α−1
acn 3.20
which gives
φn a cα
1−βlogD
1D1−β/α−1n , β /1. 3.21
Again ifK /0, we have the relation obtained from3.17and3.14:
the general continuous solution of which are
An 0 which we neglectfor alln,
An Dtn, 3.23
wheretis an arbitrary nonzero constant.
This by using3.17,3.15in3.13, gives
φ
D1−β/αn−1
D1−β/α−1
aDtnK−1 3.24
which gives
φn a
D1−β/α−1n1tα/1−β−1
K , β /1, t /0, α >0. 3.25
The value ofφgiven by3.21and3.25determine the following two nonadditive measures of length defined by2.5, that is,
L∗P, N; 1, α, βD1−β/α−1−1D1−β/αni1pini−1 , 3.26
L∗P, N;t, α, βD1−β/α−1−1D1−β/αtlogDni1piDnit−1 . 3.27
These code lengths denoted byL∗P, N; 1, α, βandL∗P, N;t, α, βmay be named as nonad-ditive typeα, βlengths of order 1 andt, respectively.
These results are contained in the following theorem.
Theorem 3.1. The mean length given by2.5of a sequence of lengthsni, i 1,2, . . . , nformed of
the code alphabet of sizeDof a probability distributionP p1, p2, . . . , pn, pi ≥0, ni1pi1
sat-isfyingni1D−ni ≤1 and nonadditivity relation can be only of one of the two forms given in3.26
and3.27.
3.1. Limiting and Particular Cases
It is immediate to see the following.
1
lim
t→0L
∗P, N;t, α, βL∗P, N; 1, α, β. 3.28
2
lim
β→1L
∗P, N; 1, α, βn
i1
piniL, 3.29
3
lim
β→1L
∗P, N;t, α, β 1
tlog
n
i1
piDtni, 3.30
length of ordertdefined by Campbell2.
4Forn1 n2· · ·nn nsay, both the expressions for length given by3.26and
3.27reduce to
D1−β/α−1−1D1−β/αn−1 , 3.31
which in the limiting case, whenβapproaches unity, reduces ton.
5Forαβ, the expression3.26becomes
L∗P, N; 1, α D1−α/α−1−1D1−α/αni1pini−1 . 3.32
6Forαβ, the expression3.27becomes
L∗P, N;t, α D1−α/α−1−1D1−α/αtlogDni1piDnit−1 . 3.33
7For α → 1 the expression 3.32 and 3.33 reduces to ni1pini L and
1/tlogni1piDtni, respectively.
Thus, we have shown thatL∗P, N; 1, α, βandL∗P, N;t, α, βare typeαandβ gen-eralizations of
Ln
i1
pini, Lt 1tlogD
n
i1
piDtni, 3.34
respectively. We now prove the following theorem.
Theorem 3.2. Ifn1n2· · ·nnndenote the lengths of an instantaneous/uniquely decipherable
code formed of code alphabet of size D.
i
L∗P, N; 1, α, β≥HP; 1, α, β, 3.35
with equality if and only ifni−logDpifor alli,
ii
with equality if and only if
ni−αlogDpilogD
n
i1
pα
i
, 3.37
whereα 1t−1.
Proof. As in Shannon’s case, refer Feinstien6,
n
i1
pini≥ −
n
i1
pilogDpi. 3.38
NowD1−β/α−1≷0 according asβ≶1.
Therefore, from the above after suitable manipulation, we get the inequality
D1−β/αni1pini−1
D1−β/α−1 ≥
Dβ−1/αni1pilogDpi−1
D1−β/α−1 , β /1, 3.39
which is the result3.35.
The case of equality can be discussed, as for Shannon’s, which holds only whenni −logDpi, for eachi.
We now proceed to partii. Ift0, the result is the same proved in parti. For other values, we use Holder’s inequality
n
i1
xpi
1/p
·
n
i1
yqi
≤xiyi, 3.40
wherep−1q−11 andp <1.
Making the substitutionsp−t, q 1−α, xip−1/ti D−niandyip1/t
i in3.40, we
get after suitable manipulations
n
i1
piDtni
1/t
≥
n
i1
pα
i
1/1−α
3.41
withα 1t−1
Raising the power to1−β/αof both sides of3.41and using thatD1−β/α−1≷0 according asβ≶1, we get result3.36after simple manipulation.
Hence, the theorem is proved.
References
1 S. Kumar, “Some more results on R-norm information measure,” Tamkang Journal of Mathematics, vol.
2 L. L. Campbell, “Definition of entropy by means of a coding problem,” vol. 6, pp. 113–118, 1966.
3 L. L. Campbell, “A coding theorem and Renyi’s entropy,” Information and Computation, vol. 8, pp. 423–
429, 1965.
4 G. H. Hardy, J. E. Littlewood, and G. Polya, Inequalities, Cambridge University Press, New York, NY,
USA, 1952.
5 C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27,
pp. 379–423, 623–656, 1948.
Submit your manuscripts at
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Mathematics
Journal ofHindawi Publishing Corporation
http://www.hindawi.com Volume 2014
in Engineering
Hindawi Publishing Corporation http://www.hindawi.com
Differential Equations International Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Probability and Statistics
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014 Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Mathematical PhysicsAdvances in
Complex Analysis
Journal of Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Optimization
Journal ofHindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Combinatorics
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
International Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Operations Research
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Function Spaces
Abstract and Applied Analysis Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporation http://www.hindawi.com Volume 2014
The Scientific
World Journal
Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Decision Sciences
Discrete Mathematics
Journal ofHindawi Publishing Corporation
http://www.hindawi.com Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014