• No results found

Some Coding Theorems for Nonadditive Generalized Mean Value Entropies

N/A
N/A
Protected

Academic year: 2020

Share "Some Coding Theorems for Nonadditive Generalized Mean Value Entropies"

Copied!
11
0
0

Loading.... (view fulltext now)

Full text

(1)

International Journal of Mathematics and Mathematical Sciences Volume 2012, Article ID 315686,10pages

doi:10.1155/2012/315686

Research Article

Some Coding Theorems for Nonadditive

Generalized Mean-Value Entropies

Satish Kumar,

1

Gurdas Ram,

2

and Vishal Gupta

3

1Department of Mathematics, Geeta Institute of Management and Technology, Kurukshetra,

Kanipla 136131, India

2Department of Applied Sciences, Maharishi Markandeshwar University, Solan 173229, India

3Department of Mathematics, Maharishi Markandeshwar University, Mullana 133203, India

Correspondence should be addressed to Satish Kumar,drsatish74@rediffmail.com

Received 8 July 2011; Accepted 7 December 2011

Academic Editor: Andrei Volodin

Copyrightq2012 Satish Kumar et al. This is an open access article distributed under the Creative

Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We give an optimality characterization of nonadditive generalized mean-value entropies from suit-able nonadditive and generalized mean-value properties of the measure of average length. The results obtained cover many results obtained by other authors as particular cases, as well as the

ordinary length due to Shannon 1948. The main instrument is thelnifunction of the word lengths

in obtaining the average length of the code.

1. Introduction

Given a discrete random variableXtaking a finite number of valuesx1, x2, . . . , xnwith prob-abilities P p1, p2, . . . , pn,pi ≥ 0,ni1pi 1, the Shannon’s entropy of the probability

distribution is given by

HP n

i1

pilogpi, 1.1

where the base of the logarithm is in general arbitrary.

(2)

that is encoded into words of lengthsN {n1, n2, . . . , nn} forming an instantaneous code, then

n

i1

D−ni 1, 1.2

whereDis the size of the code alphabet.

The average lengthL, for the instantaneous code is such that

Lpini≥ −n

i1

pilogDpi 1.3

with equality if and only if for eachi

piD−ni. 1.4

Result1.4,refer Shannon5, characterizes Shannon’s entropy as a measure of opti-mality of a linear function, namely,L, under the relation1.2.

Several generalizations of Shannon’s entropy have been studied by many authors in different ways. Here we will need the nonadditive generalized mean value measures such as

Hp1, p2, . . . , pn; 1, α, βDβ−1/α−1−1Dβ−1/αni1pilogDpi1 , α, β >0, β /1,

1.5

Hp1, p2, . . . , pn;α, β

D1−β/α1−1

⎡ ⎣

n

i1

i

1/αβ−1/α−1

−1

⎤ ⎦,

β >0, β /1, α >0, α /1; α /β.

1.6

These quantities satisfy the “nonadditivity”

HPQ HP HQ D1−β/α1HPHQ. 1.7

(3)

2. Nonadditive Measure of Code Length

Let us consider two independent sourcesX {x1, x2, . . . , xn}andY {y1, y2, . . . , ym}with

associated probability distributionP p1, p2, . . . , pnandQ q1, q2, . . . , qm. Then the

prob-ability distribution of the productXY {xi, yi}isPQ p1q1, p1q2, . . . , p1qm, . . . , pnqm. Let the sourceYbe encoded with a code of lengthM{m1, m2, . . . , mm}and the pairxi, yj

be represented by a sequence forxi andyj put side by side, so that the product source has

code length sequence:

NM{n1m1, . . . , n1mn, . . . , nnmm}. 2.1

The additive measure of mean length is required to satisfy the requirement, refer Campbell

2,

LPQ, NM, φLP, N, φLQ, M, φ, 2.2

where

LP, N, φφ−1

n

i1

piφni

, 2.3

φbeing a continuous strictly monotonic increasing function.

Campbell 3proved a noiseless coding theorem for Renyi’s entropy of order α in terms of mean length2.3defined forφni Dni.

The mean length concerning Shannon’s entropy and orderαentropy of Renyi are both additive as they satisfy additivity of type2.2.

Here we deal with nonadditive measures of length denoted byL∗which satisfy “non-additivity relation”

LPQ, NM, φLP, N, φLQ, M, φλLP, N, φLQ, M, φ,

2.4

and the mean value property

LP, N, φφ−1ni1piφlni

n

i1pi

, 2.5

wherelniis the function of length of a single element with code word length ni, which is

nonadditive.

3. Characterization of Nonadditive Measures of Code Length

(4)

First of all we will determine the nonadditive length functionlof the code word length in satisfying the nonadditivity relation

lnm ln lm λlnlm, λ /0. 3.1

This by takingfn 1λlngives

fnm fnfm. 3.2

The most general nonzero solution of3.2is

fn D1−β/αn, 3.3

whereα >0,β /1 are arbitrary constantswe have taken baseDwith a purpose here. So that

ln D1−β/αnλ −1, λ /0, 3.4

at this stage we make a proper choice of the constantλ. By analogy, its value is dictated as

λD1−β/α1, β /1 asλ /0. 3.5

Another purpose served with this value ofλis that when it tends to zero, that is, when

β → 1, the function of lengthlnshould reduce to additive one which isln n. This value ofλcan be obtained by imposing a boundary conditionl1 1 also.

So that finally we have

ln D1−β/αn−1

D1−β/α1 , β /1, α >0. 3.6

Next we proceed to determineL∗P, N, φby first evaluating the values ofφ. To achieve this we put the value oflnfrom3.6in2.5and then use the relation2.4withλ D1−β/α− 1,β /1 to get

φ−1

n

i1mj1piqjφ

D1−β/αnimj1/D1−β/α−1

n

i1

m

j1piqj

φ−1

n

i1piφD1−β/αni−1/D1−β/α−1

n

i1pi

φ−1

m

j1qjφ

D1−β/αmj1/D1−β/α−1

m

j1qj

(5)

D1−β/α1φ−1

n

i1piφD1−β/αni−1/D1−β/α−1

n

j1pi

×φ−1

m

j1qjφD1−β/αmj−1/D1−β/α−1

m

j1qj

.

3.7

Now let us takeQ{q}andM{m}so that forP p1, p2, . . . , pn, pi≥0, ni1pi1,3.7

gives after some simplification

φ−1

n

i1

piφ

D1−β/αnim−1

D1−β/α1

φ−1

n

i1

piφ

D1−β/αni1

D1−β/α1

D1−β/αmD1−β/αm−1

D1−β/α1 ,

3.8 or ψ−1 m n i1 piψm

D1−β/αni1

D1−β/α1

φ−1 n i1 piφ

D1−β/αni1

D1−β/α1

D1−β/αmD1−β/αm−1

D1−β/α1 ,

3.9

where

ψm

D1−β/αni1

D1−β/α1

φ

D1−β/αnim−1

D1−β/α1

. 3.10

Now, refer Hardy et al.4, there must be a linear relation betweenψmandφ, that is,

ψmn Amφn Bm, 3.11

whereAmandBmare independent ofn. Using3.10and3.11, we have

gnm Amgn Bm, 3.12

where

gn φ

D1−β/αn1

D1−β/α1

(6)

or

Gnm AmGn Gm, 3.14

where

Gn gng0 gna, ag0. 3.15

From the symmetry of3.14, we get

AmGn Gm AnGm Gn

AGn

n−1

Gm

Am−1

1

K

say. 3.16

Thus,

An−1KGn 3.17

for all real values ofn.

There are two cases, namely,K0 andK /0. IfK0,An 1, and3.14gives

Gnm Gn Gm, 3.18

the most general continuous solution of which is given by

Gn cn, 3.19

wherecis an arbitrary constant. This, by3.15and3.13, gives

φ

D1−β/αn1

D1−β/α1

acn 3.20

which gives

φn a

1−βlogD

1D1−β/α−1n , β /1. 3.21

Again ifK /0, we have the relation obtained from3.17and3.14:

(7)

the general continuous solution of which are

An 0 which we neglectfor alln,

An Dtn, 3.23

wheretis an arbitrary nonzero constant.

This by using3.17,3.15in3.13, gives

φ

D1−β/αn1

D1−β/α1

aDtnK−1 3.24

which gives

φn a

D1−β/α1n1tα/1−β1

K , β /1, t /0, α >0. 3.25

The value ofφgiven by3.21and3.25determine the following two nonadditive measures of length defined by2.5, that is,

LP, N; 1, α, βD1−β/α1−1D1−β/αni1pini1 , 3.26

LP, N;t, α, βD1−β/α1−1D1−β/αtlogDni1piDnit1 . 3.27

These code lengths denoted byL∗P, N; 1, α, βandL∗P, N;t, α, βmay be named as nonad-ditive typeα, βlengths of order 1 andt, respectively.

These results are contained in the following theorem.

Theorem 3.1. The mean length given by2.5of a sequence of lengthsni, i 1,2, . . . , nformed of

the code alphabet of sizeDof a probability distributionP p1, p2, . . . , pn, pi ≥0, ni1pi1

sat-isfyingni1D−ni1 and nonadditivity relation can be only of one of the two forms given in3.26

and3.27.

3.1. Limiting and Particular Cases

It is immediate to see the following.

1

lim

t→0L

P, N;t, α, βLP, N; 1, α, β. 3.28

2

lim

β→1L

P, N; 1, α, βn

i1

piniL, 3.29

(8)

3

lim

β→1L

P, N;t, α, β 1

tlog

n

i1

piDtni, 3.30

length of ordertdefined by Campbell2.

4Forn1 n2· · ·nn nsay, both the expressions for length given by3.26and

3.27reduce to

D1−β/α1−1D1−β/αn1 , 3.31

which in the limiting case, whenβapproaches unity, reduces ton.

5Forαβ, the expression3.26becomes

LP, N; 1, α D1−α/α1−1D1−α/αni1pini1 . 3.32

6Forαβ, the expression3.27becomes

LP, N;t, α D1−α/α1−1D1−α/αtlogDni1piDnit1 . 3.33

7For α → 1 the expression 3.32 and 3.33 reduces to ni1pini L and

1/tlogni1piDtni, respectively.

Thus, we have shown thatL∗P, N; 1, α, βandL∗P, N;t, α, βare typeαandβ gen-eralizations of

Ln

i1

pini, Lt 1tlogD

n

i1

piDtni, 3.34

respectively. We now prove the following theorem.

Theorem 3.2. Ifn1n2· · ·nnndenote the lengths of an instantaneous/uniquely decipherable

code formed of code alphabet of size D.

i

LP, N; 1, α, βHP; 1, α, β, 3.35

with equality if and only ifni−logDpifor alli,

ii

(9)

with equality if and only if

ni−αlogDpilogD

n

i1

i

, 3.37

whereα 1t−1.

Proof. As in Shannon’s case, refer Feinstien6,

n

i1

pini≥ −

n

i1

pilogDpi. 3.38

NowD1−β/α−1≷0 according asβ≶1.

Therefore, from the above after suitable manipulation, we get the inequality

D1−β/αni1pini1

D1−β/α1

Dβ−1/αni1pilogDpi1

D1−β/α1 , β /1, 3.39

which is the result3.35.

The case of equality can be discussed, as for Shannon’s, which holds only whenni −logDpi, for eachi.

We now proceed to partii. Ift0, the result is the same proved in parti. For other values, we use Holder’s inequality

n

i1

xpi

1/p

·

n

i1

yqi

xiyi, 3.40

wherep−1q−11 andp <1.

Making the substitutionsp−t, q 1−α, xip−1/ti D−niandyip1/t

i in3.40, we

get after suitable manipulations

n

i1

piDtni

1/t

n

i1

i

1/1−α

3.41

withα 1t−1

Raising the power to1−β/αof both sides of3.41and using thatD1−β/α−1≷0 according asβ≶1, we get result3.36after simple manipulation.

Hence, the theorem is proved.

References

1 S. Kumar, “Some more results on R-norm information measure,” Tamkang Journal of Mathematics, vol.

(10)

2 L. L. Campbell, “Definition of entropy by means of a coding problem,” vol. 6, pp. 113–118, 1966.

3 L. L. Campbell, “A coding theorem and Renyi’s entropy,” Information and Computation, vol. 8, pp. 423–

429, 1965.

4 G. H. Hardy, J. E. Littlewood, and G. Polya, Inequalities, Cambridge University Press, New York, NY,

USA, 1952.

5 C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27,

pp. 379–423, 623–656, 1948.

(11)

Submit your manuscripts at

http://www.hindawi.com

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Mathematics

Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

in Engineering

Hindawi Publishing Corporation http://www.hindawi.com

Differential Equations International Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Probability and Statistics

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014 Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Mathematical PhysicsAdvances in

Complex Analysis

Journal of Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Optimization

Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Combinatorics

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

International Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Operations Research

Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Function Spaces

Abstract and Applied Analysis Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporation http://www.hindawi.com Volume 2014

The Scientific

World Journal

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Decision Sciences

Discrete Mathematics

Journal of

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Hindawi Publishing Corporation

http://www.hindawi.com Volume 2014

Stochastic Analysis

References

Related documents

Researchers have found that people’s conceptualizations of the nature of humans and machines matter for how they make sense of and interact with technology (e.g., A. Edwards, 2018

Conclusions: Postoperative hypertension requiring IVMED after CEA is associated with increased perioperative mortality, stroke, and cardiac complications, whereas

Steele of Stanford University Online High said that shift is already underway: In a recent survey of ASCA members, she and her colleagues found that more than one-fourth of

Charge amplifier systems benefit from the very wide dynamic range of PE accelerometers by offering flexibility in adjusting the electrical output characteristics such as sensitivity

In this way, ownership economises on transaction costs by avoiding market contracting for whichever “patron” is assigned control rights. If it were not for CO, it

Commonly agreed, CSR is thus viewed as the relationship between a business organization and the society it operates in (consisting of the organization’s

Porter and Kramer (2006) emphasize the interrelationship between a corporation and society in CSR value chains and identify two forms of inside-out linkages and