Digital Theory
5.6 What Is a Bit of Data?
The transmission of communications signals is accomplished by means of a transmission of energy, generally of electromagnetic or of acoustic energy.
In contrast to the case of power trans-mission, it is not energy itself which is of interest, but rather the changes in this energy in the course of time. The more
complicated the function which repre-sents, as a function of time, the change in voltage, current, pressure, or any other carrier, the greater is the amount of information carried by the trans-mitted energy.(J. Ville)
Communications theory has up to now developed mainly on mathematical lines, taking for granted the physical significance of the quantities which figure in this formalism. But communi-cation is the transmission of physical effects from one system to another, hence communication theory should be considered as a branch of physics. Thus it is necessary to embody in its founda-tions such fundamental physical data as a quantum of action, and the discrete-ness of electrical charges. This is not only of theoretical interest. With the progress of electrical communications toward higher and higher frequencies we are approaching a region in which quantum effects become all-important.
Nor must one forget that vision, one of the most important paths of communica-tion, is based essentially on quantum effects. (D. Gabor)
Reading from the book The Mathematical Theory of Communication by Claude E Shannon:
If the number of messages in the set is finite then this number or any mono-tonic function of this number can be regarded as a measure of the informa-tion produced when one messages chosen is from the set, all choices being equally likely. As was pointed out by Hartley the most natural choice is the logarithmic function. Although this defi-nition must be generalized considerably when we consider the influence of the statistics of the message and when we have a continuous range of messages, we will in all cases use an essentially logarithmic measure.
The logarithmic measure is more convenient for various reasons:
1. It is practically more useful. Parameters of engi-neering importance such as time, bandwidth, number of relays, etc., tend to vary linearly with the logarithm of the number of possibilities. For example, adding one relay to a group doubles the number of possible states of the relays. It adds 1 to the base 2 logarithm of this number. Doubling
Hexadecimal base 16, log16 MSB most significant bit LSB least significant bit
Bit rate bits × sampling frequency × channels SNR signal-to-noise ratio, 10 × log10 (6 × (2(b − 2)) FLOP floating point operations per second MIPS million instructions per second MIPW million instructions per watt RISC reduced instruction set computer
WAV wave format
ASCII American Standard Code for Information Interchanges
CD compact disc
DVD digital versatile disc MPEG motion picture experts group
ISO International Organization for Standardization AAC advanced audio coding
MDCT modified discrete cosine transform TNS temporal noise shaping
V ⁄ 2bits Quantization level DSP digital signal processor ADC analog to digital converter DAC digital to analog converter ERB equivalent rectangular bandwidth HTML hypertext markup language
BAUD a rate defined as Xbaud × Y bits ⁄ baud = Z bits/s. (Baud is changes of state/s) Named after French engineer Jean Maurice Emile Baudot.
Table 5-1. (cont.) Digital Nomenclature
44 Chapter 5 the time roughly squares the number of possible messages, or doubles the logarithm, etc.
2. It is nearer to our intuitive feeling as to the proper measure. This is closely related to (1) above, since we intuitively measure entities by a linear comparison with the common standards.
One feels, for example, that two punch cards should have twice the capacity of one for infor-mation storage, and two identical channels twice the capacity of one for transmitting information.
3. It is mathematically more suitable. Many of the limiting operations are simple in terms of the logarithm but would require clumsy restatement in terms of the number of possibilities.
The choice of a logarithmic base corresponds to the choice of a unit for measuring information. If the base 2 is used, the resulting units may be called binary digits, or more briefly bits, a word suggested by J. W. Tukey. A device with two stable positions, such as a relay or a flip-flop circuit can store one bit of information. N such devices can store N bits, since the total number of possible states is 2N and log22N = N. If the base 10 is used the units may be called decimal digits. Fig. 5-6 gives the definition of binary.
S i n c e , a
decimal digit is about 31⁄3bits. A digital wheel on a desk computing machine has ten stable positions and therefore has a storage capacity of one decimal digit. In analytical work where integration and differentiation are involved, the base e is sometimes useful. The resulting units of information will be called natural units (Nats). Change from the base a to base b merely requires multiplication by logba.
Mathematically the number of bits is defined by:
(5-14) and
2Bits = P (5-15)
where,
P is the number of possibilities.
5.6.1 The Physical Dimensions of One Bit One Nat of information has the area of a square exactly two Planck lengths on a side.
One Nat equals 0.693 bits, One Planck length equals
(5-16)
where,
G is the gravitational constant,
is the reduced Planck constant, where C is the speed of light.
Planck area equals
The simplest numbering scheme possible, there are only two symbols:
1 and 0 B. Logically:
A system of thought in which there are only two states:
True and False
C. Binary information is not subject to misinterpretation:
Black White In Out Guilty Innocent D. Variables or non-binary terms:
Somewhat Undecided
Digital Theory 45 the length of one side
of the bit square
the length of one side of the Nat square (5-20)
(5-21)
bits per Nat. (5-22)
5.6.2 Reading Binary Numbers
Table 5-2, Binary to Decimal to Hexadecimal to Octal allows you to see the essential simplicity of
binary coding. Pick any decimal number and see how the 1s add up on the exponential scales at the top of the chart. For example the decimal number 27 in binary is found to be 16+8+2+1 =27. The hexa-decimal number 1B is in the second rotation through the hexadecimal encoding, and octal number 33 is in the third rotation for octal encoding, see Table 5-2.
Fortunately today many scientific calculators include easy conversions from decimal to octal to hexadecimal to binary code. Children can bring this kind of coding home from school for your help in solving their problems in high school mathematics classes. Cryptography is replete with many system bases including the Ban that was used in World War II by the English code breakers, see Fig. 5-7 defining Bits, Nats, and Bans. Digital test equipment is expensive can be complex and requires training in its use. We increasingly see requirements in specifi-cations demanding certification of the engineers setting these systems into operation.
1 bit = 3.88263 10× –35m
1 Nat = 3.232506 3.88263( ) 10× –35m
bitP
l
NatP
l
--- = 1.201122 2P× l 2.402244 Pl
=
2.4Pl
( )2
2Pl2
--- = 0.693
Table 5-2. Binary to Decimal to Hexadecimal to Octal Binary
MSB LSB
25 (32) 24 (16) 23 (8) 22 (4) 21 (2) 20 (1) Decimal Hex Octal
0 0 0 0 0 0 0 0 0
0 0 0 0 0 1 1 1 1
0 0 0 0 1 0 2 2 2
0 0 0 0 1 1 3 3 3
0 0 0 1 0 0 4 4 4
0 0 0 1 0 1 5 5 5
0 0 0 1 1 0 6 6 6
0 0 0 1 1 1 7 7 7
0 0 1 0 0 0 8 8 10
0 0 1 0 0 1 9 9 11
0 0 1 0 1 0 10 A 12
0 0 1 0 1 1 11 B 13
0 0 1 1 0 0 12 C 14
0 0 1 1 0 1 13 D 15
0 0 1 1 1 0 14 E 16
0 0 1 1 1 1 15 F 17
0 1 0 0 0 0 16 10 20
0 1 0 0 0 1 17 11 21
0 1 0 0 1 0 18 12 22
0 1 0 0 1 1 19 13 23
0 1 0 1 0 0 20 14 24
0 1 0 1 0 1 21 15 25
0 1 0 1 1 0 22 16 26
0 1 0 1 1 1 23 17 27
0 1 1 0 0 0 24 18 30
46 Chapter 5
5.6.3 Text into Binary, Octal, Hexadecimal
ASCII printable characters
Codes 32–127 are common for all the different vari-ations of the ASCII table; they are called printable characters and represent letters, digits, punctuation marks, and a few miscellaneous symbols. ASCII control characters number from 0–31 and are unprintable control codes that are used to control peripherals such as printers, Table 5-3.
The ASCII code of a character is found by combining its Column Number (given in 3-bit binary) with its Row Number (given in 4-bit binary).
The Column Number forms bits 6, 5 and 4 of the ASCII, and the Row Number forms bits 3, 2, 1 and 0 of the ASCII.
Example of use: to get ASCII code for letter “n”, locate it in Column 110, Row 1110. Hence its ASCII code is 1101110.
The Control Code mnemonics are given in italics above; e.g. CR for Carriage Return, LF for Line Feed, BELL for the Bell, DEL for Delete. The Space is ASCII 0100000, and is shown as ◊ here.
To write Don Davis in binary, octal, and hexadec-imal, refer to Table 5-3.
The text, Don Davis, in binary code is
01000100 01101111 01101110 00100000 01000100 01100001 01110110 01101001 01110011.
The text, Don Davis, in octal code is 104 157 156 040 104 141 166 151 163.
The text, Don Davis, in hexadecimal code is 44 6f 6e 20 44 61 76 69 73.
The binary code is called “machine language”
inasmuch as it is the code the computer understands.
The compactness that base 8 and base 16 offers over base 2 for use in program editors that then convert these more compact codes into machine language is apparent.
Sound System Engineering in binary code is 01010011 01101111 01110101 01101110 01100100 00100000 01010011 01111001 01110011 01110100 01100101 01101101 00100000 01000101 01101110 01100111 01101001 01101110 01100101 01100101 01110010 01101001 01101110 01100111.
In octal code is
123 157 165 156 144 040 123 171 163 164 145 155 040 105 156 147 151 156 145 145 162 151 156 147.
In hexadecimal code is
53 6f 75 6e 64 20 53 79 73 74 65 6d 20 45 6e 67 69 6e 65 65 72 69 6e 67.
0 1 1 0 0 1 25 19 31
0 1 1 0 1 0 26 1A 32
0 1 1 0 1 1 27 1B 33
0 1 1 1 0 0 28 1C 34
0 1 1 1 0 1 29 1D 35
0 1 1 1 1 0 30 1E 36
0 1 1 1 1 1 31 1F 37
0 1 0 0 0 0 0 32 20 40
MSB most significant bit LSB least significant bit Table 5-2. (cont.) Binary to Decimal to Hexadecimal to Octal
Table 5-3. ASCII Code Row
Number
Column Number
000 001 010 011 100 101 110 111
0000 NUL DLE à 0 @ P ` p
0001 SOH DC1 ! 1 A Q a q
0010 STX DC2 " 2 B R b r
0011 ETX DC3 # 3 C S c s
0100 EOT DC4 $ 4 D T d t
0101 ENQ NAK % 5 E U e u
0110 ACK SYN & 6 F V f v
0111 BELL ETB ' 7 G W g w
1000 BS CAN ( 8 H X h x
1001 HT EM ) 9 I Y i y
1010 LF SUB 0 : J Z j z
1011 VT ESC 0 ; K [ k {
1100 FF FS , < L \ l |
1101 CR GS - = M ] m }
1110 SO RS . > N ^ n ~
1111 SI US / ? O _ o DEL
Table 5-3. (cont.) ASCII Code
Digital Theory 47