The cross entropy or Patterson entropy as it is sometimes referred to, uses a criterion in which the entropy measure
E = −X i filn fi wi
is maximized where wi is some weighting function based on any available a priori information of the structure of fi. If the computation described earlier is re-worked using this definition of the cross entropy, then we obtain the result
fi = wiexp[−1 + 2λ(si pi− pi⊗ fi pi)].
The cross entropy method has a synergy with the Wilkinson test in statistics in which a discrete PDF or histogram Pi say of a stochastic field pi is tested against a histogram Qi representative of a stochastic field qi. A standard test to quantify how close the stochastic behaviour of pi is to qi (the null-hypothesis test) is to use the Chi-squared test in which we compute
χ2=X i Pi− Qi Qi 2 . The Wilkinson test uses the metric
E = −X i Piln Pi Qi . Further reading includes [111]-322.
3
Data Encryption Algorithms and Standards
The majority of cryptosystems are based on a series of ‘round transformations’ which are ‘driven’ by some iteration function. The role of this function is to transform information (the plaintext) into an unpredictable form (plaintext) which is uniquely dependent upon a particular ‘key’ as illustrated in Figure 11.
Figure 11: A basic cryptosystem transforms plaintext (input) to ciphertext (output) using a known iteration function and a key.
The iteration functions used in many cryptosystems have a unique design which is often based on a historical lagacy determined by performance criteria, applica- tions dependency, up-grades and modifications, user profile and so on. However, in general, the design of such iteration functions have a common goal in terms of their ability to output data that is unpredictable, data that is commonly referred to as ‘noise’.
In any application involving the detection and measurement of a signal, the re- sulting data must always be assumed have errors ideally within some acceptable tolernce. Such errors are the result of the natural noise that accompanies and, in some cases, characterises the detection of a signal. Thus, the analysis of natural noise and its simulation holds an important place in many areas of science and engineering. There are numerous techniques used to quantify natural noise but of specific importance is the use of the Probability Density Function (PDF) and the Power Spectrum (PS). These functions provide information on the probabilities of occurance of data (histogram for discrete data) and its frequency spectrum (discrete Fourier transform for discrete data) respectively. However, apart from some special- ist applications which use natural noise generated through radioactive, for example, most cryptosystem are based on simulated noise that is a characteristic of the itera- tion function used, i.e. data that is not random but pseudorandom. In otherwords, in cryptography, we have control (through the design of the iteration function) over the type of noise that is produced to covert plaintext to ciphertext, a ‘control’ that is, of course, necessary to recover the plaintext in a key dependent way. This control
(the iteration function(s) that is used) must be exercised in a way that produces a cryptographically secure ciphertext, i.e. a ‘control’ that transforms the plaintext to ciphertext with maximum possible diffusion (1 bit of the key influences all bits of the ciphertext) and confusion (ciphertext stream is uniformly distrbuted).
There are a wide range of PDFs and PSDFs that characterise a noise field and numerous physical models have evolved for the puspose of simulating such fields. Fig- ure 12 shows examples of simulated signals together with the histograms and power spectra for three different (discrete) noise fields (uniformly distributed, normally (Gaussian) distributive and fractal noise) that are illustrative of their diversity. The Figure also includes an example of a (discrete) chaotic signal that, by comparison with the noise signals, has an descernable regularity. In this sense, chaotic signals are not the same as noise signals and must be interpreted differently. The origin of chaotic fields and their interpretation is considered later after first introducing the simulation of noise through the computation of pseudorandom number streams.
3.1 Pseudo Random Number Generators
The security of a number of cryptographic algorithms depends on the generation of unpredictable random numbers [128] [143], even though it is difficult to generate a truly random number generator using software-based algorithms [146].
Good random number generators enhance the strength of cryptography and many different methods of generating random numbers have been developed for this purpose. An interesting and relatively simple method is called the Diceware passphrase [134]. In this method, a list of words is generated and each word num- bered. The numbers are generated from an dice, which acts as a random number generator, and are assembled as a five digit number, e.g. 43146. This number is then used to look up a word in a word list. A major advantage of the Diceware approach is that the level of unpredictability in the passphrase can be easily calculated. Each diceware word adds 12.9 bits of information entropy to the passphrase, i.e. log2(65) bits where five words (slightly over 64 bits) are considered a minimum length.
The best random numbers are created by harnessing natural physical processes, such as radioactive decay, which is known to exhibit truly random behaviour [123]. Emissions may be detected in rapid succession or with relatively long delays between emissions, delays that are unpredictable and random [141]. An emission detector cycles through the alphabet at a fixed rate and outputs a letter when an emission is detected. The cycle then continues until the next emission is detected providing another randomly selected letter and so on, a process generates genuinely random numbers. For example, HotBit [144] random numbers are generated using a radiation source involving beta decay. A user contacts the server, where upon the output can be downloaded over the web. The random numbers provided by HotBits are ideal for stream cipher. However, because they are not generated by some key dependent encryption algorithm, the entire random number stream needs to be exchanged
between sender and receiver rather then the key itself.
The term ‘random’ must be used loosely because software based random number generators as used in cryptography are basically pseudorandom, i.e. simulations of random processes at best. A pseudorandom generator is a deterministic algorithm that expands short random seeds into much longer bit sequences that appear to be random. In other words, although the output of a pseudorandom generator is not really random, there is no easy method of telling the difference [125]. The better the pseudorandom number generator, the better the design of an encryption engine. [142] In turn, most generators used for encryption exploit the properties of prime numbers and hence are prime number dependent, hence the importance of prime numbers in applied cryptography.
The term ‘random’ must be used loosely because software based random number generators as used in cryptography are basically pseudo-random, i.e. simulations of random processes at best. In other words, although the output of a pseudo-random generator is not really random, there is no easy method of telling the difference [125]. The better the pseudo-random number generator, the better the design of an encryption engine [142]. In turn, most generators used for encryption exploit the properties of prime numbers and hence, are prime number dependent. This is the principal reason behind, the importance of prime numbers in applied cryptography. A Pseudo Random Number Generator (PRNG) is an algorithm that output a discrete array of numbers that appear to be random, a randomness that is quan- tifiable in terms of a statistical distribution. However, PRNGs do not produce real random numbers because they do not have to - hence the use of the work Pseudo. Most simple applications, such as in computer games or in the application of Monte- Carlo simulations, for example, need relatively few random numbers to be effective. Nevertheless, the use of a poor random number generators can lead to results that are compounded in terms of spurious correlations. Indeed, some PRNGs do not nec- essarily produce anything that looks even remotely like natural random sequences. However, with some careful design contraints, they can be made to approximate such sequences, approximations can be applied to produce noise fields that are used to ‘confuse’ plaintext. A fundamental issue concerning PRNGs is that a digital com- puter can only be in a finite number of states (a large finite number, but a finite number nonetheless), and the data that is output can only be a deterministic func- tion of the input data and the current state of the digital computer. This means that any PRNG on a computer (at least, on a finite-state machine) is, by definition, periodic and thus, all PRNGs are cyclical. Anything that is periodic is, by definition, predictable and can not therefore be classed as truely random. Thus, the best that a digital computer can produce is a pseudorandom sequence or series, i.e. a sequence of numbers that ‘looks’ random - the output of PRNG. The period of the sequence should be long enough so that a finite sequence of reasonable length (i.e. one that is actually used) is not periodic. If, for example, a billion random bits are required, then a random bit sequence generator should not be designed that repeats after only
Figure 12: Examples of 100 element noisy digital signals (left) together with the characteristic 16-bit histograms (centre) and logarithmic power spectra (right). The types of noise are (from top to bottom) uniform, Gaussian or normally distributed, fractal noise and a chaotic signal.
sixteen thousand bits. These relatively short non-periodic sequences should be as statistically indistinguishable as possible from real random sequences. In addition, they should not be compressible, e.g. the distribution of run lengths (a sequence containing the same bit type) for 0s and 1s should be the same. These properties can be empirically measured and then compared with statistical expectations.
In practice, a number sequence generator is pseudorandom if it looks random and passes all the statistical tests of randomness that can be found and imple- mented in practice. Considerable effort has and continues to go into producing good pseudorandom sequences on a computer. Discussions of generators abound in the literature, along with various tests of randomness. However, all of these generators are periodic - there is no exception. However, with potential periods of 2256bits and higher, they can be used for the largest applications. The problem with all pseu- dorandom sequences is the correlations that result from their inevitable periodicity. Every pseudorandom sequence generator will produce them if the sequence is long enough.
Within a cycle, a pseudorandom sequence must have the property that it is unpredictable. In other words, it must be computationally impossible to predict what the next random bit will be, given complete knowledge of the algorithm or hardware generating the sequence and all of the previous bits in the stream. Thus, a pseudorandom sequence is really random if, in addition to looking random and passing all known statistical tests for randomness, it has another additional property which is that it cannot be reliably reproduced. For example, if the sequence gener- ator is run twice with the exactly same inputs (at least as exact as computationally possible), then the sequences are completely unrelated, i.e. their cross correlation functions are effectively zero. However, this property is not usually possible to produce on a finite state machine and for some applications of random number se- quences, is not desirable, one of the most important example being cryptography. In crytography, for example, it is essential that we are able to reproduce exactly the same random number sequence from the same key in order to decrypt a ciphertext. Thus, we refer to those processes that produce number streams which look random (and passes appropriate statistical tests) and are unpredictable but reproducable as Pseudo Random Number Generators. Pseudorandom numbers are therefore not numbers generated by a random process but are numbers generated by a completely deterministic arithmetic process that is based on the execution of a given algorithm.