Linear systems - A linear systems toolbox

A linear systems toolbox

2.2 Linear systems

2.2.1 First ideas

Imagine one day climbing the bell tower of a church and, having gained the belfry, you softly tap the lip of a large bell with a small hammer you have been carrying in your backpack. The bell emits a deep, resonant ‘clang’ and you leave, satisfied with your work. The next day, you repeat the exercise and find that the bell performs in exactly the same way. The day after that, you hit the bell slightly harder. You note an increase in the sound level, but the quality or timbre of the sound remains the same.

The day after that, you hit it so hard that the iron cracks, the note produced is horrible, and you are arrested and charged with trespassing and causing criminal damage.

There are several lessons to be drawn from this exercise in semi-quantitative campanology (apart from the obvious one about breaking and entering): the church bell is, to a good approximation, an example of a physical linear system. Linear

systems possess certain very important properties that are central to the subject of DSP, and a good understanding of them, and their ramifications, is essential for anyone embarking on a study of this kind. To return to the matter of the bell for a moment, we found that when it was struck in the same way on different days, it responded in the same manner. This is a property called temporal invariance. This is one of the most significant features of linear systems, because it guarantees that the response will be the same at different times. Second, if we assume a gap of 24 h between the first and second strike, then we know that the bell has not only completed one rotation around the earth’s axis, but it has also moved around the sun (along with the tower and the rest of the planet) at a speed of approximately 107,000 km/h. Like all true linear systems, the bell manifests spatial invariance, and this property is once again pivotal to both their understanding and their manipulation.

Another property you will have noted is that the harder you hit the bell, the louder the sound became, up to the point at which the bell cracked. Thus there was a law of proportionality at work, until the input energy became so high that the deformation it induced exceeded the elastic limit of the material. Only when this happened did the system’s response become non-linear. There are endless examples of physical systems that are linear (or very nearly so) in science, engineering and the natural world. These include most acoustic musical instruments, audio amplifiers, ultrasonic transducers, electronic filters, the acoustic properties of a room, pendulums, springs and suspension systems, to name but a very few. Note that we put in the caveat or very nearly so, because all of them are associated with small non-linearities that can for the most part be ignored when describing their behaviours. Just for interest, Figure 2.1 shows Fourier spectra resulting from plucking the top (E) string on an acoustic guitar, which has a fundamental at 329.6 Hz. The sound was recorded at a sample rate of 22.05 kHz with 16-bit resolution. Although each spectrum was calculated over a 0.2 s range, part (a) was taken right at the start of the note and part (b) was taken some 6 s later. Notice how part (a) is far richer in harmonics, indicating how we often identify musical instruments from the initial transient behaviour.

But a linear system need not necessarily be a physical thing – it could, for example, be conceptual, such as in the case of a mathematical process or transform. As long as the transform conforms to the properties that define linear systems, then it can be analysed in the same way. We can go further and express the linear system (either physical or conceptual) as a computer algorithm, which is when things start to get very interesting indeed. In fact, algorithms of this nature, when coded correctly, are extraordinarily powerful, because they allow us to dispense entirely with the physical system, and replace it with computer code. But what about the input and output signals? Simple. These are expressed as sets of numbers. The input signals can either be synthesised, or they can be acquired from a suitable transducer and fed to an analog-to-digital converter (ADC), which in turn presents the binary values to the computer. Similarly, the output signals can either be stored in digital form, or they can be fed to a digital-to-analog converter (DAC), and thence back to the real world as a continuous electrical signal. Not all DSP algorithms are linear, but many are. Table 2.1 shows a few important algorithms and mathematical transforms that are so.

0 5000 10,000

Frequency, Hz Frequency, Hz

dB magnitudedB magnitude

5000

0 10,000

Figure 2.1 Spectra of the top (E) string of an acoustic guitar, taken at (a) the note onset and (b) 6 s later

Table 2.1 Linear algorithms and mathematical operations (not related)

Examples of algorithms Examples of mathematical as linear systems operations as linear systems

Filtering Integration

Equalisation Differentiation

Deconvolution Multiplication by a constant

Reverberation Convolution

Echo Fourier transform

Weighting Laplace and z-transforms

One of the most beautiful aspects of algorithms of this kind, unlike their physical counterparts, is that they can be perfect linear systems, since everything exists as number. This may seem obvious, since a computer that multiplies x by 0.3337 will always yield 0.3337x, no matter how many times it does it, or where or when it does it.

But think of this: if we now take the output from our computer-based linear system and convert it into a real signal, then we have a new physical linear system that performs better than the original. To use a simple example of an electronic analog filter, with digital systems, we can create real world and real-time filters that out-perform their analog counterparts in almost every respect. We can make them as frequency selective and gain specific as we like, all with unconditional stability. Really? Yes indeed. In Chapter 11, we will see how we can make a DSP filter with the performance of a 50th order Butterworth filter, with phase control of one hundred thousandth of a degree and with no risk whatsoever of instability.

2.2.2 Continuous and discrete signals: some definitions

So far in our preliminary dealings with linear systems, we have switched between discussing things like bells and springs on the one hand, and linear DSP algorithms on the other. To get a little more precise, these are examples, respectively, of con-tinuous and discrete linear systems, which respond to and produce concon-tinuous and discrete signals. Think once more of an analog filter. Its input is in the form of a continuous electrical signal that is infinitesimally contiguous and extant for all points of time over the period of its duration (this dispenses with the philosoph-ical problem of a real signal that lasts forever). Its output signal may similarly be described. In this case the signal is a function of time, t, but depending on the system, it might be almost any conceivable independent variable; distance is another common one. In contrast, discrete signals are generated and used by digital processing systems, such as computers. They comprise a series of discrete values corresponding to the values of their continuous time counterparts at regularly spaced time intervals. In other words, the discrete signal can be thought of as representing a sampled version of the continuous signal. For a computer to process the continuous signal generated by a sensor for example, it must first be converted into a discrete form by digitising it. The independent variable is now parameterless, and is usually denoted as n.

Now, gentle reader, you might be objecting to the inclusion in our discussions, of continuous time systems. After all, you reasonably claim, DSP systems only work on discrete data. Quite so, but that is not the end of the story. The reasons are twofold. First, many DSP algorithms have their roots in analog systems. During the 1960s, much effort was spent in converting the mathematics associated with elec-tronic filter design into digital equivalents. To understand the way these algorithms work, a knowledge of continuous systems is indispensable. Second, many DSP algo-rithms are designed to replicate or model physical systems that exist in continuous space – think for example of digital reverberation units, artificial speech systems and musical instrument emulators. Once more, effective models cannot be devised without understanding the way continuous signals and systems behave.

Before we investigate more deeply some of the more important characteristics of linear systems, it would be worthwhile at this stage to define some mathematical nomenclature. The terminology we will adopt is not universal, but it is the one most commonly encountered in the literature (Lynn, 1986), and will be adhered to throughout the remainder of this book. Therefore:

• A time domain, continuous input signal is denoted by x(t). A time domain, discrete input signal is denoted by x[n], and is only defined for integer values of n.

• A time domain, continuous output signal is denoted by y(t). A time domain, discrete output signal is denoted by y[n], and is only defined for integer values of n.

• The Fourier transform of the time domain, continuous input signal is denoted by X(ω). The Fourier transform of the time domain, discrete input signal is denoted by X[k].

• The Fourier transform of the time domain, continuous output signal is denoted by Y (ω). The Fourier transform of the time domain, discrete output signal is denoted by Y [k].

• The impulse response of a continuous linear system is denoted by h(t). The Fourier transform of the continuous impulse response is termed the frequency response and is denoted by H (ω).

• The discrete-space equivalents of the impulse and frequency response are denoted by h[n] and H [k], respectively.

The final two definitions in the list above introduce the impulse response, h(t) (or h[n] for discrete linear systems), and the frequency response, H (ω) (or H [k]), which is obtained by taking the Fourier transform of the impulse response (we will investigate the details of the Fourier transform later in the book). The Fourier trans-form of a signal is often referred to as its Fourier spectrum, the frequency spectrum or just the spectrum. The impulse response is the most important feature of a linear system, because it characterises it completely. As its name suggests, it is how the systems responds to being stimulated with an infinitesimally short pulse, an impulse function (also known as a delta or Dirac function, in honour of Paul Dirac, one of the great theoretical physicists of the early twentieth century). If you know the impulse response, then you know exactly how the system will respond to any given input signal. Because of this, the term h(t) is often used as a shorthand expression to rep-resent the linear system itself; more on this later, and indeed on how we obtain the impulse response.

A continuous signal with its discrete counterpart is shown in Figure 2.2(a) and (b).

This figure shows two signals represented as functions of time. Typically, although not always, the continuous signal is shown as a continuous line and the discrete signal is depicted as a set of vertical lines, with square or rounded blobs. It is worth mentioning that sometimes, if the sample points are very close together, discrete signals are also shown as continuous lines because to do otherwise would just end in confusion. Any signal can also be represented as a function of frequency. Think of a chord played on a piano; not only will it have a particular amplitude (loudness) and duration, it will also

x[n]

n x(t)

a b

Figure 2.2 (a) Continuous and (b) discrete time signals

k – k

X [k]

Figure 2.3 A discrete Fourier spectrum (amplitude only), showing positive and negative harmonics

comprise a wide range of different harmonics, or frequencies. To obtain the frequency domain representation of a signal, we perform Fourier transformation, and hence we often refer to the frequency domain as the Fourier domain.

Now you might be forgiven for thinking that just as time domain signals can be represented in continuous or discrete space, the same would strictly apply for their Fourier transforms, that is, continuous signals have continuous Fourier trans-forms and discrete signals have discrete Fourier transtrans-forms. Well yes, but it is a bit more complicated than that. Let us deal with the discrete case first, since this is the most straightforward. In this instance, the Fourier transform is indeed represented as a set of discrete harmonics, as Figure 2.3 shows. Note that it also includes negative frequencies, a concept that we will deal with later, when we find out how to compute the transform. Also, we are showing only the amplitudes, not the phases, which will also be examined later. When we Fourier process continuous signals, the transform that is applied depends on the mathematical description of the signal. For example, think about these three continuous signals:

• The electrical output of an analog first order low-pass filter, produced in response to an impulse function.

• A fully rectified sine wave voltage.

• The electrical output of a microphone, produced in response to the sound of a smashing wine glass.

Table 2.2 Fourier relationships for various signals

Signal Fourier transform

Continuous signal Continuous Fourier series Continuous Fourier transform Discrete signal Discrete Fourier series

Discrete Fourier transform

The impulse response of a low-pass filter is given by the expression y(t) = 1

RCe^(−t/RC), (2.1)

where R and C denote the values of the resistor in ohms and the capacitor in farads.

Since it has an analytical description extending to±∞, we can use the continuous Fourier transform to obtain its frequency response, and this too will be a continuous, analytical function. In contrast, the electrical signal of the second example is contin-uous but periodic, with discontinuities at the end of each cycle. In this case, we apply a Fourier series analysis, which is analytically defined but is calculated for discrete harmonics of the fundamental. The final situation involves a signal that is completely arbitrary in nature, and despite being continuous, cannot be described by any analyti-cal mathematianalyti-cal function. To analyti-calculate its Fourier representation, we need to digitise it and apply the discrete Fourier transform. Again, we end up with a spectrum that com-prises a discrete set of harmonics. These relationships are summarised in Table 2.2.

2.2.3 The impulse function and step function

The impulse function, shown in Figure 2.4(a), and in shifted, weighted form in Figure 2.4(b), is the most basic and the most important of all signals. Not only can it characterise the behaviour of any linear system, it can also be used to construct any arbitrary signal, through application of the superposition principle. The contin-uous time impulse function, denoted by δ(t), is theoretically a pulse of infinitesimal duration and infinite height, that is, it has an area of unity. By convention, it is nor-malised to unit height at t = 0, and is zero everywhere else. Similarly, the discrete time impulse function δ[n] has unit height when n = 0 and is zero elsewhere. Formally therefore, the definition of the impulse function is shown on the left-hand side of Equation (2.2):

a b

c t

t or n t or n

u(t) u[n]

(t) or [n]

A(t – 2) or A[n – 2]

Figure 2.4 Basic signals at the heart of DSP. (a) The continuous or discrete impulse function; (b) a continuous or discrete impulse function weighted by A and delayed by 2 intervals; (c) the continuous step function; (d) the discrete step function

Another important function in signal processing and analysis is the continuous or discrete step function, often denoted by the terms u(t) or u[n], respectively. It is shown in continuous form in Figure 2.4(c) and in discrete form in Figure 2.4(d). In straightforward terms, this function is an edge, whose value is zero when t or n is less than zero and of unit height when t or n is greater than zero. The definition for the continuous time step function differs slightly from its discrete time counterpart in only one respect, when t or n equals 0. In this situation, we say that the value of u(t) is undefined or indeterminate, but the value of u[n] is 1. The mathematical definition of this function is shown on the right-hand side of Equation (2.2). The impulse function and the step function are closely related: if we differentiate the step function, we obtain the impulse function; conversely, integration of the impulse function yields the step function. When we are dealing with discrete signals, the two operations are termed differencing and summation, and below we will see exactly how we go about achieving this.

2.2.4 Shifting and weighting

Intuitively, an impulse function is still an impulse function if we advance or delay it in time. Similarly, it retains it essential identity (its shape), if we weight it to alter its amplitude. Figure 2.4(b) shows such a weighted, shifted discrete time impulse function, that is, it has a gain of A and has been delayed in time by 2 sample intervals.

Therefore, its definition is Aδ(t − 2) (continuous version) or Aδ[n − 2] (discrete version). Shifting and weighting of this kind can also be applied to signals; look at Figures 2.5(a)–(f ), which shows a discrete signal x[n] transformed in one of five different ways (note that here k = 1).

n n n

x[n] x[n + k] x[n – k]

x[–n] x[–n + k] x[–n – k]

Figure 2.5 Shifted and weighted discrete signals (k = 1). (a) Original signal, x[n].

(b) Phase advance (shift to the left) by k intervals, that is, x[n + k].

(c) Phase delay (shift to the right) by k intervals, that is, x[n − k].

(d) Signal reversal, that is, x[−n]. (e) Phase delay of reversed signal, that is, x[−n + k]. ( f ) Phase advance of reversed signal, that is, x[−n − k]. See Table 2.3 for numerical values

Table 2.3 Time domain transformations of signal shown in Figure 2.5, with k = 1

Signal Advance Delay Reverse Reverse and delay Reverse and advance

n x[n] n + k x[n + k] n − k x[n − k] −n x[−n] −n + k x[−n + k] −n − k x[−n − k]

−3 0 −2 0 −4 (0) 3 0 4 0 2 −1

−2 0 −1 1 −3 0 2 −1 3 0 1 2

−1 1 0 −2 −2 0 1 2 2 −1 0 −2

0 −2 1 2 −1 1 0 −2 1 2 −1 1

1 2 2 −1 0 −2 −1 1 0 −2 −2 0

2 −1 3 0 1 2 −2 0 −1 1 −3 0

3 0 4 0 2 −1 −3 0 −2 0 −4 (0)

4 0 5 (0) 3 0 −4 (0) −3 0 −5 (0)

If the signals seem a bit confusing, look at Table 2.3, which shows the index variable n, the original signal x[n] and the transformed variants. It is worthwhile stepping through the values, because it confirms in a practical way what we have just discussed. Note also the zero values that appear in parentheses; for these

values of the index variable the signal is undefined, so by convention it is set equal to zero.

2.2.5 Properties and characteristics of linear systems

Now that we have established the nomenclature, we can turn our attention to enu-merating some of the features of linear systems (Lynn, 1986). Many of the properties listed below are interdependent since they are necessary consequences of one another.

We list them anyway because they give us a fuller, immediate appreciation of how linear systems behave.

1. Temporal and spatial invariance. The behaviour of a linear system does not change over time, nor is it dependent upon its position in space. In the strictest sense, almost all supposedly linear physical systems exhibit some temporal and spatial variability. Think once more of the church bell alluded to earlier; over time its note will change, simply because the material will oxidise; it will alter as the temperature rises or falls, since the iron’s density will fluctuate; it will be modified according to whether the bell is at the top of a mountain or at sea level, both because the atmosphere is thinner at high altitudes (less damping) and earth’s gravity is very slightly weaker. However, when judged over relatively short time scales and distances, it is quite appropriate to think of the bell as a linear system.

We shall return to musical instrument emulation in a later chapter.

2. Proportionality. If an input to a linear system causes it to respond with a given output, then the same input weighted (scaled) by a factor will cause the system to respond with the original output, also weighted by the same amount (Smith, 1997). In other words we have for continuous systems, the relationships:

x(t) → h(t) → y(t),

ax(t) → h(t) → ay(t) (2.3)

and for discrete systems:

x[n] → h[n] → y[n],

ax[n] → h[n] → ay[n] (2.4)

where a is the weighting factor. This may seem obvious, but this property lies

In document Foundation of Digital Signal Processing (Page 38-52)