EE432: Digital Signal Processing Fall AY2022 Project 02: Convolution & Audio Processing
Assigned: Tuesday 09/07/2021 Due: Tuesday 09/16/2021
Introduction
In this project we will explore convolution using MATLAB. You will be reading and writing audio files. Finally, you will experiment with adding echoes and reverberations to an audio signal to simulate different acoustic environments.
I. Convolution
The convolution sum, represented by the operator ∗, takes two sequences x[n] and h[n] and produces a third sequence y[n] given by the expression
The response of a linear time invariant system to a unit impulse is known an as its impulse response h[n]. Since any input sequence x[n] can be represented as the sum of weighted, shifted impulses, linearity and time invariance allow us to calculate a system’s output as y[n] = x[n] ∗ h[n].
1. Convolution by Hand. For input x[n] and impulse response h[n] depicted in Figure 1, compute output y[n]
by hand using the table method. Submit the convolution table to show how you computed y[n].
Figure 1: Sequences x[n] and h[n]
.
2. Convolution in MATLAB. Use the conv function to compute the convolution of x[n] and h[n] above and confirm your previous hand calculations. Create a stem plot of the resulting sequence y[n]. Be sure to use the
’filled’ option so that the heads of the stems are filled circles. Also, use the axis (or xlim) function to set the axis limits so that the stem heads all fall within the display. You will need to specify the time axis by supplying vector n of time indices. As well, the horizontal limits should be set so that all points are visible.
You would use them like this:
>> stem(n,x,'filled','MarkerSize',4); xlim([xmin xmax]); grid on; or
>> stem(n,x,'filled','MarkerSize',4); axis([xmin xmax ymin ymax]);
3. An improved Convolution Function. Consider computing the convolution of the sequences shown in in Figure 2. A problem that arises from using the conv function is that both sequences are assumed to start at time n = 0.
0
x[n]
1 2 n 3
3
-2 1
1
h[n]
n 2
0
1 3
4 2 -1
Figure 2: Sequences x[n] and h[n].
.
To address this shortcoming, write an enhanced version of the conv function called conv432 invoked as [y,n] = conv432(x,xstart,h,hstart) which returns both the convolved sequence and the associated time index array. Note, that each input sequence is specified by both its values and its starting time index. For example, for x[n] in Figure 2 would be represented by the pair
x = [3,0,4,0,0,−2] and xstart = -2.
For simplicity, your function should use the existing MATLAB conv function. See the notes for Lesson #03 to aid in determining how to find the starting and ending index and/or the length of the convolution.
Error checking: if xstart or hstart are not integers, this function should return y = [ ] and n = [ ] (which is the NULL value) and display a warning message.
Note: this function does NOT create a stem plot of the convolution result, only provides the vectors needed to create a plot.
Helpful considerations: the output time index vector can be created using the : (colon) operator, e.g., if you type n = -3:6, MATLAB will create a vector of sequential integers starting with -3 and ending with 6. Also, the MATLAB length function may prove useful here.
Test your function using the x[n] and h[n] from Figures 1 and 2, and plot the results in a stem plot. TURN IN your m-file and a 2x1 subplot, well-labeled, of both of these convolutions (into your MS Word document).
II. Audio File Functions
Frequently in this course we will be using audio files in signal processing examples, so let us take a moment to introduce/review some relevant MATLAB audio file functions. The primary MATLAB functions for audio files include audiowrite, audioinfo, and audioplayer. These functions are more flexible than wavread and wavwrite which operate only on .wav files (and are no longer in MATLAB). Note the following:
1. The audioread function will read in an audio file, and will return variables containing (1) the music signal, (2) the sampling frequency and (3) the number of bits/sample. There are several different ways to call audioread; in this course you should always use it to return at least these three variables. If the audio file contains a stereo recording (left and right channels) audioread will return a matrix that is m × 2 in dimensions (where m is the number of samples in each channel). Type >> help
audioread (or doc audioread) to learn how to use it. Also, you may elect not to read the entire audio file into MATLAB; a file with 44,100 samples per second results in really large vectors. Refer to the audioread documentation to learn how to read just a portion of a file.
2. Use the audioinfo function to query the parameters in an audio file (SampleRate, TotalSamples, NumChannels, Duration, BitsPerSample, etc.).
x[n]
n 3 4
1
h[n]
n 3
0 1 1
0 2
3
-2
2 2
-1 -2
3. I would recommend not using the native MATLAB audioplayer function to play audio files.
Instead, you can read in the file using audioread, then use soundsc with the correct sample frequency to play it, or use the default audio player for your operating system.
4. The audiowrite function will allow you to create an audio file given an audio vector, the sampling frequency, and the number of bits per sample. The audio file format written will depend on the filename extension that you specify. While the .wav format is an option that can be played in most music players (e.g. Windows Media Player), .wav files are completely uncompressed, hence very large. I suggest using .mp4 or the more modern MPEG-4 AAC .m4a format, both of which are significantly more compressed. If you are using MATLAB on a Windows machine, the only allowable sample frequencies for an m4a or mp4 file are 48,000 or 44,100. IMPORTANT: Also ensure the signal you are trying to write out has values in the range −1 ≤ 𝑥 ≤ 1 or it will be clipped! You can ensure this by dividing your signal by its maximum absolute value before using audiowrite.
Experimenting with audio. There are two files (flappynews.wav and LMFAO-Party Rock Anthem-clip) available in the shared Projects/Project02 folder on the Google drive for you to use in the portion of the project.
1. Download, load into MATLAB and listen to the sample speech file flappynews.wav. Can you guess what TV show this clip is from? What is the sampling rate of this recording? Is this a stereo or mono recording?
What are the maximum and minimum signal values? Plot the waveform over the period between 0.25 and 0.75 seconds. Be sure to label your time axis correctly.
2. Play the flappynews clip backwards (time reverse). Note that MATLAB has two functions to reverse vectors or matrices from left-to-right (fliplr) or up-to-down (flipud), depending on if the inputs are row or column vectors. Time-reverse the audio and write it to a file using audiowrite as a .wav file. SUBMIT this file in your shared drive folder.
3. Observe the effect of using the incorrect sampling rate by writing audio file and specifying the sampling rate as twice the correct sampling rate when using soundsc. Listen to the resulting recording and describe the effect.
4. Determine the sampling rate and the total number of samples for the music file WhipIt-short.m4a. Listen to the clip, then listen to it backwards.
III. Echoes and Reverberations
A. Simple Echoes
We begin with simple echoes, formed by convolving an audio signal with an impulse response that consists of a value of 1 at time 0 sec (which, when played, will result in the audio signal unchanged), followed by some decreasing values, each decreasing value separated by N seconds (these will be the dying echoes when the signal is played).
There are two files (doh.wav and ping-mono.wav) available in the shared Projects/Project02 folder on the Google drive for you to use in the portion of the project.
1. Download the doh.wav file from the shared Projects/Project02 folder on the Google drive, then read it into MATLAB and use soundsc to play it (note: you must use soundsc with the correct sample frequency!). Can you guess the TV show character that made this sound famous?
2. Create an impulse response that will, when convolved with the doh signal, will produce echoes. Try this (where fs is the sample frequency of the doh.wav file):
h=zeros(1,2*fs);
h(1)=1;
h(5000)=0.5;
h(10000)=0.25;
h(15000)=0.125;
For the rest of this project, both the signals and the impulse responses are assumed to start at index 0, so YOU DO NOT NEED TO USE THE conv432 FUNCTION. Convolve the input doh with this impulse response (using the conv function), and use soundsc to play the result. You can experiment with changing the impulse response parameters, but ensure you set all the values of h to zero first using the zeros function as above. Ensure that the non-zero values are evenly spread out, and the non-zero values are decreasing. If the non-zero values are spread out by 5000 samples (as shown in the code above), how many seconds are there in between echoes?
________________ sec Now try the same experiment with the ping-mono.wav file, and listen to the results.
B. Reverberations
When a singer performs in a concert hall, part of what you hear in the audience is due to the sound coming directly from the performer, but you also experience the sound reflecting off the surfaces in the environment prior to arriving at your ears. This is known as reverberation, and it conveys a sense of the space to the listener. The effect of the acoustics of the room is well modeled as a linear time-invariant system, thus an acoustic environment can be characterized by its impulse response. Figure 3 shows the magnitude of the impulse response of a room and illustrates the typical stages of sound arriving at the listener.
Sound absent any reverberation is commonly referred to as “dry” sound. Dry sound may be generated artificially from a synthesizer, or it could be actual recordings from a recording booth. Artificial reverberation is commonly added digitally using dedicated equipment (i.e., a reverb) in order to produce more pleasing recordings. There are a variety of approaches to generating synthetic reverberation using digital filters. Filter parameters and configurations can be used to simulate various environments, and we will consider some of these in future projects. However, a conceptually simple (though computationally intensive) alternative to introducing reverberation is to convolve a dry sound with the impulse response of an environment.
There are two impulse responses posted in the /Projects/Project02 shared Google folder, one from the Milan, Italy symphony hall and the other from a cave. These files were originally downloaded from the website http://www.voxengo.com/impulses/ and were stereo impulse responses, but for this project I’ve made them mono.
These reverberation impulse responses are considerably more complex than a simple echo.
Figure 3: Example room impulse response magnitude in decibels.
1. Read in the two impulse responses (including sample rate) using audioread, and generate stem plots for the impulse responses in a single 2 x 1 subplot. Label the time axis appropriately and display only 0.2 seconds of each impulse response (or if it doesn’t last that long, plot the entire impulse response). Note:
these files also come with sampling frequencies, which will help you plot the time axis.
2. Listen to each of the impulse responses. Remember: the impulse response is the response of the system to an impulse…in the audio case, you might think of an impulse as something like a single clap of the hands onstage, and what you’re listening to is the clap from somewhere in the audience. Examine the
corresponding plots of the impulse responses. For each of the impulse responses, write your thoughts about any information you can infer about the space based on its impulse response.
C. Adding Reverberation with Convolution
Now we will add reverberation to “dry” sounds using convolution. Two dry sound files available in the Projects/Project02 shared folder are dogbark-mono.wav and doh.wav.
1. Create an audio file of the dogbark-mono in the Milan Opera House (IR_milan.wav) using convolution. Save the resulting output in .m4a format. Repeat this with the doh. Put these m4a files in your Projects folder along with the other items you’ll turn in.
2. Now let’s add more reverberations to a music clip (which already has reverberations from being recorded in a recording studio). Download either the Devo-WhipIt-clip.m4a file or the LMFAO-Party Rock Anthem- clip.wav. Convolve it with either of the reverberation impulse responses, listen to the result, and write it out as an m4a file. Include somewhere in the filename which space the song was convolved with. Put this m4a file in your Projects folder along with the other items you’ll turn in.
