Experimental guided spherical harmonics based head related transfer function modeling

(1)

Experimental Guided Spherical

Harmonics based Head-Related

Transfer Function Modeling

Mengqiu Zhang

M.E. (Xidian University, Xi’an, Shanxi, China)

B.E. (China Jiliang University, Hangzhou, Zhejiang, China)

June 2012

A thesis submitted for the degree of Doctor of Philosophy of The Australian National University

Research School of Engineering

(2)

(3)

I lovingly dedicate this thesis to my family members, who supported me each step of the way.

(4)

(5)

Declaration

The contents of this thesis are the results of original research and have not been submitted for a higher degree to any other university or institution.

Much of the work in this thesis has been published or has been submitted for publication as journal papers or conference proceedings. These papers are:

Journal articles

J1. Wen Zhang, Mengqiu Zhang, Rodney A. Kennedy, and Thushara D. Abhaya-pala, “On high resolution head-related transfer function measurements: An efficient sampling scheme”, IEEE Trans. on Audio, Speech, and Language Processing, vol. 20, no. 2, pp. 575-584, February 2012.

J2. Mengqiu Zhang, Rodney A. Kennedy, and Thushara D. Abhayapala, “Towards Optimal Continuous Frequency Representation in Spherical Harmonics based HRTF Modeling”, submitted to IEEE Trans. on Audio, Speech, and Lan-guage Processing, January 2013.

Conference papers

C1. Mengqiu Zhang, Wen Zhang, Rodney A. Kennedy, and Thushara D. Abhaya-pala, “HRTF Measurement on KEMAR Manikin”, in Proc. ACOUSTICS 2009 (Australian Acoustic Society), Adelaide, Australia, November 2009, pp. 1-8.

C2. Mengqiu Zhang, Rodney A. Kennedy, Wen Zhang, and Thushara D. Abhaya-pala, “Internal Structure Identification of Random Process by Using Principal Component Analysis”, in Proc. The 4th Int. Conf. on Signal Processing and Communication Systems (ICSPCS), Gold Coast, Australia, December 2010, pp. 1-6.

C3. Mengqiu Zhang, Rodney A. Kennedy, and Thushara D. Abhayapala, “Effi-ciency evaluation and orthogonal basis determination in functional HRTF

(6)

modeling”, in Proc. The 36th IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP 2011), Prague, Czech Republic, May 2011, pp. 53-56.

C4. Mengqiu Zhang, Rodney A. Kennedy, Wen Zhang, and Thushara D. Ab-hayapala, “Statistical method to identify key anthropometric parameters in HRTF individualization”, in Proc. The 3rd Joint Workshop on Hands-free Speech Communication and Microphone Arrays (HSCMA), Edinburgh, UK, May 2011, pp. 213-218.

The following submitted papers are also the results from my Ph.D. study but not included in this thesis:

Conference papers

C5. Klaus Reindl, Mengqiu Zhang, and Walter Kellermann, “On the limitations of binaural reproduction of monaural blind source separation output signals” (invited paper), in Proc. the 2012 European Signal Processing Conference (EUSIPCO 2012), Bucharest, Romania, August 2012, pp. 305-309.

The research work presented in this thesis has been performed jointly with Prof. Rodney A. Kennedy (The Australian National University) and A/Prof. Thushara D. Abhayapala (The Australian National University). The substantial majority of this work was my own.

Mengqiu Zhang

Research School of Engineering, The Australian National University, Canberra,

(7)

Acknowledgements

The work presented in this thesis would not have been possible without the support, hard work, and endless efforts of a number of individuals and organizations and they are gratefully acknowledged below:

• I would like to express my deeply-felt gratitude to my supervisors Prof. Rodney A. Kennedy, A/Prof. Thushara D. Abhayapala, and Dr. Paras-too Sadeghi for their thoughtful guidance, invaluable support throughout my doctoral studies. Their technical insight, sharp focus, and on-going encour-agement was a constant source of inspiration and motivation resulting in outputs which I would not have previously thought possible. Their sense of humour and genuine concern for their students made working with them a memorable and pleasurable experience.

• I would like to thank Prof. Walter Kellermann from Friedrich-Alexander-University Erlangen-Nuremberg (FAU) and Prof. Richard H. Y. So from Hong Kong University of Science and Technology (HKUST) for kindly wel-coming me to visit their research groups. During the three-month visit at FAU, Prof. Kellermann introduced me to a new field of binaural processing in hearing aids. I would also like to thank PhD student Klaus Reindl at FAU for his kind collaboration. The one-month visit at HKUST was also very valuable. The discussions with Prof. So stimulated many interesting ideas in the area of headphone based virtual sound synthesis.

• It is my great pleasure to study in the applied signal processing (ASP) group at the Research School of Engineering. I would like to thank everyone for making ASP group a friendly and relaxing research environment. Special thanks must go to Dr. Wen Zhang for her friendship as well as many helpful suggestions and fruitful discussions during my PhD studies. I am especially grateful for Dr. Xiangyun (Sean) Zhou’s friendship and kindly help with ev-erything when I came to him. I would also like to acknowledge Ms. Lesley Goldburg, Ms. Marie Katselas, and Ms. Elspeth Davies, our student

(8)

istrator, Mr. James Ashton and Peter Shevchenko, our IT administrator, for all their kindly assistances.

• I would like to thank my various sources of financial support. I would like to thank Australian National University (ANU) for providing me the scholar-ship throughout my PhD studies and the travel grant to support my visit at HKUST. I would like to thank ANU College of Engineering and Computer Science for providing me the Dean’s Travel Award to support my overseas con-ference attendances and my visit at FAU. I would also like to thank National ICT Australia and Dr. Bradley Treeby for providing me the experimental instruments, the KEMAR manikin and the optical pose tracker.

(9)

Abstract

In this thesis we investigate the experimental guided spherical harmonics based Head-Related Transfer Function (HRTF) modeling where HRTFs are parameter-ized as frequency and source location. We focus on efficiently representing the HRTF variations in sufficient detail by mathematical modeling and the experimen-tal measurements. The goal of this work is towards an optimal functional HRTF modeling taking into account the demands of decreasing the computational cost and alleviating the HRTF interpolation and/or extrapolation in the headphone based binaural systems.

To represent HRTF by models, we firstly consider the high variability of HRTFs among individuals caused by the differentiation of the scattering effects of the individual bodies on the sound waves. We conduct a series of statistical analyses on an experimental HRTF database of human subjects to reveal the correlation between the physical features of human beings, especially pinna, head, and torso, and the corresponding HRTFs. The strategy enables us to identify a minimal set of physical features which strongly influence the HRTFs in a direct physical way. We next consider the continuity of the HRTF representation in both spatial and frequency domain. We define a functional HRTF model class in which the HRTF spatial representation has been justified to be well approximated by a finite number of spherical harmonics while HRTF frequency representation remains the focus of this thesis. In order to seek an efficient representation for HRTF frequency portion, we derive a metric that is able to numerically evaluate the efficiency of different complete orthonormal bases. We show that the complex exponentials form the most efficient basis. Given the identified basis, we then provide a solution to determine the dimensionality of the representation.

To represent HRTF by measurements, we firstly consider the required angular resolution and the most suitable sampling scheme taking into account the two di-mensional angular direction and the wide audio frequency range. We review the spherical harmonic analysis of the HRTF from which the least required number of spatial samples for HRTF measurement is derived. Considering how the HRTF

(10)

(11)

List of Acronyms

BSS Blind Source Separation DFT Discrete Fourier Transform DTF Directional Transfer Functions FFT Fast Fourier Transform

FIR Finite Impulse Response

HRTF Head-Related Transfer Function HRIR Head-Related Impulse Response IIR Infinite Impulse Response IR Impulse Response

ILD Interaural Level Difference ITD Interaural Time Difference

KEMAR Knowles Electronics Mannequin for Acoustic Research KL Expansion Karhunen-Lo`eve Expansion

MLS Maximum Length Sequence

MNAM Minimum Number of Azimuthal Measurements MSE Mean Square Error

PC Principal Component

PCA Principal Component Analysis PRBS Pseudo Random Binary Signal

SFER Spatial Feature Extraction and Regularization SIR Signal-to-Interference Ratio

SNR Signal-to-Noise Ratio SPL Sound Power Level

SVD Singular Value Decomposition

(12)

(13)

Notations and Functions

(_·) Complex Conjugate

(_·)† _{Complex Conjugate Transpose} h·,_·i Inner Product

k · k Norm Operator

(_·)T _{Transpose Operator}

argmin(_·) Argument of the Minimum Ave(_·) Average Operator

cov(_·,_·) Covariance Operator E_{·} Statistical Expectation max(_·) Maximum Operator

· Integer Ceiling Function e(·) _{Exponential Function}

Jn(·) Bessel Function of n-th Order

jn(·) Spherical Bessel Function of n-th Order

hn(·) Spherical Hankel function ofn-th Order

Pn(·) Legendre polynomial of n-th Degree

Pm

n (·) Associated Legendre function of Degree n and Order m

Pm

n (·) Normalized Legendre Function of Degree n and Order m

U(_·) Chebyshev Polynomial of then-th Degree yn(·) Spherical Neumann function of n-th Order

Ym

n (·) Spherical Harmonics of Degreen and Order m

(14)

(15)

List of Figures

1.1 Using ITD and ILD to estimate the azimuth angle of a sound source. 4

1.2 HRTF magnitude response of KEMAR manikin at three angles on median plane. . . 5

1.3 HRIRs on horizontal plane of (a) left ear and (b) right ear (from CIPIC HRTF dataset [1]) . . . 7

1.4 HRTFs on median plane of (a) left ear and (b) right ear (from CIPIC HRTF dataset [1]) . . . 9

1.5 HRTFs of different subjects (from CIPIC HRTF dataset [1]) . . . . 10

1.6 The spatial sampling arrangement in (a) MIT HRTF database and (b) CIPIC HRTF database. . . 12

2.1 Random Process and random variable. The solid curves are realiza-tions of the random process. The dash curve is the mean value of the random process. The stars present the distribution of the random variable. The dot shows the mean value of the random variable. . . 24

3.1 The spatial sampling locations of (a) front view and (b) side view in CIPIC HRTF database (This figure is partly from [1].). . . 35

3.2 Correlation among HRIRs at different directions of the same subject: (a) Left ear HRIRs on horizontal plane; (b) Right ear HRIRs on horizontal plane; (c) Left ear HRIRs on median plane; (d) Right ear HRIRs on median plane. . . 37

3.3 Correlation among HRIRs of different subjects at a particular direc-tion: (a) Left ear HRIRs at (0◦_{, 0}◦_{); (b) Right ear HRIRs at (0}◦_,

0◦_{). . . .} ₃₈

3.4 PCs at the direction of φ = ₋80◦ _and _θ _{= 0}◦_{. (a) First 5 PCs for}

reconstructing the left ear HRIRs; (b) First 5 PCs for reconstructing the right ear HRIRs. . . 39

(20)

3.5 Sample distribution of weights across 27 subjects over all PCs at the direction of φ = ₋80◦ _and _θ _{= 0}◦_{. (a) Weights for reconstructing}

the left ear HRIRs; (b) Weights for reconstructing the right ear HRIRs (We use a different weight scale from the left one to reveal the interaural level difference details). . . 41

3.6 The original and reconstructed HRIRs of the subject with the ID number of 33 at the direction of φ=₋80◦ _and_θ _{= 0}◦_{. (a) The}

orig-inal and reconstructed HRIRs of the left ear. (b) The origorig-inal and reconstructed HRIRs of the right ear (We use a different amplitude scale from the upper one to reveal the details). . . 42

3.7 The original and reconstructed HRTFs of the subject with the ID number of 33 at the direction of φ = ₋80◦ _and _θ _{= 0}◦_{. (a) The}

magnitude response of the original and reconstructed HRTFs of the left ear. (b) The phase response of the original and reconstructed HRTFs of the left ear. (c) The magnitude response of the original and reconstructed HRTFs of the right ear. (d) The phase response of the original and reconstructed HRTFs of the right ear. . . 43

3.8 Inter-parameter correlation. (a) Correlations among 37 anthropo-metric parameter measurements. (b) Highlights of 22 pairs of an-thropometric parameters with strong linear correlation. The num-bers are the Pearson correlation coefficients between two anthropo-metric parameters. . . 45

3.9 Correlation between 4 pairs of anthropometric parameters. (a) Cavum concha width of left ear and right ear with RG = 0.789; (b) Pinna

offset down and head height with RG = −0.763; (c) Cavum

con-cha height of left ear and Cymba concon-cha height of right ear with RG = 0.382; (d) Cavum concha height of left ear and head depth

with RG = 0.139. . . 46

3.10 The overall test results of the model adequacy for the left ear weights of the first four PCs at all directions with 15 head and torso param-eters: (a) q = 1; (b) q = 2; (c)q = 3; (d) q = 4. Stars indicate that the value of Fi

(21)

List of Figures xix

3.11 The effect of four head and torso parameters on the 1st principal component weights of left ear at azimuth angle of 80 degrees and all elevation angles. Blue dots indicatetvalues of head width, red stars for head depth, black squares for neck height, green wedge for torso top height. The red dash line is t = 2.20. . . 51

3.12 The overall test results of the model adequacy for right ear weights of 3 principal components at all directions with 12 pinna parameters: (a) q = 1; (b) q = 2; (c) q = 3. Stars indicate that the value of F_b_sq is greater than 2.53. (For the 2nd and 3rd principal component analysis, we focus on azimuth angle of 5 degrees and 20 degrees, respectively.) . . . 52

3.13 The effect of six pinna parameters on the 1st principal component weights of right ear at all directions: (a) Cavum Concha Width; (b) Fossa Height; (c) Pinna Height; (d) Pinna Width; (e) Pinna Rotation Angle; (f) Pinna Flare Angle; (g) Intertragal Incisure Width; (h) Cymba Concha Height. Stars indicate that the t value is greater than 2.145 at those directions. . . 54

3.14 The effect of 5 pinna parameters on the 2nd principal component weights of right ear at azimuth angle of 5 degrees and all elevation angles. Blue dots indicate tvalues of cavum concha width, red stars for fossa height, black squares for pinna width, green wedge for pinna rotation angle, and purple diamonds for pinna offset back. The red dash line is t = 2.14. . . 55

3.15 The effect of 12 pinna parameters on the 3rd principal component weights of right ear at azimuth angle of 20 degrees and all elevation angles. Blue dots indicate the t value of pinna width and the red stars indicate the t value of pinna flare angle. The red dash line is t = 2.145. . . 56

3.16 The linear relationship between head depth and the weights of left ear at azimuth of 80 degrees. (a) The scatter diagram for head depth and weight at azimuth of 80 degrees and elevation of 67.5 degrees. The red line is the straight-line relationship between the two variables with the slope of 0.124. (b) The slope of the straight-line between head depth and weights at azimuth of 80 degrees and all elevations. . . 57

(22)

4.1 The cumulative variance of the reordered Fourier coefficients . . . . 72

4.2 The magnitude and the phase response of the original (the solid curve) and the reconstructed (the dashed curve) γnm(k) (n = 2 and

m= 0) by (a) complex exponentials (b) spherical Bessel functions (c) spherical Hankel functions (c) Chebyshev polynomials (d) Legendre polynomials . . . 73

4.3 The variance of all Fourier coefficients when fs = 44.1 kHz, fmax =

20kHz, and NDFT = 512 . . . 76

4.4 The inherent correlation between the phase properties of the HRTFs and the exponentials in the sub-basis. Upper panel: The sum of the variance over all n and m of the 30 Fourier coefficients. Lower panel: The solid line is the average phase response of HRTFs over all directions and the dotted lines are the phase responses of complex exponentials with the bottom one corresponding to Q = 1, then Q= 5, 10, 15, 20,25, and Q= 30 on the top. . . 77

4.5 Validity of the numerical orthogonality relation of complex exponen-tials when the DFT point is set to 128. The orthogonality error is the difference between the theoretical value of the integral (1 for the squared norm of the exponentials and 0 otherwise) and the numerical value is at a pixel gray level. . . 79

4.6 The measured (the solid line) and the reconstructed (the dashed line) left ear HRTFs at azimuth of₋30◦ _{(left ear side) and elevation}

of 0◦ _{. . . .} ₈₀

5.1 Picture of the 3:6:3 equal area division, which divides the sphere into twelve base regions, three at either cap and six 60◦_×₆₀◦ _equatorial

regions. Here, each base region is sampled with 64 points. . . 90

5.2 Picture of the polar cap region in the igloo scheme, showing the subdivision with Md = 2. The dot indicates the sampling position

(23)

List of Figures xxi

5.3 The validity of the orthogonality relation´ Ym

n (s)Ym

′

n′ (s)ds over the full-sphere and the grid with the bottom part cut out using dif-ferent quadrature rules: (a) full-sphere, trapezoidal rule; (b) part-sphere, trapezoidal rule; (c) full-part-sphere, Boole’s rule; (d) part-part-sphere, Boole’s rule. The orthogonality error is the difference between the theoretical value of the integral (1 when degree and order are equal and 0 in all other cases) and the numerical value, for the 192

sam-pling points at a pixel gray level. . . 94

5.4 An example of synthetic sampled HRTFs based on the IGLOO scheme and the reconstruction results at elevation of 79◦ _and az-imuth of 90◦_{. Sampled data: solid line; Reconstruction: dot-dash} line. . . 97

5.5 An example of HRTF interpolation at elevation of 62◦ _{and azimuth} of 14◦ compared with reference of the synthetic solutions. Reference data: solid line; Interpolation: dot-dash line. . . 98

5.6 Synthetic HRTFs reconstruction performance at different elevations. (a) θ = 30◦ _(b) _θ _{= 60}◦ _(c) _θ _{= 90}◦ _(d) _θ_{= 120}◦_{. . . .} ₉₉

5.7 Operation diagram of HRTF measurement. . . 100

5.8 An example of KEMAR left ear HRTF sampled based on the IGLOO scheme and the reconstruction results at elevation of 45◦_{and azimuth} of 160◦_{. Sampled data: solid line; Reconstruction: dot-dash line. . . 101}

5.9 An example of KEMAR left ear HRTF interpolation at elevation of 85◦ _{and azimuth of 0}◦ _{compared with measurement reference.} Sampled data: solid line; Interpolation: dot-dash line. . . 102

5.10 KEMAR HRTFs reconstruction error performance at different ele-vations: (a) Right ear θ = 60◦_{; (b) Right ear} _θ _{= 90}◦_{; (c) Right ear} θ = 120◦_{; (d) Left ear} _θ _{= 60}◦_{; (e) Left ear} _θ _{= 90}◦_{; (f) Left ear} θ = 120◦_{. . . 103}

6.1 The spherical coordinates system in HRTF measurement. . . 107

6.2 The mechanical setup in HRTF measurement. . . 108

6.3 HRTF measurement procedure . . . 110

6.4 Pre-emphasis filter: (a) magnitude Response; (b) phase property. . 112

6.5 Test signal: (a) original sweep in Time Domain (b) spectrum of the original sweep (c) pre-emphasized sweep in time domain (d) spectrum of the pre-emphasized sweep . . . 113

6.6 The test signal . . . 114

(24)

6.8 Received signals with different response delay . . . 115 6.9 Signals after alignment. Compared with Fig. 6.8, the signals are

synchronized to the same step. . . 116 6.10 Alignment procedure: B — the start point of the received test signal;

B′ — B′ = B + 184; D — the start point of the reverberation signal; A — the start point for calculating the correlation to confirm B; A′

— the start point for calculating the correlation to confirm D. . . . 117 6.11 Low pass filtering (LPF): (a) magnitude response of LPF; (b) phase

property of LPF; (c) signal before filtering; (d) signal after filtered. 118 6.12 Equalization (right ear HRTF at direction of θ = 0◦_{, φ} _{= 90}◦_{): (a)}

system frequency response; (b) rough HRTF before equalization; (c) final HRTF after equalization. . . 120 6.13 Main spectral characteristics of HRTF . . . 121 6.14 Frequency-Domain comparison of HRTFs measured at different

ele-vation angles on the median plane (φ= 0◦_{): (a) left ear HRTFs; (b)}

right ear HRTFs. . . 122 6.15 Frequency-Domain comparison of HRTFs measured at 36 azimuth

angles on the horizontal plane (θ = 0◦_{): (a) left ear HRTFs; (b)}

right ear HRTFs. . . 123 6.16 Differences of HRTFs (measured at direction ofθ = 0◦_{, φ}_{= 0}◦_{): (a)}

Left Small Ear HRTF; (b) Right Small Ear HRTF; (c) Left Large Ear HRTF; (d) Right Large Ear HRTF. . . 124

B.1 Spherical Bessel function of n = 0 (blue), n = 1 (red), n = 2 (yellow), n= 3 (green). . . 135

C.1 Spherical Neumann function of n = 0 (blue), n = 1 (red), n = 2 (yellow), n= 3 (green). . . 138

E.1 The first five Chebyshev polynomials of the second kind in the do-main [-1, 1]: q= 0 (blue), q= 1 (red), q = 2 (yellow),q = 3 (green), and q = 4 (magenta). . . 142

(25)

List of Tables

3.1 Anthropometric Parameters . . . 44 3.2 Data for Multiple Linear Regression . . . 47

4.1 Definitions of Five Orthogonal Bases . . . 68 4.2 Efficiency Properties of Five spherical harmonics based HRTF

fre-quency representations . . . 72

5.1 Comparison of different methods for HRTF sampling over sphere . . 89 5.2 The IGLOO scheme based HRTF sampling for 20 kHz audible

band-width . . . 93 5.3 Reconstruction Comparison of Different HRTF Sampling Schemes. . 104

(26)

Experimental guided spherical harmonics based head related transfer function modeling