On Microphone-Array Beamforming from a MIMO Acoustic Signal Processing Perspective

(1)

On Microphone On Microphone On Microphone On Microphone On Microphone On Microphone On Microphone

On Microphone---Array Beamforming Array Beamforming Array Beamforming Array Beamforming Array Beamforming Array Beamforming Array Beamforming Array Beamforming from a MIMO Acoustic Signal

from a MIMO Acoustic Signal from a MIMO Acoustic Signal from a MIMO Acoustic Signal from a MIMO Acoustic Signal from a MIMO Acoustic Signal from a MIMO Acoustic Signal from a MIMO Acoustic Signal

Processing Perspective Processing Perspective Processing Perspective Processing Perspective Processing Perspective Processing Perspective Processing Perspective Processing Perspective

Jacob Benesty, Jingdong Chen, Yiteng (Arden) Haung and Jacek Dmochowski _________________________________________________________________________________

__________________________________________________________________________________________________________________________________________________________________

_________________________________________________________________________________

Presented by

Osnat Goren-Peyser 24

24 24

24 June June June 2007 June 2007 2007 2007

(2)

Outline Outline Outline Outline Outline Outline Outline Outline

• Introduction

• Problem Description

• Beamforming

• Filer and sum technique

• Estimation algorithms

• Experimental results

• Further work

• Summery

(3)

Abbreviation Abbreviation Abbreviation Abbreviation Abbreviation Abbreviation Abbreviation Abbreviation

• BF – Beamforming

• DOA – Direction Of Arrival

• LS – Least Squares

• MN – Minimum Norm

• LCMV – Linearly Constrained Minimum Variance

• MVDR – Minimum Variance Distortionless Response

• GSC – Generalize Sidelobe Canceller

• SIR – Signal to Interference Ratio

(4)

Introduction Introduction Introduction Introduction Introduction Introduction Introduction Introduction

• Signal enhancement problem using microphone array.

• The objective of the array processing is to estimate the desired signals from the given microphones outputs.

• Noise and interferences:

– Noise

– Reverberation – the persistence of sound in a particular space after the original sound is removed.

– Interferers – signals with the same frequency as the desired signal that do not arrive from the desired signal DOA.

• Microphone arrays consist of sets of microphone sensors that are spatially arranged in specific pattern

– Different microphones receive different signals

– Can be exploited to separate the desired signals from the interferers using a beamformer.

(5)

Problem Description Problem Description Problem Description Problem Description Problem Description Problem Description Problem Description Problem Description

• M sources

– P desired signals – Q interferers – M = P+Q

• N microphones

• We assume N≥M

• s^m(k) - The mth source signal

s₈(k)

x₈(k)

x₉(k)

h11

h₂₁

h31

Sources Channel Microphones

• s^m(k) - The mth source signal

• xⁿ(k) - The output of the nth microphone

• h^nm - The acoustic channel impulse response from source m to microphone n

• L^h – The length of the longest channel impulse response

• bⁿ(k) - The noise observed at the nth microphone

s₉(k)

x_:(k)

sM(k)

x_N(k)

h_NM h31

hN1

(6)

Mathematical Model Mathematical Model Mathematical Model Mathematical Model Mathematical Model Mathematical Model Mathematical Model Mathematical Model

• The output of the nth microphone at time k:

• Assumptions:

– N≥M

– LTI system

– Discarding the noise terms: b(k)=0 for all k

– The first P signals are the desired sources, and the rest Q sources are interferers.

– Source signals in far-field

(7)

Interferences Interferences Interferences Interferences Interferences Interferences Interferences Interferences

• Reverberation

– Reverberation is the collection of all reflected sounds in a room.

– Reverberation time is the time required for a sound in a room to decay by 60 dB (called RT₆₀)

– Self interferences – Proportional to L^h – Proportional to L^h

• Interferers

– signals with the same frequency as the desired signal that do not arrive from the desired signal DOA.

– External interferences – Proportional to Q

• Measurements

– Speech dereverberation – Interference suppression

(8)

The main goal The main goal The main goal The main goal The main goal The main goal The main goal The main goal

• We want to perfectly estimate the desired signals from the microphone observations using weights

• Find s.t .

••

• Constraints:

(9)

Beamforming Beamforming Beamforming Beamforming Beamforming Beamforming Beamforming Beamforming

• A Beamformer is an array of sensors which can do spatial filtering.

• The objective is to estimate the signal arriving from the desired direction in the presence of noise and other interfering signals.

• A beamformer does spatial filtering in the sense that it separates two signals with overlapping frequency content originating from different that it separates two signals with overlapping frequency content originating from different directions.

• Beamforming is applicable to either transmission or reception of energy.

– The paper discuss formation of beams for reception

• By weighting or filtering and then summing signals from different microphones, the beamformer can produce virtual directivity pattern (=weighted sum of individual patterns)

(10)

Beamformer Classification Beamformer Classification Beamformer Classification Beamformer Classification Beamformer Classification Beamformer Classification Beamformer Classification Beamformer Classification

• Beamformers are classified as either data independent or statistically optimum, depending on how the weights are chosen.

• The weights in a data independent

beamformer do not depend on the array data and are chosen to present a

specified response for all signal and specified response for all signal and interference scenarios.

• The weights in a statistically optimum beamformer are chosen based on the statistics of the array data to optimize the array response.

– The weighs can be either fixed or adaptively determined.

– The statistics of the array data are not

usually known and may change over time so adaptive algorithms are typically used.

(11)

Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming

• Microphone signals are filtered and then summed together.

• {g

^n,l

} are the lth

coefficient of microphone n FIR filter

n FIR filter

• L = FIR filter length

– L=8 Delay and sum

• Filter coefficients and

filter length are chosen

depends on the desired

performance.

(12)

Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming

Estimation problem Estimation problem

• In order to estimate the desired signals from the microphone observation , we should determine NP L

^g

-length FIR filters

• Find the optimal BF weights

– Determine the optimization criterion – Determine the optimization criterion

• Determine the BF algorithm

• Different optimization criterion Different performance !

– Determine the set of linear equations describing the estimation

• Full channel knowledge

• Partial channel knowledge

• Find bounds on FIR filter length

(13)

Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming

Analysis Analysis

• The following linear set of equations describing the estimation problem:

H

^T

g ^p =u

^p

’

Where:

– H^T MLxNL^gchannel matrix – g^p NL^g length column vector – g^p NL^g length column vector – u^p’ ML length column vector – L = L^g+L^h-8

• Channel knowledge:

– Full knowledge of the impulse response matrix H

– Partial Knowledge of the impulse responses from source p to the N microphones

(14)

Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming

Analysis (cont.) Analysis (cont.)

• The following linear sub set of equations describing the estimation problem in the case that only the impulse responses from source p to the N microphones are known:

H

^:pT

g

p

=u

p

Where:

– H:p^T LxNLgchannel matrix – gp NLg length column vector – up L length column vector – up L length column vector – L = Lg+Lh-8

• The following linear sub set of equations describing the estimation problem in the case that only the impulse responses from source p to the N microphones are known:

h

^:pT

(k

^p

)g

^p

=8

Where:

– h:p(kp) the kpth column of H:p

– h:p(kp)^T NLglength channel vector – gp NLg length column vector

(15)

Deterministic characterization

of estimation problem I of estimation problem I of estimation problem I of estimation problem I of estimation problem I of estimation problem I of estimation problem I of estimation problem I

(16)

Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization of estimation problem II

of estimation problem II

(17)

Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization Deterministic characterization of estimation problem III

of estimation problem III

(18)

Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming Filter and sum Beamforming

Stochastic characterization Stochastic characterization Stochastic characterization Stochastic characterization Stochastic characterization Stochastic characterization Stochastic characterization Stochastic characterization

• R

^ss

is the sources covariance matrix

– R

^ss

is MLxML matrix – R

^ss

has full rank

– H

^T

has full column rank – H has full column rank

• R

^xx

is the microphones covariance matrix

– R

^xx

=HR

^ss

H

^T

– R

^xx

is NL

^g

xNL

^g

– R

^xx

is invertible

(19)

Beamforming Algorithms

Beamforming Algorithms Beamforming Algorithms

Beamforming Algorithms

(20)

Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system of linear equations

of linear equations of linear equations of linear equations of linear equations of linear equations of linear equations of linear equations

• We want to find x for the equation Ax=b

– A is a known MxN matrix (usually M>N) – x is an unknown Nx8 vector

– b is a known Mx8 vector

• If A is a square matrix (M=N) with full rank the

• If A is a square matrix (M=N) with full rank the solution x=A

^-8

b is unique.

• For M>N

– Ax≈b

– Least Squares (LS) solution

– Minimum norm (MN) solution

(21)

Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system Known solutions for a system of linear equations (cont.)

of linear equations (cont.) of linear equations (cont.) of linear equations (cont.) of linear equations (cont.) of linear equations (cont.) of linear equations (cont.) of linear equations (cont.)

LS LS LS LS LS LS LS LS

• Minimizing the Euclidean norm squared of Ax-b.

• Optimization criterion:

MN MN MN MN MN MN MN MN

• Minimizing the L9 norm of x

• Optimization criterion:

• Solution:

• Assumptions:

– M>N

– A has full column rank : rank (A)=N

• Solution:

• Assumptions:

– M>N

– A has full column rank : rank (A)=N

(22)

LS LS LS LS LS LS LS LS

• LS=Least Squares

• Estimation problem: H

^T

g

p

=u

p

’

• Minimizing the Euclidian distance of the residual error: H

^T

g

^p

-u

^p

’

• The optimization problem:

• The solution is: g

^p

=(HH

^T

)

^-8

Hu

p

’

• Bound on L

^g – M=N

• No upper bound on L^g

– N>M

• rank(H^T)=NL^g

•

(23)

LS (cont.) LS (cont.) LS (cont.) LS (cont.) LS (cont.) LS (cont.) LS (cont.) LS (cont.)

• Special case: N>M s.t and integer

– H

^T

is a square matrix and g

^p

=(H

^T

)

^-8

u

^p

’

– Can perfectly estimated the source signals!

– It is better to have more microphones than sources!

• Advantage

– Data independent BF

• Disadvantage

– Not flexible – the whole impulse response matrix H must

be known.

(24)

LCMV Filter LCMV Filter LCMV Filter LCMV Filter LCMV Filter LCMV Filter LCMV Filter LCMV Filter

• LCMV=Linearly Constrained Minimum Variance

• Minimizing the BF output variance (power) while maintaining m linear constraints

– Forcing signals from the direction of interest to pass with specific gain and phase – Forcing zero gain in interferer’s direction

• Each linear constraint uses one degree of freedom in the weight vector Only n-m degree of freedom available for minimizing output variance.

• Definitions:

– A is mxn constraints matrix – b is mx8 desired response vector

• Assumptions:

– m linear independent constraints rank(A)=m – m<n

• Problem:

• Solution (based on Lagrange multipliers method):

(25)

LCMV LCMV LCMV LCMV LCMV LCMV LCMV LCMV

• Advantage

– Flexible – forming the beam using general constraints.

• Disadvantage

– Complexity – Computation of constrained weigh vector.

• Estimation problem:

– LCMV8: Partial channel knowledge and N≥M

– LCMV9: Full channel knowledge and N>M

(26)

LCMV LCMV LCMV LCMV LCMV LCMV LCMV LCMV1 1 1 1 1 1 1 1

• Estimation problem: H^:p^Tg^p=u^p

• The optimization problem:

• The solution is:

• Bounds on L^g:

– N=M

••

• No upper bound

– N>M

•

• Special case: N=M s.t and integer H:p is square matrix and gp=(H:p^T)^-8up

(27)

LCMV LCMV LCMV LCMV LCMV LCMV LCMV LCMV2 2 2 2 2 2 2 2

• Estimation problem: H

^T

g

^p

=u

^p

’

• The optimization problem:

• The solution is:

• Bound on Lg:

• Special case: for and integer

– H is square matrix and g =(H

^T

)

^-8

u ’

(28)

GSC GSC GSC GSC GSC GSC GSC GSC

• GSC=Generalized Sidelobe Canceller

• Transforms the LCMV 8 algorithm from a constrained problem into an unconstrained form.

• gp is divided into two components operating on orthogonal subspaces:

gp=fp-Bpwp – Where:

– Where:

– fpis the MN solution of H:pTfp=up f^p=H:p(H:pTH:p)^-8up

– B^pis blocking matrix that spans the null space of H:p H^:pT Bp=0

• The unconstraint optimization problem:

• The solution is:

– f^pis a data independent BF (MN BF) – w^pis a statistically optimum BF

• It can be shown that: g^p^LCMV8=g^p^GSC

(29)

GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.)

• LCMV8 is the sum of two orthogonal vectors: f^p and B^pw^p

– The objective of fp is to perform dereverberation – fixed BF is sufficient to perform dereverberation – The objective of -Bpwp is to reduce interference

• Increasing Lg:

– will not change the dereverberation performance.

– A better interference suppression is expected.

• As reverberation of the room increases (Lh↑), interference suppression decreases.

– Perfect dereverberation is possible (if H^:pcan be accurately estimated) BUT perfect interference suppression is not!

– One way for improvement:

LCMV filter for dereverberation followed by Wiener filter for noise reduction.

• The dimension of the nullspace of H:p^T=NL -L

(30)

GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.)

• Bounds on L

^w^p

– M=N

• No upper bound on Lg L^w^pcan be as large as we wish

• Lg↑ better interference suppression

– M>N

• Lwp=(N-8)Lg-Lh-8

• Based on the bound on L^g:

• There is a limit to interference suppression!!

• Special case: M=Q+8=N-K,K>0

– The upper bound of L^w^p depends on: N,Q,L^h

– Q,N are fixed: L^g↑ we have to use a larger L^w^p – L^h,N are fixed: Q↑ we have to use a larger L^w^p

(31)

GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.) GSC (cont.)

• Disadvantage:

– Can be done only for LCMV8

– As L

^h

and Q increases the GSC problem (L

^w^p

) becomes more difficult to solve we should expect performance

degradation related to LCMV8 performance.

• Advantages:

– Easy for analysis

– Helps to understand the LCMV8 problem

– Less complicated implementation than LCMV8 BF.

(32)

MVDR MVDR MVDR MVDR MVDR MVDR MVDR MVDR

• MVDR=Minimum Variance Distortionless Response

• A special case of LCMV8 where we maintaining only the desired signal gain.

• The constraint: h

^:p^T

(k

^p

)g

^p

=8

• The constraint: h

^:p^T

(k

^p

)g

^p

=8

• The aim of the constraint is to align the desired source at the BF output

• The optimization problem:

• The solution is:

(33)

MVDR (cont.) MVDR (cont.) MVDR (cont.) MVDR (cont.) MVDR (cont.) MVDR (cont.) MVDR (cont.) MVDR (cont.)

• Bound on L

^g

: L

^g

≥k

^p

– Special case: L

^g

=k

^p

classical delay and sum BF

• Advantages:

– Most useful in practice since it only requires the – Most useful in practice since it only requires the

knowledge of the relative delays among microphones

• Disadvantages:

– Not the best performance!

(34)

Experiments Experiments Experiments Experiments Experiments Experiments Experiments Experiments

• Goals

– Studying the effect of filter length on beamforming performance

– Comparing the different algorithms via simulations in realistic acoustic environments

• Setup

– Rectangular Room 6.7m long by 6.8m wide by 9.9m high

– Linear array of 4 omni-directional microphones

microphones – M=: (P=8,Q=9) – N=4

– s8 – desired source(mail speaker)

– s9,s: – interferences from the same female speaker.

– Lh was truncated

– Reverberation time: T60=0.:5s

• Experiment 1 – a priori knowledge

• Experiment 2 – no knowledge (using blind technique)

(35)

Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results

Beamformer output Desired Signal & output of microphone 1

S¹(k) y(k)

x¹(k)

(36)

M=:

N=4 K=8 Lh=64 Lg=889 Lg=889

(37)

Measurement Measurement Measurement Measurement Measurement Measurement Measurement

Measurement Tools Tools Tools Tools Tools Tools Tools Tools

• SIR gain in dB

• SIR[dB]=SIR^out[dB]-SIRⁱⁿ[dB]

• Where:

• IS

• Performs a comparison of the

spectral envelope between the clean and the desired signal

• IS 0 means perfectly dereverberation

• SIR gain > 800dB means perfectly interference suppression

(38)

Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results Experimental Results

A –priori knowledgeA knowledgelind nique

(39)

Conclusions Conclusions Conclusions Conclusions Conclusions Conclusions Conclusions Conclusions

• LS and LCMV9 performance are the same

• As L^h increases (with the maximum L^g) SIR gain decreases

• When L^g is set to its maximum value both the LS and LCMV9 can achieve almost perfect interference suppression and speech dereverberation (SIR gain > 800dB & IS 0)

• LCMV8 and GSC can perform perfect speech dereverberation (IS 0 for all L^g,L^h) BUT their interference suppression performance is limited

all L^g,L^h) BUT their interference suppression performance is limited

• For a fixed L^h, reducing L^g decreases the amount of interference suppression significantly for all the methods, except MVDR.

• For all algorithms IS 0 indicating that these techniques have accomplished good speech dereverberation

• MVDR methods is very robust to the changes of both Lh and Lg and when Lg is small this method can achieve more interference suppuration then the other methods BUT the IS measures are very large

(40)

Conclusions (cont.) Conclusions (cont.) Conclusions (cont.) Conclusions (cont.) Conclusions (cont.) Conclusions (cont.) Conclusions (cont.) Conclusions (cont.)

• When all the techniques suffer from some but not significant performance degradation

• When

– LS and LCMV9 suffer significantly performance – LS and LCMV9 suffer significantly performance

degradation

– LCMV8 and GSC suffer some but not serious performance degradation

• MVDR performance does not deteriorate much as

L

^h

decreases (comparing to case)

(41)

Further Work Further Work Further Work Further Work Further Work Further Work Further Work Further Work

• Option I – Resistance of GSC algorithm to channel dynamic.

• Option II – Comparison between the far-field GSC performance and the near-field GSC

GSC performance and the near-field GSC

algorithm.

(42)

Option I Option I Option I Option I Option I Option I Option I Option I

• Define a model for channel matrix changes

• Comparison between the GSC performance with and without channel dynamic.

• Based on Israel Cohen work: “Analysis of Two-

Channel Generalized Sidelobe Canceller(GSC)

with Post-Filtering”.

(43)

Option I (cont.) Option I (cont.) Option I (cont.) Option I (cont.) Option I (cont.) Option I (cont.) Option I (cont.) Option I (cont.)

• Reference case – fixed s⁸,s⁹

– M=9 (P=8,Q=8) – N=:

• Case study 8 – s⁸ was moved, fixed s⁹

• Case study 9 – s⁸,s⁹ were moved

(44)

Option II Option II Option II Option II Option II Option II Option II Option II

• Define the near-field GSC problem

• Comparison between the far-filed and the near-field GSC algorithm performance.

• Based on Iain A. McCowan, Darren C. Moore

ans S. Sridharan work: “Near-field Adaptive

Beamformer for Robust Speech Recognition”

(45)

םוכיס םוכיס –

ת – ת הנכה תולאשל תובוש הנכה תולאשל תובוש

• היעבה דה –

ע שער תשג

"

םינופורקימ ךרעמב שומיש י

• הרטמ עש –

ע רוקמ תוא ךור

"

םיעירפמ יוכידו םידה לוטיב י

• היעבה ןורתפל תומייק תושיג רואית תוקינכט –

המולא בוציע (Beamforming)

• רמאמב עצומה ןורתפה תגצה תאוושה –

תושיגב םידה לוטיבו םיעירפמ יוכיד ןבומב םיעוציב

beamforming םינוש הערפה יראתמ תחת תונוש

• השיג לכב שמתשהל יאדכ יתמ

?

– לש םלהל הבוגתה ךרוא רשאכ עודי ץורעה

ב שומישל תופידע הנשי ו LS

LCMV9

ה ןנסמשכ LCMV9 ו LS ב שומישל תופידע הנשי עודי ץורעה לש םלהל הבוגתה ךרוא רשאכ – ה ןנסמשכ

beamformer וכרעב

ילמיסקמה

– לש םלהל הבוגתה ךרוא רשאכ עודי וניא ץורעה

ב שומישל תופידע הנשי LCMV8

ו GSC

ה ןנסמשכ beamformer

וכרעב ילמיסקמה

– רתוי הברה MVDR

יטסובור רצק ןנסמ תריחבלו ץורע ךורעש תואיגשל

, לוטיב תלוכי םלוא

ידמל השלח ולש םידהה ,

הדובע יאנתב םג םיילמיטפוא

) רצק Lh , ךורעשו ךורא Lg קייודמ Lh

.(

רתויב ןטקה אוה ולש םיצוליאה טסש איה ךכל הביסה ,

שומימל רתוי לק םג אוה ןכל .

יאדכו

םידה לוטיבל תפסונ הקינכט םע בולישב קר וב שמתשהל .

• תירשפא הבחרה תגצה

(46)

References References References References References References References References

• Jacob Benesty, Jingdong Chen, Yiteng (Arden) Haung and Jacek Dmochowski, "On Microphone-Array Beamforming from a MIMO Acoustic Signal Processing Perspective", IEEE Trans. On audio, speech and language processing, VOL. 85, NO. :, March 9007.