As we have seen a matrix is just a rectangular array of scalars (real numbers).

(1)

As we have seen a matrix is just a rectangular array of scalars (real numbers).

The size of a matrix indicates its number of rows and columns. A matrix with m rows and n columns is said to be of size m × n. The order is important here. The number of rows is alway specified first. Thus,

1 2 3 4 5 6

is a 2 × 3 matrix, while



 1 2 3 4 5 6





is a 3 × 2 matrix. We add two matrices simply by adding corresponding entries.

This means that if A is an m × n matrix, then we may add it to any other m × n matrix, but not to a matrix of a different size. For example, we have

1 2 3 4 5 6

+ 3 0 −1 2 2 −3

= 1 + 3 2 + 0 3 − 1 4 + 2 5 + 2 6 − 3

= 4 2 2 6 7 3

. On the other hand we can’t add

1 2 3 4 5 6

and



 1 2 3 4 5 6





because they differ in size. We say that matrix addition is performed entrywise because we add corresponding entries. If we want to subtract one matrix from another, we simply subtract the corresponding entries.

The notion of the product of a scalar and a matrix is similar. To multiply an matrix A by a scalar c, we simply multiply entrywise by c. For example

2 1 2

−3 0

=

2 · 1 2 · 2 2 · −3 2 · 0

=

2 4

−6 0

These operations are intuitive and easy to grasp. On the other hand the matrix product is not intuitive. Moreover, it’s complicated and involves enough calculation so that it is easy to make a mistake. There are several ways to view the matrix product and so there are several ways to define it. Two definitions will be presented here. The first one is useful theoretically although it is a bit more difficult to compute. The second definition is easier to compute, but it obscures some of the product’s theoretical implications.

For both of the reasons mentioned above, it is useful to define an “intermediate”

notion of this product. We shall begin by defining the product of an m × n matrix and an n × 1 matrix. Now an n × 1 matrix with entries x

¹

, x

2

, . . . , x

n

has the form





 x

1

x

2

.. . x

n







1

(2)

and as such is just a (vertically displayed) list of numbers. In other words it is a vector . For this reason we shall call n × 1 matrices column matrices or column vectors . Readers who are put off by calling such objects vectors can just think of them as column matrices. Such readers should begin to feel more comfortable when we come to the formal study of vectors.

In order to emphasize the special nature of column vectors we use a special notation for them. They are denoted by lower case letters decorated with a half–

arrow. Thus, we may write

~ x =





 x

¹

x

2

.. . x

n





 , ~ v =



 1 2 3



 , ~ u =

3 −4

, etc.

Armed with this new notation, we may use it to view a matrix as being determined by its columns. The columns of the matrix

1 2 3 4 5 6

are

~c

1

= 1 4

, ~c

2

= 2 5

and ~c

3

= 3 6

and so we may write

1 2 3 4 5 6

= [ ~c

¹

~c

²

~c

³

] . If we have a general m × n matrix

A =







a

11

a

12

. . . a

1n

a

21

a

22

. . . a

2n

.. . .. . . .. .. . a

m1

a

m2

. . . a

mn





 ,

then its n columns are

~c

1

=





 a

11

a

²¹

.. . a

m1







, ~c

2

=





 a

12

a

²²

.. . a

m2







, . . . , ~c

n

=





 a

1n

a

2n

.. . a

mn





 .

With these preparations, we may now define the matrix–vector product. Just

as with matrix addition it is possible to perform this multiplication only when the

matrix and column vector have the “right” respective sizes. Specifically, if A is an

m × n matrix, then the column vector must have size n × 1. The formal definition

is as follows.

(3)

Definition 1. If A is an m ×n matrix with columns ~c

¹

, ~c

²

, . . . , ~c

n

and ~ x is an n ×1 column vector with entries x

1

, x

2

, . . . , x

n

, then the product of ~ x by A is the m × 1 column vector determined by the formula

A~ x = [~c

¹

~c

²

· · · ~ c

n

]





 x

1

x

2

.. . x

n







= x

¹

~c

¹

+ x

²

~c

²

+ · · · + x

n

~c

n

.

Here’s an example. Suppose

A = 1 2 3 4 5 6

.

Since A is a 2 × 3 matrix, it can multiply only 3 × 1 column vectors. So if

~ x =



 3 2 1



 ,

then

A~ x = 1 2 3 4 5 6



 3 2 1



 = 3 1 4

+ 2 2 5

+ 1 3 6

= 10 28

.

Now let’s discuss the “computational” definition of the matrix–vector product.

The notion of a row matrix or row vector will be useful here. By definition a row matrix or row vector is 1 × n matrix. Note that if

~ r

1

= [ r

1

r

2

. . . r

n

] and ~x =



 x

¹

x

2

.. .x

n



 ,

then

~ r

1

~ x = [ r

1

r

2

. . . r

n

]





 x

1

x

2

.. . x

n







= x

1

[ r

1

] + x

2

[ r

2

] + · · · + x

n

[ r

n

]

= [ x

1

r

1

+ x

2

r

2

+ · · · x

n

r

n

] For example

[ 1 2 3 ]



 3 2 1



 = [ 3 · 1 + 2 · 2 + 1 · 3 ] = [ 10 ] = 10.

(4)

The point is that the product of a 1 × n matrix and an n × 1 column vector is a 1 × 1 matrix which is just a real number. Note that

3 1 4

+ 2 2 5

+ 1 3 6

= 3 · 1 + 2 · 2 + 1 · 3 3 · 4 + 2 · 5 + 1 · 6

= 10 28

. In our new notation, we may write

3 · 1 + 2 · 2 + 1 · 3 = [ 1 2 3 ]



 3 2 1



 and

3 · 4 + 2 · 5 + 1 · 6 = [ 4 5 6 ]



 3 2 1





Hence, we have

1 2 3 4 5 6



 3 2 1



 =







[ 1 2 3 ]



 3 2 1





[ 4 5 6 ]



 3 2 1











= 10 28

.

In other words the first entry in A~x is the product of the first row of A and ~x and the second entry in A~x is the product of the second row and ~x. In still other words if we define

~

r

1

= [ 1 2 3 ] and ~r

²

= [ 4 5 6 ] then we have

~ r

1

~ x = [ 1 2 3 ]



 3 2 1



 = [ 10 ] = 10 and

~ r

²

~ x = [ 4 5 6 ]



 3 2 1



 = [ 28 ] = 28

and so the matrix product takes the form A~ x = ~r

1

~ x

~ r

2

~ x

= 10 28

. Here are some more examples.

1 2 5 3

1 −1

= 1 − 2 5 − 3

=

−1

−2

(5)



 1 3 2 4 0 6





2 3

=



 2 + 9 4 + 12 0 + 18



 =



 11 16 18









2 −1 3

1 0 −4

5 −3 7









−3 2 4



 =





−6 − 2 + 12

−3 + 0 − 16

−15 − 6 + 28



 =



 4

−19 7





It is worth emphasizing that we may only form the product of m × n matrices and n × 1 column vectors and that the result is an m × 1 column vector.

Another dividend is that the problem of solving a system of linear equations has an equivalent formulation in the language of the matrix vector product. To illustrate this let’s consider an example. Suppose we have the linear system

x

1

− 2x

²

− 3x

³

= 4 2x

1

+ x

3

= −1 3x

¹

+ 2x

²

− x

³

= 2 If we write

A =





1 −2 −3

2 0 1

3 2 3



 , ~ x =



 x

1

x

2

x

3



 and ~b =



 4

−1 2



 , then

A~ x = ~b is the same as





1 −2 −3

2 0 1

3 2 3







 x

1

x

2

x

3



 =



 4

−1 2



 or





x

1

− 2x

²

− 3x

³

2x

1

+ x

3

3x

¹

+ 2x

²

− x

³



 =



 4

−1 2





Thus the equation A~x = ~b is just another way of encoding the information determined by the linear system. Since the matrix A is determined by the coefficients of the equations in the linear system, it is called the coefficient matrix for the system. In general if we are given a system of m equations in n unknowns of the form

a

11

x

1

+ a

12

x

2

+ . . . + a

1n

x

n

= b

1

a

21

x

1

+ a

22

x

2

+ . . . + a

2n

x

n

= b

2

.. .

a

m1

x

1

+ a

m2

x

2

+ . . . + a

mn

x

n

= b

n

, then the associated coefficient matrix is







a

11

a

12

. . . a

1n

a

21

a

22

. . . a

2n

.. . .. . . .. .. . a

m1

a

m2

. . . a

mn







.

(6)

If we also write

~ x =





 x

1

x

2

.. . x

n





 and





 b

1

b

2

.. . b

n





 ,

then the equation A~x = ~b contains the same information as the original linear system.

Now we are ready to tackle the product of two matrices. We’ve done the hard work. This next step is relatively easy.

Definition 2. Suppose A is an m × n matrix and B is an m × p matrix. In this case it is possible to form the product AB as follows. Write B in column vector notation

B = [ ~c

¹

~c

²

. . . ~c

p

] ,

where ~c

1

, ~c

²

, . . . , ~c

p

are the column vectors determined by the columns of B. Since each column is an n × 1 column vector, we may form the matrix–vector products A~c

¹

, A~c

,

. . . , A~c

p

. The product is by definition

AB = A [ ~c

¹

~c

2

. . . ~c

p

] = [ A~c

¹

A~c

2

· · · A~c

p

] .

Thus, AB is the m × p matrix whose columns are A~c

¹

, A~c

2

, . . . , A~c

p

. Here is an example.

1 2 3 4

5 6 7 8

= 1 2 3 4

5 7

1 2 3 4

8 6

=







[ 1 2 ] 5 7

[ 1 2 ] 8 6

[ 3 4 ] 5 7

[ 3 4 ] 8 6







= 5 + 14 8 + 12 15 + 28 24 + 24

= 19 20 43 48

.

Note that in the product

the (1, 1)–entry is the product of the first row and the first column, the (2, 1)–entry is the product of the second row and the first column, the (1, 2)–entry is the product of the first row and the second column and the (2, 2)–entry is the product of the second row row and the second column.

In general, the (i, j)– entry in a matrix product is the matrix–vector product of

the i

^th

row and the j

^th

column.

(7)

Here are some more examples.

1 3 0 2 2 4 6 1





 3 2 0 1 4 5 1 2





 =







[ 1 3 0 2 ]





 3 0 4 1





 [ 1 3 0 2 ]





 2 1 5 2







[ 2 4 6 1 ]





 3 0 4 1





 [ 2 4 6 1 ]





 2 1 5 2













= 3 + 0 + 0 + 2 2 + 3 + 0 + 4 6 + 0 + 24 + 1 4 + 4 + 30 + 2

= 5 9 31 40

(1)





 3 2 0 1 4 5 1 2







1 3 0 2 2 4 6 1

=







[ 3 2 ] 1 2

[ 3 2 ] 3 4

[ 3 2 ] 0 6

[ 3 2 ] 2 1

[ 0 1 ] 1 2

[ 0 1 ] 3 4

[ 0 1 ] 0 6

[ 0 1 ] 2 1

[ 4 5 ] 1 2

[ 4 5 ] 3 4

[ 4 5 ] 0 6

[ 4 5 ] 2 1

[ 1 2 ] 1 2

[ 1 2 ] 3 4

[ 1 2 ] 0 6

[ 1 2 ] 2 1







=







7 17 12 8

2 4 6 1

14 32 30 13

5 11 12 4



 (2) 

Notice that in (1) we have the product of a 2 × 4 and a 4 × 2 matrix and the result is a 2 × 2 matrix. In (2), the product of the same matrices in reverse order is given.

Now we are calculating the product of a 4 × 2 and a 2 × 4 and the result is a 4 × 4

matrix.

As we have seen a matrix is just a rectangular array of scalars (real numbers).