[[1 2 3]
[4 5 6]
[7 8 9]]
-9.51619735393e-16
Listing 11.8: Sample output from creating the determinant of a matrix.
Like the trace operation, alone, the determinant operation is not interesting, but it offers a simpler notation and it is used as an element in other key matrix operations.
11.6 Rank
The rank of a matrix is the estimate of the number of linearly independent rows or columns in a matrix. The rank of a matrix M is often denoted as the function rank().
rank(A) (11.10)
An intuition for rank is to consider it the number of dimensions spanned by all of the vectors within a matrix. For example, a rank of 0 suggest all vectors span a point, a rank of 1 suggests all vectors span a line, a rank of 2 suggests all vectors span a two-dimensional plane. The rank is estimated numerically, often using a matrix decomposition method. A common approach is to use the Singular-Value Decomposition or SVD for short. NumPy provides the matrix rank() function for calculating the rank of an array. It uses the SVD method to estimate the rank. The example below demonstrates calculating the rank of a matrix with scalar values and another vector with all zero values.
# vector rank
Listing 11.9: Example of calculating the rank of vectors.
Running the example prints the first vector and its rank of 1, followed by the second zero vector and its rank of 0.
[1 2 3]
1
[0 0 0 0 0]
0
11.6. Rank 86
Listing 11.10: Sample output from creating the rank of vectors.
The next example makes it clear that the rank is not the number of dimensions of the matrix, but the number of linearly independent directions. Three examples of a 2 × 2 matrix are provided demonstrating matrices with rank 0, 1 and 2.
# matrix rank
Listing 11.11: Example of creating the rank of matrices.
Running the example first prints a zero 2 × 2 matrix followed by the rank, then a 2 × 2 with a rank 1 and finally a 2 × 2 matrix with a rank of 2.
Listing 11.12: Sample output from creating the rank of matrices.
11.7. Extensions 87
11.7 Extensions
This section lists some ideas for extending the tutorial that you may wish to explore.
Modify each example using your own small contrived array data.
Write your own functions to implement one operation.
Research one example where each operation was used in machine learning.
If you explore any of these extensions, I’d love to know.
11.8 Further Reading
This section provides more resources on the topic if you are looking to go deeper.
11.8.1 Books
Section 3.4 Determinants. No Bullshit Guide To Linear Algebra, 2017.
http://amzn.to/2k76D4
Section 3.5 Matrix inverse. No Bullshit Guide To Linear Algebra, 2017.
http://amzn.to/2k76D4
Section 5.1 The Properties of Determinants, Introduction to Linear Algebra, Fifth Edition, 2016.
http://amzn.to/2AZ7R8j
Section 2.3 Identity and Inverse Matrices, Deep Learning, 2016.
http://amzn.to/2B3MsuU
Section 2.11 The Determinant, Deep Learning, 2016.
http://amzn.to/2B3MsuU
Section 3.D Invertibility and Isomorphic Vector Spaces, Linear Algebra Done Right, Third Edition, 2015.
http://amzn.to/2BGuEqI
Section 10.A Trace, Linear Algebra Done Right, Third Edition, 2015.
http://amzn.to/2BGuEqI
Section 10.B Determinant, Linear Algebra Done Right, Third Edition, 2015.
http://amzn.to/2BGuEqI
11.9. Summary 88
11.8.2 API
numpy.ndarray.T API.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.ndarray.T.
html
numpy.linalg.inv() API.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.inv.
html
numpy.trace() API.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.trace.html
numpy.linalg.det() API.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.det.
html
numpy.linalg.matrix rank() API.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.matrix_
rank.html
11.8.3 Articles
Transpose on Wikipedia.
https://en.wikipedia.org/wiki/Transpose
Invertible matrix on Wikipedia.
https://en.wikipedia.org/wiki/Invertible_matrix
Trace (linear algebra) on Wikipedia.
https://en.wikipedia.org/wiki/Trace_(linear_algebra)
Determinant on Wikipedia.
https://en.wikipedia.org/wiki/Determinant
Rank (linear algebra) on Wikipedia.
https://en.wikipedia.org/wiki/Rank_(linear_algebra)
11.9 Summary
In this tutorial, you discovered important linear algebra matrix operations used in the description of machine learning methods. Specifically, you learned:
The Transpose operation for flipping the dimensions of a matrix.
The Inverse operations used in solving systems of linear equations.
The Trace and Determinant operations used as shorthand notation in other matrix operations.
11.9. Summary 89
11.9.1 Next
In the next chapter you will discover sparsity and sparse matrices.
Chapter 12
Sparse Matrices
Matrices that contain mostly zero values are called sparse, distinct from matrices where most of the values are non-zero, called dense. Large sparse matrices are common in general and especially in applied machine learning, such as in data that contains counts, data encodings that map categories to counts, and even in whole subfields of machine learning such as natural language processing. It is computationally expensive to represent and work with sparse matrices as though they are dense, and much improvement in performance can be achieved by using representations and operations that specifically handle the matrix sparsity. In this tutorial, you will discover sparse matrices, the issues they present, and how to work with them directly in Python. After completing this tutorial, you will know:
That sparse matrices contain mostly zero values and are distinct from dense matrices.
The myriad of areas where you are likely to encounter sparse matrices in data, data preparation, and sub-fields of machine learning.
That there are many efficient ways to store and work with sparse matrices and SciPy provides implementations that you can use directly.
Let’s get started.
12.1 Tutorial Overview
This tutorial is divided into 5 parts; they are:
1. Sparse Matrix
2. Problems with Sparsity
3. Sparse Matrices in Machine Learning 4. Working with Sparse Matrices
5. Sparse Matrices in Python
90