Appendix B. Linear algebra

To understand advanced machine learning topics, you need to know some linear algebra. If you want to take an algorithm from an academic paper and implement it in code or investigate algorithms outside of this book, you’ll probably need a basic understanding of linear algebra. This appendix should serve as a light refresher or introduction if you’ve had this material before but it’s been a while and you need a reminder. If you’ve never had this material before, I recommend that you take a course at a university, work through a self-study book, or watch a video. Free tutorial videos are available on the internet^[1] as well as full recordings of semester-long courses.^[2] Have you ever heard “Math is not a spectator sport”? It’s true. Working through examples on your own is necessary to reinforce what you’ve watched others do in a book or video.

^[1] Gilbert Strang has some lectures that are free to view at http://www.youtube.com/watch?v=ZK3O402wf1c. You can also get the course materials at http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/. His lectures aren’t difficult to follow and communicate the key points of linear algebra. In addition, his graduate-level course on computational science is very good: http://www.youtube.com/watch?v=CgfkEUOFAj0.

^[2] I’ve heard many great things about the linear algebra videos on Kahn Academy’s website: http://www.khanacademy.org/#linear-algebra.

We’ll first discuss the basic building block of linear algebra, the matrix. Then we’ll discuss some basic operations on matrices, including taking the matrix inverse. We’ll address the vector norm, which often appears in machine learning, and we’ll conclude by discussing how we can apply calculus to linear algebra.