Appendix A: A Brief Introduction to Matrices and Vectors

INTRODUCTION

Modern regression and related methods have been developed using notation and concepts of matrix algebra. Indeed, many of the underlying theoretical results could not have been developed without matrix algebra. Although we have, as much as possible, developed most of the concepts in this book without resorting to matrix notation, some of the more advanced topics require presenting the equations in matrix notation. In addition, most of the results associated with multiple regression and related topics can be presented more compactly and simply using matrix notation. This appendix reviews the basic definitions and notation necessary to understand the use of matrices in this book. We will concentrate on notational, as opposed to computational or theoretical, issues because computers take care of the computations and there are many excellent books on the theory of linear algebra for those who would like to understand linear algebra in more detail.

DEFINITIONS

A scalar is an ordinary number, such as 17.

A matrix is a rectangular array of numbers with r rows and c columns. For example, let X be the 4 × 3 matrix

Each number in the matrix is called an element. Typically (but not always), the matrix is denoted with a boldface letter and the elements are denoted with the same letter in regular type. The subscripts on the symbol for the element denote its row and column. x_ij is the element in the ith row and jth column. For example, the element x₂₃ is the element in the second row and the third column, 9.

A vector is a matrix with one column. For example, let y be the 4 × 1 matrix, or 4 vector

Because all vectors have a single column, we only need one subscript to define an element in a vector. For example, the first element in the vector y is y₁ = 17. There are no fundamental differences between matrices and vectors; a vector is just a matrix with one column.

A square matrix is one in which the number of rows equals the number of columns. A square matrix S is symmetric if s_ij = s_ji. A square matrix D is diagonal if all the off-diagonal elements are zero, that is, if d_ij = 0 unless i = j. For example,

is a 3 × 3 symmetric matrix, and

is a 3 × 3 diagonal matrix. A diagonal matrix in which all the diagonal elements are 1 is called the identity matrix and denoted with I. For example, the 4 × 4 identity matrix is

The identity matrix plays a role similar to 1 in scalar arithmetic.

ADDING ...